From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD0CC83F09 for ; Wed, 9 Jul 2025 01:53:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 25DBC8D000C; Tue, 8 Jul 2025 21:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 20DBB8D0001; Tue, 8 Jul 2025 21:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0FCFF8D000C; Tue, 8 Jul 2025 21:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EB9A78D0001 for ; Tue, 8 Jul 2025 21:53:22 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BD8761289A5 for ; Wed, 9 Jul 2025 01:53:22 +0000 (UTC) X-FDA: 83643053844.01.F72CD94 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf26.hostedemail.com (Postfix) with ESMTP id DAD3A140006 for ; Wed, 9 Jul 2025 01:53:20 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dgElVc07; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752026000; a=rsa-sha256; cv=none; b=1VR2DSoEg6sm/jfiU7wbCZHUTl9jbvyIxVLpdwA02eDkME0kCenSKG638oe91+npd45Fb1 o5Bo/zkhTjX/f2AO+Hy1nEJA5UETCti05lpLSirqQ7u0aGxKPEcFRmKAKXTWQjJe9td9m4 0yttZ2uEf5vhy3wrJu4uBy6ivmfQAbA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dgElVc07; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752026000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=swPCx2ZCPdr3kVviURV1cSakbdre5MF5VQyGIctNo/0=; b=RdIvqvY8IVR4Ik6QYE3+i+oNB/7ZT/jDEc3geY3cgY33CK7ip0Xs51RWhk8Ib3YXV09BKP SCH/TRk+inbPo6i+bxjGPG2jW1GfaX3VDCD0dC+Ean8KZNrjXNkea/FcIbNAEVagJOdqKy hLIuo21eDLYgjuwfh2IvePvthBprRE0= Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-7399a2dc13fso5812608b3a.2 for ; Tue, 08 Jul 2025 18:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752026000; x=1752630800; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=swPCx2ZCPdr3kVviURV1cSakbdre5MF5VQyGIctNo/0=; b=dgElVc07ATHmvoU+TqDlRi3+rC4TSKypwEF/VcGxAz88TaKlA9Ebvm1UBf8Y42BOrM xG8rpezOQ2PS3Ev/90p8biJiRQkhbenK2R+yYqZAgpiEqUb3Ed7LT1Posv2GnLTvHA0x HsSXVCAooaHBnUwd4bwsZAJBMui3jkex8BRZpbPzER0WAkICAYLYl0cd+b3B0zXbhic9 K2aHVYJm1xVo6IAvSY2HMKHcxVknxxo+cbNnLXG4LeLm04UOq1CM+Z7ZaGfcVw9Efs46 DG4H4yP+FT6aTJZ9D+cskI3zvhqa+4SDma7fcD957FQVh3u+qj2UMmh+IB1xNNy5vUgZ di0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752026000; x=1752630800; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=swPCx2ZCPdr3kVviURV1cSakbdre5MF5VQyGIctNo/0=; b=OXo3kOCF4hGx5ycs3mUPjzPZqrHCDsy0iPWprgcJ81xhKxqKpzJv9RhGhXNG/L0w6a PpAIZYFoPn4znCxlG2qhQw7uDq2wvAGe4m22NtQU75NeuhuP92anN7dyPD9XOhzSdZPz 6zI7zyvp/ZqVWmFW7wDZoa4vm67/d0mniw97JyPqk6ntPy6pO3pT6fYTw8Khqtby8GM2 IJ+EYZRNo3u6+OhEJJahc+CYAXs+ytq4mq/zCEbmR+WX1/HPs9+kh7hDIFRQWno51pnD uYveCwg9evVDr/TEgmC6CWBGQ4YUO07af9daFzI7OvScHkdjt+q7VWKfldREjNkw0C+o pF5g== X-Forwarded-Encrypted: i=1; AJvYcCU5aM6bU4rwSneFL9uCch7WlDD375nZvAx5pMgw/824vyt99vAbCMS9uI4ff5JOjaotDfXuMHbm5w==@kvack.org X-Gm-Message-State: AOJu0Yym5/aiqtD/QVFW2iTFYYrS+P3ot2+QDl3m+hhMNeGBtAkAA2DW G41kDdT73WbafK6Rwb81j53FmszIh+R4xHTuYBBCTqtuHkP0qx38khwZRc9ifg== X-Gm-Gg: ASbGncvgTwhjyEjOooUhleiJ4IiCwXA+DDZAIdOEDihogWDuFNwrGLikFUamSoLufhO N44Clno6764+Ybb8ujT9jrMp3Qnt9EGDR55xaYnCpCiyyQ31MsiRqQY1gwzRY5GQYCr6mrcOE2i clvmjZio9vhA2dGXfD+Pnr1OzzIV9UlZLCOMw+MXw8SCVHijzfE0EFGAjKfw6M5fXCezUCvB31R Lc+qoXnfvdgR7d019+kY3jFOvaLMh0nU71gvATCGFFWWg13H9Kgpd957atJxU+tbdN66YHVsKib oKmzGO89Ep4Oxpud4K3jCi2xVa0fieHQJI9ZniSftjwIeHBum/tuiMpqb5kWDddo13YK7g1/fp3 7h8G0wWxvuKqiqTxwbySnuPi4YBE= X-Google-Smtp-Source: AGHT+IFyHvkwCKA3WtV4emwIkj9KrLzmSuKcCHFtpkTnsKskKbbrpP9oMuPYme/O7vMCfh2JX9n7Rg== X-Received: by 2002:a05:6a00:22d5:b0:747:aa79:e2f5 with SMTP id d2e1a72fcca58-74ea60b6b52mr1390164b3a.0.1752025999720; Tue, 08 Jul 2025 18:53:19 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74ce417e842sm13620416b3a.79.2025.07.08.18.53.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 08 Jul 2025 18:53:19 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v2 5/6] mm: Introduce alloc_frozen_pages_nolock() Date: Tue, 8 Jul 2025 18:53:02 -0700 Message-Id: <20250709015303.8107-6-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250709015303.8107-1-alexei.starovoitov@gmail.com> References: <20250709015303.8107-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DAD3A140006 X-Stat-Signature: dbrw4ssp6q8d3kcxdb986b8ese6t6wq1 X-HE-Tag: 1752026000-784932 X-HE-Meta: U2FsdGVkX1+4m13mFjL3VE//q+UVonHNU3axE7STrHPAEMSXAOxhFofTDOEJSfhDGT0ByUL73Y5Lv8ylnq+eA385109oTa4Ehk278dczy+lOazHZDnRBLOJhRHgizu46rfgycRxNWLeBoUwXJXSI7FlTBUTRtJmNqMLDDySTIob0qAmLtmBhO7I+3dJaMhT0IDt4FyaCC9UnfNo/VbDpYgrYWxgsiG4To5Gs5iBshKjbYvvqJpqdZEK0CPYOppfIbeYWRyLV/X6/t+de4ainimkf8daBsvy0UX7ZwH0QkYvnbvPjcdAn9e1+iJQ31CQw3ePrtnn61n26RevkBQXMC5epL1lEUmI2oPRxfOtPe4w8Rb3LhQgRNpAr5hwdgrQQiozn2Oo8Cy6AAJM2dU9zThTf1wlqJQ4cQyDlzTLFRDbabSqVLdWndeffR9rJhJs84s3kdTHZx1QXhi5Nh6dznew9orcK/3TJeChCRtMQGJ6uHmZMVvYGCyrdXF/CE1rYNEkCTadtLDD6Kz9yLorH5nG8XFq4vOdxQzspFl3inTLs80ZBATfdqyd+AKVNYmHDTjFdipNVCPRIYdVzZgLIbNhlhKowAp6RnpKI5FF9kw/Pxbf3fZXbAtkHCDCWSlBwiYPGdkzDshUKgogUQOWbrjR88PGhe3ju4Wp1hj/RWyovHlpVHZ2gJX9d2cftHit/ovzf8TkeVQpEwzSh3qfsrPqpXOmaVKOPobbC5avpmE//gOdcNB66jymy9a6au8QvSIyy7FMoDk9+meXpEt8SnJoWOzzTU8t4N1SLUclYZdNYSzI1mFiP0P7POpGutfNS13fM/fbUGxC20t815a/dfIfxcGhclmzOfno4Q+hoVnNxhWugT9AvTtZupLfRc+r5RCqlvJFIBsknQVdfeeeiR2K2ZZNLtAUte1KYEDLyho7UYt1v44mkGxQeaK+gWbrakm7UmlKgEhTwZwBlMaj IOWA5Zn2 URTL4bUCs6xoaWXhLQie8wS+QasovwTuo9a8QQtpRFqSulMxQuv4ws+H9tHTeqi2/O7TrGm0kQZJZ9pcmoylE0JWwOtYGfzufp7xROfniM46iMjh1FOdVXoiMrvFgTpiaeX5qF9cIGrDRtuAwM7K3Hb33wQw9ktm1ZmNSHw+jSmNmBEzRR48vgrXXwidGGHNY499bTNpW/L3zLxY9D7heZ/X8CnyqG6vrT3Aw/lijlRFEe5/HCLt0JJxxTwsnBQDiXD+SFYc7fn/WqQFoVmeXAysJ6CgFZxRG1XiYXENMa+SsbcGSdEabbkdLBL/+2qu2PYRUsckUY7ULQHfeA2RJCOa9qRINn4F0MQHtqcLgdOvhtKb7r86OdNHD6W2Dp4bKIxZ7bTc8PD3SAlE/8RGj3q2q6j7XIkcocxza5DnlFCGGnLsxjcDvwar0YrtGyP+dGGnYO96EJ/9FRyFlfrljO3wGW0lJbC4RkVGpaX1Zbx55HoUmcbX57oOWO6iKrNfKWIdrkzng4djNUqVjbdUpZZxz8rZTydFO4SEq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Split alloc_pages_nolock() and introduce alloc_frozen_pages_nolock() to be used by alloc_slab_page(). Signed-off-by: Alexei Starovoitov --- mm/internal.h | 4 ++++ mm/page_alloc.c | 48 +++++++++++++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 21 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5b0f71e5434b..ea85cf703331 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -837,6 +837,10 @@ static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int ord #define alloc_frozen_pages(...) \ alloc_hooks(alloc_frozen_pages_noprof(__VA_ARGS__)) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order); +#define alloc_frozen_pages_nolock(...) \ + alloc_hooks(alloc_frozen_pages_nolock_noprof(__VA_ARGS__)) + extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); extern void zone_pcp_enable(struct zone *zone); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 148945f0b667..11a184bab03c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7504,23 +7504,7 @@ static bool __free_unaccepted(struct page *page) #endif /* CONFIG_UNACCEPTED_MEMORY */ -/** - * alloc_pages_nolock - opportunistic reentrant allocation from any context - * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. - * @nid: node to allocate from - * @order: allocation order size - * - * Allocates pages of a given order from the given node. This is safe to - * call from any context (from atomic, NMI, and also reentrant - * allocator -> tracepoint -> alloc_pages_nolock_noprof). - * Allocation is best effort and to be expected to fail easily so nobody should - * rely on the success. Failures are not reported via warn_alloc(). - * See always fail conditions below. - * - * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. - * It means ENOMEM. There is no reason to call it again and expect !NULL. - */ -struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) { /* * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. @@ -7583,16 +7567,38 @@ struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int or /* Unlike regular alloc_pages() there is no __alloc_pages_slowpath(). */ - if (page) - set_page_refcounted(page); - if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) && unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) { - free_pages_nolock(page, order); + __free_frozen_pages(page, order, FPI_TRYLOCK); page = NULL; } trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype); kmsan_alloc_page(page, order, alloc_gfp); return page; } +/** + * alloc_pages_nolock - opportunistic reentrant allocation from any context + * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. + * @nid: node to allocate from + * @order: allocation order size + * + * Allocates pages of a given order from the given node. This is safe to + * call from any context (from atomic, NMI, and also reentrant + * allocator -> tracepoint -> alloc_pages_nolock_noprof). + * Allocation is best effort and to be expected to fail easily so nobody should + * rely on the success. Failures are not reported via warn_alloc(). + * See always fail conditions below. + * + * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. + * It means ENOMEM. There is no reason to call it again and expect !NULL. + */ +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +{ + struct page *page; + + page = alloc_frozen_pages_nolock_noprof(gfp_flags, nid, order); + if (page) + set_page_refcounted(page); + return page; +} EXPORT_SYMBOL_GPL(alloc_pages_nolock_noprof); -- 2.47.1