From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 584D4C83F17 for ; Wed, 16 Jul 2025 02:30:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2278A6B00C4; Tue, 15 Jul 2025 22:30:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B0E06B00C6; Tue, 15 Jul 2025 22:30:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09F7A6B00C7; Tue, 15 Jul 2025 22:30:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E09876B00C4 for ; Tue, 15 Jul 2025 22:30:07 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id ACBE412B22C for ; Wed, 16 Jul 2025 02:30:07 +0000 (UTC) X-FDA: 83668548054.06.A4C2CB4 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf30.hostedemail.com (Postfix) with ESMTP id CC29580006 for ; Wed, 16 Jul 2025 02:30:05 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=UUorm37n; spf=pass (imf30.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752633005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=PSdW12rwOjI+YdIGUsiU9yoI9O2QTfiD7jmphZWWO3exe+1QMivRuk4fXIs2stjXsNv1NA T0+ty3xOqkz7wpkBIB0nXcS5VXUVI6B/Ogyc23HNW5nYVvsbwUbN2RdIQ4n5/nNCYyVaac 0aCmeRW7nPUKaj2W5AcEuDATVvG4t/Y= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=UUorm37n; spf=pass (imf30.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752633005; a=rsa-sha256; cv=none; b=Gh8nm8wKXquJusPIJUCoLHm7/4zvSrvGaPPtwhBn7dyKLY/aGYNItOS+ZsVAIqsr21s1Jz 5iPOX/qT6hS2FtSTFrI/+B28BuTdZQxO/qUPD+LT5Gxw+xD3vyX2Y2zSg2lR6cI+sF5l5h znKto2WJYMcO2+5NG9NIzct72rcgNjQ= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-234bfe37cccso50721635ad.0 for ; Tue, 15 Jul 2025 19:30:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752633005; x=1753237805; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=UUorm37ngNXCA2X4lUV7HCiWCR4i4G/Po18vOzC2kRtK7fdh8+cCy7/npQyafBDCwP iKdEangSJ+PjxW/i6sWo7NBX2LvUF/yNTtu1Fong4G267Quf/dLy3QNp8xwKplKKQU09 /IP5/5T/XmZvJ6YsyYAj/kAuUwF34AMCmjv7pCjdWtPdRMbIUfT4OB0O5+FpsgXCcCpl R7QDfkmpm0VH0G+i4u/sEwzeJfTefTVZsTGApq/r1ZRRyduGBZJZaCKP0EYrW266MA3+ 9baHL2lO1b8LzFAywo+uM041Hs52RobXAKLSxeM06ZCONnOWs/ImnMKA3iD4T57hC4XQ itVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752633005; x=1753237805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=qfLH0tGysYu1HaIzKTw/w1wG+v1FPBEapCbi4sSwDzWQp6mlzQMF+7pEZ7FUTDE/ZH bsiEqasIf+VDFncF9or/GfkI9/7QPTOQsG9itW5Kx51iBwylxXcAD13KCgM9tcTQhi7X PoKFDnjgK5T7eSSzN22tr+K26ILuJqHz7h09H37cST4Z1KkaLl04SWeN8b9UuDenlAtj V9tlO89jvBxtpk4o/no6cMbohzOylT8iNBnTJFXL7I8TP/TKYITnXi7tjBU0lAi6J/dR nWa4LYHFVmCGyzEMXoZ7ZMKMej0rqDbN9m5a+CdbOOZZAyM2JnnMnZMKBrKEp6zuKUMq bbSw== X-Forwarded-Encrypted: i=1; AJvYcCWzwgE+BdpVUrZ4TcfLJWFTUVLwcl2RHlHx0agOWrkjPnt8SLJu72p8e8P5y0gWSWBtiYEyex8Ucw==@kvack.org X-Gm-Message-State: AOJu0YxZ0ttzZeR58bxrXA2ZzDtbeGOFzhKM8BLTIb+j9BsbTLvfRmAx 9J3izAcVBFlRlFmw0r2JPAm/2Ia/aVrPuWj51EsgDmGEH5YDbeIg6lAB X-Gm-Gg: ASbGnctoZHlzh1/oikkKkKz/dBStzDIn7KUHj5hdBR6IU+y3UhMBgNIarNxiUf34bHn eTwiIlV41gtwMl9yxCVmn64/b1vvJgJEzH+6X6+eq4lCUzUzu6ck4/PvRxpxnRfhvPPMYwwRcL7 TeBt1UcuDA+KLVHHRDJfFB1SYYVs8vI8AAMivXxrX1Os+Z3H4IPZVQDhWixk3MuieL3qCsUX59k Qaice/Xevb9vMd8stSAMXrHyT9kt73mFh8OIdGIWlHuxjOt2kK+pBv4dNUcFH/eTZxFOkrWfvF/ sCvnhGV7JTSqj53FLEiXgY1iYkWKHiNPzN+uaQUHNm9vQRmivlQfsG3hJu2C7Gx7MrW7f98UY8l tnjWkt/+yv8QxCd9IXe1CxLaTyL3u9zYcbczrgJArinTQz+RMIr25b5WcTY4vdt0= X-Google-Smtp-Source: AGHT+IHn6Byr5+wZ+pbkTuno1o31ixBo3a4hfO63c4om8gQKe5/SQIwhVu31aNzj2EmKlKVC4PYZGA== X-Received: by 2002:a17:903:230a:b0:234:bfcb:5c21 with SMTP id d9443c01a7336-23e24ed6f39mr16153555ad.19.1752633004527; Tue, 15 Jul 2025 19:30:04 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-23de42aea81sm116743265ad.82.2025.07.15.19.30.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 15 Jul 2025 19:30:04 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v3 4/6] mm: Introduce alloc_frozen_pages_nolock() Date: Tue, 15 Jul 2025 19:29:48 -0700 Message-Id: <20250716022950.69330-5-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250716022950.69330-1-alexei.starovoitov@gmail.com> References: <20250716022950.69330-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CC29580006 X-Stat-Signature: 7sz3eetbqjoiagmsdoi7hf7ghq8u4dey X-HE-Tag: 1752633005-417080 X-HE-Meta: U2FsdGVkX1+IFbqH697+2nABCZNs1bGA4PPgctOji9H8xEVlKEbwX6f/TY36+ZYm08i3dj2Ig9h6UFrwXu+qxL8JeFvha0BWRQcFeVGQVcUIChn3gBxe7nPDgF3fYWEpiVRNiVl8RKOL7nAxz1HrZH74tY1ENvFAS32hvFWJ2cOrM1L/x5QCHr0n5pPWC0MW40QnVjpqmw7GlkmMmk1pLFg6+YfkAzVhj4s6SfyAfHpg4f3zGZsX5Y9AmFj9yznm8Rp6RT5/wqB3YGgiYUEUP6QUShrlgElezgzZ4P0VNGbfgLUKGPFwSL/EfcEsGxlKRMjdicM/scfQoNNZ0P92ofWaUujElHxF3GuCzhS4UIQoSZ+4Qa/SfPIlRVT6nIjahffGlvxFavhYxXm0IbrHifYmf5+3T3feTTmE+HvtPz8/jRX7IoOu45khYrVaSJGlAbEg24vIK7OzEkqsk8lsIsznFYPdxBO+40qH3E7gnRx0/nrdmazBpLBZA4GejozynZ1wpmB3VraC83e+MYLj8I9YTZGlWVTwW+a7LANOX515gip6JZQ78FMnSz/4nwBXHD3Su8VraDh0k4i30nyVnOk7My5fxZaS70L3KOWQoWD/FLQJbLoWkguowalpjkhCINMlZbYe9/ZkYAoEkK3k3vOUxNd4GyYj0c+taZ+gPF+7tOVFQmtYkz6uP7tEubUcG3CILv/kCC+hKVWYI45yEtVIRGVc7DmBGfE3W/N0gs8fK2KVXHwlvNVIQ2rk3LhRZPSirbBrzXlJ6NDposwkNQ6+Kg7niWuYoq0f0Yt7VwyYvdOsAmgtL2AKIIRmajLDR0zJf9ogY2Qr0RuFVDMJLzWl3qcGkaMRWVSOxcqfoZL02ZJ1Bv0dY641waX2Re5XcC6qFh7jU9UURy9ECbg02O5pWpjGd1WnnJ792AT7V46zgayXiLQlX6m8zfPgSXNWeFD/2zBD8lmkeCxfyFe Mo1rnfoO ZdhB+61INNnWtaS1B6RGtSsSW9aHTm5HQTxywlPERoEMD60H50+ckWXQCXRTr9zRn4qFHc8mbE/s2CNTllM88PpOI7H6ejJTcne+x3nbLVilzPHr0CnQytEPCI9phPlBqiyFHFLDC/y/JaFICvn4NWtBcz6MaENiHVmj1o1AslTmdySRgOkyW3O2jYIezwJ3xqtos5xt5l9jl4kBbTEfc8NQs/hIWgdlnXCyNjwFlIUP70ls1cX/Kfyac07ZyLgaoXzmZgdJkg1Eifyd3JVo6/H+LYMXel9hcB3uv7bHD0vVXcM9ZwaM+cFtPEHZ0H0XhLfAVDAmffN0xzQ4ALKXFN1AfJ24kW6LpTewi4zEYQW+qH01ASNsQrWdm1uQzhLEALSYeeJV2RorRXdXuzXaWm/BPVC7Y5sDQa0gjMuK2xRbCyIIDU8l1rehTBT5RZy0Mep5mAWx6f4EUU8M9IZ+EmL5iMvKwpiDax5amNupfryyp5blBE6hIu3lVn6bzJLsi4h7fxEuFVhx2tQFapNTTlAHpKMUss3UaF7Bf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Split alloc_pages_nolock() and introduce alloc_frozen_pages_nolock() to be used by alloc_slab_page(). Reviewed-by: Vlastimil Babka Signed-off-by: Alexei Starovoitov --- mm/internal.h | 4 ++++ mm/page_alloc.c | 48 +++++++++++++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 21 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5b0f71e5434b..ea85cf703331 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -837,6 +837,10 @@ static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int ord #define alloc_frozen_pages(...) \ alloc_hooks(alloc_frozen_pages_noprof(__VA_ARGS__)) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order); +#define alloc_frozen_pages_nolock(...) \ + alloc_hooks(alloc_frozen_pages_nolock_noprof(__VA_ARGS__)) + extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); extern void zone_pcp_enable(struct zone *zone); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 148945f0b667..11a184bab03c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7504,23 +7504,7 @@ static bool __free_unaccepted(struct page *page) #endif /* CONFIG_UNACCEPTED_MEMORY */ -/** - * alloc_pages_nolock - opportunistic reentrant allocation from any context - * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. - * @nid: node to allocate from - * @order: allocation order size - * - * Allocates pages of a given order from the given node. This is safe to - * call from any context (from atomic, NMI, and also reentrant - * allocator -> tracepoint -> alloc_pages_nolock_noprof). - * Allocation is best effort and to be expected to fail easily so nobody should - * rely on the success. Failures are not reported via warn_alloc(). - * See always fail conditions below. - * - * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. - * It means ENOMEM. There is no reason to call it again and expect !NULL. - */ -struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) { /* * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. @@ -7583,16 +7567,38 @@ struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int or /* Unlike regular alloc_pages() there is no __alloc_pages_slowpath(). */ - if (page) - set_page_refcounted(page); - if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) && unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) { - free_pages_nolock(page, order); + __free_frozen_pages(page, order, FPI_TRYLOCK); page = NULL; } trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype); kmsan_alloc_page(page, order, alloc_gfp); return page; } +/** + * alloc_pages_nolock - opportunistic reentrant allocation from any context + * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. + * @nid: node to allocate from + * @order: allocation order size + * + * Allocates pages of a given order from the given node. This is safe to + * call from any context (from atomic, NMI, and also reentrant + * allocator -> tracepoint -> alloc_pages_nolock_noprof). + * Allocation is best effort and to be expected to fail easily so nobody should + * rely on the success. Failures are not reported via warn_alloc(). + * See always fail conditions below. + * + * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. + * It means ENOMEM. There is no reason to call it again and expect !NULL. + */ +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +{ + struct page *page; + + page = alloc_frozen_pages_nolock_noprof(gfp_flags, nid, order); + if (page) + set_page_refcounted(page); + return page; +} EXPORT_SYMBOL_GPL(alloc_pages_nolock_noprof); -- 2.47.1