From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06EBDC83F1A for ; Fri, 18 Jul 2025 02:17:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 996CE6B00B2; Thu, 17 Jul 2025 22:17:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F8D76B00B3; Thu, 17 Jul 2025 22:17:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C0376B00B7; Thu, 17 Jul 2025 22:17:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 68FAB6B00B2 for ; Thu, 17 Jul 2025 22:17:04 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DA49AC02D3 for ; Fri, 18 Jul 2025 02:17:03 +0000 (UTC) X-FDA: 83675772726.19.9F773EB Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf29.hostedemail.com (Postfix) with ESMTP id F316A120007 for ; Fri, 18 Jul 2025 02:17:01 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="chvi65v/"; spf=pass (imf29.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752805022; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=PPX8ySbtpACnKo9HS5LIZAbXcTyPigAbVwUpYGyHddl75dCz7wAtq+kN87o83/PvLMcmKq ttDBZmDa+8kJzFRpxMmbTJNldzT6fdY1q/hDhxtwwjAeqfXbhcO6ZAS+AR4PIy/I+b4Kht 4OtEYjJfjO/VeXdm0Vp2AZ01bIRuJZI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="chvi65v/"; spf=pass (imf29.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752805022; a=rsa-sha256; cv=none; b=2SOzvq4CBOtmpmyc8JyTMmRxHp8zIxakff0XtpsJcAcsEiZJ7SAyY1KBg8YImara4jFUzC E0Ka9LvggWPgPKCt8SQRr8dqllg3qLRs3gypNuP9xMoRyyRX9WHT3Dxx4aYdl99xn4apd5 kre0LF9799PTxDXv9Etva2hTZ8yojX8= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-7426c44e014so1686823b3a.3 for ; Thu, 17 Jul 2025 19:17:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752805021; x=1753409821; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=chvi65v/3gIq1CiEkOkKAMjKjCOIt7QOo8eauSJ16O4Z1QjHhOvIy77x12EBI0sB2d 0EsO8s08KcPkpI08RquQo44mrkudJC8e8IIlMTbyJvtD/4Q1rcxgaGmPJaG1bZUOsoL8 Hgr2CQeawFgFIh8JMud6qhVTK787AyLxHZB5kyU8aASpV/2ArjKgG6w2sNLLNdDDmpko qSo+/6rhusyIub77K45jtThMqCz1p1tlobse+NL0opCgiqXD/0c/MpOnuLkYOcTlWLPb WtDFkhJILaXTmC6WTNxadkZFmghw0Kl5o28u889lQSJv5DpnsaaB9f/vxJIK0rWkkhxp dnDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752805021; x=1753409821; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i518bbzfZO/aJkAgPJDTpr5hRhX5KSVRvbb0iT/IJ8Q=; b=RzDKLkp0VhrQTfzpi2QDwY5lKd5dWzJiBK77GnPghawa9i1e+jFrQj9CJNaJn7WNDJ q1VwvolJjDW43LjU0jeFOA9W8UeenBcGW5sTs+Hgskmsq5AYBonnu/isNqqDxANiacFc XojeUaJGLxrSEnAYaWy/EVTCvMomhWrCkfnWb8ckrZJtTkfcDgwYtzm0HBDwzTyZiUpe a6vsM27ufCd/sXOJ+xOpwHZsmZS4GgwPGf45zWHsFnne6VcVZXmM6/HUr+d0TK7z545m 275TGM0VS+LeKFG1quUdgBYYhzTt83nSVr11xIYZ59ErgFhSolSZQ/DNYwYsZ9MhQsV9 br4A== X-Forwarded-Encrypted: i=1; AJvYcCU6UXywHWFmXk1RkXgau18p/RMBC3eAh0eq0/5CvH/SeKuwNqAT3gEND1n0no4lo4CcGH/HNJmmsw==@kvack.org X-Gm-Message-State: AOJu0Ywob4EDzv7YHgOHvWMdJznVYLOb4qEm8P/Ki+uIMl9MRKEpRhzV WRhYX4PYB+nEvZDKNoJpRuNweWV3xlyb4t/zvZViSio/encdyq1yKemj X-Gm-Gg: ASbGncthlRnxiL/4Jxy3EaZyMTib08Eja1Vt2jdt4/ekRL2JIzl8Brh+Hu/63XinqNk QseuN/eUJo57Q7aRrwRFvLRmFttvqRRRZf/RVDuiBsAV2QsASM8EeVUSxH3MhCQBPzyZ+p5dzAE Kyg/KR1VmdJNJdTnpgOsMFCA7f23D/TPlHEqZH+5FGNQRcZWQnP7Tcx9O57AAj1E6wycnqWKXDB iHq1j4ny3m09s7GM9GnxBK6D4RSkm4zSeLh26tyRlcyw9kkSoroL3oA0lBHV9MYys6cP8KbL7GV udIib430pEWQXobqEm3MQ8Ud1nh2p0VLtcuYZb0p5TvYM6ADos2LIXywV063+tGT5IdlV2+nPi6 ji5yYgC1tg4XSZbaFQp8mA0S++mksNZeQSRFXbHXV5o0qt8e9omfP87Hk63xXmkMIC4mWazto5w == X-Google-Smtp-Source: AGHT+IFcY2VsA7LkPF499uUm0R86UTS1mtNIOHPycskWH/JF5YYqWIqSstbyRkbU87Fg4yiMkJvBRQ== X-Received: by 2002:a05:6a20:7290:b0:235:b6de:4470 with SMTP id adf61e73a8af0-237d5a04312mr16499209637.13.1752805020753; Thu, 17 Jul 2025 19:17:00 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b3f2ffb69b8sm300281a12.71.2025.07.17.19.16.59 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 17 Jul 2025 19:17:00 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v4 4/6] mm: Introduce alloc_frozen_pages_nolock() Date: Thu, 17 Jul 2025 19:16:44 -0700 Message-Id: <20250718021646.73353-5-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250718021646.73353-1-alexei.starovoitov@gmail.com> References: <20250718021646.73353-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: 1hzd9fi5pgjcadboguo4swhdyibcqxob X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: F316A120007 X-Rspam-User: X-HE-Tag: 1752805021-361481 X-HE-Meta: U2FsdGVkX19sUt0Ss6Mq2ucnWQ5B4EkK5vfGGnWN4ww4r11qTo+KN4Bv/puydiKgHsXJpd8w4XghlHLhbC18wpAmZMLidZrpsclyeI+5s8+XEnUG0VyrgFlNiTvF2iNJQA4lTAk+xts6e+GR4WRnsGY8Q37QVvNW9y9moPyB+vv95xC2uqlCmd0aWs9bHBeM1j5TovROG/njvUM4o/KW/danB5f6e2sslJO920ghf8sZnQeczwW1cxsIfG6f3ucbWx3WYlh2w6d6nyc6NTIgELPVzwXgKzu3q9b6kxKzB4+P2BZQTt8g/MRkO1hjwV+8kimQEswoGvbz2hUaCDdhuWnjoe8YpQc2TGUgUHDcG3jgpazUp5xo7ImjGi2/H/z45J9Iz3IW5DkvaqVgcSSDTqiGzJ1D3Lj+N3wo9gl9Y/vurU3Qg7S+zwMD8QgiZ6JRNQ6XcgBjhcnz52u9yP5xLlQUZ7U6vMJkiW9Ov55F5jLgnm4pMxEGIMkijmtXKO3HRW+KrmlJ0E+wLm3ZwBpNWzafXAPLettq5MeTXpk8s0M1v4V7dj2nk7QeZrrR+DMczAHv2lNCYs/vTgJDoUNGvSeo/63a7yZ2ANvr9hgVen3nuUD/LAZmHZ9PyzrY753DE5wmTpiT5yUGN2qahynttVC1JquErmi3zC0FK5mbXiPHt4vbfJ9Cf3/bY7xICrG38YzKV/SXb8omB6Df2Enc4fi1VqTXvT8f30CmwCWpD02Z6PSZjs+9ovatKb2z4vS4Ep50z98wB4/SY0v32dI1bejZPFBlwN0Nr+P1mNbnH6FMU4AiGBFOmeBEsbtOJz8290nUtjKZMIm/z7pbbprsld9J3FKX2WFk8Uwk7SM5b37CUIz2xKW3su3eRRG9TK94Mjx4XrhywmrGIeueW9qggTopMVVSWBPRQs879uyNil3SWl3MSLLafWyz51Vc/I0Qmh1dHomQdrijra/hb72 OG2V8Z2p gChAJ2WUJ77C9Py3Gi7QvP0e4/qXUmRfccoKwdhqxh4xw06BWRrlihuZnvvqORqDlgwOpL9+ff5NInN7CybkITSeEDY+vAMrfUZvulUQG3lY2w9MJuSnS+Xe6GFFRVLvy9I0gIb1W+xZ5nvmfNHPmVyf1jcqPeCDkoOf+I7YqWkCp4qO7cv+9f4Z8X5SHq8PKc+5ghqBxFwukdsVZ+2yYuC7wniMxYEU2Gd3TeLc+CePeocLR2txrT9yVs214ot9U0XmlLjW5u0kjKaQDuQ87D1o6VzZvuGHXSiXdb43pbFRkkLX7KlG4icGjWBj/eUegnTwuIFmDshCk3m3C3P9+7o1Yl+Ss6gXu7ZPL7UvwUzhnlpxHu7QTVvZT8z3SE9flqdC36qv0spFRj1c7aSaTnNPLB4XZ028L7fCsNzpSo4Oa+4jPRKHbU3ZzvIYR5lkF5WuTN1zXwuFiJ1ElZoXAPj0LhFzO9hn4chiw6+as/Jrd+GfHekFPDewrWLksiFETa0WQ40FFF9J4ASX5LQ+Ois7DBfwfSDJ4Dm70wS6tnfWnp11RRkvejiBjfvoy1o7IuP0t/UMo2XsfH+1Cbdf8klSJnVG2+fltL7B8EDpYpaPaXy2TYGdn3I89w65ewtAkZ8SqBjNoSApwFokmGxxwAmx+Lw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Split alloc_pages_nolock() and introduce alloc_frozen_pages_nolock() to be used by alloc_slab_page(). Reviewed-by: Vlastimil Babka Signed-off-by: Alexei Starovoitov --- mm/internal.h | 4 ++++ mm/page_alloc.c | 48 +++++++++++++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 21 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5b0f71e5434b..ea85cf703331 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -837,6 +837,10 @@ static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int ord #define alloc_frozen_pages(...) \ alloc_hooks(alloc_frozen_pages_noprof(__VA_ARGS__)) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order); +#define alloc_frozen_pages_nolock(...) \ + alloc_hooks(alloc_frozen_pages_nolock_noprof(__VA_ARGS__)) + extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); extern void zone_pcp_enable(struct zone *zone); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 148945f0b667..11a184bab03c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7504,23 +7504,7 @@ static bool __free_unaccepted(struct page *page) #endif /* CONFIG_UNACCEPTED_MEMORY */ -/** - * alloc_pages_nolock - opportunistic reentrant allocation from any context - * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. - * @nid: node to allocate from - * @order: allocation order size - * - * Allocates pages of a given order from the given node. This is safe to - * call from any context (from atomic, NMI, and also reentrant - * allocator -> tracepoint -> alloc_pages_nolock_noprof). - * Allocation is best effort and to be expected to fail easily so nobody should - * rely on the success. Failures are not reported via warn_alloc(). - * See always fail conditions below. - * - * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. - * It means ENOMEM. There is no reason to call it again and expect !NULL. - */ -struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) { /* * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. @@ -7583,16 +7567,38 @@ struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int or /* Unlike regular alloc_pages() there is no __alloc_pages_slowpath(). */ - if (page) - set_page_refcounted(page); - if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) && unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) { - free_pages_nolock(page, order); + __free_frozen_pages(page, order, FPI_TRYLOCK); page = NULL; } trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype); kmsan_alloc_page(page, order, alloc_gfp); return page; } +/** + * alloc_pages_nolock - opportunistic reentrant allocation from any context + * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. + * @nid: node to allocate from + * @order: allocation order size + * + * Allocates pages of a given order from the given node. This is safe to + * call from any context (from atomic, NMI, and also reentrant + * allocator -> tracepoint -> alloc_pages_nolock_noprof). + * Allocation is best effort and to be expected to fail easily so nobody should + * rely on the success. Failures are not reported via warn_alloc(). + * See always fail conditions below. + * + * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. + * It means ENOMEM. There is no reason to call it again and expect !NULL. + */ +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) +{ + struct page *page; + + page = alloc_frozen_pages_nolock_noprof(gfp_flags, nid, order); + if (page) + set_page_refcounted(page); + return page; +} EXPORT_SYMBOL_GPL(alloc_pages_nolock_noprof); -- 2.47.1