From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA988C3ABA9 for ; Thu, 1 May 2025 03:27:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4BF396B009E; Wed, 30 Apr 2025 23:27:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4214E6B00A1; Wed, 30 Apr 2025 23:27:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29E756B00A2; Wed, 30 Apr 2025 23:27:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 07D6E6B009E for ; Wed, 30 Apr 2025 23:27:49 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6A597BF6BE for ; Thu, 1 May 2025 03:27:49 +0000 (UTC) X-FDA: 83392904658.05.840A8B3 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf16.hostedemail.com (Postfix) with ESMTP id A995118000B for ; Thu, 1 May 2025 03:27:47 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=REeGrru4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746070067; a=rsa-sha256; cv=none; b=20WFzru/jvMBflTb5NwUT6D0uhcu2XwKbd87RHHbAQdUgM4fK+o7JWJu2Kxom7pZJX2Zav Uv16+Mlcj7nVAdXmte2Js6OAVSo6RyXHHEyoDLMzxXWQkOGDbtlwtxknkAeLU8q3a10iVV eCAwaPL4qy3sJZy7+oSRGLwm16aC9HU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=REeGrru4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746070067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x+EUZuBpCtEkxSo+g5YTm/FD1JFtDCLsEwWX610EiQk=; b=wPLm6kuzomNYrEx9g3lThTYei+AR1LXXKN03HelpeQvhkf/GkWvK8hP8R5WmSXSigYzytB D0yaZP+s8sPVojLFnVymM/ocUPnof3QS5Y7iZQWXF9hghwQjEglqMXxYAtN/Pkzsa/83I4 ZSuJ4LQfNEg1y1KxLOKyY/KDFyIo5Sc= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-73bb647eb23so524768b3a.0 for ; Wed, 30 Apr 2025 20:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746070066; x=1746674866; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x+EUZuBpCtEkxSo+g5YTm/FD1JFtDCLsEwWX610EiQk=; b=REeGrru4bBox12tLK7cMy4Zegc28/T1JM4ofmElwnf5KQxVULvSqp8j7PNJk2SE5mn 3lsj6F5YWRGjuFJcvMz1iXFRnAgiXW5TfsuVd5dweto1ynkI0663jGOdh1IApQe2nlkP /8uQm+2fZ/DbbuyMGfdq0S4OfpgtqZZQ743oqdPW+OltoUdiJ6jIn8IjM5kO5KbMZwH+ Q/HSBPnrJlm2zLRYBtDMO2NREZAG01aToYta1zZx7VMJQhf8OMI1K9piE8KToFHbUW6a lws2l82LZRSmYA8ahXia6EL905MXq1G8+ogAiKWyPD46PfDTwMJP9Zsl7+cp1AEmQsJS aRng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746070066; x=1746674866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x+EUZuBpCtEkxSo+g5YTm/FD1JFtDCLsEwWX610EiQk=; b=F0xty4fStoIXv2fnkmdGXWVerfm+GZqvS2Oswd5UpJUf4RSr8ShrfEhvb5uCVwgqMr ot+7SaA2zmx8gvurqWqalVFfSQIX2lG5xhu4S1YhYYiYqG+9uQMysDfvi0cA2vntoCdB f0KWXe1cFTJfEjH5JCMGJ0oBre95hlUlLTVN8SXt55Ibl65jJc4ipaB3AWtNjZO6k0oA F9V7eEowvgclaAECcuEuEwRlQ/afs1w9Cr8kfmPMioC1ip7CvxYWUhw5PLCFQrPdaCFJ EaxntZQkQxJ+I+jEnSXzvT6t9SV37W6zRp0ERD6e5oAGVT4Ho0gk6aSa4UJNT4FU6Vjg aJUA== X-Forwarded-Encrypted: i=1; AJvYcCXDolVCZtHLm4mXtMwZWK4ojyNblmUGbtbpuEAZ6A0OrGUKu3h6mclzSckpjfrXW4+DeAJcmIWYyg==@kvack.org X-Gm-Message-State: AOJu0YxvfYM887melOLgzz1seTsRv1vK+51zCbjaZnvgnPPV962AB79G 9pm+QdRn9U6D6Ujiubv1n/s9QmpcCBzIZOpQAFKTYa/XbHVpRzB1 X-Gm-Gg: ASbGnctBMpV5x0NV2udTg9KvdTl5xS+ix7y0jx4KYsjau56dRWWgInwhK98/smbthzF Cs1LDC0V9MELg728O6MPqyd4qvCXCuvihsUg+uy0fLeX4xS7WoPT02IPhj/nrMEGA7N7UP32BZw E3YPtnRxVM/d5U+ArEuRc3CcPvTnDUwnUQ7d78pz90+yOIEijbML23RLe3UNkHW4aNdLMcdqJXq e+B3SploPrldJc0GPlP7RpGhi/3/Rpv9E56vGweoJu9SBJX6mU1eipHhlzaEm/QoSTyGmkCA4Pq Hsyr1KCXi+cOxMKCiGFdAa6RqFx0OBtweKEaZKsGdl8l/j32TFXO2MJnWgS0BXBiD/RI X-Google-Smtp-Source: AGHT+IHeLJ2NXP14CarSZr4yDkwP+sPREACfOArzqc/wYp/BDY0aljYBICiyQDeQSwmr+N9W4BBA8Q== X-Received: by 2002:a05:6a00:9282:b0:72d:3b2e:fef9 with SMTP id d2e1a72fcca58-7403a828934mr7589622b3a.20.1746070066598; Wed, 30 Apr 2025 20:27:46 -0700 (PDT) Received: from localhost.localdomain ([2620:10d:c090:400::5:13f8]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8d256sm2512273b3a.157.2025.04.30.20.27.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 30 Apr 2025 20:27:46 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org, willy@infradead.org Subject: [PATCH 5/6] mm: Allow GFP_ACCOUNT and GFP_COMP to be used in alloc_pages_nolock(). Date: Wed, 30 Apr 2025 20:27:17 -0700 Message-Id: <20250501032718.65476-6-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250501032718.65476-1-alexei.starovoitov@gmail.com> References: <20250501032718.65476-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A995118000B X-Stat-Signature: x18d4ekzumbsy345tnz5cmioungnsr7x X-Rspam-User: X-HE-Tag: 1746070067-700671 X-HE-Meta: U2FsdGVkX1+/4SSvP1H46m7lIS/D3Edj2SHQhxi8b/FBE4R7Fo5PnhYHd67wCvvylmyW/nqxsZIP5EH5zHkTD6OtFXKKIBUuBRzxVWQQq3xWvPHjn7QYVnlcyapam1rn4tygA5TJENlj+qzpufEYoi5ePoEqs5aaNVIAsCY/P/0DkPF2LW83o3euMQMFGfv3lHaZkUFKDiUCx7sEgxA9W/VlHYK9BiVPnMCUZT1coGXCdvDR+7UUWtmViRMFxTunNKfxuUNjYyPTONt6GJXm41DSSccLAUx5O7ZaiqK18sg+fI5Fvq58TalyfZK+wfz5Hz36rm66ajZKB9mzOR0t6Rr6xxDlAAzCkXffDvV+qUvWV4LRGqG9L1T8Vnitzz4BfNn50ZJFiuTQsm3JBXCCFQose6meUdv6KGSvezHqH7Qf/26w+LPbergWQU2J2hSP/YTmJxZoLuI7BkMASi3ONMmCmfqQ34sje1Thm0zW1hrsSAMQEkXmMsBvlUMa16Jv/GtZqQgyTGJQj8m1QzbWaGh4cRNgxH7WxmxFDYWaxuDOI4iPKhEz9MinNi8bDCpBrnE0j4YGCfkyODvRLKSyMhLhqsRnvUCPXhhUT74IX3jDiVJof+9k6XcXsMOxLeTHxy6PZSaDE5lkjjpRYdnL5sH9h162TwVJZHQiAV7fEnjAJpRjZLS6wMhMvSve5CI559t55AOlIR5YeQI+gzlEco9tQ1FOUeqYZETWKFT9uYeEDQXc4cWRXQMdOJaP+wzrE/oKueOHosF7DNoDOkQNBGSnmJu9L1FDkt+2NTINN5RelTHp+Eu23GLPj0hBwS8r3WAQLf0cbGT9RBvOSOtcL1642iOukg0HYLdNiMWBXIM/x3G0SWYfpRlv1LnjG3VMj0bp3pf6dvyNtMeeRjYxlT9GAiC/ngSo3MQvwPhVeC0GO+4c22e6bkeP8suhDTVKme/jwKJIWjMo+0lffXz 4m+hHze/ /STUPT3+fa3mXWZ8nhIQqC1b1zv29AWLj2X3kj1KSAhJd/NplnICJTghc/sB40nY4cauFsjYkLPNQCwdEv65nbDPpPpyWgGRS6PZD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Allow __GFP_ACCOUNT and __GFP_COMP flags to be specified when calling alloc_pages_nolock(), since upcoming reentrant alloc_slab_page() needs to allocate __GFP_COMP pages while BPF infra needs __GFP_ACCOUNT. Signed-off-by: Alexei Starovoitov --- include/linux/gfp.h | 2 +- kernel/bpf/syscall.c | 2 +- mm/page_alloc.c | 8 +++++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index be160e8d8bcb..9afbe5b3aef6 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -354,7 +354,7 @@ static inline struct page *alloc_page_vma_noprof(gfp_t gfp, } #define alloc_page_vma(...) alloc_hooks(alloc_page_vma_noprof(__VA_ARGS__)) -struct page *alloc_pages_nolock_noprof(int nid, unsigned int order); +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order); #define alloc_pages_nolock(...) alloc_hooks(alloc_pages_nolock_noprof(__VA_ARGS__)) extern unsigned long get_free_pages_noprof(gfp_t gfp_mask, unsigned int order); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index d0ddba2a952b..83af8fa9db3f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -578,7 +578,7 @@ static bool can_alloc_pages(void) static struct page *__bpf_alloc_page(int nid) { if (!can_alloc_pages()) - return alloc_pages_nolock(nid, 0); + return alloc_pages_nolock(__GFP_ACCOUNT, nid, 0); return alloc_pages_node(nid, GFP_KERNEL | __GFP_ZERO | __GFP_ACCOUNT diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1d77a07b0659..303df205ca7d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7379,6 +7379,7 @@ static bool __free_unaccepted(struct page *page) /** * alloc_pages_nolock - opportunistic reentrant allocation from any context + * @gfp_flags: GFP flags. Only __GFP_ACCOUNT, __GFP_COMP allowed. * @nid: node to allocate from * @order: allocation order size * @@ -7392,7 +7393,7 @@ static bool __free_unaccepted(struct page *page) * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. * It means ENOMEM. There is no reason to call it again and expect !NULL. */ -struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) { /* * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. @@ -7415,11 +7416,12 @@ struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) * doesn't want to deplete reserves. */ gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC - | __GFP_ACCOUNT; + | gfp_flags; unsigned int alloc_flags = ALLOC_TRYLOCK; struct alloc_context ac = { }; struct page *page; + VM_WARN_ON_ONCE(gfp_flags & ~(__GFP_ACCOUNT | __GFP_COMP)); /* * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is * unsafe in NMI. If spin_trylock() is called from hard IRQ the current @@ -7462,7 +7464,7 @@ struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) if (page) set_page_refcounted(page); - if (memcg_kmem_online() && page && + if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) && unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) { free_pages_nolock(page, order); page = NULL; -- 2.47.1