From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B991C83F03 for ; Wed, 9 Jul 2025 17:24:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 18EE76B0146; Wed, 9 Jul 2025 13:24:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 13FA16B0147; Wed, 9 Jul 2025 13:24:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02E786B0148; Wed, 9 Jul 2025 13:24:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E329D6B0146 for ; Wed, 9 Jul 2025 13:24:37 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 954BA10B821 for ; Wed, 9 Jul 2025 17:24:37 +0000 (UTC) X-FDA: 83645400594.14.A6DADF2 Received: from mailrelay-egress16.pub.mailoutpod3-cph3.one.com (mailrelay-egress16.pub.mailoutpod3-cph3.one.com [46.30.212.3]) by imf08.hostedemail.com (Postfix) with ESMTP id C01E3160004 for ; Wed, 9 Jul 2025 17:24:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b=T9LYSHLw; dkim=pass header.d=konsulko.se header.s=ed1 header.b="LT8DH/c+" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752081876; a=rsa-sha256; cv=none; b=eFn/KxxNTO3O6Vph0SPtBpls3a5d6pHjuwvFomBiK6wXCzLZHi7k3mlGdTGrMU+krRKTsY jyut1scyLtT4D77PnVjzN7BOoOspp0gN8EkHDX1R9B08j2ZbCOM/M0umTbDC3PZPdbkcPt a6oqgl2yoz/LoVobO4QuF6TL8tHS5Tc= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b=T9LYSHLw; dkim=pass header.d=konsulko.se header.s=ed1 header.b="LT8DH/c+"; dmarc=none; spf=none (imf08.hostedemail.com: domain of vitaly.wool@konsulko.se has no SPF policy when checking 46.30.212.3) smtp.mailfrom=vitaly.wool@konsulko.se ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752081876; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LrAktGzEr99plww7RQsKYhjMWGxj5SAMk+RHJzmK6Ug=; b=7kTZEP2KQBJe0x1oZgqzfgH1tRWx3NBzsrdReqxn1ewZ4ljocGGzv/z7TBakXBfeojbTEw uj55xOzhlpc2LXVZWThRvUezD5xpuYbJL09lJOjkxPNijZkefYEne44n4/mBxZkm6LtKxB Z8C3OIvDi0GAE4ZKv3eVNbG95zxpex4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1752081873; x=1752686673; d=konsulko.se; s=rsa1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=LrAktGzEr99plww7RQsKYhjMWGxj5SAMk+RHJzmK6Ug=; b=T9LYSHLwSRbrEMwdcrSDsva5TQrosqkDOAIqW7sbTvOJjCj7RxwznqswRAjTOmLQI/EtCaqf7n/GX ceBbmJlBtLMnY2WVwWrqAFc+voF0Z5Yo8ZLSdGfn+CXfoGXDXGkMCWkCHy84GavWbhiDGNp/JflR5N 2Vd44aHo4Pc5XNugP1c9iYVXntZIGgejWObk1RZ0KyGi+HCGVHaaOaGta0vaNQZ7l3+IrHh5vjSZ89 2TaHQ5cYEXQwonYUePBTKan+i9ARXBUftpzAN21IQDPb8OL8nygcZ7PqqjRekz1tCwFm69bXKJ08Se Ea638j0U+eOCgf5/GQiWuaK7AKS4LJA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1752081873; x=1752686673; d=konsulko.se; s=ed1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=LrAktGzEr99plww7RQsKYhjMWGxj5SAMk+RHJzmK6Ug=; b=LT8DH/c+ol1Lg+cQsN1EzKer+bHvDwp5dgp8b9IyDOistvlz0aXBgD9tsSStPUgmAAnuU+vawb8xb sKMBOdGAA== X-HalOne-ID: 938077d2-5ce9-11f0-86af-f78b1f841584 Received: from slottsdator.home (host-90-238-19-233.mobileonline.telia.com [90.238.19.233]) by mailrelay1.pub.mailoutpod2-cph3.one.com (Halon) with ESMTPSA id 938077d2-5ce9-11f0-86af-f78b1f841584; Wed, 09 Jul 2025 17:24:33 +0000 (UTC) From: Vitaly Wool To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Uladzislau Rezki , Danilo Krummrich , Alice Ryhl , Vlastimil Babka , rust-for-linux@vger.kernel.org, Lorenzo Stoakes , "Liam R . Howlett" , Kent Overstreet , linux-bcachefs@vger.kernel.org, bpf@vger.kernel.org, Herbert Xu , Jann Horn , Pedro Falcato , Vitaly Wool Subject: [PATCH v12 1/4] mm/vmalloc: allow to set node and align in vrealloc Date: Wed, 9 Jul 2025 19:24:16 +0200 Message-Id: <20250709172416.1031970-1-vitaly.wool@konsulko.se> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250709172345.1031907-1-vitaly.wool@konsulko.se> References: <20250709172345.1031907-1-vitaly.wool@konsulko.se> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C01E3160004 X-Stat-Signature: n7xgirrost8p63md5ce93sr4giw9chzk X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1752081875-833288 X-HE-Meta: U2FsdGVkX197ioi4L1HEVJizqlM5ArfzbR3aRkOjrB4tBcuVpr6UylK1x9xyclPerVJauzmU9xIaiMoyqN8XzjzrfzTC+ho173yhSwIQJc5gq3/FlTYvrKBAEAgPUcQ908yG23kE5cJgWI0r4RC3XEySgemCeQrq/H5kG6hrLMzh9dX2QHSHoEUmLFnXQ2Ja3wkAcB/0N4Rsw4eDlCJxuQ0L4X+Xpb3IfG2On0vRkGmrMtlpiM3rFfuMIRxoObtS45AWWXlxz+5Apz8bPnghXzl9VmZLmzcDWj0A3gdkPSUU3uCNQw2KiV7/UyGwqPN+Y54UQ4fmPUHcLdCCurShSNNibMrkdOfeg1grp6v+THhOj1tY1pa0yNOlbPq8o5SbHiNl6fkqwhsIfnQnLmcNpl663YtAHp43+Tv/EzP3v6pcjJObOyiDcHseBOD9rkRx5HPX3h4lPCaGHxvU931caYvTIa8wDdIQm5uiqjETojvOwVX3+PGViFJLzVFWxPnwmwif1RXUHxSae4USz9+7SHenVOlNYlpqX9CSGKckWHmDxeGLhzC+e2i/CPMQeNxOwMBCLCZZ34BIdlPVww53aY5R/3uh3iBRAFa2u2NTUpdoJsm0yGcpZrP6E/tluONvTRCgK4vnBG+KTvs3s9b3uQU0RBK91WMprMGLRqOlFPod78en8S7vrgL/cs2SeDNvZiuVMu6gp+DVVtNbD0/nlootfitvlANiFwJRafrldoNtGZh+tA8d5KSP/pqfU9D3C4wMaCwVSWSo+RCGnoSYagwPd/JvS1fjDpfjhRVlvY0uQe5kI6xgB5fMMTVjpephOU/wk4Qufyqkl1wR+011DOoPNwslrKjW9x1KNyMWMsdRr989iXM9zg5HLv2dj0+exbnhngganPieiDYQv1QE7w9eNDCQUwjmrndWJiy+QpdoyVIjxxQ9DtPK6SqwqHABI6wbBSv8rqTsQP5kgLK 55J0VvTs objeC2Csln3kJ7MzO7EL449ZPbLwBGl8VFeBQCz4Sae17pKGZgpqNMUKb7Dy5BmLvLSNayiRo1Y3TCigkK6XQrEWZuWVxh43XI+OPbunNHgN3K4jQiy+P9atAPcxwBDZb30wE0qo7nu5HvtcJpA32wI5R51aNn4nA8lFVWjrSS4SVT2n0QymEhgOvOi42eREUgoGPiYqRiJcpNUg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reimplement vrealloc() to be able to set node and alignment should a user need to do so. Rename the function to vrealloc_node_align() to better match what it actually does now and introduce macros for vrealloc() and friends for backward compatibility. With that change we also provide the ability for the Rust part of the kernel to set node and alignment in its allocations. Signed-off-by: Vitaly Wool Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: Vlastimil Babka --- include/linux/vmalloc.h | 12 +++++++++--- mm/nommu.c | 3 ++- mm/vmalloc.c | 31 ++++++++++++++++++++++++++----- 3 files changed, 37 insertions(+), 9 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index fdc9aeb74a44..68791f7cb3ba 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -197,9 +197,15 @@ extern void *__vcalloc_noprof(size_t n, size_t size, gfp_t flags) __alloc_size(1 extern void *vcalloc_noprof(size_t n, size_t size) __alloc_size(1, 2); #define vcalloc(...) alloc_hooks(vcalloc_noprof(__VA_ARGS__)) -void * __must_check vrealloc_noprof(const void *p, size_t size, gfp_t flags) - __realloc_size(2); -#define vrealloc(...) alloc_hooks(vrealloc_noprof(__VA_ARGS__)) +void *__must_check vrealloc_node_align_noprof(const void *p, size_t size, + unsigned long align, gfp_t flags, int nid) __realloc_size(2); +#define vrealloc_node_noprof(_p, _s, _f, _nid) \ + vrealloc_node_align_noprof(_p, _s, 1, _f, _nid) +#define vrealloc_noprof(_p, _s, _f) \ + vrealloc_node_align_noprof(_p, _s, 1, _f, NUMA_NO_NODE) +#define vrealloc_node_align(...) alloc_hooks(vrealloc_node_align_noprof(__VA_ARGS__)) +#define vrealloc_node(...) alloc_hooks(vrealloc_node_noprof(__VA_ARGS__)) +#define vrealloc(...) alloc_hooks(vrealloc_noprof(__VA_ARGS__)) extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); diff --git a/mm/nommu.c b/mm/nommu.c index 87e1acab0d64..8359b2025b9f 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -119,7 +119,8 @@ void *__vmalloc_noprof(unsigned long size, gfp_t gfp_mask) } EXPORT_SYMBOL(__vmalloc_noprof); -void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) +void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align, + gfp_t flags, int node) { return krealloc_noprof(p, size, (flags | __GFP_COMP) & ~__GFP_HIGHMEM); } diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 6dbcdceecae1..03dd06097b25 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -4089,19 +4089,31 @@ void *vzalloc_node_noprof(unsigned long size, int node) EXPORT_SYMBOL(vzalloc_node_noprof); /** - * vrealloc - reallocate virtually contiguous memory; contents remain unchanged + * vrealloc_node_align_noprof - reallocate virtually contiguous memory; contents + * remain unchanged * @p: object to reallocate memory for * @size: the size to reallocate + * @align: requested alignment * @flags: the flags for the page level allocator + * @nid: node number of the target node + * + * If @p is %NULL, vrealloc_XXX() behaves exactly like vmalloc(). If @size is + * 0 and @p is not a %NULL pointer, the object pointed to is freed. * - * If @p is %NULL, vrealloc() behaves exactly like vmalloc(). If @size is 0 and - * @p is not a %NULL pointer, the object pointed to is freed. + * if @nid is not NUMA_NO_NODE, this function will try to allocate memory on + * the given node. If reallocation is not necessary (e. g. the new size is less + * than the current allocated size), the current allocation will be preserved + * unless __GFP_THISNODE is set. In the latter case a new allocation on the + * requested node will be attempted. * * If __GFP_ZERO logic is requested, callers must ensure that, starting with the * initial memory allocation, every subsequent call to this API for the same * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that * __GFP_ZERO is not fully honored by this API. * + * If the requested alignment is bigger than the one the *existing* allocation + * has, this function will fail. + * * In any case, the contents of the object pointed to are preserved up to the * lesser of the new and old sizes. * @@ -4111,7 +4123,8 @@ EXPORT_SYMBOL(vzalloc_node_noprof); * Return: pointer to the allocated memory; %NULL if @size is zero or in case of * failure */ -void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) +void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align, + gfp_t flags, int nid) { struct vm_struct *vm = NULL; size_t alloced_size = 0; @@ -4135,6 +4148,12 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) if (WARN(alloced_size < old_size, "vrealloc() has mismatched area vs requested sizes (%p)\n", p)) return NULL; + if (WARN(!IS_ALIGNED((unsigned long)p, align), + "will not reallocate with a bigger alignment (0x%lx)\n", align)) + return NULL; + if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE && + nid != page_to_nid(vmalloc_to_page(p))) + goto need_realloc; } /* @@ -4165,8 +4184,10 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) return (void *)p; } +need_realloc: /* TODO: Grow the vm_area, i.e. allocate and map additional pages. */ - n = __vmalloc_noprof(size, flags); + n = __vmalloc_node_noprof(size, align, flags, nid, __builtin_return_address(0)); + if (!n) return NULL; -- 2.39.2