From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 577ADC83F17 for ; Tue, 15 Jul 2025 13:58:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EAAD18D0011; Tue, 15 Jul 2025 09:58:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E5AD08D0001; Tue, 15 Jul 2025 09:58:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D49FD8D0011; Tue, 15 Jul 2025 09:58:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C351B8D0001 for ; Tue, 15 Jul 2025 09:58:09 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 878821A04BF for ; Tue, 15 Jul 2025 13:58:09 +0000 (UTC) X-FDA: 83666653098.08.23C78EE Received: from mailrelay-egress16.pub.mailoutpod3-cph3.one.com (mailrelay-egress16.pub.mailoutpod3-cph3.one.com [46.30.212.3]) by imf01.hostedemail.com (Postfix) with ESMTP id B15A140007 for ; Tue, 15 Jul 2025 13:58:07 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b="ZYkj/tH0"; dkim=pass header.d=konsulko.se header.s=ed1 header.b="K7s/z8ke" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752587887; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RvpazCzVxvXnIveADtjt84U407dyEylMhRtOiCnmbUA=; b=QtVlO003FrBqq2pX+sZoeHCCNPzAl9OPHNpEI33Wb5vZhPwM6Erhvhn+zzyNl8jQnGmRLh 50sqV6vIF0Mks7aY8894A/tmHnVUr3hkV/wp3brGAYnJt4MvLXftDigtWI4oraMrtbK1m9 hupyg2/HRPIYIvhzGc6uI3q2W8zKHiU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752587887; a=rsa-sha256; cv=none; b=semwOEGoBcDJ3vXCxtaro1Rbs1FiUrF/733GlxjWyp9lv5DYdWP5iZ3cpqazVq/DA6QwTr ksUBmXZ41/Fc4qwvN2cGyRU3Gbkc1aBr4hH2/fGI4DjYKt+KDUrP+mQAujRdvjlcUY6Gih vDUhHlx8+yMvPc+hZS1VeK5sH1VUP9c= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b="ZYkj/tH0"; dkim=pass header.d=konsulko.se header.s=ed1 header.b="K7s/z8ke"; spf=none (imf01.hostedemail.com: domain of vitaly.wool@konsulko.se has no SPF policy when checking 46.30.212.3) smtp.mailfrom=vitaly.wool@konsulko.se; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1752587886; x=1753192686; d=konsulko.se; s=rsa1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=RvpazCzVxvXnIveADtjt84U407dyEylMhRtOiCnmbUA=; b=ZYkj/tH0/s0Co1qLPu9jhXAVZgLaM16V863OL7E+GKPk8A9exPRCUxW93Y1I9tNPVuS7PBk6ppW9b KGpn3f2TvWHpvpIxxhLVXunPZTyR/p58I7sdMBlQOpQM1wh1swpWRcmW2x8pYLEBo5KSz0NdpSuadb WD56pXT2bX4CDROy/8oa+iBijp/bCPKSevSHll86blURiM7fkCb92xCS4Br61Xy1EnXhn7VuASYESE JYStkaBCe18bu3jK3OfFEIffBrLxUxAOi31cEgQy1eMwicvLDcJFJFeWkP3XoaGqbiPd3Hef/a+B53 phyA+TJ5I6o/F3LGLm6bUjO6lvMtRvg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1752587886; x=1753192686; d=konsulko.se; s=ed1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=RvpazCzVxvXnIveADtjt84U407dyEylMhRtOiCnmbUA=; b=K7s/z8keFCDg2jxD/UnJCssqJywHAo1+Lm1JkRas/9lTcIvPj0eP5MtupCAWiBSiFDEoGzOqfGUrp iTJRct0CA== X-HalOne-ID: b99010bc-6183-11f0-8923-e90f2b8e16ca Received: from slottsdator.home (host-90-238-19-233.mobileonline.telia.com [90.238.19.233]) by mailrelay2.pub.mailoutpod2-cph3.one.com (Halon) with ESMTPSA id b99010bc-6183-11f0-8923-e90f2b8e16ca; Tue, 15 Jul 2025 13:58:05 +0000 (UTC) From: Vitaly Wool To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Uladzislau Rezki , Danilo Krummrich , Alice Ryhl , Vlastimil Babka , rust-for-linux@vger.kernel.org, Lorenzo Stoakes , "Liam R . Howlett" , Kent Overstreet , linux-bcachefs@vger.kernel.org, bpf@vger.kernel.org, Herbert Xu , Jann Horn , Pedro Falcato , Vitaly Wool Subject: [PATCH v13 1/4] mm/vmalloc: allow to set node and align in vrealloc Date: Tue, 15 Jul 2025 15:58:03 +0200 Message-Id: <20250715135803.2230193-1-vitaly.wool@konsulko.se> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250715135645.2230065-1-vitaly.wool@konsulko.se> References: <20250715135645.2230065-1-vitaly.wool@konsulko.se> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B15A140007 X-Stat-Signature: cms5kuh8hj7hgttrsguiwiq6r4q9m3hb X-Rspam-User: X-HE-Tag: 1752587887-661939 X-HE-Meta: U2FsdGVkX1/rL21+sEx7t/xXavvk5YYO0i0766o4de8UuIq+PP6JJDxjG85EnpeIKaU9kuKwybxeVGkqAM6MV9JzPboWHJNICYplRUkfhzZK6TTJimbD1a4QV1itzDaawICLNSYh4CIfG7hISaMBakKm9bmcfRBbb/MJa2MfsQmyMgUuUNoF8RJ2IQjFSvMbmYcH3HF/7jd/rF54B1t9HgVL+o8E9dva2F9ng89dwR1aDEdCQNcQuBgM5eFpA5j0g7VUVSlWjcChsFTu7M7QnFfyp8u0xcF81OwivryHqKf8p+Pj60qRPp/X9hWE7L4p76EhMHPC8z3U2eBjQKODv+CyTkHqXq12PK3Z++oU8/W/tzp8fe53bUBUBDwOqgQ168HSkQ+9PTVpwbjL8QxhPYo6SbuYB7IN2fjJCp9paG4fCx/sXZ165Lo8GrUh2rQoDtm2f+eBLqBrcpymJ/E7aZlYqhT8/WxUBG8kbPPBQ4CJPFNbiyPKb1DUH5HWt6t44N4nhGPyEvMwctqIQ1YpFZUqrQoHjJxi6GtYQxcVoEv6Zu0MAaTBrM5kFb+yg9g3uXRacW8nNW69UPsgZkfGXedbTcUaJ6Z5iCB9iT1ReN1ozY0JCgeLyWOpvVcAP45GRt6EUI0Ntvr+tbaibrbW9bGJCFrNP2qnYQ54gAFqEOnX2YjvMKhWyVvSuedw5IAjRSsLP4kZT9eS+FV1hcSfMnaMScuM6YH0o3ugxkuiX03airFPaUhlIz5irMsRcyK9wBeHs80LU++6mJDxHdjYiaHm4EsDS+IueTv2LoA6d5v/qQHkpMsKyahdk3UMR9jLeUKfzQnc6jKOcyIQN2WUxQvXCxOIcncIlQGb7pLYaxRy9Z8Ibu2nKWYQ1UHs9ZTWda8bODJzUd61n5of3itelmQaDk0qu+ETa2PitACt877H00WN6OnguB6EIAr5xrzAjZSRskb9aoVZIugi4zL MaGi+3uj PFKGMMR7DANAQDfWf774Qn9ezVjFzkdsgqNCdZRNrUPUDjhiDr1+hUA4Gq/x4E6BgkNwkUW648sq97Do4yXeBfGTRJ1GD413xJrC17dgiYkSZ6WIPPc5n4qoTBoheeVhrGiuTZocxX5a2UVcPPprH6Cp7FaaidXw9jRsKJxy2/wimCx+kkUO07NHJIsaAL5YBePQmT95cJnIQSM0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reimplement vrealloc() to be able to set node and alignment should a user need to do so. Rename the function to vrealloc_node_align() to better match what it actually does now and introduce macros for vrealloc() and friends for backward compatibility. With that change we also provide the ability for the Rust part of the kernel to set node and alignment in its allocations. Signed-off-by: Vitaly Wool Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: Vlastimil Babka --- include/linux/vmalloc.h | 12 +++++++++--- mm/nommu.c | 3 ++- mm/vmalloc.c | 29 ++++++++++++++++++++++++----- 3 files changed, 35 insertions(+), 9 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index fdc9aeb74a44..68791f7cb3ba 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -197,9 +197,15 @@ extern void *__vcalloc_noprof(size_t n, size_t size, gfp_t flags) __alloc_size(1 extern void *vcalloc_noprof(size_t n, size_t size) __alloc_size(1, 2); #define vcalloc(...) alloc_hooks(vcalloc_noprof(__VA_ARGS__)) -void * __must_check vrealloc_noprof(const void *p, size_t size, gfp_t flags) - __realloc_size(2); -#define vrealloc(...) alloc_hooks(vrealloc_noprof(__VA_ARGS__)) +void *__must_check vrealloc_node_align_noprof(const void *p, size_t size, + unsigned long align, gfp_t flags, int nid) __realloc_size(2); +#define vrealloc_node_noprof(_p, _s, _f, _nid) \ + vrealloc_node_align_noprof(_p, _s, 1, _f, _nid) +#define vrealloc_noprof(_p, _s, _f) \ + vrealloc_node_align_noprof(_p, _s, 1, _f, NUMA_NO_NODE) +#define vrealloc_node_align(...) alloc_hooks(vrealloc_node_align_noprof(__VA_ARGS__)) +#define vrealloc_node(...) alloc_hooks(vrealloc_node_noprof(__VA_ARGS__)) +#define vrealloc(...) alloc_hooks(vrealloc_noprof(__VA_ARGS__)) extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); diff --git a/mm/nommu.c b/mm/nommu.c index b624acec6d2e..afde6c626b07 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -119,7 +119,8 @@ void *__vmalloc_noprof(unsigned long size, gfp_t gfp_mask) } EXPORT_SYMBOL(__vmalloc_noprof); -void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) +void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align, + gfp_t flags, int node) { return krealloc_noprof(p, size, (flags | __GFP_COMP) & ~__GFP_HIGHMEM); } diff --git a/mm/vmalloc.c b/mm/vmalloc.c index ab986dd09b6a..e0a593651d96 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -4081,19 +4081,29 @@ void *vzalloc_node_noprof(unsigned long size, int node) EXPORT_SYMBOL(vzalloc_node_noprof); /** - * vrealloc - reallocate virtually contiguous memory; contents remain unchanged + * vrealloc_node_align_noprof - reallocate virtually contiguous memory; contents + * remain unchanged * @p: object to reallocate memory for * @size: the size to reallocate + * @align: requested alignment * @flags: the flags for the page level allocator + * @nid: node number of the target node + * + * If @p is %NULL, vrealloc_XXX() behaves exactly like vmalloc_XXX(). If @size + * is 0 and @p is not a %NULL pointer, the object pointed to is freed. * - * If @p is %NULL, vrealloc() behaves exactly like vmalloc(). If @size is 0 and - * @p is not a %NULL pointer, the object pointed to is freed. + * If the caller wants the new memory to be on specific node *only*, + * __GFP_THISNODE flag should be set, otherwise the function will try to avoid + * reallocation and possibly disregard the specified @nid. * * If __GFP_ZERO logic is requested, callers must ensure that, starting with the * initial memory allocation, every subsequent call to this API for the same * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that * __GFP_ZERO is not fully honored by this API. * + * Requesting an alignment that is bigger than the alignment of the existing + * allocation will fail. + * * In any case, the contents of the object pointed to are preserved up to the * lesser of the new and old sizes. * @@ -4103,7 +4113,8 @@ EXPORT_SYMBOL(vzalloc_node_noprof); * Return: pointer to the allocated memory; %NULL if @size is zero or in case of * failure */ -void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) +void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align, + gfp_t flags, int nid) { struct vm_struct *vm = NULL; size_t alloced_size = 0; @@ -4127,6 +4138,12 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) if (WARN(alloced_size < old_size, "vrealloc() has mismatched area vs requested sizes (%p)\n", p)) return NULL; + if (WARN(!IS_ALIGNED((unsigned long)p, align), + "will not reallocate with a bigger alignment (0x%lx)\n", align)) + return NULL; + if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE && + nid != page_to_nid(vmalloc_to_page(p))) + goto need_realloc; } /* @@ -4157,8 +4174,10 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags) return (void *)p; } +need_realloc: /* TODO: Grow the vm_area, i.e. allocate and map additional pages. */ - n = __vmalloc_noprof(size, flags); + n = __vmalloc_node_noprof(size, align, flags, nid, __builtin_return_address(0)); + if (!n) return NULL; -- 2.39.2