From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B79B2EB87B; Wed, 3 Dec 2025 23:30:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764804637; cv=none; b=g2+cMtoMK/lyiYpveHSMtdkDf1VMrRfpKA4/iBapJk9mq4bag+FCWrfuS3A0zyuyYKH8hZXkEoiXwdvoB2juzdAtv4LewnZNtXgu0d5Ox7UNm3m8VKDvi/RDo4JULCb50I9PXGsoXWGDb9T6qweB5ys8dC3zgSbdGUNmKb9KGnY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764804637; c=relaxed/simple; bh=/3oEfKp5bU/+SWlxMLCCWi/qv6HDY8MmV0RMn5cIJmw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PLs86OB0KRVzKVAc901T2tku1HKRdWJWRpDYsz35W23P8b3mtvTRNVD1dNmyyfzvyTGbEkWZdX4fnDLujsfmXuKeuz+QK8CkRkYtKEcj4paH6CsadZrZgmPSvSJGE58HYS6h/lQwuY0JS6jBPXoemb1AFyr9y3R1W7MM5aRarmE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QdDp1s8C; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QdDp1s8C" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30DBEC2BCB2; Wed, 3 Dec 2025 23:30:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1764804637; bh=/3oEfKp5bU/+SWlxMLCCWi/qv6HDY8MmV0RMn5cIJmw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QdDp1s8CleK4hn41r/2RClG5pRP24c3+GUC2GorVnlaRiJkA+OO5i3A0z+CAyKZ0Q dG0gZzAzwJFADJPJ53r65BfarPkFk1HJq9+WX8GHmuhrmPtZZ70HSbAKWlCrWqd+tK A8eJdl9ouGG1RD2Plye9Fj4BbMJNDMFIWDJ+U/vyWcG0RErdZ1/uj6bfAoEQ+uzVHZ qv3xUYM8tlNoRM7f3sZ29ueuvPbH+lxrzBPFV9RsRea/n/9+V3EYFbOWRz1Wk0jyRL A52Og2BDbN7t/zU1B4M8VCbaxeTVh7D5TtyXtNScg0JKvYKGhHi5Y3d+OMF/D4lgeO 5hUSu6sPYr1GA== From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Jonathan Corbet , Andrew Morton , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , "Gustavo A. R. Silva" , workflows@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, Linus Torvalds , Randy Dunlap , Miguel Ojeda , Przemek Kitszel , Matthew Wilcox , John Hubbard , Joe Perches , Christoph Lameter , Marco Elver , Vegard Nossum , Pekka Enberg , Joonsoo Kim , Bill Wendling , Justin Stitt , Jann Horn , Greg Kroah-Hartman , Sasha Levin , Nathan Chancellor , Peter Zijlstra , Nick Desaulniers , Jakub Kicinski , Yafang Shao , Tony Ambardar , Alexander Lobakin , Jan Hendrik Farr , Alexander Potapenko , linux-kernel@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH v6 4/5] slab: Introduce kmalloc_flex() and family Date: Wed, 3 Dec 2025 15:30:34 -0800 Message-Id: <20251203233036.3212363-4-kees@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20251203233029.it.641-kees@kernel.org> References: <20251203233029.it.641-kees@kernel.org> Precedence: bulk X-Mailing-List: workflows@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7374; i=kees@kernel.org; h=from:subject; bh=/3oEfKp5bU/+SWlxMLCCWi/qv6HDY8MmV0RMn5cIJmw=; b=owGbwMvMwCVmps19z/KJym7G02pJDJkGJ6TKdwfPPHTuSsniQ00Mj6pWurkzsn85a/0uzETQn uPgkx1hHaUsDGJcDLJiiixBdu5xLh5v28Pd5yrCzGFlAhnCwMUpABOJU2Jk+KHcPLc3LHhHPr+l vjW/yYcIiz9m39hdXxyZNddfTuV9NCPD0vlfvwUqCsoHBT15EVrWv8Lyz9ulQnNSdlxrfZh8qdG IHQA= X-Developer-Key: i=kees@kernel.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Content-Transfer-Encoding: 8bit As done for kmalloc_obj*(), introduce a type-aware allocator for flexible arrays, which may also have "counted_by" annotations: ptr = kmalloc(struct_size(ptr, flex_member, count), gfp); becomes: ptr = kmalloc_flex(*ptr, flex_member, count, gfp); The internal use of __flex_counter() allows for automatically setting the counter member of a struct's flexible array member when it has been annotated with __counted_by(), avoiding any missed early size initializations while __counted_by() annotations are added to the kernel. Additionally, this also checks for "too large" allocations based on the type size of the counter variable. For example: if (count > type_max(ptr->flex_counter)) fail...; size = struct_size(ptr, flex_member, count); ptr = kmalloc(size, gfp); ptr->flex_counter = count; becomes (n.b. unchanged from earlier example): ptr = kmalloc_flex(*ptr, flex_member, count, gfp); ptr->flex_count = count; Note that manual initialization of the flexible array counter is still required (at some point) after allocation as not all compiler versions support the __counted_by annotation yet. But doing it internally makes sure they cannot be missed when __counted_by _is_ available, meaning that the bounds checker will not trip due to the lack of "early enough" initializations that used to work before enabling the stricter bounds checking. For example: ptr = kmalloc_flex(*ptr, flex_member, count, gfp); fill(ptr->flex, count); ptr->flex_count = count; This works correctly before adding a __counted_by annotation (since nothing is checking ptr->flex accesses against ptr->flex_count). After adding the annotation, the bounds sanitizer would trip during fill() because ptr->flex_count wasn't set yet. But with kmalloc_flex() setting ptr->flex_count internally at allocation time, the existing code works without needing to move the ptr->flex_count assignment before the call to fill(). (This has been a stumbling block for __counted_by adoption.) Signed-off-by: Kees Cook --- Cc: Jonathan Corbet Cc: Vlastimil Babka Cc: Andrew Morton Cc: Christoph Lameter Cc: David Rientjes Cc: Roman Gushchin Cc: Harry Yoo Cc: "Gustavo A. R. Silva" Cc: Cc: Cc: Cc: --- Documentation/process/deprecated.rst | 7 ++++ include/linux/slab.h | 48 ++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/Documentation/process/deprecated.rst b/Documentation/process/deprecated.rst index 91c628fa2d59..fed56864d036 100644 --- a/Documentation/process/deprecated.rst +++ b/Documentation/process/deprecated.rst @@ -387,6 +387,7 @@ allocations. For example, these open coded assignments:: ptr = kzalloc(sizeof(*ptr), gfp); ptr = kmalloc_array(count, sizeof(*ptr), gfp); ptr = kcalloc(count, sizeof(*ptr), gfp); + ptr = kmalloc(struct_size(ptr, flex_member, count), gfp); ptr = kmalloc(sizeof(struct foo, gfp); become, respectively:: @@ -395,4 +396,10 @@ become, respectively:: ptr = kzalloc_obj(*ptr, gfp); ptr = kmalloc_objs(*ptr, count, gfp); ptr = kzalloc_objs(*ptr, count, gfp); + ptr = kmalloc_flex(*ptr, flex_member, count, gfp); __auto_type ptr = kmalloc_obj(struct foo, gfp); + +If `ptr->flex_member` is annotated with __counted_by(), the allocation +will automatically fail if `count` is larger than the maximum +representable value that can be stored in the counter member associated +with `flex_member`. diff --git a/include/linux/slab.h b/include/linux/slab.h index 726457daedbd..2656ea610b68 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -982,6 +982,33 @@ void *kmalloc_nolock_noprof(size_t size, gfp_t gfp_flags, int node); (TYPE *)KMALLOC(__obj_size, GFP); \ }) +/** + * __alloc_flex - Allocate an object that has a trailing flexible array + * @KMALLOC: kmalloc wrapper function to use for allocation. + * @GFP: GFP flags for the allocation. + * @TYPE: type of structure to allocate space for. + * @FAM: The name of the flexible array member of @TYPE structure. + * @COUNT: how many @FAM elements to allocate space for. + * + * Returns: Newly allocated pointer to @TYPE with @COUNT-many trailing + * @FAM elements, or NULL on failure or if @COUNT cannot be represented + * by the member of @TYPE that counts the @FAM elements (annotated via + * __counted_by()). + */ +#define __alloc_flex(KMALLOC, GFP, TYPE, FAM, COUNT) \ +({ \ + const size_t __count = (COUNT); \ + const size_t __obj_size = struct_size_t(TYPE, FAM, __count); \ + TYPE *__obj_ptr; \ + if (WARN_ON_ONCE(overflows_flex_counter_type(TYPE, FAM, __count))) \ + __obj_ptr = NULL; \ + else \ + __obj_ptr = KMALLOC(__obj_size, GFP); \ + if (__obj_ptr) \ + __set_flex_counter(__obj_ptr->FAM, __count); \ + __obj_ptr; \ +}) + /** * kmalloc_obj - Allocate a single instance of the given type * @VAR_OR_TYPE: Variable or type to allocate. @@ -1005,23 +1032,44 @@ void *kmalloc_nolock_noprof(size_t size, gfp_t gfp_flags, int node); #define kmalloc_objs(VAR_OR_TYPE, COUNT, GFP) \ __alloc_objs(kmalloc, GFP, typeof(VAR_OR_TYPE), COUNT) +/** + * kmalloc_flex - Allocate a single instance of the given flexible structure + * @VAR_OR_TYPE: Variable or type to allocate (with its flex array). + * @FAM: The name of the flexible array member of the structure. + * @COUNT: How many flexible array member elements are desired. + * @GFP: GFP flags for the allocation. + * + * Returns: newly allocated pointer to @VAR_OR_TYPE on success, NULL on + * failure. If @FAM has been annotated with __counted_by(), the allocation + * will immediately fail if @COUNT is larger than what the type of the + * struct's counter variable can represent. + */ +#define kmalloc_flex(VAR_OR_TYPE, FAM, COUNT, GFP) \ + __alloc_flex(kmalloc, GFP, typeof(VAR_OR_TYPE), FAM, COUNT) + /* All kzalloc aliases for kmalloc_(obj|objs|flex). */ #define kzalloc_obj(P, GFP) \ __alloc_objs(kzalloc, GFP, typeof(P), 1) #define kzalloc_objs(P, COUNT, GFP) \ __alloc_objs(kzalloc, GFP, typeof(P), COUNT) +#define kzalloc_flex(P, FAM, COUNT, GFP) \ + __alloc_flex(kzalloc, GFP, typeof(P), FAM, COUNT) /* All kvmalloc aliases for kmalloc_(obj|objs|flex). */ #define kvmalloc_obj(P, GFP) \ __alloc_objs(kvmalloc, GFP, typeof(P), 1) #define kvmalloc_objs(P, COUNT, GFP) \ __alloc_objs(kvmalloc, GFP, typeof(P), COUNT) +#define kvmalloc_flex(P, FAM, COUNT, GFP) \ + __alloc_flex(kvmalloc, GFP, typeof(P), FAM, COUNT) /* All kvzalloc aliases for kmalloc_(obj|objs|flex). */ #define kvzalloc_obj(P, GFP) \ __alloc_objs(kvzalloc, GFP, typeof(P), 1) #define kvzalloc_objs(P, COUNT, GFP) \ __alloc_objs(kvzalloc, GFP, typeof(P), COUNT) +#define kvzalloc_flex(P, FAM, COUNT, GFP) \ + __alloc_flex(kvzalloc, GFP, typeof(P), FAM, COUNT) #define kmem_buckets_alloc(_b, _size, _flags) \ alloc_hooks(__kmalloc_node_noprof(PASS_BUCKET_PARAMS(_size, _b), _flags, NUMA_NO_NODE)) -- 2.34.1