Re: [PATCH] rust: slab: add basic slab module

Re: [PATCH] rust: slab: add basic slab module

[John Hubbard]

On 9/24/25 12:36 PM, Elijah Wright wrote:
> this patch adds a basic slab module for kmem_cache, primarily wrapping
> kmem_cache_create, kmem_cache_alloc, kmem_cache_free, and kmem_cache_destroy.

+Cc linux-mm

Maybe that's not required, but if so I'll be quite surprised. :)

thanks,
John Hubbard

> 
> Signed-off-by: Elijah Wright <git@elijahs.space>
> ---
>  rust/helpers/slab.c | 10 ++++++
>  rust/kernel/lib.rs  |  1 +
>  rust/kernel/slab.rs | 85 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 96 insertions(+)
>  create mode 100644 rust/kernel/slab.rs
> 
> diff --git a/rust/helpers/slab.c b/rust/helpers/slab.c
> index a842bfbddcba..799de7bc1405 100644
> --- a/rust/helpers/slab.c
> +++ b/rust/helpers/slab.c
> @@ -13,3 +13,13 @@ rust_helper_kvrealloc(const void *p, size_t size, gfp_t flags)
>  {
>  	return kvrealloc(p, size, flags);
>  }
> +
> +struct kmem_cache * rust_helper_kmem_cache_create(const char *name, unsigned int size, unsigned int align, gfp_t flags, void (*ctor)(void *))
> +{
> +	return kmem_cache_create(name, size, align, flags, NULL);
> +}
> +
> +void * rust_helper_kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags)
> +{
> +	return kmem_cache_alloc(cachep, flags);
> +}
> \ No newline at end of file
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index fef97f2a5098..bd76eadbe297 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -116,6 +116,7 @@
>  pub mod security;
>  pub mod seq_file;
>  pub mod sizes;
> +pub mod slab;
>  mod static_assert;
>  #[doc(hidden)]
>  pub mod std_vendor;
> diff --git a/rust/kernel/slab.rs b/rust/kernel/slab.rs
> new file mode 100644
> index 000000000000..8b418f9db7cb
> --- /dev/null
> +++ b/rust/kernel/slab.rs
> @@ -0,0 +1,85 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Slab bindings.
> +//!
> +//! C header: [`include/linux/slab.h`](srctree/include/linux/slab.h)
> +
> +use core::{marker::PhantomData, mem, ptr::NonNull};
> +
> +use crate::{
> +    alloc::Flags,
> +    bindings,
> +    error::{code::ENOMEM, Result},
> +    str::CStr,
> +};
> +
> +/// A wrapper for kmem_cache that allocates objects of type `T`.
> +#[repr(transparent)]
> +pub struct Slab<T> {
> +    cache: NonNull<bindings::kmem_cache>,
> +    _p: PhantomData<T>,
> +}
> +
> +impl<T> Slab<T> {
> +    /// Creates a cache for objects of type `T`.
> +    pub fn try_new(name: &CStr, flags: Flags) -> Result<Self> {
> +        let size = mem::size_of::<T>();
> +        let align = mem::align_of::<T>();
> +        debug_assert!(size <= usize::MAX);
> +        debug_assert!(align <= usize::MAX);
> +
> +        // SAFETY: `flags` is a valid impl, `name` is a valid C string, and
> +        // other arguments are plain values.
> +        let cache = unsafe {
> +            bindings::kmem_cache_create(
> +                name.as_char_ptr(),
> +                size as u32,
> +                align as u32,
> +                flags.as_raw(),
> +                None,
> +            )
> +        };
> +
> +        NonNull::new(cache)
> +            .map(|c| Slab {
> +                cache: c,
> +                _p: PhantomData,
> +            })
> +            .ok_or(ENOMEM)
> +    }
> +
> +    /// Allocates one object from the cache with the given gfp flags.
> +    #[inline]
> +    pub fn alloc(&self, flags: Flags) -> Result<NonNull<T>> {
> +        // SAFETY: `self.cache` is a valid pointer obtained from
> +        // `kmem_cache_create` and still alive because `self` is borrowed.
> +        let ptr = unsafe { bindings::kmem_cache_alloc(self.cache.as_ptr(), flags.as_raw()) };
> +        NonNull::new(ptr.cast()).ok_or(ENOMEM)
> +    }
> +
> +    /// Frees an object previously returned by `alloc()`.
> +    ///
> +    /// # Safety
> +    /// The caller must guarantee that `obj` was allocated from this cache and
> +    /// is no longer accessed afterwards.
> +    #[inline]
> +    pub unsafe fn free(&self, obj: NonNull<T>) {
> +        // SAFETY: By the safety contract the pointer is valid and unique at
> +        // this point.
> +        unsafe { bindings::kmem_cache_free(self.cache.as_ptr(), obj.cast().as_ptr()) };
> +    }
> +
> +    /// Returns the raw mutable pointer to the cache
> +    #[inline]
> +    pub fn as_ptr(&self) -> *mut bindings::kmem_cache {
> +        self.cache.as_ptr()
> +    }
> +}
> +
> +impl<T> Drop for Slab<T> {
> +    fn drop(&mut self) {
> +        // SAFETY: `self.cache` is valid and we are the final owner because
> +        // of ownership rules.
> +        unsafe { bindings::kmem_cache_destroy(self.cache.as_ptr()) };
> +    }
> +}

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

(+Cc: Lorenzo, Vlastimil, Liam, Uladzislau, MM)

On Wed Sep 24, 2025 at 9:36 PM CEST, Elijah Wright wrote:
> this patch adds a basic slab module for kmem_cache, primarily wrapping
> kmem_cache_create, kmem_cache_alloc, kmem_cache_free, and kmem_cache_destroy.

What's the motivation?

I mean, we will need kmem_cache soon. But the users will all be drivers, e.g.
the GPU drivers that people work on currently.

Drivers shouldn't use "raw" allocators (such as Kmalloc [1] or Vmalloc [2]), but
the corresponding "managed" allocation primitives, such as KBox [3], VBox [4],
KVec, etc.

Therefore, the code below shouldn't be used by drivers directly, hence the
question for motivation.

In any case, kmem_cache is a special allocator (special as in it can have a
non-static lifetime in contrast to other kernel allocators) and should be
integrated with the existing infrastructure in rust/kernel/alloc/.

I think there are multiple options for that; (1) isn't really an option, but I
think it's good to mention anyways:

  (1) Allow for non-zero sized implementations of the Allocator trait [3], such
      that we can store a reference count to the KmemCache. This is necessary to
      ensure that a Box<T, KmemCache> can't out-live the KmemCache itself.

      The reason why I said it's not really an option is because it discards the
      option for dynamic dispatch of the generic Box type.

  (2) Same as (1), but with a custom Box type. This keeps dynamic dispatch for
      the generic Box type (i.e. KBox, VBox, KVBox), but duplicates quite some
      code and still doesn't allow for dynamic dispatch for the KmemCacheBox.

  (3) Implement a macro to generate a custom KmemCache Allocator trait
      implementation for every KmemCache instance with a static lifetime.

      This makes KmemCache technically equivalent to the other allocators, such
      as Kmalloc, etc. but obviously has the downside that the KmemCache might
      live much longer than required.

      Technically, most KmemCache instances live for the whole module lifetime,
      so it might be fine though.

      (This is what I think Alice proposed.)

  (4) Solve the problem on the C side and let kmem_cache_alloc() take care of
      acquiring a reference count to the backing kmem_cache. The main question
      here would be where to store the pointer for decreasing the reference
      count on kmem_cache_free().

      Theoretically, it could be stored within the allocation itself, but it's a
      bit of a yikes.

      However, it would resolve all the mentioned problems above.

I'd like to see (3) or (4), also depending on what the MM folks think.

- Danilo

[1] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Kmalloc.html
[2] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Vmalloc.html
[3] https://rust.docs.kernel.org/kernel/alloc/kbox/type.KBox.html
[4] https://rust.docs.kernel.org/kernel/alloc/kbox/type.VBox.html
[5] https://rust.docs.kernel.org/kernel/alloc/trait.Allocator.html

Re: [PATCH] rust: slab: add basic slab module

[Elijah]

I was thinking of maybe creating something like KBox for kmem_cache but 
I didn't want to touch allocator code yet, I figured I would just create 
the groundwork for that to exist. rbtree.rs uses KBox now but I'm not 
sure it should, at least if it's going to scale to many nodes

On 9/25/2025 2:54 AM, Danilo Krummrich wrote:
> What's the motivation?
> 
> I mean, we will need kmem_cache soon. But the users will all be drivers, e.g.
> the GPU drivers that people work on currently.
> 
> Drivers shouldn't use "raw" allocators (such as Kmalloc [1] or Vmalloc [2]), but
> the corresponding "managed" allocation primitives, such as KBox [3], VBox [4],
> KVec, etc.
> 
> Therefore, the code below shouldn't be used by drivers directly, hence the
> question for motivation.
> 
> In any case, kmem_cache is a special allocator (special as in it can have a
> non-static lifetime in contrast to other kernel allocators) and should be
> integrated with the existing infrastructure in rust/kernel/alloc/.
> 
> I think there are multiple options for that; (1) isn't really an option, but I
> think it's good to mention anyways:
> 
>    (1) Allow for non-zero sized implementations of the Allocator trait [3], such
>        that we can store a reference count to the KmemCache. This is necessary to
>        ensure that a Box<T, KmemCache> can't out-live the KmemCache itself.
> 
>        The reason why I said it's not really an option is because it discards the
>        option for dynamic dispatch of the generic Box type.
> 
>    (2) Same as (1), but with a custom Box type. This keeps dynamic dispatch for
>        the generic Box type (i.e. KBox, VBox, KVBox), but duplicates quite some
>        code and still doesn't allow for dynamic dispatch for the KmemCacheBox.
> 
>    (3) Implement a macro to generate a custom KmemCache Allocator trait
>        implementation for every KmemCache instance with a static lifetime.
> 
>        This makes KmemCache technically equivalent to the other allocators, such
>        as Kmalloc, etc. but obviously has the downside that the KmemCache might
>        live much longer than required.
> 
>        Technically, most KmemCache instances live for the whole module lifetime,
>        so it might be fine though.
> 
>        (This is what I think Alice proposed.)
> 
>    (4) Solve the problem on the C side and let kmem_cache_alloc() take care of
>        acquiring a reference count to the backing kmem_cache. The main question
>        here would be where to store the pointer for decreasing the reference
>        count on kmem_cache_free().
> 
>        Theoretically, it could be stored within the allocation itself, but it's a
>        bit of a yikes.
> 
>        However, it would resolve all the mentioned problems above.
> 
> I'd like to see (3) or (4), also depending on what the MM folks think.

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Thu Sep 25, 2025 at 7:20 PM CEST, Elijah wrote:

Please don't top post [1]; use interleaved style instead.

[1] https://subspace.kernel.org/etiquette.html#do-not-top-post-when-replying

> I was thinking of maybe creating something like KBox for kmem_cache but 
> I didn't want to touch allocator code yet, I figured I would just create 
> the groundwork for that to exist. rbtree.rs uses KBox now but I'm not 
> sure it should, at least if it's going to scale to many nodes

Ok, so you want to support kmemcache for rbtree nodes. Ideally, you should also
have a use-case for that, but given that we'll also need kmemcache in other
drivers (such as Nova) anyways, I think that's fine.

However, in any case this should be integrated into the rust/kernel/alloc/
infrastructure in one of the ways described below.

As mentioned, I would like to see (3) or (4). (2) may be viable as well, but I
really hate the resulting code duplication (not having dynamic dispatch for a
kmemcache backed Box is probably not that big of a deal though).

Anyways, I'd also like to hear some more opinions, especially regarding (4), as
mentioned.

> On 9/25/2025 2:54 AM, Danilo Krummrich wrote:
>> What's the motivation?
>> 
>> I mean, we will need kmem_cache soon. But the users will all be drivers, e.g.
>> the GPU drivers that people work on currently.
>> 
>> Drivers shouldn't use "raw" allocators (such as Kmalloc [1] or Vmalloc [2]), but
>> the corresponding "managed" allocation primitives, such as KBox [3], VBox [4],
>> KVec, etc.
>> 
>> Therefore, the code below shouldn't be used by drivers directly, hence the
>> question for motivation.
>> 
>> In any case, kmem_cache is a special allocator (special as in it can have a
>> non-static lifetime in contrast to other kernel allocators) and should be
>> integrated with the existing infrastructure in rust/kernel/alloc/.
>> 
>> I think there are multiple options for that; (1) isn't really an option, but I
>> think it's good to mention anyways:
>> 
>>    (1) Allow for non-zero sized implementations of the Allocator trait [3], such
>>        that we can store a reference count to the KmemCache. This is necessary to
>>        ensure that a Box<T, KmemCache> can't out-live the KmemCache itself.
>> 
>>        The reason why I said it's not really an option is because it discards the
>>        option for dynamic dispatch of the generic Box type.
>> 
>>    (2) Same as (1), but with a custom Box type. This keeps dynamic dispatch for
>>        the generic Box type (i.e. KBox, VBox, KVBox), but duplicates quite some
>>        code and still doesn't allow for dynamic dispatch for the KmemCacheBox.
>> 
>>    (3) Implement a macro to generate a custom KmemCache Allocator trait
>>        implementation for every KmemCache instance with a static lifetime.
>> 
>>        This makes KmemCache technically equivalent to the other allocators, such
>>        as Kmalloc, etc. but obviously has the downside that the KmemCache might
>>        live much longer than required.
>> 
>>        Technically, most KmemCache instances live for the whole module lifetime,
>>        so it might be fine though.
>> 
>>        (This is what I think Alice proposed.)
>> 
>>    (4) Solve the problem on the C side and let kmem_cache_alloc() take care of
>>        acquiring a reference count to the backing kmem_cache. The main question
>>        here would be where to store the pointer for decreasing the reference
>>        count on kmem_cache_free().
>> 
>>        Theoretically, it could be stored within the allocation itself, but it's a
>>        bit of a yikes.
>> 
>>        However, it would resolve all the mentioned problems above.
>> 
>> I'd like to see (3) or (4), also depending on what the MM folks think.

Re: [PATCH] rust: slab: add basic slab module

[Liam R. Howlett]

* Danilo Krummrich <dakr@kernel.org> [250925 13:43]:
> On Thu Sep 25, 2025 at 7:20 PM CEST, Elijah wrote:
> 

...

> 
> > I was thinking of maybe creating something like KBox for kmem_cache but 
> > I didn't want to touch allocator code yet, I figured I would just create 
> > the groundwork for that to exist. rbtree.rs uses KBox now but I'm not 
> > sure it should, at least if it's going to scale to many nodes
> 
> Ok, so you want to support kmemcache for rbtree nodes. Ideally, you should also
> have a use-case for that, but given that we'll also need kmemcache in other
> drivers (such as Nova) anyways, I think that's fine.

This seems different than what exists on the C side, at least to me.
The rbtree is implemented by embedding the links to the prev/next into
the struct which is using the tree.

The above sounds like the nodes of the rbtree are allocated on its own
and not part of another allocation on the rust side?

That is, the kmemcache would allocate the struct that contains the
rbtree linkage (or nodes, I guess), but not the nodes alone.

Thanks,
Liam

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Thu Sep 25, 2025 at 8:02 PM CEST, Liam R. Howlett wrote:
> * Danilo Krummrich <dakr@kernel.org> [250925 13:43]:
>> On Thu Sep 25, 2025 at 7:20 PM CEST, Elijah wrote:
>> 
>
> ...
>
>> 
>> > I was thinking of maybe creating something like KBox for kmem_cache but 
>> > I didn't want to touch allocator code yet, I figured I would just create 
>> > the groundwork for that to exist. rbtree.rs uses KBox now but I'm not 
>> > sure it should, at least if it's going to scale to many nodes
>> 
>> Ok, so you want to support kmemcache for rbtree nodes. Ideally, you should also
>> have a use-case for that, but given that we'll also need kmemcache in other
>> drivers (such as Nova) anyways, I think that's fine.
>
> This seems different than what exists on the C side, at least to me.
> The rbtree is implemented by embedding the links to the prev/next into
> the struct which is using the tree.
>
> The above sounds like the nodes of the rbtree are allocated on its own
> and not part of another allocation on the rust side?
>
> That is, the kmemcache would allocate the struct that contains the
> rbtree linkage (or nodes, I guess), but not the nodes alone.

On the Rust side the rbtree's Node structure looks like this:

	struct Node<K, V> {
	    links: bindings::rb_node,
	    key: K,
	    value: V,
	}

The allocation would be for the entire rbtree::Node and not only for the C
struct rb_node.

Re: [PATCH] rust: slab: add basic slab module

[Boqun Feng]

On Thu, Sep 25, 2025 at 02:02:55PM -0400, Liam R. Howlett wrote:
> * Danilo Krummrich <dakr@kernel.org> [250925 13:43]:
> > On Thu Sep 25, 2025 at 7:20 PM CEST, Elijah wrote:
> > 
> 
> ...
> 
> > 
> > > I was thinking of maybe creating something like KBox for kmem_cache but 
> > > I didn't want to touch allocator code yet, I figured I would just create 
> > > the groundwork for that to exist. rbtree.rs uses KBox now but I'm not 
> > > sure it should, at least if it's going to scale to many nodes
> > 
> > Ok, so you want to support kmemcache for rbtree nodes. Ideally, you should also
> > have a use-case for that, but given that we'll also need kmemcache in other
> > drivers (such as Nova) anyways, I think that's fine.
> 
> This seems different than what exists on the C side, at least to me.
> The rbtree is implemented by embedding the links to the prev/next into
> the struct which is using the tree.
> 
> The above sounds like the nodes of the rbtree are allocated on its own
> and not part of another allocation on the rust side?
> 

For current in-tree API, yes, because we have to start with something
simple and the user (binder I believe?) doesn't require
external-allocated nodes.

Regards,
Boqun

> That is, the kmemcache would allocate the struct that contains the
> rbtree linkage (or nodes, I guess), but not the nodes alone.
> 
> Thanks,
> Liam

Re: [PATCH] rust: slab: add basic slab module

[Elijah]

On 9/25/2025 10:43 AM, Danilo Krummrich wrote:

 > Anyways, I'd also like to hear some more opinions, especially 
regarding (4), as
 > mentioned.

ok I just looked it over. I think (4) isn't really viable, the cache 
pointer storage overhead is probably too much (in addition to some 
complex issues). I don't really like having a macro (so (3) basically) 
and I think there are issues with non-static caches, I don't know 
specifically though. obviously (1) doesn't work. (2) is exactly what I 
was thinking specifically for rbtree. IMO the only useful options are 
(2) and (3). I would say maybe implement (2) first and if you want 
macro-generated ZST allocators they can wrap whatever that Box ends up 
being.

 > not having dynamic dispatch for a
kmemcache backed Box is probably not that big of a deal though

I agree

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Thu Sep 25, 2025 at 8:05 PM CEST, Elijah wrote:
> On 9/25/2025 10:43 AM, Danilo Krummrich wrote:
>
>  > Anyways, I'd also like to hear some more opinions, especially 
> regarding (4), as
>  > mentioned.
>
> ok I just looked it over. I think (4) isn't really viable, the cache 
> pointer storage overhead is probably too much (in addition to some 
> complex issues).

Probably, given that kmemcache is meant to be used when we have a lot of (small)
allocations.

However, I'd like to hear from the MM folks, if there may be options in the
existing infrastructure, or if it's really a dead end.

What are the other complex issues you are referring to?

> I don't really like having a macro (so (3) basically) 
> and I think there are issues with non-static caches, I don't know 
> specifically though. obviously (1) doesn't work. (2) is exactly what I 
> was thinking specifically for rbtree. IMO the only useful options are 
> (2) and (3). I would say maybe implement (2) first and if you want 
> macro-generated ZST allocators they can wrap whatever that Box ends up 
> being.

I'm not sold on duplicating the Box code just yet. What issues are you referring
to?

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Thu Sep 25, 2025 at 8:15 PM CEST, Danilo Krummrich wrote:
> On Thu Sep 25, 2025 at 8:05 PM CEST, Elijah wrote:
>> On 9/25/2025 10:43 AM, Danilo Krummrich wrote:
>>
>>  > Anyways, I'd also like to hear some more opinions, especially 
>> regarding (4), as
>>  > mentioned.
>>
>> ok I just looked it over. I think (4) isn't really viable, the cache 
>> pointer storage overhead is probably too much (in addition to some 
>> complex issues).
>
> Probably, given that kmemcache is meant to be used when we have a lot of (small)
> allocations.
>
> However, I'd like to hear from the MM folks, if there may be options in the
> existing infrastructure, or if it's really a dead end.
>
> What are the other complex issues you are referring to?
>
>> I don't really like having a macro (so (3) basically) 
>> and I think there are issues with non-static caches, I don't know 
>> specifically though. obviously (1) doesn't work. (2) is exactly what I 
>> was thinking specifically for rbtree. IMO the only useful options are 
>> (2) and (3). I would say maybe implement (2) first and if you want 
>> macro-generated ZST allocators they can wrap whatever that Box ends up 
>> being.
>
> I'm not sold on duplicating the Box code just yet. What issues are you referring
> to?

Also, if we argue that (4) has a problem with scalability, then (2) has the
problem as well.

Re: [PATCH] rust: slab: add basic slab module

[Elijah]

On 9/25/2025 11:51 AM, Elijah wrote:
> On 9/25/2025 11:15 AM, Danilo Krummrich wrote:
>> What are the other complex issues you are referring to?
> 
> I think I misunderstood what was involved for (4). I was thinking of 
> keeping the extra reference in the object but I think it can be done per 
> CPU. I'm not totally sure how much would be changed in the kernel but no 
> matter the implementation for (4) there will be some complexity
> 
>> I'm not sold on duplicating the Box code just yet. What issues are you 
>> referring
>> to?
> 
> mainly where the cache isn't static. if you have kmem accounting enabled 
> and a ZST that encodes the original cache's address then that would 
> break, unless I'm misunderstanding (3), but that's just an example. I'm 
> just thinking of potential code in the kernel now

Re: [PATCH] rust: slab: add basic slab module

[Alice Ryhl]

On Thu, Sep 25, 2025 at 11:54 AM Danilo Krummrich <dakr@kernel.org> wrote:
>   (3) Implement a macro to generate a custom KmemCache Allocator trait
>       implementation for every KmemCache instance with a static lifetime.
>
>       This makes KmemCache technically equivalent to the other allocators, such
>       as Kmalloc, etc. but obviously has the downside that the KmemCache might
>       live much longer than required.
>
>       Technically, most KmemCache instances live for the whole module lifetime,
>       so it might be fine though.
>
>       (This is what I think Alice proposed.)

Yes, this is what I proposed. Most kmemcaches live for the duration of
the module, so it's no issue if they are global.

Alice

Re: [PATCH] rust: slab: add basic slab module

[Vlastimil Babka]

On 9/25/25 11:54, Danilo Krummrich wrote:
> (+Cc: Lorenzo, Vlastimil, Liam, Uladzislau, MM)
> 
> On Wed Sep 24, 2025 at 9:36 PM CEST, Elijah Wright wrote:
>> this patch adds a basic slab module for kmem_cache, primarily wrapping
>> kmem_cache_create, kmem_cache_alloc, kmem_cache_free, and kmem_cache_destroy.
> 
> What's the motivation?
> 
> I mean, we will need kmem_cache soon. But the users will all be drivers, e.g.
> the GPU drivers that people work on currently.
> 
> Drivers shouldn't use "raw" allocators (such as Kmalloc [1] or Vmalloc [2]), but
> the corresponding "managed" allocation primitives, such as KBox [3], VBox [4],
> KVec, etc.
> 
> Therefore, the code below shouldn't be used by drivers directly, hence the
> question for motivation.
> 
> In any case, kmem_cache is a special allocator (special as in it can have a
> non-static lifetime in contrast to other kernel allocators) and should be
> integrated with the existing infrastructure in rust/kernel/alloc/.
> 
> I think there are multiple options for that; (1) isn't really an option, but I
> think it's good to mention anyways:
> 
>   (1) Allow for non-zero sized implementations of the Allocator trait [3], such
>       that we can store a reference count to the KmemCache. This is necessary to
>       ensure that a Box<T, KmemCache> can't out-live the KmemCache itself.
> 
>       The reason why I said it's not really an option is because it discards the
>       option for dynamic dispatch of the generic Box type.
> 
>   (2) Same as (1), but with a custom Box type. This keeps dynamic dispatch for
>       the generic Box type (i.e. KBox, VBox, KVBox), but duplicates quite some
>       code and still doesn't allow for dynamic dispatch for the KmemCacheBox.
> 
>   (3) Implement a macro to generate a custom KmemCache Allocator trait
>       implementation for every KmemCache instance with a static lifetime.
> 
>       This makes KmemCache technically equivalent to the other allocators, such
>       as Kmalloc, etc. but obviously has the downside that the KmemCache might
>       live much longer than required.

I don't know enough of Rust to follow why is that. What does mean "much longer"?

>       Technically, most KmemCache instances live for the whole module lifetime,
>       so it might be fine though.

I think so, yeah.

>       (This is what I think Alice proposed.)
> 
>   (4) Solve the problem on the C side and let kmem_cache_alloc() take care of
>       acquiring a reference count to the backing kmem_cache. The main question
>       here would be where to store the pointer for decreasing the reference
>       count on kmem_cache_free().

Pointer to what, the ref counter? If it's part of struct kmem_cache, then we
have pointer to that in kmem_cache_free(). Even when kfree() is used, it can
be (or rather already is) obtained. So that's not the issue (unless I
misunderstand).

But we wouldn't want to pay the cost of the atomic updates of the refcount
for all caches. If done only for Rust-created caches, the implementation
also would better not to add branches to all allocation/free fastpaths.

But also why is this refcounting needed? What would happen on the Rust side
if someone destroyed the cache too early? In the underlying C implementation
we notice it (reliably AFAICT), warn and abort the destroy (leaving it as a
kind of zombie) so I'd say it's safe. So if Rust needs to know if the
destroy was successful or premature, we could probably just return the
result instead of the current void.

>       Theoretically, it could be stored within the allocation itself, but it's a
>       bit of a yikes.
> 
>       However, it would resolve all the mentioned problems above.
> 
> I'd like to see (3) or (4), also depending on what the MM folks think.
> 
> - Danilo
> 
> [1] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Kmalloc.html
> [2] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Vmalloc.html
> [3] https://rust.docs.kernel.org/kernel/alloc/kbox/type.KBox.html
> [4] https://rust.docs.kernel.org/kernel/alloc/kbox/type.VBox.html
> [5] https://rust.docs.kernel.org/kernel/alloc/trait.Allocator.html

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Fri Sep 26, 2025 at 5:00 PM CEST, Vlastimil Babka wrote:
> On 9/25/25 11:54, Danilo Krummrich wrote:
>> (+Cc: Lorenzo, Vlastimil, Liam, Uladzislau, MM)
>> 
>> On Wed Sep 24, 2025 at 9:36 PM CEST, Elijah Wright wrote:
>>> this patch adds a basic slab module for kmem_cache, primarily wrapping
>>> kmem_cache_create, kmem_cache_alloc, kmem_cache_free, and kmem_cache_destroy.
>> 
>> What's the motivation?
>> 
>> I mean, we will need kmem_cache soon. But the users will all be drivers, e.g.
>> the GPU drivers that people work on currently.
>> 
>> Drivers shouldn't use "raw" allocators (such as Kmalloc [1] or Vmalloc [2]), but
>> the corresponding "managed" allocation primitives, such as KBox [3], VBox [4],
>> KVec, etc.
>> 
>> Therefore, the code below shouldn't be used by drivers directly, hence the
>> question for motivation.
>> 
>> In any case, kmem_cache is a special allocator (special as in it can have a
>> non-static lifetime in contrast to other kernel allocators) and should be
>> integrated with the existing infrastructure in rust/kernel/alloc/.
>> 
>> I think there are multiple options for that; (1) isn't really an option, but I
>> think it's good to mention anyways:
>> 
>>   (1) Allow for non-zero sized implementations of the Allocator trait [3], such
>>       that we can store a reference count to the KmemCache. This is necessary to
>>       ensure that a Box<T, KmemCache> can't out-live the KmemCache itself.
>> 
>>       The reason why I said it's not really an option is because it discards the
>>       option for dynamic dispatch of the generic Box type.
>> 
>>   (2) Same as (1), but with a custom Box type. This keeps dynamic dispatch for
>>       the generic Box type (i.e. KBox, VBox, KVBox), but duplicates quite some
>>       code and still doesn't allow for dynamic dispatch for the KmemCacheBox.
>> 
>>   (3) Implement a macro to generate a custom KmemCache Allocator trait
>>       implementation for every KmemCache instance with a static lifetime.
>> 
>>       This makes KmemCache technically equivalent to the other allocators, such
>>       as Kmalloc, etc. but obviously has the downside that the KmemCache might
>>       live much longer than required.
>
> I don't know enough of Rust to follow why is that. What does mean "much longer"?

Nothing Rust specific, I just point out that if the kmemcache does not need to
be tied to the module lifetime (but a more narrow scope for whatever reason), it
would still need to live as long as the module lives with (3). Which for
built-in means forever, which would likely be "much longer". :)

>>       Technically, most KmemCache instances live for the whole module lifetime,
>>       so it might be fine though.
>
> I think so, yeah.
>
>>       (This is what I think Alice proposed.)
>> 
>>   (4) Solve the problem on the C side and let kmem_cache_alloc() take care of
>>       acquiring a reference count to the backing kmem_cache. The main question
>>       here would be where to store the pointer for decreasing the reference
>>       count on kmem_cache_free().
>
> Pointer to what, the ref counter? If it's part of struct kmem_cache, then we
> have pointer to that in kmem_cache_free(). Even when kfree() is used, it can
> be (or rather already is) obtained. So that's not the issue (unless I
> misunderstand).

Yes, the reference count.

> But we wouldn't want to pay the cost of the atomic updates of the refcount
> for all caches. If done only for Rust-created caches, the implementation
> also would better not to add branches to all allocation/free fastpaths.

Indeed, that's why I was wondering if you see other options to deal with this in
the MM layer, and reading the below, it seems like you even already do.

> But also why is this refcounting needed? What would happen on the Rust side
> if someone destroyed the cache too early?

Well, that's the question. I thought that if the cache is destroyed potential
remaining allocations become invalid.

If that's the case we have to ensure that all allocations are freed before the
cache is destroyed.

> In the underlying C implementation
> we notice it (reliably AFAICT), warn and abort the destroy (leaving it as a
> kind of zombie) so I'd say it's safe. So if Rust needs to know if the
> destroy was successful or premature, we could probably just return the
> result instead of the current void.

The only thing we need on the Rust side is that existing allocations remain
valid even if the cache is destroyed. Or the other way around the cache is
silently kept alive internally.

>>       Theoretically, it could be stored within the allocation itself, but it's a
>>       bit of a yikes.
>> 
>>       However, it would resolve all the mentioned problems above.
>> 
>> I'd like to see (3) or (4), also depending on what the MM folks think.
>> 
>> - Danilo
>> 
>> [1] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Kmalloc.html
>> [2] https://rust.docs.kernel.org/kernel/alloc/allocator/struct.Vmalloc.html
>> [3] https://rust.docs.kernel.org/kernel/alloc/kbox/type.KBox.html
>> [4] https://rust.docs.kernel.org/kernel/alloc/kbox/type.VBox.html
>> [5] https://rust.docs.kernel.org/kernel/alloc/trait.Allocator.html

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Fri Sep 26, 2025 at 5:33 PM CEST, Danilo Krummrich wrote:
> The only thing we need on the Rust side is that existing allocations remain
> valid even if the cache is destroyed. Or the other way around the cache is
> silently kept alive internally.

Or to express it in C code:

	struct kmem_cache *cache = kmem_cache_create();
	struct Foo *foo = kmem_cache_alloc();

	// After this call cache will never be accessed; leaves a zombie cache,
	// since foo is still alive.
	kmem_cache_destroy(cache);

	// This must still be valid.
	foo->bar = 42;

	// Frees foo and causes the "zombie" cache to actually be destroyed.
	kmem_cache_free(foo);

Re: [PATCH] rust: slab: add basic slab module

[Vlastimil Babka]

On 9/26/25 17:55, Danilo Krummrich wrote:
> On Fri Sep 26, 2025 at 5:33 PM CEST, Danilo Krummrich wrote:
>> The only thing we need on the Rust side is that existing allocations remain
>> valid even if the cache is destroyed. Or the other way around the cache is
>> silently kept alive internally.
> 
> Or to express it in C code:
> 
> 	struct kmem_cache *cache = kmem_cache_create();
> 	struct Foo *foo = kmem_cache_alloc();
> 
> 	// After this call cache will never be accessed; leaves a zombie cache,
> 	// since foo is still alive.
> 	kmem_cache_destroy(cache);

This will cause a WARN.

> 	// This must still be valid.
> 	foo->bar = 42;

Yes this will be safe.

> 	// Frees foo and causes the "zombie" cache to actually be destroyed.
> 	kmem_cache_free(foo);

The free will be fine. But not cause the cache destruction, as that would
require checks on each free. But should be fine wrt safety if we only leak
some memory due to a wrong usage, no?

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Fri Sep 26, 2025 at 6:32 PM CEST, Vlastimil Babka wrote:
> On 9/26/25 17:55, Danilo Krummrich wrote:
>> On Fri Sep 26, 2025 at 5:33 PM CEST, Danilo Krummrich wrote:
>>> The only thing we need on the Rust side is that existing allocations remain
>>> valid even if the cache is destroyed. Or the other way around the cache is
>>> silently kept alive internally.
>> 
>> Or to express it in C code:
>> 
>> 	struct kmem_cache *cache = kmem_cache_create();
>> 	struct Foo *foo = kmem_cache_alloc();
>> 
>> 	// After this call cache will never be accessed; leaves a zombie cache,
>> 	// since foo is still alive.
>> 	kmem_cache_destroy(cache);
>
> This will cause a WARN.
>
>> 	// This must still be valid.
>> 	foo->bar = 42;
>
> Yes this will be safe.

That's great!

>> 	// Frees foo and causes the "zombie" cache to actually be destroyed.
>> 	kmem_cache_free(foo);
>
> The free will be fine. But not cause the cache destruction, as that would
> require checks on each free. But should be fine wrt safety if we only leak
> some memory due to a wrong usage, no?

Yes, technically that's safe, but we wouldn't prevent the leak, which still
is not desirable (and not our ambition for a Rust API).

From a C standpoint, both the warning and the cache leak could be solved by
making kmem_cache_destroy() fallible as you mentioned previously.

On the Rust side the cache would be represented with a struct KmemCache<T>
(where T is the type that should be allocated by the cache).

kmem_cache_destroy() would be called from KmemCache<T>::drop(), which is not
fallible. But even if it were, we can't enforce that users keep the KmemCache
instance alive as long as there are allocations.

So, either we always keep the KmemCache<T> alive for the whole module lifetime
(which depending on whether its built-in or not could be considered a memory
leak as well). Or we ensure that the last kmem_cache_free() also frees the cache
if kmem_cache_destroy() was called previously.

OOC, does the cache pointer remain valid if kmem_cache_destroy() is called while
allocations still exist? I.e. is this, except for the WARN(), valid code?

	kmem_cache_destroy(cache);
	kmem_cache_free(foo);
	kmem_cache_destroy(cache);

At a quick glance it appears to me that things would go south in
kmem_cache_release(). Anyways, I don't think it would help, if it would be the
case.

Re: [PATCH] rust: slab: add basic slab module

[Vlastimil Babka]

On 9/26/25 18:58, Danilo Krummrich wrote:
> On Fri Sep 26, 2025 at 6:32 PM CEST, Vlastimil Babka wrote:
>> On 9/26/25 17:55, Danilo Krummrich wrote:
> 
>>> 	// Frees foo and causes the "zombie" cache to actually be destroyed.
>>> 	kmem_cache_free(foo);
>>
>> The free will be fine. But not cause the cache destruction, as that would
>> require checks on each free. But should be fine wrt safety if we only leak
>> some memory due to a wrong usage, no?
> 
> Yes, technically that's safe, but we wouldn't prevent the leak, which still
> is not desirable (and not our ambition for a Rust API).
> 
> From a C standpoint, both the warning and the cache leak could be solved by
> making kmem_cache_destroy() fallible as you mentioned previously.
> 
> On the Rust side the cache would be represented with a struct KmemCache<T>
> (where T is the type that should be allocated by the cache).
> 
> kmem_cache_destroy() would be called from KmemCache<T>::drop(), which is not
> fallible. But even if it were, we can't enforce that users keep the KmemCache
> instance alive as long as there are allocations.
> 
> So, either we always keep the KmemCache<T> alive for the whole module lifetime
> (which depending on whether its built-in or not could be considered a memory
> leak as well). Or we ensure that the last kmem_cache_free() also frees the cache
> if kmem_cache_destroy() was called previously.

The rust wrapper side could do that so we don't have to add that check in
all kmem_cache_free() calls, maybe?
Also either way it could perhaps be difficult/expensive (memory barriers
etc) to properly handle a racing kmem_cache_free() and kmem_cache_destroy()
in a way that ensures the cache is always being destroyed, and not have the
kmem_cache_destroy() observe the destroy was premature, while the racing
kmem_cache_free() doesn't yet observe that destroy was attempted, and not
try to remove it.
> OOC, does the cache pointer remain valid if kmem_cache_destroy() is called while
> allocations still exist? I.e. is this, except for the WARN(), valid code?
> 
> 	kmem_cache_destroy(cache);
> 	kmem_cache_free(foo);
> 	kmem_cache_destroy(cache);
Currently not, but making it valid hopefully should not be difficult,
without affecting fast path.

> At a quick glance it appears to me that things would go south in
> kmem_cache_release(). Anyways, I don't think it would help, if it would be the
> case.

It would help if it was safe and the rust wrapper side did this retry?

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Fri Sep 26, 2025 at 7:11 PM CEST, Vlastimil Babka wrote:
> The rust wrapper side could do that so we don't have to add that check in
> all kmem_cache_free() calls, maybe?
> Also either way it could perhaps be difficult/expensive (memory barriers
> etc) to properly handle a racing kmem_cache_free() and kmem_cache_destroy()
> in a way that ensures the cache is always being destroyed, and not have the
> kmem_cache_destroy() observe the destroy was premature, while the racing
> kmem_cache_free() doesn't yet observe that destroy was attempted, and not
> try to remove it.

Yeah, it's probably not much better than just using a reference count, but more
error prone.

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Thu Sep 25, 2025 at 11:54 AM CEST, Danilo Krummrich wrote:
>   (3) Implement a macro to generate a custom KmemCache Allocator trait
>       implementation for every KmemCache instance with a static lifetime.

Thinking about it a bit more, I think we should go with option (3) for now.

With its only limitation being that it always binds the lifetime of a kmemcache
to the module lifetime, it still seems to be the best option, considering that
the alternatives require additional synchronization in hot paths, may *silently*
leak the kmemcache, cause significant code duplication or break dynamic
dispatch.

Tieing the kmemcache to the module lifetime should cover the vast majority of
use-cases; should we ever really need something else we can still revisit the
options.

Thanks,
Danilo

Re: [PATCH] rust: slab: add basic slab module

[Vlastimil Babka]

On 9/28/25 16:47, Danilo Krummrich wrote:
> On Thu Sep 25, 2025 at 11:54 AM CEST, Danilo Krummrich wrote:
>>   (3) Implement a macro to generate a custom KmemCache Allocator trait
>>       implementation for every KmemCache instance with a static lifetime.
> 
> Thinking about it a bit more, I think we should go with option (3) for now.
> 
> With its only limitation being that it always binds the lifetime of a kmemcache
> to the module lifetime, it still seems to be the best option, considering that
> the alternatives require additional synchronization in hot paths, may *silently*
> leak the kmemcache, cause significant code duplication or break dynamic
> dispatch.
> 
> Tieing the kmemcache to the module lifetime should cover the vast majority of
> use-cases; should we ever really need something else we can still revisit the
> options.

Sounds good.

> Thanks,
> Danilo

Re: [PATCH] rust: slab: add basic slab module

[Elijah]

On 9/28/2025 7:47 AM, Danilo Krummrich wrote:
> On Thu Sep 25, 2025 at 11:54 AM CEST, Danilo Krummrich wrote:
>>    (3) Implement a macro to generate a custom KmemCache Allocator trait
>>        implementation for every KmemCache instance with a static lifetime.
> 
> Thinking about it a bit more, I think we should go with option (3) for now.

do you want me to implement that now? if so I can take a look in a few hours

Re: [PATCH] rust: slab: add basic slab module

[Danilo Krummrich]

On Mon Sep 29, 2025 at 4:11 PM CEST, Elijah wrote:
> On 9/28/2025 7:47 AM, Danilo Krummrich wrote:
>> On Thu Sep 25, 2025 at 11:54 AM CEST, Danilo Krummrich wrote:
>>>    (3) Implement a macro to generate a custom KmemCache Allocator trait
>>>        implementation for every KmemCache instance with a static lifetime.
>> 
>> Thinking about it a bit more, I think we should go with option (3) for now.
>
> do you want me to implement that now? if so I can take a look in a few hours

I think it is what we settle with, so if you want to work on that, that'd be
great! :)

[PATCH v2] rust: slab: add basic slab module

[Elijah Wright]

this revision adds gen_kmem_cache_allocator, a macro that implements
Allocator::realloc for kmem_cache. the one concern that I did have was realloc()
for resizing, since that obviously isn't possible for slab

Signed-off-by: Elijah Wright <git@elijahs.space>
---
 rust/kernel/slab.rs | 52 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/rust/kernel/slab.rs b/rust/kernel/slab.rs
index 8b418f9db7cb..3f1310f309c5 100644
--- a/rust/kernel/slab.rs
+++ b/rust/kernel/slab.rs
@@ -83,3 +83,55 @@ fn drop(&mut self) {
         unsafe { bindings::kmem_cache_destroy(self.cache.as_ptr()) };
     }
 }
+
+// SAFETY: The pointer does not change after creation, so `Slab<T>` may
+// be used from multiple threads.
+unsafe impl<T> Send for Slab<T> {}
+unsafe impl<T> Sync for Slab<T> {}
+
+/// Generates a zero-sized allocator type that allocates from a given
+/// `Slab<T>`.
+#[macro_export]
+macro_rules! gen_kmem_cache_allocator {
+    (struct $name:ident for $cache:expr $(,)?) => {
+        #[derive(Clone, Copy, Default)]
+        pub struct $name;
+
+        // SAFETY: Allocation and free happen through kernel APIs which
+        // provide guarantees. The ZST carries no state, so it can be
+        // duplicated freely.
+        unsafe impl $crate::alloc::Allocator for $name {
+            #[inline]
+            unsafe fn realloc(
+                ptr: Option::<::core::ptr::NonNull<u8>>,
+                layout: ::core::alloc::Layout,
+                old_layout: ::core::alloc::Layout,
+                flags: $crate::alloc::Flags,
+            ) -> ::core::result::Result<::core::ptr::NonNull<[u8]>, $crate::alloc::AllocError> {
+                if layout.size() == 0 {
+                    if let Some(p) = ptr {
+                        // SAFETY: Caller promises `p` came from this allocator.
+                        unsafe {
+                            $crate::bindings::kmem_cache_free($cache.as_ptr(), p.as_ptr().cast());
+                        }
+                    }
+                    let dang = $crate::alloc::dangling_from_layout(layout);
+                    let slice = ::core::ptr::NonNull::slice_from_raw_parts(dang, 0);
+                    return Ok(slice);
+                }
+
+                if ptr.is_some() {
+                    return Err($crate::alloc::AllocError);
+                }
+
+                let raw_ptr = unsafe {
+                    $crate::bindings::kmem_cache_alloc($cache.as_ptr(), flags.as_raw())
+                };
+                let nn = ::core::ptr::NonNull::new(raw_ptr.cast())
+                    .ok_or($crate::alloc::AllocError)?;
+                let slice = ::core::ptr::NonNull::slice_from_raw_parts(nn.cast::<u8>(), layout.size());
+                Ok(slice)
+            }
+        }
+    };
+}
\ No newline at end of file
-- 
2.49.1

Re: [PATCH v2] rust: slab: add basic slab module

[Alice Ryhl]

On Wed, Oct 1, 2025 at 6:45 AM Elijah Wright <git@elijahs.space> wrote:
>
> this revision adds gen_kmem_cache_allocator, a macro that implements
> Allocator::realloc for kmem_cache. the one concern that I did have was realloc()
> for resizing, since that obviously isn't possible for slab
>
> Signed-off-by: Elijah Wright <git@elijahs.space>

When you send a new version, please don't reply to the previous
version. It's too easy to miss the ne version if you do.

> ---
>  rust/kernel/slab.rs | 52 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
>
> diff --git a/rust/kernel/slab.rs b/rust/kernel/slab.rs
> index 8b418f9db7cb..3f1310f309c5 100644
> --- a/rust/kernel/slab.rs
> +++ b/rust/kernel/slab.rs
> @@ -83,3 +83,55 @@ fn drop(&mut self) {
>          unsafe { bindings::kmem_cache_destroy(self.cache.as_ptr()) };
>      }
>  }
> +
> +// SAFETY: The pointer does not change after creation, so `Slab<T>` may
> +// be used from multiple threads.
> +unsafe impl<T> Send for Slab<T> {}
> +unsafe impl<T> Sync for Slab<T> {}
> +
> +/// Generates a zero-sized allocator type that allocates from a given
> +/// `Slab<T>`.
> +#[macro_export]
> +macro_rules! gen_kmem_cache_allocator {
> +    (struct $name:ident for $cache:expr $(,)?) => {
> +        #[derive(Clone, Copy, Default)]
> +        pub struct $name;
> +
> +        // SAFETY: Allocation and free happen through kernel APIs which
> +        // provide guarantees. The ZST carries no state, so it can be
> +        // duplicated freely.
> +        unsafe impl $crate::alloc::Allocator for $name {
> +            #[inline]
> +            unsafe fn realloc(
> +                ptr: Option::<::core::ptr::NonNull<u8>>,
> +                layout: ::core::alloc::Layout,
> +                old_layout: ::core::alloc::Layout,
> +                flags: $crate::alloc::Flags,
> +            ) -> ::core::result::Result<::core::ptr::NonNull<[u8]>, $crate::alloc::AllocError> {
> +                if layout.size() == 0 {
> +                    if let Some(p) = ptr {
> +                        // SAFETY: Caller promises `p` came from this allocator.
> +                        unsafe {
> +                            $crate::bindings::kmem_cache_free($cache.as_ptr(), p.as_ptr().cast());
> +                        }
> +                    }
> +                    let dang = $crate::alloc::dangling_from_layout(layout);
> +                    let slice = ::core::ptr::NonNull::slice_from_raw_parts(dang, 0);
> +                    return Ok(slice);
> +                }
> +
> +                if ptr.is_some() {
> +                    return Err($crate::alloc::AllocError);
> +                }
> +
> +                let raw_ptr = unsafe {
> +                    $crate::bindings::kmem_cache_alloc($cache.as_ptr(), flags.as_raw())
> +                };
> +                let nn = ::core::ptr::NonNull::new(raw_ptr.cast())
> +                    .ok_or($crate::alloc::AllocError)?;
> +                let slice = ::core::ptr::NonNull::slice_from_raw_parts(nn.cast::<u8>(), layout.size());
> +                Ok(slice)

Hm, this is kind of tricky. We specify a size when calling this, but
kmem caches only support one single size. I don't know what to do
about that.

Alice