Re: [LSF/MM/BPF TOPIC] Ways to mitigate limitations of percpu memory allocator

[Pedro Falcato <pfalcato@suse.de>]

On Fri, Feb 27, 2026 at 03:41:50PM +0900, Harry Yoo wrote:
> Hi folks, I'd like to discuss ways to mitigate limitations of
> percpu memory allocator. 
> 
> While the percpu memory allocator has served its role well,
> it has a few problems: 1) its global lock contention, and
> 2) lack of features to avoid high initialization cost of percpu memory.
> 
> Global lock contention
> =======================
> 
> Percpu allocator has a global lock when allocating or freeing memory.
> Of course, caching percpu memory is not always worth it, because
> it would meaningfully increase memory usage.
> 
> However, some users (e.g., fork+exec, tc filter) suffer from
> the lock contention when many CPUs allocate / free percpu memory
> concurrently.
> 
> That said, we need a way to cache percpu memory per cpu, in a selective
> way. As an opt-in approach, Mateusz Guzik proposed [1] keeping percpu
> memory in slab objects and letting slab cache them per cpu,
> with slab ctor+dtor pair: allocate percpu memory and
> associate it with slab object in constructor, and free it when
> deallocating slabs (with resurrecting slab destructor feature).
> 
> This only works when percpu memory is associated with slab objects.
> I would like to hear if anybody thinks it's still worth redesigning
> percpu memory allocator for better scalability.

I think this (make alloc_percpu actually scale) is the obvious suggestion.
Everything else is just papering over the cracks.

> Slab constructor + destructor Pair
> ----------------------------------
> 
> Percpu allocator doesn't distinguish types of objects
> unlike slab and it doesn't support constructors that could avoid
> re-initializing them on every allocation.
> One solution to this is using slab ctor+dtor pair; as long as a certain
> state is preserved on free (e.g. sum of percpu counter is zero),
> initialization needs to be done only once on construction.

As I said way back when, making an object permanently accessible 
a-la TYPESAFE_BY_RCU) is screwey and messes with the object lifetime
too much. Not to mention the locking problems that we discussed back-and-forth.

-- 
Pedro