Re: [LSF/MM/BPF TOPIC] Ways to mitigate limitations of percpu memory allocator

[Pedro Falcato <pfalcato@suse.de>]

On Thu, Mar 05, 2026 at 12:48:21PM +0100, Jan Kara wrote:
> On Thu 05-03-26 11:33:21, Pedro Falcato wrote:
> > On Fri, Feb 27, 2026 at 03:41:50PM +0900, Harry Yoo wrote:
> > > Hi folks, I'd like to discuss ways to mitigate limitations of
> > > percpu memory allocator. 
> > > 
> > > While the percpu memory allocator has served its role well,
> > > it has a few problems: 1) its global lock contention, and
> > > 2) lack of features to avoid high initialization cost of percpu memory.
> > > 
> > > Global lock contention
> > > =======================
> > > 
> > > Percpu allocator has a global lock when allocating or freeing memory.
> > > Of course, caching percpu memory is not always worth it, because
> > > it would meaningfully increase memory usage.
> > > 
> > > However, some users (e.g., fork+exec, tc filter) suffer from
> > > the lock contention when many CPUs allocate / free percpu memory
> > > concurrently.
> > > 
> > > That said, we need a way to cache percpu memory per cpu, in a selective
> > > way. As an opt-in approach, Mateusz Guzik proposed [1] keeping percpu
> > > memory in slab objects and letting slab cache them per cpu,
> > > with slab ctor+dtor pair: allocate percpu memory and
> > > associate it with slab object in constructor, and free it when
> > > deallocating slabs (with resurrecting slab destructor feature).
> > > 
> > > This only works when percpu memory is associated with slab objects.
> > > I would like to hear if anybody thinks it's still worth redesigning
> > > percpu memory allocator for better scalability.
> > 
> > I think this (make alloc_percpu actually scale) is the obvious suggestion.
> > Everything else is just papering over the cracks.
> 
> I disagree. There are two separate (although related) issues that need
> solving. One issue is certainly scalability of the percpu allocator.
> Another issue (which is also visible in singlethreaded workloads) is that
> a percpu counter creation has a rather large cost even if the allocator is
> totally uncontended - this is because of the initialization (and final
> summarization) cost. And this is very visible e.g. in the fork() intensive
> loads such as shell scripts where we currently allocate several percpu
> arrays for each fork() and significant part of the fork() cost is currently
> the initialization of percpu arrays on larger machines. Reducing this
> overhead is a separate goal.

I agree that it's a separate issue. But it's as much of an issue for
single-threaded processes as much as multi-threaded. Say you have a 64 core
CPU. Why should you pay for 64 separate cores when you only spawned 2 threads?
(and, yes, this is a not-so-rare situation, like lld which spawns up to 16
threads (https://reviews.llvm.org/D147493), even if you have hundreds of CPUs)

So perhaps the best way to go about this problem would be to go back to
per-task RSS accounting. This one had problems with many-task RSS accuracy,
but the current one has problems for many-cpu RSS accuracy. A single-threaded
optimization could patch over the problem for the vast majority of programs,
but exceptions exist.

Or another possible idea: lazily initialize these cpu counters somehow,
on task switch.

I'm afraid that while the solution presented by Mathieu fixes a problem with
the current scheme (insane inaccuracy on large-cpu-count), it might also add
to the percpu allocation + init problem (this might not be true, I have not
paid too much attention).

-- 
Pedro