Re: [LSF/MM/BPF TOPIC] Ways to mitigate limitations of percpu memory allocator

[Gabriel Krisman Bertazi <krisman@suse.de>]

Pedro Falcato <pfalcato@suse.de> writes:

> On Thu, Mar 05, 2026 at 12:48:21PM +0100, Jan Kara wrote:
>
>> On Thu 05-03-26 11:33:21, Pedro Falcato wrote:
>> > On Fri, Feb 27, 2026 at 03:41:50PM +0900, Harry Yoo wrote:
>> > > Hi folks, I'd like to discuss ways to mitigate limitations of
>> > > percpu memory allocator. 
>> > > 
>> > > While the percpu memory allocator has served its role well,
>> > > it has a few problems: 1) its global lock contention, and
>> > > 2) lack of features to avoid high initialization cost of percpu memory.
>> > > 
>> > > Global lock contention
>> > > =======================
>> > > 
>> > > Percpu allocator has a global lock when allocating or freeing memory.
>> > > Of course, caching percpu memory is not always worth it, because
>> > > it would meaningfully increase memory usage.
>> > > 
>> > > However, some users (e.g., fork+exec, tc filter) suffer from
>> > > the lock contention when many CPUs allocate / free percpu memory
>> > > concurrently.
>> > > 
>> > > That said, we need a way to cache percpu memory per cpu, in a selective
>> > > way. As an opt-in approach, Mateusz Guzik proposed [1] keeping percpu
>> > > memory in slab objects and letting slab cache them per cpu,
>> > > with slab ctor+dtor pair: allocate percpu memory and
>> > > associate it with slab object in constructor, and free it when
>> > > deallocating slabs (with resurrecting slab destructor feature).
>> > > 
>> > > This only works when percpu memory is associated with slab objects.
>> > > I would like to hear if anybody thinks it's still worth redesigning
>> > > percpu memory allocator for better scalability.
>> > 
>> > I think this (make alloc_percpu actually scale) is the obvious suggestion.
>> > Everything else is just papering over the cracks.
>> 
>> I disagree. There are two separate (although related) issues that need
>> solving. One issue is certainly scalability of the percpu allocator.
>> Another issue (which is also visible in singlethreaded workloads) is that
>> a percpu counter creation has a rather large cost even if the allocator is
>> totally uncontended - this is because of the initialization (and final
>> summarization) cost. And this is very visible e.g. in the fork() intensive
>> loads such as shell scripts where we currently allocate several percpu
>> arrays for each fork() and significant part of the fork() cost is currently
>> the initialization of percpu arrays on larger machines. Reducing this
>> overhead is a separate goal.
>
> I agree that it's a separate issue. But it's as much of an issue for
> single-threaded processes as much as multi-threaded. Say you have a 64 core
> CPU. Why should you pay for 64 separate cores when you only spawned 2 threads?
> (and, yes, this is a not-so-rare situation, like lld which spawns up to 16
> threads (https://reviews.llvm.org/D147493), even if you have hundreds
> of CPUs)

True.  Still, being an up-front initialization cost, it is the most
relevant the shortest the task lives.  I'd imagine that even for
something as lld doing 16 clone syscalls, the overhead of a single
percpu counter initialization is a very small blip in the profile, not
worth special-casing for.  The single-threaded case is the obvious
optimizable-case in this sense.

> So perhaps the best way to go about this problem would be to go back to
> per-task RSS accounting. This one had problems with many-task RSS accuracy,
> but the current one has problems for many-cpu RSS accuracy.

The current pcpu one has a much smaller accuracy error than per-task,
which justified its inclusion in the first place, no?  IIRC, there was a
real use case where the worse accuracy mattered for process selection
during OOM.

> A single-threaded
> optimization could patch over the problem for the vast majority of programs,
> but exceptions exist.

>
> Or another possible idea: lazily initialize these cpu counters somehow,
> on task switch.

>
> I'm afraid that while the solution presented by Mathieu fixes a problem with
> the current scheme (insane inaccuracy on large-cpu-count), it might also add
> to the percpu allocation + init problem (this might not be true, I have not
> paid too much attention).

-- 
Gabriel Krisman Bertazi