Re: [LSF/MM/BPF TOPIC] SLUB allocator, mainly the sheaves caching layer

[Vlastimil Babka <vbabka@suse.cz>]

On 2/26/25 01:17, Christoph Lameter (Ampere) wrote:
> 
> Let me just express my general concern. SLUB was written because SLAB
> became a Byzantine mess with layer upon layer of debugging and queues

I don't recall it having much debugging. IIRC it was behind some config that
nobody enabled. SLUB's debugging that can be dynamically enabled on boot is
so much better.

> here and there and with "maintenance" for these queues going on every 2
> seconds staggered on all processors. This caused a degree of OS noise that
> caused HPC jobs (and today we see similar issues with AI jobs) to not be
> able to accomplish a deterministic rendezvous. On some large machines

Yeah, I don't want to reintroduce this, hence sheaves intentionally don't
support NUMA restricted allocations so none of the flushed alien arrays are
necessary.

> we had ~10% of the whole memory vanish into one of the other queue on boot
> up with  the customers being a bit upset were all the expensive memory
> went.
> 
> It seems that were have nearly recreated the old nightmare again.

I don't see it that bleak.

> I would suggest rewriting the whole allocator once again trying to
> simplify things as much as possible and isolating specialized allocator
> functionality needed for some subsystems into different APIs.

Any specific suggestions? Some things are hard to isolate i.e. make them
work on top of the core allocator because not interacting with the internals
would not allow some useful functionality, or efficiency.

> The main allocation / free path needs to be as simple and as efficient as
> possible. It may not be possible to accomplish something like that given
> all the special casing that we have been pushing into it. Also consider the

I see some possibilities for simplification in not trying to support KASAN
together with slab_debug anymore. KASAN should be superior for that purpose
(of course you pay the extra cost) and it's tricky to not have it step on
each other's toes with slab_debug.

> runtime security measures and verification stuff that is on by default at
> runtime as well.

Yeah more and more hardening seems to be the current trend. But also not
realistically possible to isolate away from the core. I at least tried to
always compile all of it away completely when the respective CONFIG is not
enabled. OTOH I'd like to see some of that to support boot parameters (via
static keys etc) so it can be compiled in but not enabled. That would not
completely eliminate the overhead of passing e.g. the bucket parameter or
performing kmalloc random index evaluation, but would not allocate the
separate caches if not enabled, so the memory overhead of that would not be
imposed.