Re: [LSF/MM/BPF TOPIC] SLOB+SLAB allocators removal and future SLUB improvements

[Vlastimil Babka <vbabka@suse.cz>]

On 4/5/23 21:54, Binder Makin wrote:
> I'm still running tests to explore some of these questions.
> The machines I am using are roughly as follows.
> 
> Intel dual socket 56 total cores
> 192-384GB ram
> LEVEL1_ICACHE_SIZE                 32768
> LEVEL1_DCACHE_SIZE                 32768
> LEVEL2_CACHE_SIZE                  1048576
> LEVEL3_CACHE_SIZE                  40370176
> 
> Amd dual socket 128 total cores
> 1TB ram
> LEVEL1_ICACHE_SIZE                 32768
> LEVEL1_DCACHE_SIZE                 32768
> LEVEL2_CACHE_SIZE                  524288
> LEVEL3_CACHE_SIZE                  268435456
> 
> Arm single socket 64 total cores
> 256GB rma
> LEVEL1_ICACHE_SIZE                 65536
> LEVEL1_DCACHE_SIZE                 65536
> LEVEL2_CACHE_SIZE                  1048576
> LEVEL3_CACHE_SIZE                  33554432

So with "some artifact of different cache layout" I didn't mean the
different cache sizes of the processors, but possible differences how
objects end up placed in memory by SLAB vs SLUB causing them to collide in
the cache of cause false sharing less or more. This kind of interference can
make interpreting (micro)benchmark results hard.

Anyway, how I'd hope to approach this topic would be that SLAB removal is
proposed, and anyone who opposes that because they can't switch from SLAB to
SLUB would describe why they can't. I'd hope the "why" to be based on
testing with actual workloads, not just benchmarks. Benchmarks are then of
course useful if they can indeed distill the reason why the actual workload
regresses, as then anyone can reproduce that locally and develop/test fixes
etc. My hope is that if some kind of regression is found (e.g. due to lack
of percpu array in SLUB), it can be dealt with by improving SLUB.

Historically I recall that we (SUSE) objected somwhat to SLAB removal as our
distro kernels were using it, but we have switched since. Then networking
had concerns (possibly related to the lack percpu array) but seems bulk
allocations helped and they use SLUB these days [1]. And IIRC Google was
also sticking to SLAB, which led to some attempts to augment SLUB for those
workloads years ago, but those were never finished. So I'd be curious if we
should restart those effors or can just remove SLAB now.

[1] https://lore.kernel.org/all/93665604-5420-be5d-2104-17850288b955@redhat.com/