Re: [RFC PATCH] slub: spill refill leftover objects into percpu sheaves

[Hao Li <hao.li@linux.dev>]

On Wed, Apr 15, 2026 at 01:55:54PM -0700, Vinicius Costa Gomes wrote:
> Hao Li <hao.li@linux.dev> writes:
> 
> > When performing objects refill, we tend to optimistically assume that
> > there will be more allocation requests coming next; this is the
> > fundamental assumption behind this optimization.
> >
> > When __refill_objects_node() isolates a partial slab and satisfies a
> > bulk allocation from its freelist, the slab can still have a small tail
> > of free objects left over. Today those objects are freed back to the
> > slab immediately.
> >
> > If the leftover tail is local and small enough to fit, keep it in the
> > current CPU's sheaves instead. This avoids pushing those objects back
> > through the __slab_free slowpath.
> >
> > Add a helper to obtain both the freelist and its free-object count, and
> > then spill the remaining objects into a percpu sheaf when:
> > - the tail fits in a sheaf
> > - the slab is local to the current CPU
> > - the slab is not pfmemalloc
> > - the target sheaf has enough free space
> >
> > Otherwise keep the existing fallback and free the tail back to the slab.
> >
> > Also add a SHEAF_SPILL stat so the new path can be observed in SLUB
> > stats.
> >
> > On the mmap2 case in the will-it-scale benchmark suite, this patch can
> > improve performance by about 2~5%.
> >
> > Signed-off-by: Hao Li <hao.li@linux.dev>
> > ---
> >
> > This patch is an exploratory attempt to address the leftover objects and
> > partial slab issues in the refill path, and it is marked as RFC to warmly
> > welcome any feedback, suggestions, and discussion!
> >
> 
> I was also looking at these regressions, but I went from a different
> direction, and ended up with 3 patches:
> 
> 1. the regressions showed a lot of increase in the cache misses,
>    which gave me the idea that a cache would help (and it seemed to help)
> 
> 2. Allowing smaller refills (but potentially more frequent);
> 
> 3. A cute (but with small impact) use of prefetch();
> 

Great!
Thanks for sharing those infos!

> The numbers are here (the commentary from the bot are very hit or miss,
> so don't pay too much attention to them):
> 
> https://github.com/vcgomes/linux/commit/c898c39ee8def5252942281353eda6acdd83d4ea
> 
> I am re-running the tests against a more recent tree, but if you
> want to take a look:
> 
> https://github.com/vcgomes/linux/tree/mm-sheaves-regression-timerfd
> 
> Also, if you feel it's useful, I can send a RFC.


I also tried stashing leftover objects into the PCS before, but at the time I
observed that this could quickly drain the node partial list, which then led to
slab alloc/free churn, and the end result was a performance regression. So I
gave up this direction :/

I took a quick look at the code and performance report in your GitHub repo, and
the performance gains you showed there are really interesting to me!
I'm going to try testing it on my own machine as well.

Also, the idea of a warm_slab looks very interesting. I had previously thought
about something along similar lines, except that I was considering stashing
leftover untouched slabs at the per-CPU level. But that would effectively bring
back the old stale CPU partial list in another form, so I ended up dropping
that direction. But you direction is really interesting!

I'd be very happy to see a new approach come out of this. If it's convenient
for you, I'd love to see an RFC. Thanks!

-- 
Thanks,
Hao