Re: [PATCH v2] slub: keep empty main sheaf as spare in __pcs_replace_empty_main()

[Hao Li <hao.li@linux.dev>]

On Tue, Jan 20, 2026 at 04:21:16PM +0800, Zhao Liu wrote:

Hi, Zhao,

Thanks again for your thorough testing and detailed feedback - I really
appreciate your help.

> > 1. Machine Configuration
> > 
> > The topology of my machine is as follows:
> > 
> > CPU(s):              384
> > On-line CPU(s) list: 0-383
> > Thread(s) per core:  2
> > Core(s) per socket:  96
> > Socket(s):           2
> > NUMA node(s):        2
> 
> It seems like this is a GNR machine - maybe SNC could be enabled.

Actually, my cpu is AMD EPYC 96-Core Processor. SNC is disabled, and
there's only one NUMA node per socket.

> 
> > Since my machine only has 192 cores when counting physical cores, I had to
> > enable SMT to support the higher number of tasks in the LKP test cases. My
> > configuration was as follows:
> > 
> > will-it-scale:
> >   mode: process
> >   test: mmap2
> >   no_affinity: 0
> >   smt: 1
> 
> For lkp, smt parameter is disabled. I tried with smt=1 locally, the
> difference between "with fix" & "w/o fix" is not significate. Maybe smt
> parameter could be set as 0.

Just to confirm: do you mean that on your machine, when smt=1, the performance
difference between "with fix" and "without fix" is not significant - regardless
of whether it's a gain or regression? Thanks.

> 
> On another machine (2 sockets with SNC3 enabled - 6 NUMA nodes), there's
> the similar regression happening when tasks fill up a socket and then
> there're more get_partial_node().

From a theoretical standpoint, it seems like having more nodes should reduce
lock contention, not increase it...

By the way, I wanted to confirm one thing: in your earlier perf data, I noticed
that the sampling ratio of native_queued_spin_lock_slowpath and get_partial_node
slightly increased with the patch. Does this suggest that the lock contention
you're observing mainly comes from kmem_cache_node->list_lock rather than
node_barn->lock?

If possible, could you help confirm this using "perf report -g" to see where the
contention is coming from?

> 
> > Here's the "perf report --no-children -g" output with the patch:
> > 
> > ```
> > +   30.36%  mmap2_processes  [kernel.kallsyms]     [k] perf_iterate_ctx
> > -   28.80%  mmap2_processes  [kernel.kallsyms]     [k] native_queued_spin_lock_slowpath
> >    - 24.72% testcase
> >       - 24.71% __mmap
> >          - 24.68% entry_SYSCALL_64_after_hwframe
> >             - do_syscall_64
> >                - 24.61% ksys_mmap_pgoff
> >                   - 24.57% vm_mmap_pgoff
> >                      - 24.51% do_mmap
> >                         - 24.30% __mmap_region
> >                            - 18.33% mas_preallocate
> >                               - 18.30% mas_alloc_nodes
> >                                  - 18.30% kmem_cache_alloc_noprof
> >                                     - 18.28% __pcs_replace_empty_main
> >                                        + 9.06% barn_replace_empty_sheaf
> >                                        + 6.12% barn_get_empty_sheaf
> >                                        + 3.09% refill_sheaf
> 
> this is the difference with my previous perf report: here the proportion
> of refill_sheaf is low - it indicates the shaeves are enough in the most
> time.
> 
> Back to my previous test, I'm guessing that with this fix, under extreme
> conditions of massive mmap usage, each CPU now stores an empty spare sheaf
> locally. Previously, each CPU's spare sheaf was NULL. So memory pressure
> increases with more spare sheaves locally.

I'm not quite sure about this point - my intuition is that this shouldn't
consume a significant amount of memory.

> And in that extreme scenario,
> cross-socket remote NUMA access incurs significant overhead — which is why
> regression occurs here.

This part I haven't fully figured out yet - still looking into it.

> 
> However, testing from 1 task to max tasks (nr_tasks = nr_logical_cpus)
> shows overall significant improvements in most scenarios. Regressions
> only occur at the specific topology boundaries described above.

It does look like there's some underlying factor at play, triggering a
performance tipping point. Though I haven't yet figured out the exact pattern.

> 
> I believe the cases with performance gains are more common. So I think
> the regression is a corner case. If it does indeed impact certain
> workloads in the future, we may need to reconsider optimization at that
> time. It can now be used as a reference.

Agreed — this seems to be a corner case, and your test results have been really
helpful as a reference. Thanks again for the great support and insightful
discussion.

-- 
Thanks,
Hao