Re: [PATCH RFC v2 00/10] SLUB percpu sheaves

[Suren Baghdasaryan <surenb@google.com>]

On Mon, Mar 17, 2025 at 4:08 AM Vlastimil Babka <vbabka@suse.cz> wrote:
>
> On 3/14/25 18:10, Suren Baghdasaryan wrote:
> > On Tue, Mar 4, 2025 at 11:08 AM Liam R. Howlett <Liam.Howlett@oracle.com> wrote:
> >>
> >> * Vlastimil Babka <vbabka@suse.cz> [250304 05:55]:
> >> > On 2/25/25 21:26, Suren Baghdasaryan wrote:
> >> > > On Mon, Feb 24, 2025 at 1:12 PM Suren Baghdasaryan <surenb@google.com> wrote:
> >> > >>
> >> > >> >
> >> > >> > > The values represent the total time it took to perform mmap syscalls, less is
> >> > >> > > better.
> >> > >> > >
> >> > >> > > (1)                  baseline       control
> >> > >> > > Little core       7.58327       6.614939 (-12.77%)
> >> > >> > > Medium core  2.125315     1.428702 (-32.78%)
> >> > >> > > Big core          0.514673     0.422948 (-17.82%)
> >> > >> > >
> >> > >> > > (2)                  baseline      control
> >> > >> > > Little core       7.58327       5.141478 (-32.20%)
> >> > >> > > Medium core  2.125315     0.427692 (-79.88%)
> >> > >> > > Big core          0.514673    0.046642 (-90.94%)
> >> > >> > >
> >> > >> > > (3)                   baseline      control
> >> > >> > > Little core        7.58327      4.779624 (-36.97%)
> >> > >> > > Medium core   2.125315    0.450368 (-78.81%)
> >> > >> > > Big core           0.514673    0.037776 (-92.66%)
> >> > >
> >> > > (4)                   baseline      control
> >> > > Little core        7.58327      4.642977 (-38.77%)
> >> > > Medium core   2.125315    0.373692 (-82.42%)
> >> > > Big core           0.514673    0.043613 (-91.53%)
> >> > >
> >> > > I think the difference between (3) and (4) is noise.
> >> > > Thanks,
> >> > > Suren.
> >> >
> >> > Hi, as we discussed yesterday, it would be useful to set the baseline to
> >> > include everything before sheaves as that's already on the way to 6.15, so
> >> > we can see more clearly what sheaves do relative to that. So at this point
> >> > it's the vma lock conversion including TYPESAFE_BY_RCU (that's not undone,
> >> > thus like in scenario (4)), and benchmark the following:
> >> >
> >> > - baseline - vma locking conversion with TYPESAFE_BY_RCU
> >> > - baseline+maple tree node reduction from mm-unstable (Liam might point out
> >> > which patches?)
> >>
> >> Sid's patches [1] are already in mm-unstable.
> >>
> >>
> >> > - the above + this series + sheaves enabled for vm_area_struct cache
> >> > - the above + full maple node sheaves conversion [1]
> >> > - the above + the top-most patches from [1] that are optimizations with a
> >> > tradeoff (not clear win-win) so it would be good to know if they are useful
> >> >
> >> > [1] currently the 4 commits here:
> >> > https://web.git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slub-percpu-sheaves-v2-maple
> >> > from "maple_tree: Sheaf conversion" to "maple_tree: Clean up sheaf"
> >> > but as Liam noted, they won't cherry pick without conflict once maple tree
> >> > node reduction is backported, but he's working on a rebase
> >>
> >> Rebased maple tree sheaves, patches are here [2].
> >
> > Hi Folks,
> > Sorry for the delay. I got the numbers last week but they looked a bit
> > weird, so I reran the test increasing the number of iterations to make
> > sure noise is not a factor. That took most of this week. Below are the
> > results. Please note that I had to backport the patchsets to 6.12
> > because that's the closest stable Android kernel I can use. I measure
> > cumulative time to execute mmap syscalls, so the smaller the number
> > the better mmap performance is:
>
> Is that a particular benchmark doing those syscalls, or you time them within
> actual workloads?

I time them inside my workload.

>
> > baseline: 6.12 + vm_lock conversion and TYPESAFE_BY_RCU
> > config1: baseline + Sid's patches [1]
> > config2: sheaves RFC
> > config3: config1 + vm_area_struct with sheaves
> > config4: config2 + maple_tree Sheaf conversion [2]
> > config5: config3 + 2 last optimization patches from [3]
> >
> >                config1     config2     config3     config4     config5
> > Little core    -0.10%      -10.10%     -12.89%     -10.02%     -13.64%
> > Mid core       -21.05%     -37.31%     -44.97%     -15.81%     -22.15%
> > Big core       -17.17%     -34.41%     -45.68%     -11.39%     -15.29%
>
> Thanks a lot, Suren.
>
> > [1] https://lore.kernel.org/linux-mm/20250227204823.758784-1-sidhartha.kumar@oracle.com/
> > [2] https://www.infradead.org/git/?p=users/jedix/linux-maple.git;a=shortlog;h=refs/heads/sheaves_rebase_20250304
> > [3] https://web.git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slub-percpu-sheaves-v2-maple
> >
> > From the numbers, it looks like config4 regresses the performance and
> > that's what looked weird to me last week and I wanted to confirm this.
> > But from sheaves POV, it looks like they provide the benefits I saw
> > before. Sid's patches which I did not test separately before also look
> > beneficial.
>
> Indeed, good job, Sid. It's weird that config4 isn't doing well. The problem
> can be either in sheaves side (the sheaves preallocation isn't effective) or
> maple tree side doing some excessive work. It could be caused by the wrong
> condition in kmem_cache_return_sheaf() that Harry pointed out, so v3 might
> improve if that was it. Otherwise we'll probably need to fill the gaps in
> sheaf-related stats and see what are the differences between config3 and
> config4.
>
> > Thanks,
> > Suren.
> >
> >>
> >>
> >> >
> >> >
> >> ...
> >>
> >> Thanks,
> >> Liam
> >>
> >> [1]. https://lore.kernel.org/linux-mm/20250227204823.758784-1-sidhartha.kumar@oracle.com/
> >> [2]. https://www.infradead.org/git/?p=users/jedix/linux-maple.git;a=shortlog;h=refs/heads/sheaves_rebase_20250304
>