Re: [RFC PATCH v2 0/5] mm, bpf: BPF based THP adjustment

[Yafang Shao <laoar.shao@gmail.com>]

On Tue, May 20, 2025 at 11:54 PM David Hildenbrand <david@redhat.com> wrote:
>
> >> I totally agree with you that the key point here is how to define the
> >> API. As I replied to David, I believe we have two fundamental
> >> principles to adjust the THP policies:
> >> 1. Selective Benefit: Some tasks benefit from THP, while others do not.
> >> 2. Conditional Safety: THP allocation is safe under certain conditions
> >> but not others.
> >>
> >> Therefore, I believe we can define these APIs based on the established
> >> principles - everything else constitutes implementation details, even
> >> if core MM internals need to change.
> >
> > But if we're looking to make the concept of THP go away, we really need to
> > go further than this.
>
> Yeah. I might be wrong, but I also don't think doing control on a
> per-process level etc would be the right solution long-term.

The reality is that achieving truly 'automatic' THP behavior requires
process-level control. Given that THP provides no benefit for certain
workloads, there's no justification for incurring the overhead of
allocating higher-order pages in those cases.

>
> In a world where we do stuff automatically ("auto" mode), we would be
> much smarter about where to place a (m)THP, and which size we would use.

We still have considerable ground to cover before reaching this goal.

>
> One might use bpf to control the allocation policy. But I don't think
> this would be per-process or even per-VMA etc. Sure, we might give
> hints, but placement decisions should happen on another level (e.g.,
> during page faults, during khugepaged etc).

Nico has proposed introducing a new 'defer' mode to address this.
However, I argue that we could achieve the same functionality through
BPF instead of adding a dedicated policy mode. [0]

[0]. https://lore.kernel.org/linux-mm/CALOAHbAa7DY6+hO4RJtjg-MS+cnUmsiPXX8KS1MKSfgy6HLYAQ@mail.gmail.com/

>
> >
> > The second we have 'bpf program that figures out whether THP should be
> > used' we are permanently tied to the idea of THP on/off being a thing.
> >
> > I mean any future stuff that makes THP more automagic will probably involve
> > having new modes for the legacy THP
> > /sys/kernel/mm/transparent_hugepage/enabled and
> > /sys/kernel/mm/transparent_hugepage/hugepages-xxkB/enabled
>
> Yeah, the plan is to have "auto" in
> /sys/kernel/mm/transparent_hugepage/enabled and just have all other
> sizes "inherit" that option. And have a Kconfig that just enables that
> as default. Once we're there, just phase out the interface long-term.
>
> That's the plan. Now we "only" have to figure out how to make the
> placement actually better ;)
>
> >
> > But if people are super reliant on this stuff it's potentially really
> > limiting.
> >
> > I think you said in another post here that you were toying with the notion
> > of exposing somehow the madvise() interface and having that be the 'stable
> > API' of sorts?
> >
> > That definitely sounds more sensible than something that very explicitly
> > interacts with THP.
> >
> > Of course we have Usama's series and my proposed series for extending
> > process_madvise() along those lines also.
>
> Yes.
>


-- 
Regards
Yafang