Re: [RFC PATCH v2 0/5] mm, bpf: BPF based THP adjustment

["Liam R. Howlett" <Liam.Howlett@oracle.com>]

* Liam R. Howlett <Liam.Howlett@oracle.com> [250526 10:54]:
> * David Hildenbrand <david@redhat.com> [250526 06:49]:
> > On 26.05.25 11:37, Yafang Shao wrote:
> > > On Mon, May 26, 2025 at 4:14 PM David Hildenbrand <david@redhat.com> wrote:
> > > > 
> > > > > Hi all,
> > > > > 
> > > > > Let’s summarize the current state of the discussion and identify how
> > > > > to move forward.
> > > > > 
> > > > > - Global-Only Control is Not Viable
> > > > > We all seem to agree that a global-only control for THP is unwise. In
> > > > > practice, some workloads benefit from THP while others do not, so a
> > > > > one-size-fits-all approach doesn’t work.
> > > > > 
> > > > > - Should We Use "Always" or "Madvise"?
> > > > > I suspect no one would choose 'always' in its current state. ;)
> > > > 
> > > > IIRC, RHEL9 has the default set to "always" for a long time.
> > > 
> > > good to know.
> > > 
> > > > 
> > > > I guess it really depends on how different the workloads are that you
> > > > are running on the same machine.
> > > 
> > > Correct. If we want to enable THP for specific workloads without
> > > modifying the kernel, we must isolate them on dedicated servers.
> > > However, this approach wastes resources and is not an acceptable
> > > solution.
> > > 
> > > > 
> > > >   > Both Lorenzo and David propose relying on the madvise mode. However,>
> > > > since madvise is an unprivileged userspace mechanism, any user can
> > > > > freely adjust their THP policy. This makes fine-grained control
> > > > > impossible without breaking userspace compatibility—an undesirable
> > > > > tradeoff.
> > > > 
> > > > If required, we could look into a "sealing" mechanism, that would
> > > > essentially lock modification attempts performed by the process (i.e.,
> > > > MADV_HUGEPAGE).
> > > 
> > > If we don’t introduce a new THP mode and instead rely solely on
> > > madvise, the "sealing" mechanism could either violate the intended
> > > semantics of madvise(), or simply break madvise() entirely, right?
> > 
> > We would have to be a bit careful, yes.
> > 
> > Errors from MADV_HUGEPAGE/MADV_NOHUGEPAGE are often ignored, because these
> > options also fail with -EINVAL on kernels without THP support.
> > 
> > Ignoring MADV_NOHUGEPAGE can be problematic with userfaultfd.
> > 
> > What you likely really want to do is seal when you configured
> > MADV_NOHUGEPAGE to be the default, and fail MADV_HUGEPAGE later.

I am also not entirely sure how sealing a non-existing vma would work.
We'd have to seal the default flags, but sealing is one way and this
surely shouldn't be one way?

> 
> I think this works.  Take the example from a previous thread where
> containers are differentiated by allowing or not allowing THP.  If you
> set a container MADV_HOHUGEPAGE (or whatever flag we used for the same
> meaning), then if a library uses that call and it fails do we want to
> report it as a failure?  I would reason that the library shouldn't hard
> fail if its unable to use THP, so it's okay to return the failure.
> 
> Alternatively, if it is a hard requirement, then that container
> shouldn't be allowed to continue in such a state and should verify the
> return.  (If this is even a possibility?)
> 
> > 
> > > > 
> > > > The could be added on top of the current proposals that are flying
> > > > around, and could be done e.g., per-process.
> > > 
> > > How about introducing a dedicated "process" mode? This would allow
> > > each process to use different THP modes—some in "always," others in
> > > "madvise," and the rest in "never." Future THP modes could also be
> > > added to this framework.
> > 
> > We have to be really careful about not creating even more mess with more
> > modes.
> 
> Yes, and clarity would depend on the mode name, imo.  Never meaning
> never, for example.
> 
> So we'd need an answer to David's question below before agreeing on
> "process". If it survives across fork and exec calls, is it really a
> "process" setting?
> 
> I believe you are seeing it as "setting default" really doesn't mean
> setting a default if you cannot overwrite it, and if you can overwrite
> the "default" then it's not going to work for all use cases.
> 
> > 
> > How would that design look like in detail (how would we set it per process
> > etc?)?
> > 
> > -- 
> > Cheers,
> > 
> > David / dhildenb
> >