Re: [LSF/MM/BPF TOPIC] HGM for hugetlbfs

[Matthew Wilcox <willy@infradead.org>]

On Wed, Jun 07, 2023 at 09:38:35AM +0200, David Hildenbrand wrote:
> I don't want to get too involved in this discussion (busy), but I absolutely
> agree on the points that were raised at LSF/MM that
> 
> (A) hugetlb is complicated and very special (many things not integrated with
> core-mm, so we need special-casing all over the place). [example: what is a
> pte?]

This is somethign that absolutely does need to get fixed.  It's one of
the big sources of complexity and confusion around code that is supposed
to work with both hugetlb & THP.  I understand why hugetlb originally
said "everything is a pte", but THP went a different route, and I think
hugetlb now needs to follow.

Fixing pagewalk.h to not be complete garbage would be a good start.
I can elaborate more along these lines if someone's actually going to
put in the work to do it.

> (B) We added a bunch of complexity in the past that some people considered
> very important (and it was not feature frozen, right? ;) ). Looking back, we
> might just not have done some of that, or done it differently/cleaner --
> better integrated in the core. (PMD sharing, MAP_PRIVATE, a reservation
> mechanism that still requires preallocation because it fails with NUMA/fork,
> ...)

It'd be nice if people engaged seriously with the efforts to move that
functionality into the core.  eg mshare.  Saying "Oh just share the
hugetlb implementation" is not serious engagement, it's an indication
you haven't been paying attention to what the needs are.

I haven't looked at the hugetlb reservation mechanism in enough detail to
be able to understand why people use it, what they actually want, and how
it could be done better in the core.  Maybe somebody else could do that.