Re: [LSF/MM/BPF TOPIC] Beyond 2MB: Why Terabyte-Scale Machines Need 1GB Transparent Huge Pages

["David Hildenbrand (Arm)" <david@kernel.org>]

On 2/24/26 21:35, Matthew Wilcox wrote:
> On Tue, Feb 24, 2026 at 02:08:26PM -0500, Zi Yan wrote:
>> On 24 Feb 2026, at 14:03, Johannes Weiner wrote:
>>>
>>> I know this isn't your intention, but one interesting aspect of
>>> supporting PUD mapped folios natively is that it could open the door
>>> to simplifying hugetlb as well.
>>>
>>> We currently have all kinds of huge_vma checks scattered over the page
>>> table code, and entirely parallel paths for unmapping etc. With native
>>> PUD mappings, this could allow pushing the special casing out of the
>>> virtual memory layer and into where we deal with the page objects.
>>>
>>> You might be able to take it as far as the only thing left of hugetlb
>>> is the reservation pool. Such that a naive application does mmap() as
>>> per usual, and it comes down to a separate allocation policy how the
>>> backing pages are served (buddy, CMA, boot-time reservations, ...)
>>>
>>> Approaching it this way could help separate out the discussion on code
>>> impact and tech debt of PUD mappings, from the allocation technique
>>> question, which in itself is a fairly large topic.
>>
>> I agree with this 100%. Adding 1GB folio support first, we then can think
>> about what other THP features, e.g., split, migration, PMD/PTE mapping, are
>> really needed and add them one by one. It is also going to be a good way
>> of retiring hugetlb special code.
> 
> But this hasn't happened yet for PMD-sized hugetlb, and there's no need
> to wait for PUD-sized THP to start this process.  I don't think that
> introducing PUD-sized THP will actually motivate anyone to do this work.
> 
> I think we have four main things that hugetlb still offers:
> 
>  - Reserved pool (mentioned above) which we don't yet have a THP
>    replacement for
>  - shared page tables.  mshare() is the replacement here, and that
>    project is moving along nicely.
>  - Being able to allocate gigantic folios.  This is also progressing.
>  - Guaranteeing that you don't get a fallback; you either get memory in
>    the size you asked for, or you fail.
> 
> Every time this comes up, I offer the pagewalk code as an egregious
> example of where we force every user to know "oh, hugetlb is special".
> Getting rid of mm_walk_ops->hugetlb_entry() would be a great improvement.

I'm all for having a better way to walk folios, and Oscar did some work
on that in the past that got stalled (also partially my fault for being
overloaded to provide feedback).

But that would be rather replacing mm_walk_ops, not changing it.

The real challenge are not PMD-mapped or PUD-mapped hugetlb folios.
These are the easy case.

It's all about the hugetlb corner cases. PTE-batching to handle
cont-pte, we already can support. Cont-PMD-mapped hugetlb folios would
require similar support (rather easy).

But the real struggle is "fun" hugetlb things like
* PMD table sharing. I don't want ordinary page table walking code to
  mess with that monstrosity. It can mostly stay in mm/hugetlb.c where
  it can die a slow death.
* Powerpc stuff like a hugetlb folio that spwans two full page tables.
  There is no way we are going to add batching support (reading+writing)
  to generic page table walking code just for that purpose.

Instead, I think we should
* Leave all the mm_walk_ops->hugetlb_entry() stuff alone.
* Enlighten shmem to provide 2M + 1G folios from an alternative
  allocator (pool) (like guest_memfd is planning)
* Allow hugetlb to server as such a pool. Preparations happening in [1].
* Maybe enlighten shmem to be able to reserve folios in hugetlb.

Essentially: rebuild something like hugetlb for shared memory using
shmem, and reuse the existing hugetlb pool. Avoid all the nastiness
(weird architecture tweaks, odd folio sizes we don't want to support,
huge pmd sharing, etc).


[1] https://lore.kernel.org/r/cover.1770854662.git.ackerleytng@google.com

-- 
Cheers,

David