Re: [LSF/MM/BPF TOPIC] Beyond 2MB: Why Terabyte-Scale Machines Need 1GB Transparent Huge Pages

[Zi Yan <ziy@nvidia.com>]

On 19 Feb 2026, at 11:00, David Hildenbrand (Arm) wrote:

>>
>> I see 1G THPs being opportunistically used ideally at the start of the application
>> or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory
>> available and a greater chance of getting 1G THPs.
>>
>> Splitting strategy
>> ==================
>>
>> When PUD THP must be break -- for COW after fork, partial munmap, mprotect on
>> a subregion, or reclaim -- it splits directly from PUD to PTE level, converting
>> 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to
>> PMDs and only the necessary PMDs to PTEs. This is something that would hopefully
>> be possible with Davids proposal [3].

With mapping of folios > PMD with PMDs, you can use non uniform split to keep
after-split folios as large as possible.

>
> There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

This sounds more reasonable than splitting 1GB itself.

>
> However, we cannot easily do the same when remapping a 1 GiB THP to be mapped by PMDs etc. I think there are examples where that just doesn't work or is not desired.
>
> But I considered that in general (avoid folio_split()) an interesting approach. The remapping part is a bit different though.

If HW can support multiple TLB entries translating to the same physical frame
and allow translation priority of TLB entries, this remapping would be easy
and we can still keep the 1GB PUD mapping. Basically, we can have 1GB TLB entry
pointing to the 1GB folio and another 4KB TLB entry pointing to the remapped
region and overriding the part in the original 1GB vaddr region.

Without that, SW will need to split the PUD into PMDs and PTEs.


Best Regards,
Yan, Zi