Re: ANON_LARGE_FOLIOS meeting follow-up & refined proposal

[David Hildenbrand <david@redhat.com>]

On 25.09.23 10:51, Ryan Roberts wrote:
> On 23/09/2023 01:33, John Hubbard wrote:
>> On 9/22/23 08:48, Ryan Roberts wrote:
>> ...
>>> I never had any feedback on the below; I'm not sure if that means everyone is
>>> happy or that nobody read it??
>>
>> One can never really know: zero or more people read it, and of those, no
>> one hated it enough to send out a quick NAK. So that's a *possible*,
>> lukewarm endorsement of sorts. Success! :)
> 
> You really know how to fill a guy with confidence! ;-)
> 
>>
>> ...
>>
>>> BUT I've had yet another idea on the controls front, which would enable exposing
>>> this to user space as an extension to transparent_hugepage, while continuing to
>>> support THP as is and also be able to control THP and ALF (anon large folio)
>>
>> The new ALF / ANON_LARGE_FOLIO naming looks good to me. The grep aspect
>> is a nice touch.
> 
> Well if we go the route of the newest proposal, then I guess the naming is less
> important, because it all attaches to transparent_hugepage.
> 
>>
>> ...
>>
>>> Add 2 controls to sysfs:
>>>
>>> /sys/kernel/mm/transparent_hugepage/anon_orders
>>>     - bitfield where set bits are orders that will be tried during allocation
>>>     - defaults to 1<<PMD_ORDER, which gives current THP behaviour with no ALF
>>>     - For now, 1<<PMD_ORDER is highest settable bit, but easy to expand in future
>>>     - To enable ALF, set the appropriate lower bits
>>>     - To disable THP, clear 1<<PMD_ORDER
>>>     - (In future we could add an "auto" option too)
>>>
>>> /sys/kernel/mm/transparent_hugepage/anon_always_mask
>>>     - orders in (anon_orders & anon_always_mask) are not subject to madvise
>>>     - so when enabled=madvise, still try (anon_orders & anon_always_mask) orders
>>>       as if enabled=always
>>>     - defaults to 0 (all subject to madvise)
>>>
>>
>> I *think* I like this a lot,
> 
> On the weight of this lukewarm endorsement, I'm going to code it up and aim to
> post something for dicussion end of this week. ;-)
> 
>> although I have some clarifying question
>> below. It seems to address the key things that have been complicating
>> the discussions: the API is now looking more flexible, and yet still
>> easy to understand and reason about. Nice.
>>
>> A couple of questions about how this works:
>>
>>>
>>> The defaults for those controls give you "legacy THP". But you can modify the
>>> controls to generate policies like this:
>>>
>>>
>>
>> For these tables, a small key or legend would help. I've forgotten already
>> what "S" means, and am also vague about exactly what "THP>ALF>S" behavior
>> means, too.
> 
> THP:
>      transparent hugepage allocation; specifically PMD sized/aligned/mapped.
> 
> ALF:
>      anonymous large folio allocation; specifically some order between
>      [PMD_ORDER-1, 1]. Always PTE-mapped.

^ and that's exactly not where we wanted to draw the line to be 
future-proof. Ideally we'd create something that is future proof, such 
that it could be extended to any folio sizes in the future.

-- 
Cheers,

David / dhildenb