Re: [PATCH 3/5] mm: page_alloc: defrag_mode

[Zi Yan <ziy@nvidia.com>]

On 14 Mar 2025, at 16:50, Johannes Weiner wrote:

> On Fri, Mar 14, 2025 at 02:54:03PM -0400, Zi Yan wrote:
>> On 13 Mar 2025, at 17:05, Johannes Weiner wrote:
>>
>>> The page allocator groups requests by migratetype to stave off
>>> fragmentation. However, in practice this is routinely defeated by the
>>> fact that it gives up *before* invoking reclaim and compaction - which
>>> may well produce suitable pages. As a result, fragmentation of
>>> physical memory is a common ongoing process in many load scenarios.
>>>
>>> Fragmentation deteriorates compaction's ability to produce huge
>>> pages. Depending on the lifetime of the fragmenting allocations, those
>>> effects can be long-lasting or even permanent, requiring drastic
>>> measures like forcible idle states or even reboots as the only
>>> reliable ways to recover the address space for THP production.
>>>
>>> In a kernel build test with supplemental THP pressure, the THP
>>> allocation rate steadily declines over 15 runs:
>>>
>>>     thp_fault_alloc
>>>     61988
>>>     56474
>>>     57258
>>>     50187
>>>     52388
>>>     55409
>>>     52925
>>>     47648
>>>     43669
>>>     40621
>>>     36077
>>>     41721
>>>     36685
>>>     34641
>>>     33215
>>>
>>> This is a hurdle in adopting THP in any environment where hosts are
>>> shared between multiple overlapping workloads (cloud environments),
>>> and rarely experience true idle periods. To make THP a reliable and
>>> predictable optimization, there needs to be a stronger guarantee to
>>> avoid such fragmentation.
>>>
>>> Introduce defrag_mode. When enabled, reclaim/compaction is invoked to
>>> its full extent *before* falling back. Specifically, ALLOC_NOFRAGMENT
>>> is enforced on the allocator fastpath and the reclaiming slowpath.
>>>
>>> For now, fallbacks are permitted to avert OOMs. There is a plan to add
>>> defrag_mode=2 to prefer OOMs over fragmentation, but this requires
>>> additional prep work in compaction and the reserve management to make
>>> it ready for all possible allocation contexts.
>>>
>>> The following test results are from a kernel build with periodic
>>> bursts of THP allocations, over 15 runs:
>>>
>>>                                         vanilla    defrag_mode=1
>>> @claimer[unmovable]:                        189              103
>>> @claimer[movable]:                           92              103
>>> @claimer[reclaimable]:                      207               61
>>> @pollute[unmovable from movable]:            25                0
>>> @pollute[unmovable from reclaimable]:        28                0
>>> @pollute[movable from unmovable]:         38835                0
>>> @pollute[movable from reclaimable]:      147136                0
>>> @pollute[reclaimable from unmovable]:       178                0
>>> @pollute[reclaimable from movable]:          33                0
>>> @steal[unmovable from movable]:              11                0
>>> @steal[unmovable from reclaimable]:           5                0
>>> @steal[reclaimable from unmovable]:         107                0
>>> @steal[reclaimable from movable]:            90                0
>>> @steal[movable from reclaimable]:           354                0
>>> @steal[movable from unmovable]:             130                0
>>>
>>> Both types of polluting fallbacks are eliminated in this workload.
>>>
>>> Interestingly, whole block conversions are reduced as well. This is
>>> because once a block is claimed for a type, its empty space remains
>>> available for future allocations, instead of being padded with
>>> fallbacks; this allows the native type to group up instead of
>>> spreading out to new blocks. The assumption in the allocator has been
>>> that pollution from movable allocations is less harmful than from
>>> other types, since they can be reclaimed or migrated out should the
>>> space be needed. However, since fallbacks occur *before*
>>> reclaim/compaction is invoked, movable pollution will still cause
>>> non-movable allocations to spread out and claim more blocks.
>>>
>>> Without fragmentation, THP rates hold steady with defrag_mode=1:
>>>
>>>     thp_fault_alloc
>>>     32478
>>>     20725
>>>     45045
>>>     32130
>>>     14018
>>>     21711
>>>     40791
>>>     29134
>>>     34458
>>>     45381
>>>     28305
>>>     17265
>>>     22584
>>>     28454
>>>     30850
>>>
>>> While the downward trend is eliminated, the keen reader will of course
>>> notice that the baseline rate is much smaller than the vanilla
>>> kernel's to begin with. This is due to deficiencies in how reclaim and
>>> compaction are currently driven: ALLOC_NOFRAGMENT increases the extent
>>> to which smaller allocations are competing with THPs for pageblocks,
>>> while making no effort themselves to reclaim or compact beyond their
>>> own request size. This effect already exists with the current usage of
>>> ALLOC_NOFRAGMENT, but is amplified by defrag_mode insisting on whole
>>> block stealing much more strongly.
>>>
>>> Subsequent patches will address defrag_mode reclaim strategy to raise
>>> the THP success baseline above the vanilla kernel.
>>
>> All makes sense to me. But is there a better name than defrag_mode?
>> It sounds very similar to /sys/kernel/mm/transparent_hugepage/defrag.
>> Or it actually means the THP defrag mode?
>
> Thanks for taking a look!
>
> I'm not set on defrag_mode, but I also couldn't think of anything
> better.
>
> The proximity to the THP flag name strikes me as beneficial, since
> it's an established term for "try harder to make huge pages".
>
> Suggestions welcome :)
>
>>> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
>>> ---
>>>  Documentation/admin-guide/sysctl/vm.rst |  9 +++++++++
>>>  mm/page_alloc.c                         | 27 +++++++++++++++++++++++--
>>>  2 files changed, 34 insertions(+), 2 deletions(-)
>>>
>>
>> When I am checking ALLOC_NOFRAGMENT, I find that in get_page_from_freelist(),
>> ALLOC_NOFRAGMENT is removed when allocation goes into a remote node. I wonder
>> if this could reduce the anti-fragmentation effort for NUMA systems. Basically,
>> falling back to a remote node for allocation would fragment the remote node,
>> even the remote node is trying hard to not fragment itself. Have you tested
>> on a NUMA system?
>
> There is this hunk in the patch:
>
> @@ -3480,7 +3486,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
>  				continue;
>  		}
>
> -		if (no_fallback && nr_online_nodes > 1 &&
> +		if (no_fallback && !defrag_mode && nr_online_nodes > 1 &&
>  		    zone != zonelist_zone(ac->preferred_zoneref)) {
>  			int local_nid;
>
> So it shouldn't clear the flag when spilling into the next node.
>
> Am I missing something?

Oh, I missed that part. Thank you for pointing it out.

Best Regards,
Yan, Zi