Re: [PATCH] mm/hugetlb: wait for hugepage folios to be freed

[Ge Yang <yangge1116@126.com>]

在 2025/2/14 16:08, David Hildenbrand 写道:
> On 14.02.25 07:32, yangge1116@126.com wrote:
>> From: Ge Yang <yangge1116@126.com>
>>
>> Since the introduction of commit b65d4adbc0f0 ("mm: hugetlb: defer 
>> freeing
>> of HugeTLB pages"), which supports deferring the freeing of HugeTLB 
>> pages,
>> the allocation of contiguous memory through cma_alloc() may fail
>> probabilistically.
>>
>> In the CMA allocation process, if it is found that the CMA area is 
>> occupied
>> by in-use hugepage folios, these in-use hugepage folios need to be 
>> migrated
>> to another location. When there are no available hugepage folios in the
>> free HugeTLB pool during the migration of in-use HugeTLB pages, new 
>> folios
>> are allocated from the buddy system. A temporary state is set on the 
>> newly
>> allocated folio. Upon completion of the hugepage folio migration, the
>> temporary state is transferred from the new folios to the old folios.
>> Normally, when the old folios with the temporary state are freed, it is
>> directly released back to the buddy system. However, due to the deferred
>> freeing of HugeTLB pages, the PageBuddy() check fails, ultimately leading
>> to the failure of cma_alloc().
>>
>> Here is a simplified call trace illustrating the process:
>> cma_alloc()
>>      ->__alloc_contig_migrate_range() // Migrate in-use hugepage
>>          ->unmap_and_move_huge_page()
>>              ->folio_putback_hugetlb() // Free old folios
>>      ->test_pages_isolated()
>>          ->__test_page_isolated_in_pageblock()
>>               ->PageBuddy(page) // Check if the page is in buddy
>>
>> To resolve this issue, we have implemented a function named
>> wait_for_hugepage_folios_freed(). This function ensures that the hugepage
>> folios are properly released back to the buddy system after their 
>> migration
>> is completed. By invoking wait_for_hugepage_folios_freed() following the
>> migration process, we guarantee that when test_pages_isolated() is
>> executed, it will successfully pass.
> 
> Okay, so after every successful migration -> put of src, we wait for the 
> src to actually get freed.
> 
> When migrating multiple hugetlb folios, we'd wait once per folio.
> 
> It reminds me a bit about pcp caches, where folios are !buddy until the 
> pcp was drained.
> 
It seems that we only track unmovable, reclaimable, and movable pages on 
the pcp lists. For specific details, please refer to the 
free_frozen_pages() function.

> I wonder if that waiting should instead be done exactly once after 
> migrating multiple folios? For example, at the beginning of 
> test_pages_isolated(), to "flush" that state from any previous migration?
> 
Yes, this can improve performance. I will make the modification in the 
next version. Thank you.
> Thanks for all your effort around making CMA allocations / migration 
> more reliable.
>