Re: [RFC] mm: khugepaged: use largest enabled hugepage order for min_free_kbytes

[Usama Arif <usamaarif642@gmail.com>]

On 06/06/2025 15:37, Usama Arif wrote:
> On arm64 machines with 64K PAGE_SIZE, the min_free_kbytes and hence the
> watermarks are evaluated to extremely high values, for e.g. a server with
> 480G of memory, only 2M mTHP hugepage size set to madvise, with the rest
> of the sizes set to never, the min, low and high watermarks evaluate to
> 11.2G, 14G and 16.8G respectively.
> In contrast for 4K PAGE_SIZE of the same machine, with only 2M THP hugepage
> size set to madvise, the min, low and high watermarks evaluate to 86M, 566M
> and 1G respectively.
> This is because set_recommended_min_free_kbytes is designed for PMD
> hugepages (pageblock_order = min(HPAGE_PMD_ORDER, PAGE_BLOCK_ORDER)).
> Such high watermark values can cause performance and latency issues in
> memory bound applications on arm servers that use 64K PAGE_SIZE, eventhough
> most of them would never actually use a 512M PMD THP.
> 
> Instead of using HPAGE_PMD_ORDER for pageblock_order use the highest large
> folio order enabled in set_recommended_min_free_kbytes.
> With this patch, when only 2M THP hugepage size is set to madvise for the
> same machine with 64K page size, with the rest of the sizes set to never,
> the min, low and high watermarks evaluate to 2.08G, 2.6G and 3.1G

I forgot to change the other pageblock_nr_pages instance, the patch
will need the below fixlet as well. The watermark numbers will then be
the same as when 4K PAGE_SIZE is used.

commit 0c6bb4e5b3aa078949d712ab9c35e7b2a33cd8a4 (HEAD)
Author: Usama Arif <usamaarif642@gmail.com>
Date:   Fri Jun 6 15:43:25 2025 +0100

    [fixlet] mm: khugepaged: replace all instances of pageblock_nr_pages
    
    This will change the 64K page size, 2M THP hugepage madvise  min, low
    and high watermarks to 87M, 575M and 1G.
    
    Signed-off-by: Usama Arif <usamaarif642@gmail.com>

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index e64cba74eb2a..1c643f13135e 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2650,7 +2650,7 @@ static void set_recommended_min_free_kbytes(void)
        }
 
        /* Ensure 2 pageblocks are free to assist fragmentation avoidance */
-       recommended_min = pageblock_nr_pages * nr_zones * 2;
+       recommended_min = min_thp_pageblock_nr_pages() * nr_zones * 2;
 
        /*
         * Make sure that on average at least two pageblocks are almost free