[PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[Liu Shixin]

In alloc_surplus_hugetlb_folio(), we increase nr_huge_pages and
surplus_huge_pages separately. In the middle window, if we set
nr_hugepages to smaller and satisfy count < persistent_huge_pages(h),
the surplus_huge_pages will be increased by adjust_pool_surplus().

After adding delay in the middle window, we can reproduce the problem
easily by following step:

 1. echo 3 > /proc/sys/vm/nr_overcommit_hugepages
 2. mmap two hugepages. When nr_huge_pages=2 and surplus_huge_pages=1,
    goto step 3.
 3. echo 0 > /proc/sys/vm/nr_huge_pages

Finally, nr_huge_pages is less than surplus_huge_pages.

To fix the problem, call only_alloc_fresh_hugetlb_folio() instead and
move down __prep_account_new_huge_page() into the hugetlb_lock.

Fixes: 0c397daea1d4 ("mm, hugetlb: further simplify hugetlb allocation API")
Signed-off-by: Liu Shixin <liushixin2@huawei.com>
---
v2->v3: Modify the comment suggested by Oscar.
 mm/hugetlb.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 9faa1034704ff..0e08d2fff2360 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2253,11 +2253,20 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
 		goto out_unlock;
 	spin_unlock_irq(&hugetlb_lock);
 
-	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask);
+	folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL);
 	if (!folio)
 		return NULL;
 
+	hugetlb_vmemmap_optimize_folio(h, folio);
+
 	spin_lock_irq(&hugetlb_lock);
+	/*
+	 * nr_huge_pages needs to be adjusted within the same lock cycle
+	 * as surplus_pages, otherwise it might confuse
+	 * persistent_huge_pages() momentarily.
+	 */
+	__prep_account_new_huge_page(h, nid);
+
 	/*
 	 * We could have raced with the pool size change.
 	 * Double check that and simply deallocate the new page
-- 
2.34.1

Re: [PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[Peter Xu]

On Wed, Mar 05, 2025 at 11:54:09AM +0800, Liu Shixin wrote:
> In alloc_surplus_hugetlb_folio(), we increase nr_huge_pages and
> surplus_huge_pages separately. In the middle window, if we set
> nr_hugepages to smaller and satisfy count < persistent_huge_pages(h),
> the surplus_huge_pages will be increased by adjust_pool_surplus().
> 
> After adding delay in the middle window, we can reproduce the problem
> easily by following step:
> 
>  1. echo 3 > /proc/sys/vm/nr_overcommit_hugepages
>  2. mmap two hugepages. When nr_huge_pages=2 and surplus_huge_pages=1,
>     goto step 3.
>  3. echo 0 > /proc/sys/vm/nr_huge_pages
> 
> Finally, nr_huge_pages is less than surplus_huge_pages.
> 
> To fix the problem, call only_alloc_fresh_hugetlb_folio() instead and
> move down __prep_account_new_huge_page() into the hugetlb_lock.
> 
> Fixes: 0c397daea1d4 ("mm, hugetlb: further simplify hugetlb allocation API")
> Signed-off-by: Liu Shixin <liushixin2@huawei.com>

Acked-by: Peter Xu <peterx@redhat.com>

-- 
Peter Xu

Re: [PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[Oscar Salvador]

On Wed, Mar 05, 2025 at 11:54:09AM +0800, Liu Shixin wrote:
> In alloc_surplus_hugetlb_folio(), we increase nr_huge_pages and
> surplus_huge_pages separately. In the middle window, if we set
> nr_hugepages to smaller and satisfy count < persistent_huge_pages(h),
> the surplus_huge_pages will be increased by adjust_pool_surplus().
> 
> After adding delay in the middle window, we can reproduce the problem
> easily by following step:
> 
>  1. echo 3 > /proc/sys/vm/nr_overcommit_hugepages
>  2. mmap two hugepages. When nr_huge_pages=2 and surplus_huge_pages=1,
>     goto step 3.
>  3. echo 0 > /proc/sys/vm/nr_huge_pages
> 
> Finally, nr_huge_pages is less than surplus_huge_pages.
> 
> To fix the problem, call only_alloc_fresh_hugetlb_folio() instead and
> move down __prep_account_new_huge_page() into the hugetlb_lock.
> 
> Fixes: 0c397daea1d4 ("mm, hugetlb: further simplify hugetlb allocation API")
> Signed-off-by: Liu Shixin <liushixin2@huawei.com>

Acked-by: Oscar Salvador <osalvador@suse.de>


-- 
Oscar Salvador
SUSE Labs

Re: [PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[Liu Shixin]

On 2025/3/5 11:54, Liu Shixin wrote:
> In alloc_surplus_hugetlb_folio(), we increase nr_huge_pages and
> surplus_huge_pages separately. In the middle window, if we set
> nr_hugepages to smaller and satisfy count < persistent_huge_pages(h),
> the surplus_huge_pages will be increased by adjust_pool_surplus().
>
> After adding delay in the middle window, we can reproduce the problem
> easily by following step:
>
>  1. echo 3 > /proc/sys/vm/nr_overcommit_hugepages
>  2. mmap two hugepages. When nr_huge_pages=2 and surplus_huge_pages=1,
>     goto step 3.
>  3. echo 0 > /proc/sys/vm/nr_huge_pages
>
> Finally, nr_huge_pages is less than surplus_huge_pages.
>
> To fix the problem, call only_alloc_fresh_hugetlb_folio() instead and
> move down __prep_account_new_huge_page() into the hugetlb_lock.
>
> Fixes: 0c397daea1d4 ("mm, hugetlb: further simplify hugetlb allocation API")
> Signed-off-by: Liu Shixin <liushixin2@huawei.com>
> ---
> v2->v3: Modify the comment suggested by Oscar.
>  mm/hugetlb.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 9faa1034704ff..0e08d2fff2360 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2253,11 +2253,20 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
>  		goto out_unlock;
>  	spin_unlock_irq(&hugetlb_lock);
>  
> -	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask);
> +	folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL);
>  	if (!folio)
>  		return NULL;
>  
> +	hugetlb_vmemmap_optimize_folio(h, folio);
> +
>  	spin_lock_irq(&hugetlb_lock);
> +	/*
> +	 * nr_huge_pages needs to be adjusted within the same lock cycle
> +	 * as surplus_pages, otherwise it might confuse
> +	 * persistent_huge_pages() momentarily.
> +	 */
> +	__prep_account_new_huge_page(h, nid);
> +
>  	/*
>  	 * We could have raced with the pool size change.
>  	 * Double check that and simply deallocate the new page

Hi,

Sorry, there's a mistake that the nid may be mismatch.
Please use the following code to fix it, or should I send a fix patch ?

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 39f92aad7bd1..6670f9b9e07a 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2271,7 +2271,7 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
         * as surplus_pages, otherwise it might confuse
         * persistent_huge_pages() momentarily.
         */
-       __prep_account_new_huge_page(h, nid);
+       __prep_account_new_huge_page(h, folio_nid(folio));
 
        /*
         * We could have raced with the pool size change.

Re: [PATCH v3] mm/hugetlb: update nr_huge_pages and surplus_huge_pages together

[Andrew Morton]

On Tue, 1 Apr 2025 16:12:29 +0800 Liu Shixin <liushixin2@huawei.com> wrote:

> > +	/*
> > +	 * nr_huge_pages needs to be adjusted within the same lock cycle
> > +	 * as surplus_pages, otherwise it might confuse
> > +	 * persistent_huge_pages() momentarily.
> > +	 */
> > +	__prep_account_new_huge_page(h, nid);
> > +
> >  	/*
> >  	 * We could have raced with the pool size change.
> >  	 * Double check that and simply deallocate the new page
> 
> Hi,
> 
> Sorry, there's a mistake that the nid may be mismatch.
> Please use the following code to fix it, or should I send a fix patch ?
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 39f92aad7bd1..6670f9b9e07a 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2271,7 +2271,7 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
>          * as surplus_pages, otherwise it might confuse
>          * persistent_huge_pages() momentarily.
>          */
> -       __prep_account_new_huge_page(h, nid);
> +       __prep_account_new_huge_page(h, folio_nid(folio));
>  
>         /*
>          * We could have raced with the pool size change.

Yes, please send a formal patch, with

Fixes: 2273dea6b1e1 ("mm/hugetlb: update nr_huge_pages and surplus_huge_pages together")