[PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Baolin Wang]

When running the khugepaged selftest for shmem (./khugepaged all:shmem),
I encountered the following test failures:
"
Run test: collapse_full (khugepaged:shmem)
Collapse multiple fully populated PTE table.... Fail
...
Run test: collapse_single_pte_entry (khugepaged:shmem)
Collapse PTE table with single PTE entry present.... Fail
...
Run test: collapse_full_of_compound (khugepaged:shmem)
Allocate huge page... OK
Split huge page leaving single PTE page table full of compound pages... OK
Collapse PTE table full of compound pages.... Fail
"

The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
khugepaged from continuing to scan shmem VMA after khugepaged finishes
scanning in the wait_for_scan() function. Moreover, shmem requires a refault
to establish PMD mappings.

However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
refault.

To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
With this fix, the shmem test case passes.

Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
 tools/testing/selftests/mm/khugepaged.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 8a4d34cce36b..d462f62d8116 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
 		usleep(TICK);
 	}
 
-	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
-
 	return timeout == -1;
 }
 
@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
 	if (ops != &__anon_ops)
 		ops->fault(p, 0, nr_hpages * hpage_pmd_size);
 
+	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
 	if (ops->check_huge(p, expect ? nr_hpages : 0))
 		success("OK");
 	else
-- 
2.43.5

[PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[Baolin Wang]

Currently, we only test anonymous memory collapse by default. We should also
add shmem collapse as a default test item to catch issues that could break
the test cases.

Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
 tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 33fc7fafa8f9..a38c984103ce 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
 
 CATEGORY="thp" run_test ./khugepaged -s 2
 
+CATEGORY="thp" run_test ./khugepaged all:shmem
+
+CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
+
 CATEGORY="thp" run_test ./transhuge-stress -d 20
 
 # Try to create XFS if not provided
-- 
2.43.5

RE: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[Bird, Tim]

collpase in the subject line should be collapse
 -- Tim

> -----Original Message-----
> From: Baolin Wang <baolin.wang@linux.alibaba.com>
> Subject: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item
> 
> Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that
> could break the test cases. Signed-off-by: Baolin Wang <baolin. wang@ linux. alibaba. com> --- tools/testing/selftests/mm/run_vmtests. sh
> 
> Currently, we only test anonymous memory collapse by default. We should also
> add shmem collapse as a default test item to catch issues that could break
> the test cases.
> 
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>  tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 33fc7fafa8f9..a38c984103ce 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
> 
>  CATEGORY="thp" run_test ./khugepaged -s 2
> 
> +CATEGORY="thp" run_test ./khugepaged all:shmem
> +
> +CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
> +
>  CATEGORY="thp" run_test ./transhuge-stress -d 20
> 
>  # Try to create XFS if not provided
> --
> 2.43.5
> 

Re: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[Baolin Wang]

On 2025/6/12 12:20, Bird, Tim wrote:
> collpase in the subject line should be collapse

Good catch. Will fix the typo. Thanks.

>> -----Original Message-----
>> From: Baolin Wang <baolin.wang@linux.alibaba.com>
>> Subject: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item
>>
>> Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that
>> could break the test cases. Signed-off-by: Baolin Wang <baolin. wang@ linux. alibaba. com> --- tools/testing/selftests/mm/run_vmtests. sh
>>
>> Currently, we only test anonymous memory collapse by default. We should also
>> add shmem collapse as a default test item to catch issues that could break
>> the test cases.
>>
>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>> ---
>>   tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
>> index 33fc7fafa8f9..a38c984103ce 100755
>> --- a/tools/testing/selftests/mm/run_vmtests.sh
>> +++ b/tools/testing/selftests/mm/run_vmtests.sh
>> @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
>>
>>   CATEGORY="thp" run_test ./khugepaged -s 2
>>
>> +CATEGORY="thp" run_test ./khugepaged all:shmem
>> +
>> +CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
>> +
>>   CATEGORY="thp" run_test ./transhuge-stress -d 20
>>
>>   # Try to create XFS if not provided
>> --
>> 2.43.5
>>

Re: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[Dev Jain]

On 12/06/25 9:24 am, Baolin Wang wrote:
> Currently, we only test anonymous memory collapse by default. We should also
> add shmem collapse as a default test item to catch issues that could break
> the test cases.
>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>   tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 33fc7fafa8f9..a38c984103ce 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
>   
>   CATEGORY="thp" run_test ./khugepaged -s 2
>   
> +CATEGORY="thp" run_test ./khugepaged all:shmem
> +
> +CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
> +
>   CATEGORY="thp" run_test ./transhuge-stress -d 20
>   
>   # Try to create XFS if not provided

Reviewed-by: Dev Jain <dev.jain@arm.com>
Tested-by: Dev Jain <dev.jain@arm.com>

Re: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[David Hildenbrand]

On 12.06.25 05:54, Baolin Wang wrote:
> Currently, we only test anonymous memory collapse by default. We should also
> add shmem collapse as a default test item to catch issues that could break
> the test cases.
> 
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>   tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 33fc7fafa8f9..a38c984103ce 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
>   
>   CATEGORY="thp" run_test ./khugepaged -s 2
>   
> +CATEGORY="thp" run_test ./khugepaged all:shmem
> +
> +CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem

Ahh, there we have it already, nice :)

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item

[Zi Yan]

On 11 Jun 2025, at 23:54, Baolin Wang wrote:

> Currently, we only test anonymous memory collapse by default. We should also
> add shmem collapse as a default test item to catch issues that could break
> the test cases.
>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>  tools/testing/selftests/mm/run_vmtests.sh | 4 ++++
>  1 file changed, 4 insertions(+)
>

Acked-by: Zi Yan <ziy@nvidia.com>


Best Regards,
Yan, Zi

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Dev Jain]

On 12/06/25 9:24 am, Baolin Wang wrote:
> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
> I encountered the following test failures:
> "
> Run test: collapse_full (khugepaged:shmem)
> Collapse multiple fully populated PTE table.... Fail
> ...
> Run test: collapse_single_pte_entry (khugepaged:shmem)
> Collapse PTE table with single PTE entry present.... Fail
> ...
> Run test: collapse_full_of_compound (khugepaged:shmem)
> Allocate huge page... OK
> Split huge page leaving single PTE page table full of compound pages... OK
> Collapse PTE table full of compound pages.... Fail
> "
>
> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
> khugepaged from continuing to scan shmem VMA after khugepaged finishes
> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
> to establish PMD mappings.
>
> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
> refault.
>
> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
> With this fix, the shmem test case passes.
>
> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>   tools/testing/selftests/mm/khugepaged.c | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 8a4d34cce36b..d462f62d8116 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
>   		usleep(TICK);
>   	}
>   
> -	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
> -
>   	return timeout == -1;
>   }
>   
> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
>   	if (ops != &__anon_ops)
>   		ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>   
> +	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>   	if (ops->check_huge(p, expect ? nr_hpages : 0))
>   		success("OK");
>   	else

The idea looks sane to me, but do we need to add the madvise call to
madvise_retracted_page_tables() too, since that also calls wait_for_scan()?

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Baolin Wang]

On 2025/6/12 13:10, Dev Jain wrote:
> 
> On 12/06/25 9:24 am, Baolin Wang wrote:
>> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
>> I encountered the following test failures:
>> "
>> Run test: collapse_full (khugepaged:shmem)
>> Collapse multiple fully populated PTE table.... Fail
>> ...
>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>> Collapse PTE table with single PTE entry present.... Fail
>> ...
>> Run test: collapse_full_of_compound (khugepaged:shmem)
>> Allocate huge page... OK
>> Split huge page leaving single PTE page table full of compound 
>> pages... OK
>> Collapse PTE table full of compound pages.... Fail
>> "
>>
>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to 
>> prevent
>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>> scanning in the wait_for_scan() function. Moreover, shmem requires a 
>> refault
>> to establish PMD mappings.
>>
>> However, after commit 2b0f922323cc, PMD mappings are prevented if the 
>> VMA is
>> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings 
>> during
>> refault.
>>
>> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem 
>> refault.
>> With this fix, the shmem test case passes.
>>
>> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are 
>> disabled by the hw/process/vma")
>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>> ---
>>   tools/testing/selftests/mm/khugepaged.c | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tools/testing/selftests/mm/khugepaged.c 
>> b/tools/testing/selftests/mm/khugepaged.c
>> index 8a4d34cce36b..d462f62d8116 100644
>> --- a/tools/testing/selftests/mm/khugepaged.c
>> +++ b/tools/testing/selftests/mm/khugepaged.c
>> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char 
>> *p, int nr_hpages,
>>           usleep(TICK);
>>       }
>> -    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>> -
>>       return timeout == -1;
>>   }
>> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, 
>> char *p, int nr_hpages,
>>       if (ops != &__anon_ops)
>>           ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>> +    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>       if (ops->check_huge(p, expect ? nr_hpages : 0))
>>           success("OK");
>>       else
> 
> The idea looks sane to me, but do we need to add the madvise call to
> madvise_retracted_page_tables() too, since that also calls wait_for_scan()?

Yes, I also realized this after sending the patches:) Thanks.

To keep the original logic:

diff --git a/tools/testing/selftests/mm/khugepaged.c 
b/tools/testing/selftests/mm/khugepaged.c
index d462f62d8116..3452763e2fe3 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -1074,6 +1074,7 @@ static void madvise_retracted_page_tables(struct 
collapse_context *c,
                 return;
         }
         success("OK");
+       madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
         c->collapse("Install huge PMD from page cache", p, nr_hpages, ops,
                     true);
         validate_memory(p, 0, size);

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[David Hildenbrand]

On 12.06.25 05:54, Baolin Wang wrote:
> When running the khugepaged selftest for shmem (./khugepaged all:shmem),

Hmm, this combination is not run automatically through run_tests.sh, 
right? IIUC, it only runs "./khugepaged" which tests anon only ...

Should we add it there? Then I would probably have noticed that myself 
earlier :)

> I encountered the following test failures:
> "
> Run test: collapse_full (khugepaged:shmem)
> Collapse multiple fully populated PTE table.... Fail
> ...
> Run test: collapse_single_pte_entry (khugepaged:shmem)
> Collapse PTE table with single PTE entry present.... Fail
> ...
> Run test: collapse_full_of_compound (khugepaged:shmem)
> Allocate huge page... OK
> Split huge page leaving single PTE page table full of compound pages... OK
> Collapse PTE table full of compound pages.... Fail
> "
> 
> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
> khugepaged from continuing to scan shmem VMA after khugepaged finishes
> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
> to establish PMD mappings.
> 
> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
> refault.

Right. It's always problematic when we have some contradicting 
information in the VMA vs. pagecache.

> 
> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
> With this fix, the shmem test case passes.
> 
> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>   tools/testing/selftests/mm/khugepaged.c | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 8a4d34cce36b..d462f62d8116 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
>   		usleep(TICK);
>   	}
>   
> -	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
> -
>   	return timeout == -1;
>   }
>   
> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
>   	if (ops != &__anon_ops)
>   		ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>   
> +	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>   	if (ops->check_huge(p, expect ? nr_hpages : 0))
>   		success("OK");
>   	else

It's a shame we have this weird interface: there is no way we can clear 
VM_HUGEPAGE without setting VM_NOHUGEPAGE :(

But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll 
almost immediately later call cleanup_area() where we munmap(), right?

-- 
Cheers,

David / dhildenb

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Baolin Wang]

On 2025/6/12 18:08, David Hildenbrand wrote:
> On 12.06.25 05:54, Baolin Wang wrote:
>> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
> 
> Hmm, this combination is not run automatically through run_tests.sh, 
> right? IIUC, it only runs "./khugepaged" which tests anon only ...
> 
> Should we add it there? Then I would probably have noticed that myself 
> earlier :)

Yes, see patch 2.

>> I encountered the following test failures:
>> "
>> Run test: collapse_full (khugepaged:shmem)
>> Collapse multiple fully populated PTE table.... Fail
>> ...
>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>> Collapse PTE table with single PTE entry present.... Fail
>> ...
>> Run test: collapse_full_of_compound (khugepaged:shmem)
>> Allocate huge page... OK
>> Split huge page leaving single PTE page table full of compound 
>> pages... OK
>> Collapse PTE table full of compound pages.... Fail
>> "
>>
>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to 
>> prevent
>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>> scanning in the wait_for_scan() function. Moreover, shmem requires a 
>> refault
>> to establish PMD mappings.
>>
>> However, after commit 2b0f922323cc, PMD mappings are prevented if the 
>> VMA is
>> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings 
>> during
>> refault.
> 
> Right. It's always problematic when we have some contradicting 
> information in the VMA vs. pagecache.
> 
>>
>> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem 
>> refault.
>> With this fix, the shmem test case passes.
>>
>> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are 
>> disabled by the hw/process/vma")
>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>> ---
>>   tools/testing/selftests/mm/khugepaged.c | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tools/testing/selftests/mm/khugepaged.c 
>> b/tools/testing/selftests/mm/khugepaged.c
>> index 8a4d34cce36b..d462f62d8116 100644
>> --- a/tools/testing/selftests/mm/khugepaged.c
>> +++ b/tools/testing/selftests/mm/khugepaged.c
>> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char 
>> *p, int nr_hpages,
>>           usleep(TICK);
>>       }
>> -    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>> -
>>       return timeout == -1;
>>   }
>> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, 
>> char *p, int nr_hpages,
>>       if (ops != &__anon_ops)
>>           ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>> +    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>       if (ops->check_huge(p, expect ? nr_hpages : 0))
>>           success("OK");
>>       else
> 
> It's a shame we have this weird interface: there is no way we can clear 
> VM_HUGEPAGE without setting VM_NOHUGEPAGE :(

Right.

> But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll 
> almost immediately later call cleanup_area() where we munmap(), right?

I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test 
cases all passed.

However, a potential impact of removing MADV_NOHUGEPAGE is that, 
khugepaged might report 'timeout', but check_huge() would still report 
'success' (assuming khugepaged tries to scan the VMA and successfully 
collapses it after the timeout). Such test result could be confusing.

I know this kind of case is very rare and may not even be detectable in 
tests. At least, I couldn't reproduce it. But I prefer to keep the 
original test logic and fix the issue without making further changes.

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[David Hildenbrand]

On 12.06.25 13:37, Baolin Wang wrote:
> 
> 
> On 2025/6/12 18:08, David Hildenbrand wrote:
>> On 12.06.25 05:54, Baolin Wang wrote:
>>> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
>>
>> Hmm, this combination is not run automatically through run_tests.sh,
>> right? IIUC, it only runs "./khugepaged" which tests anon only ...
>>
>> Should we add it there? Then I would probably have noticed that myself
>> earlier :)
> 
> Yes, see patch 2.

Yes, was pleasantly surprised when I found that :)

> 
>>> I encountered the following test failures:
>>> "
>>> Run test: collapse_full (khugepaged:shmem)
>>> Collapse multiple fully populated PTE table.... Fail
>>> ...
>>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>>> Collapse PTE table with single PTE entry present.... Fail
>>> ...
>>> Run test: collapse_full_of_compound (khugepaged:shmem)
>>> Allocate huge page... OK
>>> Split huge page leaving single PTE page table full of compound
>>> pages... OK
>>> Collapse PTE table full of compound pages.... Fail
>>> "
>>>
>>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to
>>> prevent
>>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>>> scanning in the wait_for_scan() function. Moreover, shmem requires a
>>> refault
>>> to establish PMD mappings.
>>>
>>> However, after commit 2b0f922323cc, PMD mappings are prevented if the
>>> VMA is
>>> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings
>>> during
>>> refault.
>>
>> Right. It's always problematic when we have some contradicting
>> information in the VMA vs. pagecache.
>>
>>>
>>> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem
>>> refault.
>>> With this fix, the shmem test case passes.
>>>
>>> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are
>>> disabled by the hw/process/vma")
>>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>>> ---
>>>    tools/testing/selftests/mm/khugepaged.c | 3 +--
>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/tools/testing/selftests/mm/khugepaged.c
>>> b/tools/testing/selftests/mm/khugepaged.c
>>> index 8a4d34cce36b..d462f62d8116 100644
>>> --- a/tools/testing/selftests/mm/khugepaged.c
>>> +++ b/tools/testing/selftests/mm/khugepaged.c
>>> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char
>>> *p, int nr_hpages,
>>>            usleep(TICK);
>>>        }
>>> -    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>> -
>>>        return timeout == -1;
>>>    }
>>> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg,
>>> char *p, int nr_hpages,
>>>        if (ops != &__anon_ops)
>>>            ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>>> +    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>>        if (ops->check_huge(p, expect ? nr_hpages : 0))
>>>            success("OK");
>>>        else
>>
>> It's a shame we have this weird interface: there is no way we can clear
>> VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
> 
> Right.
> 
>> But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll
>> almost immediately later call cleanup_area() where we munmap(), right?
> 
> I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test
> cases all passed.
> 
> However, a potential impact of removing MADV_NOHUGEPAGE is that,
> khugepaged might report 'timeout', but check_huge() would still report
> 'success' (assuming khugepaged tries to scan the VMA and successfully
> collapses it after the timeout). Such test result could be confusing.

If we run into the timeout, we return "true" from wait_for_scan(), and 
in khugepaged_collapse() returns immediately.

So we wouldn't issue another check_huge() call in khugepaged_collapse().

Did I miss something?


-- 
Cheers,

David / dhildenb

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Baolin Wang]

On 2025/6/12 19:45, David Hildenbrand wrote:
> On 12.06.25 13:37, Baolin Wang wrote:
>>
>>
>> On 2025/6/12 18:08, David Hildenbrand wrote:
>>> On 12.06.25 05:54, Baolin Wang wrote:
>>>> When running the khugepaged selftest for shmem (./khugepaged 
>>>> all:shmem),
>>>
>>> Hmm, this combination is not run automatically through run_tests.sh,
>>> right? IIUC, it only runs "./khugepaged" which tests anon only ...
>>>
>>> Should we add it there? Then I would probably have noticed that myself
>>> earlier :)
>>
>> Yes, see patch 2.
> 
> Yes, was pleasantly surprised when I found that :)
> 
>>
>>>> I encountered the following test failures:
>>>> "
>>>> Run test: collapse_full (khugepaged:shmem)
>>>> Collapse multiple fully populated PTE table.... Fail
>>>> ...
>>>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>>>> Collapse PTE table with single PTE entry present.... Fail
>>>> ...
>>>> Run test: collapse_full_of_compound (khugepaged:shmem)
>>>> Allocate huge page... OK
>>>> Split huge page leaving single PTE page table full of compound
>>>> pages... OK
>>>> Collapse PTE table full of compound pages.... Fail
>>>> "
>>>>
>>>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to
>>>> prevent
>>>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>>>> scanning in the wait_for_scan() function. Moreover, shmem requires a
>>>> refault
>>>> to establish PMD mappings.
>>>>
>>>> However, after commit 2b0f922323cc, PMD mappings are prevented if the
>>>> VMA is
>>>> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings
>>>> during
>>>> refault.
>>>
>>> Right. It's always problematic when we have some contradicting
>>> information in the VMA vs. pagecache.
>>>
>>>>
>>>> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem
>>>> refault.
>>>> With this fix, the shmem test case passes.
>>>>
>>>> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are
>>>> disabled by the hw/process/vma")
>>>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>>>> ---
>>>>    tools/testing/selftests/mm/khugepaged.c | 3 +--
>>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>>
>>>> diff --git a/tools/testing/selftests/mm/khugepaged.c
>>>> b/tools/testing/selftests/mm/khugepaged.c
>>>> index 8a4d34cce36b..d462f62d8116 100644
>>>> --- a/tools/testing/selftests/mm/khugepaged.c
>>>> +++ b/tools/testing/selftests/mm/khugepaged.c
>>>> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char
>>>> *p, int nr_hpages,
>>>>            usleep(TICK);
>>>>        }
>>>> -    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>>> -
>>>>        return timeout == -1;
>>>>    }
>>>> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg,
>>>> char *p, int nr_hpages,
>>>>        if (ops != &__anon_ops)
>>>>            ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>>>> +    madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>>>>        if (ops->check_huge(p, expect ? nr_hpages : 0))
>>>>            success("OK");
>>>>        else
>>>
>>> It's a shame we have this weird interface: there is no way we can clear
>>> VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
>>
>> Right.
>>
>>> But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll
>>> almost immediately later call cleanup_area() where we munmap(), right?
>>
>> I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test
>> cases all passed.
>>
>> However, a potential impact of removing MADV_NOHUGEPAGE is that,
>> khugepaged might report 'timeout', but check_huge() would still report
>> 'success' (assuming khugepaged tries to scan the VMA and successfully
>> collapses it after the timeout). Such test result could be confusing.
> 
> If we run into the timeout, we return "true" from wait_for_scan(), and 
> in khugepaged_collapse() returns immediately.
> 
> So we wouldn't issue another check_huge() call in khugepaged_collapse().
> 
> Did I miss something?

Ah, right. Sorry for the wrong example. Now I'm fine to drop the 
MADV_NOHUGEPAGE settiing. Thanks.

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Zi Yan]

On 11 Jun 2025, at 23:54, Baolin Wang wrote:

> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
> I encountered the following test failures:
> "
> Run test: collapse_full (khugepaged:shmem)
> Collapse multiple fully populated PTE table.... Fail
> ...
> Run test: collapse_single_pte_entry (khugepaged:shmem)
> Collapse PTE table with single PTE entry present.... Fail
> ...
> Run test: collapse_full_of_compound (khugepaged:shmem)
> Allocate huge page... OK
> Split huge page leaving single PTE page table full of compound pages... OK
> Collapse PTE table full of compound pages.... Fail
> "
>
> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
> khugepaged from continuing to scan shmem VMA after khugepaged finishes
> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
> to establish PMD mappings.
>
> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is

Can you add the title of the commit? It is easier to understand the context.

2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")

> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
> refault.
>
> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
> With this fix, the shmem test case passes.
>
> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>  tools/testing/selftests/mm/khugepaged.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 8a4d34cce36b..d462f62d8116 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
>  		usleep(TICK);
>  	}
>
> -	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
> -
>  	return timeout == -1;
>  }
>
I assume you are going to just remove this madvise based on your discussion
with David. With that, feel free to add Reviewed-by: Zi Yan <ziy@nvidia.com>

Thanks.

Best Regards,
Yan, Zi

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[David Hildenbrand]

On 12.06.25 17:46, Zi Yan wrote:
> On 11 Jun 2025, at 23:54, Baolin Wang wrote:
> 
>> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
>> I encountered the following test failures:
>> "
>> Run test: collapse_full (khugepaged:shmem)
>> Collapse multiple fully populated PTE table.... Fail
>> ...
>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>> Collapse PTE table with single PTE entry present.... Fail
>> ...
>> Run test: collapse_full_of_compound (khugepaged:shmem)
>> Allocate huge page... OK
>> Split huge page leaving single PTE page table full of compound pages... OK
>> Collapse PTE table full of compound pages.... Fail
>> "
>>
>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
>> to establish PMD mappings.
>>
>> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
> 
> Can you add the title of the commit? It is easier to understand the context.
> 
> 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
>

Probably checkpatch.pl would point out the same :)

-- 
Cheers,

David / dhildenb

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Baolin Wang]

On 2025/6/12 23:46, Zi Yan wrote:
> On 11 Jun 2025, at 23:54, Baolin Wang wrote:
> 
>> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
>> I encountered the following test failures:
>> "
>> Run test: collapse_full (khugepaged:shmem)
>> Collapse multiple fully populated PTE table.... Fail
>> ...
>> Run test: collapse_single_pte_entry (khugepaged:shmem)
>> Collapse PTE table with single PTE entry present.... Fail
>> ...
>> Run test: collapse_full_of_compound (khugepaged:shmem)
>> Allocate huge page... OK
>> Split huge page leaving single PTE page table full of compound pages... OK
>> Collapse PTE table full of compound pages.... Fail
>> "
>>
>> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
>> khugepaged from continuing to scan shmem VMA after khugepaged finishes
>> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
>> to establish PMD mappings.
>>
>> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
> 
> Can you add the title of the commit? It is easier to understand the context.
> 
> 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")

Sure.

> 
>> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
>> refault.
>>
>> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
>> With this fix, the shmem test case passes.
>>
>> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
>> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>> ---
>>   tools/testing/selftests/mm/khugepaged.c | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
>> index 8a4d34cce36b..d462f62d8116 100644
>> --- a/tools/testing/selftests/mm/khugepaged.c
>> +++ b/tools/testing/selftests/mm/khugepaged.c
>> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
>>   		usleep(TICK);
>>   	}
>>
>> -	madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>> -
>>   	return timeout == -1;
>>   }
>>
> I assume you are going to just remove this madvise based on your discussion
> with David. With that, feel free to add Reviewed-by: Zi Yan <ziy@nvidia.com>

Thanks.

Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Mario Casquero]

This patch has been successfully tested. All the khugepaged shmem tests passed.

# ./khugepaged khugepaged:shmem
Save THP and khugepaged settings... OK
Allocate huge page on fault... OK
Split huge PMD on MADV_DONTNEED... OK

Run test: collapse_full (khugepaged:shmem)
Collapse multiple fully populated PTE table.... OK

Run test: collapse_single_pte_entry (khugepaged:shmem)
Collapse PTE table with single PTE entry present.... OK

Run test: collapse_full_of_compound (khugepaged:shmem)
Allocate huge page... OK
Split huge page leaving single PTE page table full of compound pages... OK
Collapse PTE table full of compound pages.... OK
Restore THP and khugepaged settings... OK

Tested-by: Mario Casquero <mcasquer@redhat.com>

On Thu, Jun 12, 2025 at 5:55 AM Baolin Wang
<baolin.wang@linux.alibaba.com> wrote:
>
> When running the khugepaged selftest for shmem (./khugepaged all:shmem),
> I encountered the following test failures:
> "
> Run test: collapse_full (khugepaged:shmem)
> Collapse multiple fully populated PTE table.... Fail
> ...
> Run test: collapse_single_pte_entry (khugepaged:shmem)
> Collapse PTE table with single PTE entry present.... Fail
> ...
> Run test: collapse_full_of_compound (khugepaged:shmem)
> Allocate huge page... OK
> Split huge page leaving single PTE page table full of compound pages... OK
> Collapse PTE table full of compound pages.... Fail
> "
>
> The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
> khugepaged from continuing to scan shmem VMA after khugepaged finishes
> scanning in the wait_for_scan() function. Moreover, shmem requires a refault
> to establish PMD mappings.
>
> However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
> set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
> refault.
>
> To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
> With this fix, the shmem test case passes.
>
> Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>  tools/testing/selftests/mm/khugepaged.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 8a4d34cce36b..d462f62d8116 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,
>                 usleep(TICK);
>         }
>
> -       madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
> -
>         return timeout == -1;
>  }
>
> @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,
>         if (ops != &__anon_ops)
>                 ops->fault(p, 0, nr_hpages * hpage_pmd_size);
>
> +       madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
>         if (ops->check_huge(p, expect ? nr_hpages : 0))
>                 success("OK");
>         else
> --
> 2.43.5
>
>