[PATCH v1 1/3] selftests/mm: Fix half_ufd_size_MB calculation

[PATCH v1 1/3] selftests/mm: Fix half_ufd_size_MB calculation

[Ryan Roberts]

$half_ufd_size_MB is supposed to be half of the available hugetlb memory
expressed in MB. But previously it was calculated in pages since
$freepgs is the number of free pages.

When huge pages are 2M it doesn't make a whole lot of difference; the
number of pages that get used is just halved. But on arm64 with 16K or
64K base pages, the PMD size (and default hugetlb size) is 32M and 512M
respectively. So in this case we end up passing a number of MB that is
smaller than a single hugetlb page and the test raises an error.

Fixes: 2e47a445d7b3 ("selftests/mm: run_vmtests.sh: fix hugetlb mem size calculation")
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
 tools/testing/selftests/mm/run_vmtests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index da7e26668103..14fa9d40d574 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -304,7 +304,7 @@ uffd_stress_bin=./uffd-stress
 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
 # Hugetlb tests require source and destination huge pages. Pass in half
 # the size of the free pages we have, which is used for *each*.
-half_ufd_size_MB=$((freepgs / 2))
+half_ufd_size_MB=$(((freepgs * hpgsize_KB / 2) / 1024))
 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
--
2.43.0

[PATCH v1 2/3] selftests/mm: uffd-unit-tests support for hugepages > 2M

[Ryan Roberts]

uffd-unit-tests uses a memory area with a fixed 32M size. Then it
calculates the number of pages by dividing by page_size, which itself is
either the base page size or the PMD huge page size depending on the
test config. For the latter, we end up with nr_pages=1 for arm64 16K
base pages, and nr_pages=0 for 64K base pages. This doesn't end well.

So let's make the 32M size a floor and also ensure that we have at least
2 pages given the PMD size. With this change, the tests pass on arm64
64K base page size configuration.

Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
 tools/testing/selftests/mm/uffd-unit-tests.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 74c8bc02b506..6973e57b227a 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -26,6 +26,8 @@
 #define ALIGN_UP(x, align_to) \
 	((__typeof__(x))((((unsigned long)(x)) + ((align_to)-1)) & ~((align_to)-1)))
 
+#define MAX(a, b) (((a) > (b)) ? (a) : (b))
+
 struct mem_type {
 	const char *name;
 	unsigned int mem_flag;
@@ -196,7 +198,8 @@ uffd_setup_environment(uffd_test_args_t *args, uffd_test_case_t *test,
 	else
 		page_size = psize();
 
-	nr_pages = UFFD_TEST_MEM_SIZE / page_size;
+	/* Ensure we have at least 2 pages */
+	nr_pages = MAX(UFFD_TEST_MEM_SIZE, page_size * 2) / page_size;
 	/* TODO: remove this global var.. it's so ugly */
 	nr_cpus = 1;
 
-- 
2.43.0

Re: [PATCH v1 2/3] selftests/mm: uffd-unit-tests support for hugepages > 2M

[Peter Xu]

On Tue, Mar 18, 2025 at 05:43:40PM +0000, Ryan Roberts wrote:
> uffd-unit-tests uses a memory area with a fixed 32M size. Then it
> calculates the number of pages by dividing by page_size, which itself is
> either the base page size or the PMD huge page size depending on the
> test config. For the latter, we end up with nr_pages=1 for arm64 16K
> base pages, and nr_pages=0 for 64K base pages. This doesn't end well.
> 
> So let's make the 32M size a floor and also ensure that we have at least
> 2 pages given the PMD size. With this change, the tests pass on arm64
> 64K base page size configuration.
> 
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>

Acked-by: Peter Xu <peterx@redhat.com>

-- 
Peter Xu

Re: [PATCH v1 2/3] selftests/mm: uffd-unit-tests support for hugepages > 2M

[Rafael Aquini]

On Tue, Mar 18, 2025 at 05:43:40PM +0000, Ryan Roberts wrote:
> uffd-unit-tests uses a memory area with a fixed 32M size. Then it
> calculates the number of pages by dividing by page_size, which itself is
> either the base page size or the PMD huge page size depending on the
> test config. For the latter, we end up with nr_pages=1 for arm64 16K
> base pages, and nr_pages=0 for 64K base pages. This doesn't end well.
> 
> So let's make the 32M size a floor and also ensure that we have at least
> 2 pages given the PMD size. With this change, the tests pass on arm64
> 64K base page size configuration.
> 
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
> ---
>  tools/testing/selftests/mm/uffd-unit-tests.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 74c8bc02b506..6973e57b227a 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -26,6 +26,8 @@
>  #define ALIGN_UP(x, align_to) \
>  	((__typeof__(x))((((unsigned long)(x)) + ((align_to)-1)) & ~((align_to)-1)))
>  
> +#define MAX(a, b) (((a) > (b)) ? (a) : (b))
> +
>  struct mem_type {
>  	const char *name;
>  	unsigned int mem_flag;
> @@ -196,7 +198,8 @@ uffd_setup_environment(uffd_test_args_t *args, uffd_test_case_t *test,
>  	else
>  		page_size = psize();
>  
> -	nr_pages = UFFD_TEST_MEM_SIZE / page_size;
> +	/* Ensure we have at least 2 pages */
> +	nr_pages = MAX(UFFD_TEST_MEM_SIZE, page_size * 2) / page_size;
>  	/* TODO: remove this global var.. it's so ugly */
>  	nr_cpus = 1;
>  
> -- 
> 2.43.0
> 
 
Acked-by: Rafael Aquini <raquini@redhat.com>

[PATCH v1 3/3] selftests/mm: Speed up split_huge_page_test

[Ryan Roberts]

create_pagecache_thp_and_fd() was previously writing a file sized at
twice the PMD size by making a per-byte write syscall. This was quite
slow when the PMD size is 4M, but completely intolerable for 32M (PMD
size for arm64's 16K page size), and 512M (PMD size for arm64's 64K page
size).

The byte pattern has a 256 byte period, so let's create a 1K buffer and
fill it with exactly 4 periods. Then we can write the buffer as many
times as is required to fill the file. This makes things much more
tolerable.

The test now passes for 16K page size. It still fails for 64K page size
because MAX_PAGECACHE_ORDER is too small for 512M folio size (I think).

Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
 tools/testing/selftests/mm/split_huge_page_test.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c
index 3f353f3d070f..499333d75fff 100644
--- a/tools/testing/selftests/mm/split_huge_page_test.c
+++ b/tools/testing/selftests/mm/split_huge_page_test.c
@@ -5,6 +5,7 @@
  */
 
 #define _GNU_SOURCE
+#include <assert.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <stdarg.h>
@@ -361,6 +362,7 @@ int create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd,
 {
 	size_t i;
 	int dummy = 0;
+	unsigned char buf[1024];
 
 	srand(time(NULL));
 
@@ -368,11 +370,12 @@ int create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd,
 	if (*fd == -1)
 		ksft_exit_fail_msg("Failed to create a file at %s\n", testfile);
 
-	for (i = 0; i < fd_size; i++) {
-		unsigned char byte = (unsigned char)i;
+	assert(fd_size % sizeof(buf) == 0);
+	for (i = 0; i < sizeof(buf); i++)
+		buf[i] = (unsigned char)i;
+	for (i = 0; i < fd_size; i += sizeof(buf))
+		write(*fd, buf, sizeof(buf));
 
-		write(*fd, &byte, sizeof(byte));
-	}
 	close(*fd);
 	sync();
 	*fd = open("/proc/sys/vm/drop_caches", O_WRONLY);
-- 
2.43.0

Re: [PATCH v1 3/3] selftests/mm: Speed up split_huge_page_test

[Peter Xu]

On Tue, Mar 18, 2025 at 05:43:41PM +0000, Ryan Roberts wrote:
> create_pagecache_thp_and_fd() was previously writing a file sized at
> twice the PMD size by making a per-byte write syscall. This was quite
> slow when the PMD size is 4M, but completely intolerable for 32M (PMD
> size for arm64's 16K page size), and 512M (PMD size for arm64's 64K page
> size).
> 
> The byte pattern has a 256 byte period, so let's create a 1K buffer and
> fill it with exactly 4 periods. Then we can write the buffer as many
> times as is required to fill the file. This makes things much more
> tolerable.
> 
> The test now passes for 16K page size. It still fails for 64K page size
> because MAX_PAGECACHE_ORDER is too small for 512M folio size (I think).
> 
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>

Acked-by: Peter Xu <peterx@redhat.com>

-- 
Peter Xu

Re: [PATCH v1 3/3] selftests/mm: Speed up split_huge_page_test

[Rafael Aquini]

On Tue, Mar 18, 2025 at 05:43:41PM +0000, Ryan Roberts wrote:
> create_pagecache_thp_and_fd() was previously writing a file sized at
> twice the PMD size by making a per-byte write syscall. This was quite
> slow when the PMD size is 4M, but completely intolerable for 32M (PMD
> size for arm64's 16K page size), and 512M (PMD size for arm64's 64K page
> size).
> 
> The byte pattern has a 256 byte period, so let's create a 1K buffer and
> fill it with exactly 4 periods. Then we can write the buffer as many
> times as is required to fill the file. This makes things much more
> tolerable.
> 
> The test now passes for 16K page size. It still fails for 64K page size
> because MAX_PAGECACHE_ORDER is too small for 512M folio size (I think).
> 
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
> ---
>  tools/testing/selftests/mm/split_huge_page_test.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c
> index 3f353f3d070f..499333d75fff 100644
> --- a/tools/testing/selftests/mm/split_huge_page_test.c
> +++ b/tools/testing/selftests/mm/split_huge_page_test.c
> @@ -5,6 +5,7 @@
>   */
>  
>  #define _GNU_SOURCE
> +#include <assert.h>
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <stdarg.h>
> @@ -361,6 +362,7 @@ int create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd,
>  {
>  	size_t i;
>  	int dummy = 0;
> +	unsigned char buf[1024];
>  
>  	srand(time(NULL));
>  
> @@ -368,11 +370,12 @@ int create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd,
>  	if (*fd == -1)
>  		ksft_exit_fail_msg("Failed to create a file at %s\n", testfile);
>  
> -	for (i = 0; i < fd_size; i++) {
> -		unsigned char byte = (unsigned char)i;
> +	assert(fd_size % sizeof(buf) == 0);
> +	for (i = 0; i < sizeof(buf); i++)
> +		buf[i] = (unsigned char)i;
> +	for (i = 0; i < fd_size; i += sizeof(buf))
> +		write(*fd, buf, sizeof(buf));
>  
> -		write(*fd, &byte, sizeof(byte));
> -	}
>  	close(*fd);
>  	sync();
>  	*fd = open("/proc/sys/vm/drop_caches", O_WRONLY);
> -- 
> 2.43.0
> 
> 

Acked-by: Rafael Aquini <raquini@redhat.com>

Re: [PATCH v1 1/3] selftests/mm: Fix half_ufd_size_MB calculation

[Peter Xu]

On Tue, Mar 18, 2025 at 05:43:39PM +0000, Ryan Roberts wrote:
> $half_ufd_size_MB is supposed to be half of the available hugetlb memory
> expressed in MB. But previously it was calculated in pages since
> $freepgs is the number of free pages.
> 
> When huge pages are 2M it doesn't make a whole lot of difference; the
> number of pages that get used is just halved. But on arm64 with 16K or
> 64K base pages, the PMD size (and default hugetlb size) is 32M and 512M
> respectively. So in this case we end up passing a number of MB that is
> smaller than a single hugetlb page and the test raises an error.
> 
> Fixes: 2e47a445d7b3 ("selftests/mm: run_vmtests.sh: fix hugetlb mem size calculation")
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>

There's a similar fix already in akpm/mm-hotfixes-stable:

67a2f86846f2 selftests/mm: run_vmtests.sh: fix half_ufd_size_MB calculation

Thanks,

> ---
>  tools/testing/selftests/mm/run_vmtests.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index da7e26668103..14fa9d40d574 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -304,7 +304,7 @@ uffd_stress_bin=./uffd-stress
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
>  # Hugetlb tests require source and destination huge pages. Pass in half
>  # the size of the free pages we have, which is used for *each*.
> -half_ufd_size_MB=$((freepgs / 2))
> +half_ufd_size_MB=$(((freepgs * hpgsize_KB / 2) / 1024))
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
> --
> 2.43.0
> 

-- 
Peter Xu

Re: [PATCH v1 1/3] selftests/mm: Fix half_ufd_size_MB calculation

[Ryan Roberts]

On 18/03/2025 19:54, Peter Xu wrote:
> On Tue, Mar 18, 2025 at 05:43:39PM +0000, Ryan Roberts wrote:
>> $half_ufd_size_MB is supposed to be half of the available hugetlb memory
>> expressed in MB. But previously it was calculated in pages since
>> $freepgs is the number of free pages.
>>
>> When huge pages are 2M it doesn't make a whole lot of difference; the
>> number of pages that get used is just halved. But on arm64 with 16K or
>> 64K base pages, the PMD size (and default hugetlb size) is 32M and 512M
>> respectively. So in this case we end up passing a number of MB that is
>> smaller than a single hugetlb page and the test raises an error.
>>
>> Fixes: 2e47a445d7b3 ("selftests/mm: run_vmtests.sh: fix hugetlb mem size calculation")
>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
> 
> There's a similar fix already in akpm/mm-hotfixes-stable:
> 
> 67a2f86846f2 selftests/mm: run_vmtests.sh: fix half_ufd_size_MB calculation

Oops, my bad, forgot to check mm branches. This was just a drive-by fix while
working in another context.

Hopefully Andrew is ok to take the other 2 patches and drop this one? If not,
let me know and I'll resend the two other patches on their own.

Thanks,
Ryan

> 
> Thanks,
> 
>> ---
>>  tools/testing/selftests/mm/run_vmtests.sh | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
>> index da7e26668103..14fa9d40d574 100755
>> --- a/tools/testing/selftests/mm/run_vmtests.sh
>> +++ b/tools/testing/selftests/mm/run_vmtests.sh
>> @@ -304,7 +304,7 @@ uffd_stress_bin=./uffd-stress
>>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
>>  # Hugetlb tests require source and destination huge pages. Pass in half
>>  # the size of the free pages we have, which is used for *each*.
>> -half_ufd_size_MB=$((freepgs / 2))
>> +half_ufd_size_MB=$(((freepgs * hpgsize_KB / 2) / 1024))
>>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
>>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
>>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
>> --
>> 2.43.0
>>
> 

Re: [PATCH v1 1/3] selftests/mm: Fix half_ufd_size_MB calculation

[Rafael Aquini]

On Tue, Mar 18, 2025 at 05:43:39PM +0000, Ryan Roberts wrote:
> $half_ufd_size_MB is supposed to be half of the available hugetlb memory
> expressed in MB. But previously it was calculated in pages since
> $freepgs is the number of free pages.
> 
> When huge pages are 2M it doesn't make a whole lot of difference; the
> number of pages that get used is just halved. But on arm64 with 16K or
> 64K base pages, the PMD size (and default hugetlb size) is 32M and 512M
> respectively. So in this case we end up passing a number of MB that is
> smaller than a single hugetlb page and the test raises an error.
> 
> Fixes: 2e47a445d7b3 ("selftests/mm: run_vmtests.sh: fix hugetlb mem size calculation")
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
> ---
>  tools/testing/selftests/mm/run_vmtests.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index da7e26668103..14fa9d40d574 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -304,7 +304,7 @@ uffd_stress_bin=./uffd-stress
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
>  # Hugetlb tests require source and destination huge pages. Pass in half
>  # the size of the free pages we have, which is used for *each*.
> -half_ufd_size_MB=$((freepgs / 2))
> +half_ufd_size_MB=$(((freepgs * hpgsize_KB / 2) / 1024))
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
>  CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
> --
> 2.43.0
> 
> 

this one is already fixed in linux-next, see commit
67a2f86846f2 ("selftests/mm: run_vmtests.sh: fix half_ufd_size_MB calculation")

-- Rafael