Re: memfd and transparent hugepages

[David Hildenbrand <david@redhat.com>]

On 18.11.24 15:13, Avi Kivity wrote:
> #define _GNU_SOURCE
> #include <sys/mman.h>
> #include <linux/memfd.h>
> #include <unistd.h>
> #include <stddef.h>
> 
> int main(int ac, char** av) {
>      size_t memsz = (size_t)2048 << 20;
>      int fd = memfd_create("memory", MFD_CLOEXEC);
>      ftruncate(fd, memsz);
>      void* p = mmap((void*)0x40000000, memsz, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FILE, fd, 0);
>      madvise(p, memsz, MADV_HUGEPAGE);
>      madvise(p, memsz, MADV_POPULATE_WRITE);
>      madvise(p, memsz, MADV_COLLAPSE);
>      pause();
>      return 0;
> }

Hi,

> 
> While memfd is documented as using anonymous pages, AnonHugePages shows
> as zero.

it's an anonymous file, not anonymous memory. memfd is shmem in diguise.

> 
> If memfd incompatible with transparent hugepages?

Shmem supports THP.

But you have one flaws in your program: you have to mmap it MAP_SHARED. 
It's shmem.

Otherwise during MADV_POPULATE_WRITE you're simply allocating a shmem 
page (pageache), and the COW it to map an anonymous page in your page 
tables, resulting in a double memory consumption (pagecache+anonymous 
memory). khugepaged does currently not operate on such VMAs.

If you use MAP_SHARED in your example above:

$ cat /proc/`pgrep tst`/smaps_rollup
00400000-7ffc2770d000 ---p 00000000 00:00 0 
[rollup]
Rss:             2098460 kB
Pss:             2097269 kB
Pss_Dirty:       2097248 kB
Pss_Anon:             96 kB
Pss_File:             21 kB
Pss_Shmem:       2097152 kB
Shared_Clean:       1200 kB
Shared_Dirty:          0 kB
Private_Clean:        12 kB
Private_Dirty:   2097248 kB
Referenced:      2098460 kB
Anonymous:            96 kB
KSM:                   0 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:  2097152 kB
FilePmdMapped:         0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB



-- 
Cheers,

David / dhildenb