[Issue] mprotect+madvise may be better than mmap for permission changes and page zeroing

[Lance Yang <ioworker0@gmail.com>]

Hello Everyone,

I've noticed that using mprotect(PROT_NONE) with
madvise(MADV_DONTNEED) is much faster than
mmap(PROT_NONE, MAP_FIXED) alone for changing
permissions and zeroing pages.

I have maintained a chunk-allocator internally at the
company. It allocates a chunk using
mmap(PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0),
releases a chunk using mmap(PROT_NONE, MAP_ANON|MAP_FIXED|MAP_PRIVATE, -1, 0),
and reuses a chunk using mprotect(PROT_READ|PROT_WRITE).
Recently, I replaced mmap(PROT_NONE, MAP_FIXED) with
mprotect(PROT_NONE) + madvise(MADV_DONTNEED) to
reduce the latency of releasing chunks.

Test code:
https://github.com/ioworker0/mmapvsmprotect/blob/main/test2.c

Here are the test results on my machine:
CPU: AMD EPYC 7R13 Processor
Kernel: 6.2.0
Elapsed Time for mprotect+madvise: 3670 nanoseconds
Elapsed Time for mmap: 5520 nanoseconds

Thanks for your time!
Lance Yang