mmap with huge page

mmap with huge page

[David Frank]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

Hi,According to the instruction, I have to mount a huge directory to hugetlbfs and create file in the huge directory to use the mmap huge page feature. But the issue is that, the files in the huge directory takes up the huge pages configured throughvm.nr_hugepages = 

even the files are not used.
When the total size of the files in the huge directory = vm.nr_hugepages * huge page size, then mmap would fail with 'can not allocate memory' if the file to be  mapped is in the huge dir or the call has HUGEPAGETLB flag.
Basically, I have to move the files off of the huge directory to free up huge pages.
Am I missing anything here?
Thanks,
David

[-- Attachment #2: Type: text/html, Size: 2187 bytes --]

Re: mmap with huge page

[Mike Kravetz]

On 07/17/2018 12:05 PM, David Frank wrote:
> Hi,
> According to the instruction, I have to mount a huge directory to hugetlbfs and create file in the huge directory to use the mmap huge page feature. But the issue is that, the files in the huge directory takes up the huge pages configured through
> vm.nr_hugepages =
> 
> even the files are not used.
> 
> When the total size of the files in the huge directory = vm.nr_hugepages * huge page size, then mmap would fail with 'can not allocate memory' if the file to be  mapped is in the huge dir or the call has HUGEPAGETLB flag.
> 
> Basically, I have to move the files off of the huge directory to free up huge pages.
> 
> Am I missing anything here?
> 

No, that is working as designed.

hugetlbfs filesystems are generally pre-allocated with nr_hugepages
huge pages.  That is the upper limit of huge pages available.  You can
use overcommit/surplus pages to try and exceed the limit, but that
comes with a whole set of potential issues.

If you have not done so already, please see Documentation/vm/hugetlbpage.txt
in the kernel source tree.
-- 
Mike Kravetz

Re: mmap with huge page

[David Frank]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

 Thanks Mike.  I read the doc, which is not explicit on the non used file taking up huge page count 
    On Tuesday, July 17, 2018, 4:57:04 PM PDT, Mike Kravetz <mike.kravetz@oracle.com> wrote:  
 
 On 07/17/2018 12:05 PM, David Frank wrote:
> Hi,
> According to the instruction, I have to mount a huge directory to hugetlbfs and create file in the huge directory to use the mmap huge page feature. But the issue is that, the files in the huge directory takes up the huge pages configured through
> vm.nr_hugepages =
> 
> even the files are not used.
> 
> When the total size of the files in the huge directory = vm.nr_hugepages * huge page size, then mmap would fail with 'can not allocate memory' if the file to be  mapped is in the huge dir or the call has HUGEPAGETLB flag.
> 
> Basically, I have to move the files off of the huge directory to free up huge pages.
> 
> Am I missing anything here?
> 

No, that is working as designed.

hugetlbfs filesystems are generally pre-allocated with nr_hugepages
huge pages.  That is the upper limit of huge pages available.  You can
use overcommit/surplus pages to try and exceed the limit, but that
comes with a whole set of potential issues.

If you have not done so already, please see Documentation/vm/hugetlbpage.txt
in the kernel source tree.
-- 
Mike Kravetz
  

[-- Attachment #2: Type: text/html, Size: 2630 bytes --]

Re: mmap with huge page

[Michal Hocko]

On Wed 18-07-18 00:01:10, David Frank wrote:
>  Thanks Mike.  I read the doc, which is not explicit on the non used file taking up huge page count 

What do you consider non user file? The file contains a data somebody
might want to read later. You cannot simply remove it. This is not
different to any other in memory filesystem (e.g. tmpfs, ramfs)
-- 
Michal Hocko
SUSE Labs