Howdy,

On Wed, May 18, 2011 at 12:34 PM, Russ Anderson <rja@sgi.com> wrote:

> If the total size of hugepages allocated on a system is
> over half of the total memory size, commitlimit becomes
> a negative number.
>
> What happens in fs/proc/meminfo.c is this calculation:
>
>        allowed = ((totalram_pages - hugetlb_total_pages())
>                * sysctl_overcommit_ratio / 100) + total_swap_pages;
>
> The problem is that hugetlb_total_pages() is larger than
> totalram_pages resulting in a negative number.  Since
> allowed is an unsigned long the negative shows up as a
> big number.
>
> A similar calculation occurs in __vm_enough_memory() in mm/mmap.c.
>
> A symptom of this problem is that /proc/meminfo prints a
> very large CommitLimit number.
>
> CommitLimit:    737869762947802600 kB
>
> To reproduce the problem reserve over half of memory as hugepages.
> For example "default_hugepagesz=1G hugepagesz=1G hugepages=64
> Then look at /proc/meminfo "CommitLimit:" to see if it is too big.
>
> The fix is to not subtract hugetlb_total_pages().  When hugepages
> are allocated totalram_pages is decremented so there is no need to
> subtract out hugetlb_total_pages() a second time.
>
> Reported-by: Russ Anderson <rja@sgi.com>
> Signed-off-by: Russ Anderson <rja@sgi.com>
>
> ---
>
> Example of "CommitLimit:" being too big.
>
> uv1-sys:~ # cat /proc/meminfo
> MemTotal:       32395508 kB
> MemFree:        32029276 kB
> Buffers:            8656 kB
> Cached:            89548 kB
> SwapCached:            0 kB
> Active:            55336 kB
> Inactive:          73916 kB
> Active(anon):      31220 kB
> Inactive(anon):       36 kB
> Active(file):      24116 kB
> Inactive(file):    73880 kB
> Unevictable:           0 kB
> Mlocked:               0 kB
> SwapTotal:             0 kB
> SwapFree:              0 kB
> Dirty:              1692 kB
> Writeback:             0 kB
> AnonPages:         31132 kB
> Mapped:            15668 kB
> Shmem:               152 kB
> Slab:              70256 kB
> SReclaimable:      17148 kB
> SUnreclaim:        53108 kB
> KernelStack:        6536 kB
> PageTables:         3704 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:    737869762947802600 kB
> Committed_AS:     394044 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:      713960 kB
> VmallocChunk:   34325764204 kB
> HardwareCorrupted:     0 kB
> HugePages_Total:      32
> HugePages_Free:       32
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:    1048576 kB
> DirectMap4k:       16384 kB
> DirectMap2M:     2064384 kB
> DirectMap1G:    65011712 kB
>
>  fs/proc/meminfo.c |    2 +-
>  mm/mmap.c         |    3 +--
>  2 files changed, 2 insertions(+), 3 deletions(-)
>
> Index: linux/fs/proc/meminfo.c
> ===================================================================
> --- linux.orig/fs/proc/meminfo.c        2011-05-17 16:03:50.935658801 -0500
> +++ linux/fs/proc/meminfo.c     2011-05-18 08:53:00.568784147 -0500
> @@ -36,7 +36,7 @@ static int meminfo_proc_show(struct seq_
>        si_meminfo(&i);
>        si_swapinfo(&i);
>        committed = percpu_counter_read_positive(&vm_committed_as);
> -       allowed = ((totalram_pages - hugetlb_total_pages())
> +       allowed = (totalram_pages
>                * sysctl_overcommit_ratio / 100) + total_swap_pages;
>
>        cached = global_page_state(NR_FILE_PAGES) -
> Index: linux/mm/mmap.c
> ===================================================================
> --- linux.orig/mm/mmap.c        2011-05-17 16:03:51.727658828 -0500
> +++ linux/mm/mmap.c     2011-05-18 08:54:34.912222405 -0500
> @@ -167,8 +167,7 @@ int __vm_enough_memory(struct mm_struct
>                goto error;
>        }
>
> -       allowed = (totalram_pages - hugetlb_total_pages())
> -               * sysctl_overcommit_ratio / 100;
> +       allowed = totalram_pages * sysctl_overcommit_ratio / 100;
>        /*
>         * Leave the last 3% for root
>         */
> --
> Russ Anderson, OS RAS/Partitioning Project Lead
> SGI - Silicon Graphics Inc          rja@sgi.com


I'm afraid this will introduce a bug on how accurate kernel will account
memory for overcommitment limits.

totalram_pages is not decremented as hugepages are allocated. Since
hugepages are reserved, hugetlb_total_pages() has to be accounted and
subtracted from totalram_pages in order to render an accurate number of
remaining pages available to the general memory workload commitment.

I've tried to reproduce your findings on my boxes,  without
success, unfortunately.

I'll keep chasing to hit this behaviour, though.

Cheers!
--aquini