From: Christophe LEROY <christophe.leroy@c-s.fr>
To: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au
Cc: linuxppc-dev@lists.ozlabs.org, linux-mm@kvack.org,
Scott Wood <oss@buserror.net>
Subject: Re: [PATCH v4 2/3] powerpc/mm/hugetlb: Add support for reserving gigantic huge pages via kernel command line
Date: Wed, 2 Aug 2017 10:10:14 +0200 [thread overview]
Message-ID: <4d4e90d3-2d5b-7a38-0575-f03e5f225af5@c-s.fr> (raw)
In-Reply-To: <877eym1l4m.fsf@linux.vnet.ibm.com>
Le 02/08/2017 A 09:31, Aneesh Kumar K.V a A(C)crit :
> Christophe LEROY <christophe.leroy@c-s.fr> writes:
>
>> Hi,
>>
>> Le 28/07/2017 A 07:01, Aneesh Kumar K.V a A(C)crit :
>>> With commit aa888a74977a8 ("hugetlb: support larger than MAX_ORDER") we added
>>> support for allocating gigantic hugepages via kernel command line. Switch
>>> ppc64 arch specific code to use that.
>>>
>>> W.r.t FSL support, we now limit our allocation range using BOOTMEM_ALLOC_ACCESSIBLE.
>>>
>>> We use the kernel command line to do reservation of hugetlb pages on powernv
>>> platforms. On pseries hash mmu mode the supported gigantic huge page size is
>>> 16GB and that can only be allocated with hypervisor assist. For pseries the
>>> command line option doesn't do the allocation. Instead pseries does gigantic
>>> hugepage allocation based on hypervisor hint that is specified via
>>> "ibm,expected#pages" property of the memory node.
>>
>> It looks like it doesn't work on the 8xx:
>>
>> root@vgoip:~# dmesg | grep -i huge
>> [ 0.000000] Kernel command line: console=ttyCPM0,115200N8
>> ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off hugepagesz=8M
>> hugepages=4
>> [ 0.416722] HugeTLB registered 8.00 MiB page size, pre-allocated 4 pages
>> [ 0.423184] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
>> root@vgoip:~# cat /proc/meminfo
>> MemTotal: 123388 kB
>> MemFree: 77900 kB
>> MemAvailable: 78412 kB
>> Buffers: 0 kB
>> Cached: 3964 kB
>> SwapCached: 0 kB
>> Active: 3788 kB
>> Inactive: 1680 kB
>> Active(anon): 1636 kB
>> Inactive(anon): 20 kB
>> Active(file): 2152 kB
>> Inactive(file): 1660 kB
>> Unevictable: 0 kB
>> Mlocked: 0 kB
>> SwapTotal: 0 kB
>> SwapFree: 0 kB
>> Dirty: 0 kB
>> Writeback: 0 kB
>> AnonPages: 1552 kB
>> Mapped: 2404 kB
>> Shmem: 152 kB
>> Slab: 0 kB
>> SReclaimable: 0 kB
>> SUnreclaim: 0 kB
>> KernelStack: 304 kB
>> PageTables: 208 kB
>> NFS_Unstable: 0 kB
>> Bounce: 0 kB
>> WritebackTmp: 0 kB
>> CommitLimit: 45308 kB
>> Committed_AS: 16664 kB
>> VmallocTotal: 866304 kB
>> VmallocUsed: 0 kB
>> VmallocChunk: 0 kB
>> HugePages_Total: 0
>> HugePages_Free: 0
>> HugePages_Rsvd: 0
>> HugePages_Surp: 0
>> Hugepagesize: 512 kB
>
> But you are printing above the default hugepaeg details. You haven't
> changed that in kernel command line. What does
> /sys/kernel/mm/hugepages/<hugepages-size>/nr_hugepages show ?
It says 4, so indeed it seems to work.
>
> To change the default hugepage size you may want to use
> default_hugepagesz=8M
>
Ah ? Documentation/admin-guide/kernel-parameters.txt says it is the same
as hugepagesz:
default_hugepagesz=
[same as hugepagesz=] The size of the default
HugeTLB page size. This is the size represented by
the legacy /proc/ hugepages APIs, used for SHM, and
default size when mounting hugetlbfs filesystems.
Defaults to the default architecture's huge page size
if not specified.
You are right, with default_hugepagesz instead of hugepagesz I get the
following in 16k page sizes mode (ie 8M is < MAX_ORDER)
root@vgoip:~# dmesg | grep -i huge
[ 0.000000] Kernel command line: console=ttyCPM0,115200N8
ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
default_hugepagesz=8M hugepages=4
[ 0.410451] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
[ 0.417091] HugeTLB registered 8.00 MiB page size, pre-allocated 4 pages
root@vgoip:~# cat /proc/meminfo
MemTotal: 123968 kB
MemFree: 73248 kB
MemAvailable: 75120 kB
Buffers: 0 kB
Cached: 5936 kB
SwapCached: 0 kB
Active: 6912 kB
Inactive: 2496 kB
Active(anon): 3664 kB
Inactive(anon): 80 kB
Active(file): 3248 kB
Inactive(file): 2416 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 3536 kB
Mapped: 3216 kB
Shmem: 272 kB
Slab: 0 kB
SReclaimable: 0 kB
SUnreclaim: 0 kB
KernelStack: 304 kB
PageTables: 672 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 45600 kB
Committed_AS: 18528 kB
VmallocTotal: 866304 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HugePages_Total: 4
HugePages_Free: 4
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 8192 kB
However, in 4K pages mode, ie (8M > MAX_ORDER), I get
root@vgoip:~# dmesg | grep -i huge
[ 0.000000] Kernel command line: console=ttyCPM0,115200N8
ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
default_hugepagesz=8M hugepages=4
[ 0.413274] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
[ 0.419844] HugeTLB registered 8.00 MiB page size, pre-allocated 0 pages
root@vgoip:~# cat /proc/meminfo
...
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 8192 kB
Looks like for gigantic pages, we have to specify both
default_hugepagesz=8M and hugepagesz=8M. When we do it it works back:
[ 0.000000] Kernel command line: console=ttyCPM0,115200N8
ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
default_hugepagesz=8M hugepagesz=8M hugepages=4
[ 0.420093] HugeTLB registered 8.00 MiB page size, pre-allocated 4 pages
[ 0.426538] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
Christophe
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-08-02 8:10 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-28 5:01 [PATCH v4 1/3] mm/hugetlb: Allow arch to override and call the weak function Aneesh Kumar K.V
2017-07-28 5:01 ` [PATCH v4 2/3] powerpc/mm/hugetlb: Add support for reserving gigantic huge pages via kernel command line Aneesh Kumar K.V
2017-08-02 6:13 ` Christophe LEROY
2017-08-02 7:31 ` Aneesh Kumar K.V
2017-08-02 8:10 ` Christophe LEROY [this message]
2017-08-02 8:52 ` Christophe LEROY
2017-07-28 5:01 ` [PATCH v4 3/3] powerpc/mm/hugetlb: Allow runtime allocation of 16G Aneesh Kumar K.V
2017-08-15 13:20 ` [PATCH v4 1/3] mm/hugetlb: Allow arch to override and call the weak function Michael Ellerman
2017-08-18 12:50 ` [v4, " Michael Ellerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4d4e90d3-2d5b-7a38-0575-f03e5f225af5@c-s.fr \
--to=christophe.leroy@c-s.fr \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=benh@kernel.crashing.org \
--cc=linux-mm@kvack.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=oss@buserror.net \
--cc=paulus@samba.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox