[PATCH] Documentation: hugetlb: Update hugetlb options

[PATCH] Documentation: hugetlb: Update hugetlb options

[Peter Xu]

The hugepage options are not documented clearly.

Firstly, default_hugepagesz= should always be specified after the
declaration of the same type of huge page using hugepagesz=.  For
example, if we boot a x86_64 system with kernel cmdline
"default_hugepagesz=2M", we'll get a very funny error message:

"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"

It's understandable from code-wise because when hugetlb_init() we
didn't have the 2M page hstate registered, so it's unsupported.
However 2M is actually the default huge page size on x86_64, so we'll
register it right after the error message.  However it's very
confusing if without these knowledges.

Secondly, hugepages= option must be used _after_ another hugepagesz=.
The word "interleave" is fine but it didn't declare the fact that
each of the hugepages= option will be applied to the previous parsed
hugepagesz= option.

State all these clear.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 .../admin-guide/kernel-parameters.txt         | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 86aae1fa099a..2a77b2b01e5e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -839,7 +839,8 @@
 			the legacy /proc/ hugepages APIs, used for SHM, and
 			default size when mounting hugetlbfs filesystems.
 			Defaults to the default architecture's huge page size
-			if not specified.
+			if not specified.  This option can only be used with a
+			pre-defined "hugepagesz=" of the same size.
 
 	deferred_probe_timeout=
 			[KNL] Debugging option to set a timeout in seconds for
@@ -1475,13 +1476,17 @@
 	hpet_mmap=	[X86, HPET_MMAP] Allow userspace to mmap HPET
 			registers.  Default set by CONFIG_HPET_MMAP_DEFAULT.
 
+	hugepagesz=	[HW,IA-64,PPC,X86-64] Declare one type of HugeTLB pages
+			with the size specified.  Valid pages sizes on x86-64
+			are 2M (when the CPU supports "pse") and 1G (when the
+			CPU supports the "pdpe1gb" cpuinfo flag).
+
 	hugepages=	[HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
-	hugepagesz=	[HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
-			On x86-64 and powerpc, this option can be specified
-			multiple times interleaved with hugepages= to reserve
-			huge pages of different sizes. Valid pages sizes on
-			x86-64 are 2M (when the CPU supports "pse") and 1G
-			(when the CPU supports the "pdpe1gb" cpuinfo flag).
+			On x86-64 and powerpc, this option must be used after
+			one hugepagesz= to allocate huge pages of that specific
+			size.  This option can also be used multiple times
+			pairing with hugepagesz= to allocate huge pages for
+			different sizes.
 
 	hung_task_panic=
 			[KNL] Should the hung task detector generate panics.
-- 
2.24.1

Re: [PATCH] Documentation: hugetlb: Update hugetlb options

[Mike Kravetz]

On 4/9/20 2:58 PM, Peter Xu wrote:
> The hugepage options are not documented clearly.
> 
> Firstly, default_hugepagesz= should always be specified after the
> declaration of the same type of huge page using hugepagesz=.  For
> example, if we boot a x86_64 system with kernel cmdline
> "default_hugepagesz=2M", we'll get a very funny error message:
> 
> "HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
> 
> It's understandable from code-wise because when hugetlb_init() we
> didn't have the 2M page hstate registered, so it's unsupported.
> However 2M is actually the default huge page size on x86_64, so we'll
> register it right after the error message.  However it's very
> confusing if without these knowledges.
> 
> Secondly, hugepages= option must be used _after_ another hugepagesz=.
> The word "interleave" is fine but it didn't declare the fact that
> each of the hugepages= option will be applied to the previous parsed
> hugepagesz= option.
> 
> State all these clear.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>

Hi Peter,

Did you happen to see this patch series?

https://lore.kernel.org/linux-mm/20200401183819.20647-1-mike.kravetz@oracle.com/

That should address the documentation issue and more.
-- 
Mike Kravetz

Re: [PATCH] Documentation: hugetlb: Update hugetlb options

[Peter Xu]

On Thu, Apr 09, 2020 at 03:08:07PM -0700, Mike Kravetz wrote:
> On 4/9/20 2:58 PM, Peter Xu wrote:
> > The hugepage options are not documented clearly.
> > 
> > Firstly, default_hugepagesz= should always be specified after the
> > declaration of the same type of huge page using hugepagesz=.  For
> > example, if we boot a x86_64 system with kernel cmdline
> > "default_hugepagesz=2M", we'll get a very funny error message:
> > 
> > "HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
> > 
> > It's understandable from code-wise because when hugetlb_init() we
> > didn't have the 2M page hstate registered, so it's unsupported.
> > However 2M is actually the default huge page size on x86_64, so we'll
> > register it right after the error message.  However it's very
> > confusing if without these knowledges.
> > 
> > Secondly, hugepages= option must be used _after_ another hugepagesz=.
> > The word "interleave" is fine but it didn't declare the fact that
> > each of the hugepages= option will be applied to the previous parsed
> > hugepagesz= option.
> > 
> > State all these clear.
> > 
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> 
> Hi Peter,
> 
> Did you happen to see this patch series?
> 
> https://lore.kernel.org/linux-mm/20200401183819.20647-1-mike.kravetz@oracle.com/
> 
> That should address the documentation issue and more.

Great!

I missed that, but I'll definitely read it (probably tomorrow).  Let's
ignore this patch then.

Thanks,

-- 
Peter Xu

Re: [PATCH] Documentation: hugetlb: Update hugetlb options

[Nitesh Narayan Lal]

[-- Attachment #1.1: Type: text/plain, Size: 3285 bytes --]


On 4/9/20 5:58 PM, Peter Xu wrote:
> The hugepage options are not documented clearly.
>
> Firstly, default_hugepagesz= should always be specified after the
> declaration of the same type of huge page using hugepagesz=.  For
> example, if we boot a x86_64 system with kernel cmdline
> "default_hugepagesz=2M", we'll get a very funny error message:
>
> "HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"

There is an ongoing discussion to fix this issue.
AFAIK it has not been settled yet and Mike was going to post a patch-set.
https://lkml.org/lkml/2020/3/9/96

>
> It's understandable from code-wise because when hugetlb_init() we
> didn't have the 2M page hstate registered, so it's unsupported.
> However 2M is actually the default huge page size on x86_64, so we'll
> register it right after the error message.  However it's very
> confusing if without these knowledges.
>
> Secondly, hugepages= option must be used _after_ another hugepagesz=.
> The word "interleave" is fine but it didn't declare the fact that
> each of the hugepages= option will be applied to the previous parsed
> hugepagesz= option.
>
> State all these clear.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  .../admin-guide/kernel-parameters.txt         | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 86aae1fa099a..2a77b2b01e5e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -839,7 +839,8 @@
>  			the legacy /proc/ hugepages APIs, used for SHM, and
>  			default size when mounting hugetlbfs filesystems.
>  			Defaults to the default architecture's huge page size
> -			if not specified.
> +			if not specified.  This option can only be used with a
> +			pre-defined "hugepagesz=" of the same size.
>  
>  	deferred_probe_timeout=
>  			[KNL] Debugging option to set a timeout in seconds for
> @@ -1475,13 +1476,17 @@
>  	hpet_mmap=	[X86, HPET_MMAP] Allow userspace to mmap HPET
>  			registers.  Default set by CONFIG_HPET_MMAP_DEFAULT.
>  
> +	hugepagesz=	[HW,IA-64,PPC,X86-64] Declare one type of HugeTLB pages
> +			with the size specified.  Valid pages sizes on x86-64
> +			are 2M (when the CPU supports "pse") and 1G (when the
> +			CPU supports the "pdpe1gb" cpuinfo flag).
> +
>  	hugepages=	[HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
> -	hugepagesz=	[HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
> -			On x86-64 and powerpc, this option can be specified
> -			multiple times interleaved with hugepages= to reserve
> -			huge pages of different sizes. Valid pages sizes on
> -			x86-64 are 2M (when the CPU supports "pse") and 1G
> -			(when the CPU supports the "pdpe1gb" cpuinfo flag).
> +			On x86-64 and powerpc, this option must be used after
> +			one hugepagesz= to allocate huge pages of that specific
> +			size.  This option can also be used multiple times
> +			pairing with hugepagesz= to allocate huge pages for
> +			different sizes.
>  
>  	hung_task_panic=
>  			[KNL] Should the hung task detector generate panics.
-- 
Nitesh


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Documentation: hugetlb: Update hugetlb options

[Nitesh Narayan Lal]

[-- Attachment #1.1: Type: text/plain, Size: 1708 bytes --]


On 4/9/20 6:18 PM, Peter Xu wrote:
> On Thu, Apr 09, 2020 at 03:08:07PM -0700, Mike Kravetz wrote:
>> On 4/9/20 2:58 PM, Peter Xu wrote:
>>> The hugepage options are not documented clearly.
>>>
>>> Firstly, default_hugepagesz= should always be specified after the
>>> declaration of the same type of huge page using hugepagesz=.  For
>>> example, if we boot a x86_64 system with kernel cmdline
>>> "default_hugepagesz=2M", we'll get a very funny error message:
>>>
>>> "HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
>>>
>>> It's understandable from code-wise because when hugetlb_init() we
>>> didn't have the 2M page hstate registered, so it's unsupported.
>>> However 2M is actually the default huge page size on x86_64, so we'll
>>> register it right after the error message.  However it's very
>>> confusing if without these knowledges.
>>>
>>> Secondly, hugepages= option must be used _after_ another hugepagesz=.
>>> The word "interleave" is fine but it didn't declare the fact that
>>> each of the hugepages= option will be applied to the previous parsed
>>> hugepagesz= option.
>>>
>>> State all these clear.
>>>
>>> Signed-off-by: Peter Xu <peterx@redhat.com>
>> Hi Peter,
>>
>> Did you happen to see this patch series?
>>
>> https://lore.kernel.org/linux-mm/20200401183819.20647-1-mike.kravetz@oracle.com/
>>
>> That should address the documentation issue and more.
> Great!
>
> I missed that, but I'll definitely read it (probably tomorrow).  Let's
> ignore this patch then.

My bad, I didn't realize that Mike has already  pointed you in the right direction.
Please ignore my earlier email. :)

>
> Thanks,
>
-- 
Nitesh


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]