From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D015C433F5 for ; Fri, 25 Mar 2022 03:15:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85A9B6B0071; Thu, 24 Mar 2022 23:15:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E2116B0073; Thu, 24 Mar 2022 23:15:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65C728D0001; Thu, 24 Mar 2022 23:15:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id 4F56C6B0071 for ; Thu, 24 Mar 2022 23:15:11 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id EAF2EA5A7A for ; Fri, 25 Mar 2022 03:15:10 +0000 (UTC) X-FDA: 79281442380.20.6436836 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf01.hostedemail.com (Postfix) with ESMTP id C1FB04002C for ; Fri, 25 Mar 2022 03:15:09 +0000 (UTC) Received: from kwepemi500011.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KPnGz3CL7zfZbd; Fri, 25 Mar 2022 11:13:31 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500011.china.huawei.com (7.221.188.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Fri, 25 Mar 2022 11:15:05 +0800 Received: from [10.174.179.19] (10.174.179.19) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Fri, 25 Mar 2022 11:15:05 +0800 Content-Type: multipart/alternative; boundary="------------yN2KDjt0pTEgW9Yjh4Ldyoj0" Message-ID: Date: Fri, 25 Mar 2022 11:15:04 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.3.2 Subject: Re: [PATCH] hugetlb: Fix hugepages_setup when deal with pernode Content-Language: en-US To: Mike Kravetz , , , , References: <20220324074037.2024608-1-liupeng256@huawei.com> <361111c6-921b-e129-edf3-367748f6497e@oracle.com> From: "liupeng (DM)" In-Reply-To: <361111c6-921b-e129-edf3-367748f6497e@oracle.com> X-Originating-IP: [10.174.179.19] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Stat-Signature: dbn4rdt7wuhfcdtegzfuehoy338f8ug7 Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=liupeng256@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C1FB04002C X-HE-Tag: 1648178109-688048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --------------yN2KDjt0pTEgW9Yjh4Ldyoj0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit On 2022/3/25 5:57, Mike Kravetz wrote: > On 3/24/22 00:40, Peng Liu wrote: >> Hugepages can be specified to pernode since "hugetlbfs: extend >> the definition of hugepages parameter to support node allocation", >> but the following two problems are observed. >> >> 1) Confusing behavior is observed when both 1G and 2M hugepage >> is set after "numa=off". >> cmdline hugepage settings: >> hugepagesz=1G hugepages=0:3,1:3 >> hugepagesz=2M hugepages=0:1024,1:1024 >> results: >> HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages >> HugeTLB registered 2.00 MiB page size, pre-allocated 1024 pages >> >> 2) Using invalid option values causes the entire kernel boot option >> string to be reported as Unknown. >> Unknown kernel command line parameters "hugepages=0:1024,1:1024" > Thank you for debugging and sending the patch! > > My first thought was "If someone is specifying 'numa=off' as well as > numa node specific allocations on the same command line, we should just > fail the allocation request". However, this same situation could exist > without the 'numa=off' option as long as an invalid node is included in > the list. We will "specifying 'numa=off' as well as numa node specific allocations" for some debugging and test cases. If the original command line can be partly effective, this will be convenient. Yet, we also test "an invalid node is included in the list", the behavior is the same with "numa=off". > With your patch, the node specific allocations are parsed (and processed) > until there is an error. So, in the example above 3 1G pages and 1024 2M > pages are allocated on node 0. That seems correct. > > Now suppose the node specific allocations are specified as: > hugepagesz=1G hugepages=1:3,0:3 > hugepagesz=2M hugepages=1:1024,0:1024 For this case, with/without this patch, huge page will be not allocated on any node. > Since node 1 is invalid, we experience an error here and do not allocate > any pages on node 0. > > I am wondering if we should just error and ignore the entire string if > ANY of the specified nodes are invalid? Thoughts? Thank you for your response. This patch only to be consistent between 2M/1G behavior, and repair "return 0" as 1d02b444b8d1 ("tracing: Fix return value of __setup handlers"). With this patch, a node could allocate huge pages until there is an error, and it will print the invalid parameter from the first parse error. So, I think this is acceptable. The following tests is added with this patch. *Case1*: settings: hugepagesz=1G hugepages=0:3,1:3,2:3 hugepagesz=2M hugepages=0:1024,1:1024,3:1024 results: HugeTLB: Invalid hugepages parameter 1:3,2:3 HugeTLB: Invalid hugepages parameter 1:1024,3:1024 HugeTLB registered 1.00 GiB page size, pre-allocated 3 pages HugeTLB registered 2.00 MiB page size, pre-allocated 1024 pages *Case2*: settings: hugepagesz=1G hugepages=1:3,0:3 hugepagesz=2M hugepages=1:1024,0:1024 results: HugeTLB: Invalid hugepages parameter 1:3,0:3 HugeTLB: Invalid hugepages parameter 1:1024,0:1024 HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages Peng Liu. --------------yN2KDjt0pTEgW9Yjh4Ldyoj0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 7bit


On 2022/3/25 5:57, Mike Kravetz wrote:
On 3/24/22 00:40, Peng Liu wrote:

Hugepages can be specified to pernode since "hugetlbfs: extend
the definition of hugepages parameter to support node allocation",
but the following two problems are observed.

1) Confusing behavior is observed when both 1G and 2M hugepage
is set after "numa=off".
 cmdline hugepage settings:
  hugepagesz=1G hugepages=0:3,1:3
  hugepagesz=2M hugepages=0:1024,1:1024
 results:
  HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
  HugeTLB registered 2.00 MiB page size, pre-allocated 1024 pages

2) Using invalid option values causes the entire kernel boot option
string to be reported as Unknown.
 Unknown kernel command line parameters "hugepages=0:1024,1:1024"

Thank you for debugging and sending the patch!

My first thought was "If someone is specifying 'numa=off' as well as
numa node specific allocations on the same command line, we should just
fail the allocation request".  However, this same situation could exist
without the 'numa=off' option as long as an invalid node is included in
the list.
We will "specifying 'numa=off' as well as numa node specific allocations"
for some debugging and test cases. If the original command line can be
partly effective, this will be convenient. Yet, we also test "an invalid
node is included in the list", the behavior is the same with "numa=off".

With your patch, the node specific allocations are parsed (and processed)
until there is an error.  So, in the example above 3 1G pages and 1024 2M
pages are allocated on node 0.  That seems correct.

Now suppose the node specific allocations are specified as:
hugepagesz=1G hugepages=1:3,0:3
hugepagesz=2M hugepages=1:1024,0:1024
For this case, with/without this patch, huge page will be not allocated
on any node.
Since node 1 is invalid, we experience an error here and do not allocate
any pages on node 0.

I am wondering if we should just error and ignore the entire string if
ANY of the specified nodes are invalid?  Thoughts?

Thank you for your response.

This patch only to be consistent between 2M/1G behavior, and repair "return 0"
as 1d02b444b8d1 ("tracing: Fix return value of __setup handlers").
With this patch, a node could allocate huge pages until there is an error, and it
will print the invalid parameter from the first parse error. So, I think this
is acceptable.

The following tests is added with this patch.

Case1:
settings:
hugepagesz=1G hugepages=0:3,1:3,2:3
hugepagesz=2M hugepages=0:1024,1:1024,3:1024

results:
HugeTLB: Invalid hugepages parameter 1:3,2:3
HugeTLB: Invalid hugepages parameter 1:1024,3:1024
HugeTLB registered 1.00 GiB page size, pre-allocated 3 pages
HugeTLB registered 2.00 MiB page size, pre-allocated 1024 pages

Case2:

settings:
hugepagesz=1G hugepages=1:3,0:3
hugepagesz=2M hugepages=1:1024,0:1024

results:
HugeTLB: Invalid hugepages parameter 1:3,0:3
HugeTLB: Invalid hugepages parameter 1:1024,0:1024
HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages

Peng Liu.


    

  

--------------yN2KDjt0pTEgW9Yjh4Ldyoj0--