From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55E47C433F5 for ; Fri, 15 Apr 2022 07:01:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADECE6B0071; Fri, 15 Apr 2022 03:01:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A8E576B0073; Fri, 15 Apr 2022 03:01:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92EC46B0074; Fri, 15 Apr 2022 03:01:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 80BC76B0071 for ; Fri, 15 Apr 2022 03:01:09 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 403CB80BDA for ; Fri, 15 Apr 2022 07:01:09 +0000 (UTC) X-FDA: 79358216658.14.BC9F0DC Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf20.hostedemail.com (Postfix) with ESMTP id A1D631C0002 for ; Fri, 15 Apr 2022 07:01:07 +0000 (UTC) Received: from kwepemi500018.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KfnH05RzbzFpts; Fri, 15 Apr 2022 14:58:36 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500018.china.huawei.com (7.221.188.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 15 Apr 2022 15:01:03 +0800 Received: from [10.174.179.19] (10.174.179.19) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 15 Apr 2022 15:01:02 +0800 Content-Type: multipart/alternative; boundary="------------oZ3qP6BvvK3R5YZQkDsAlhVj" Message-ID: <8d824db7-ef18-7dc5-7b78-72c8aebd2ca0@huawei.com> Date: Fri, 15 Apr 2022 15:01:01 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.3.2 Subject: Re: [PATCH v3 1/4] hugetlb: Fix wrong use of nr_online_nodes Content-Language: en-US To: Kefeng Wang , , , , , , , , , References: <20220413032915.251254-1-liupeng256@huawei.com> <20220413032915.251254-2-liupeng256@huawei.com> <20220415020927.x7ylevbd5uaevfyt@offworld> <08896d0c-8821-000e-4cc2-9e64beda167f@huawei.com> From: "liupeng (DM)" In-Reply-To: <08896d0c-8821-000e-4cc2-9e64beda167f@huawei.com> X-Originating-IP: [10.174.179.19] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Queue-Id: A1D631C0002 X-Stat-Signature: owog8wy6511crb5bg3qyhtkmp5xdwg5u Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=liupeng256@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspamd-Server: rspam01 X-HE-Tag: 1650006067-525234 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --------------oZ3qP6BvvK3R5YZQkDsAlhVj Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit On 2022/4/15 13:41, Kefeng Wang wrote: > > On 2022/4/15 10:09, Davidlohr Bueso wrote: >> On Wed, 13 Apr 2022, Peng Liu wrote: >> >>> Certain systems are designed to have sparse/discontiguous nodes. In >>> this case, nr_online_nodes can not be used to walk through numa node. >>> Also, a valid node may be greater than nr_online_nodes. >>> >>> However, in hugetlb, it is assumed that nodes are contiguous. Recheck >>> all the places that use nr_online_nodes, and repair them one by one. >>> >>> Suggested-by: David Hildenbrand >>> Fixes: 4178158ef8ca ("hugetlbfs: fix issue of preallocation of >>> gigantic pages can't work") >>> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages >>> parameter to support node allocation") >>> Fixes: e79ce9832316 ("hugetlbfs: fix a truncation issue in hugepages >>> parameter") >>> Fixes: f9317f77a6e0 ("hugetlb: clean up potential spectre issue >>> warnings") >>> Signed-off-by: Peng Liu >>> Reviewed-by: Baolin Wang >>> Reviewed-by: Mike Kravetz >> >> Reviewed-by: Davidlohr Bueso >> >> ... but >> >>> --- >>> mm/hugetlb.c | 12 ++++++------ >>> 1 file changed, 6 insertions(+), 6 deletions(-) >>> >>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c >>> index b34f50156f7e..5b5a2a5a742f 100644 >>> --- a/mm/hugetlb.c >>> +++ b/mm/hugetlb.c >>> @@ -2979,7 +2979,7 @@ int __alloc_bootmem_huge_page(struct hstate >>> *h, int nid) >>>     struct huge_bootmem_page *m = NULL; /* initialize for clang */ >>>     int nr_nodes, node; >>> >>> -    if (nid != NUMA_NO_NODE && nid >= nr_online_nodes) >>> +    if (nid != NUMA_NO_NODE && !node_online(nid)) >> >> afaict null_blk could also use this, actually the whole thing wants a >> helper - node_valid()? >> > This one should be unnecessary, and this patch looks has a bug, > > if a very nid passed to node_online(), it may crash,  could you > re-check it, > > see my changes below, > > 1) add tmp check against MAX_NUMNODES before node_online() check, > >     and move it after get tmp in hugepages_setup() , this could cover > both per-node alloc and normal alloc > > 2) due to for_each_online_node() usage, we can drop additional check > of nid in __alloc_bootmem_huge_page() > > > $ git diff > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index fb5a549169ce..5a3ddec181a0 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -2986,8 +2986,6 @@ int __alloc_bootmem_huge_page(struct hstate *h, > int nid) >         struct huge_bootmem_page *m = NULL; /* initialize for clang */ >         int nr_nodes, node; > > -       if (nid != NUMA_NO_NODE && nid >= nr_online_nodes) > -               return 0; >         /* do node specific alloc */ >         if (nid != NUMA_NO_NODE) { >                 m = memblock_alloc_try_nid_raw(huge_page_size(h), > huge_page_size(h), > @@ -3095,7 +3093,7 @@ static void __init > hugetlb_hstate_alloc_pages(struct hstate *h) >         } > >         /* do node specific alloc */ > -       for (i = 0; i < nr_online_nodes; i++) { > +       for_each_online_node(i) { >                 if (h->max_huge_pages_node[i] > 0) { >                         hugetlb_hstate_alloc_pages_onenode(h, i); >                         node_specific_alloc = true; > @@ -4059,7 +4057,7 @@ static int __init hugetlb_init(void) >                         default_hstate.max_huge_pages = >                                 default_hstate_max_huge_pages; > > -                       for (i = 0; i < nr_online_nodes; i++) > +                       for_each_online_node(i) > default_hstate.max_huge_pages_node[i] = > default_hugepages_in_node[i]; >                 } > @@ -4168,15 +4166,15 @@ static int __init hugepages_setup(char *s) >                 count = 0; >                 if (sscanf(p, "%lu%n", &tmp, &count) != 1) >                         goto invalid; > +               if (tmp > MAX_NUMNODES || !node_online(tmp)) > +                       goto invalid; >                 /* Parameter is node format */ >                 if (p[count] == ':') { >                         if (!hugetlb_node_alloc_supported()) { >                                 pr_warn("HugeTLB: architecture can't > support node specific alloc, ignoring!\n"); >                                 return 0; >                         } > -                       if (tmp >= nr_online_nodes) > -                               goto invalid; > -                       node = array_index_nospec(tmp, nr_online_nodes); > +                       node = array_index_nospec(tmp, MAX_NUMNODES); >                         p += count + 1; >                         /* Parse hugepages */ >                         if (sscanf(p, "%lu%n", &tmp, &count) != 1) > @@ -4304,7 +4302,7 @@ static int __init default_hugepagesz_setup(char *s) >          */ >         if (default_hstate_max_huge_pages) { >                 default_hstate.max_huge_pages = > default_hstate_max_huge_pages; > -               for (i = 0; i < nr_online_nodes; i++) > +               for_each_online_node(i) >                         default_hstate.max_huge_pages_node[i] = >                                 default_hugepages_in_node[i]; >                 if (hstate_is_gigantic(&default_hstate)) > > Yes, node_online is not a safe function which will cause panic if a very big number nid is received. So, this patch needs to be modified. Thanks. --------------oZ3qP6BvvK3R5YZQkDsAlhVj Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit


On 2022/4/15 13:41, Kefeng Wang wrote:

On 2022/4/15 10:09, Davidlohr Bueso wrote:
On Wed, 13 Apr 2022, Peng Liu wrote:

Certain systems are designed to have sparse/discontiguous nodes. In
this case, nr_online_nodes can not be used to walk through numa node.
Also, a valid node may be greater than nr_online_nodes.

However, in hugetlb, it is assumed that nodes are contiguous. Recheck
all the places that use nr_online_nodes, and repair them one by one.

Suggested-by: David Hildenbrand <david@redhat.com>
Fixes: 4178158ef8ca ("hugetlbfs: fix issue of preallocation of gigantic pages can't work")
Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
Fixes: e79ce9832316 ("hugetlbfs: fix a truncation issue in hugepages parameter")
Fixes: f9317f77a6e0 ("hugetlb: clean up potential spectre issue warnings")
Signed-off-by: Peng Liu <liupeng256@huawei.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>

Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>

... but

---
mm/hugetlb.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index b34f50156f7e..5b5a2a5a742f 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2979,7 +2979,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
    struct huge_bootmem_page *m = NULL; /* initialize for clang */
    int nr_nodes, node;

-    if (nid != NUMA_NO_NODE && nid >= nr_online_nodes)
+    if (nid != NUMA_NO_NODE && !node_online(nid))

afaict null_blk could also use this, actually the whole thing wants a
helper - node_valid()?

This one should be unnecessary, and this patch looks has a bug,

if a very nid passed to node_online(), it may crash,  could you re-check it,

see my changes below,

1) add tmp check against MAX_NUMNODES before node_online() check,

    and move it after get tmp in hugepages_setup() , this could cover both per-node alloc and normal alloc

2) due to for_each_online_node() usage, we can drop additional check of nid in __alloc_bootmem_huge_page()


$ git diff
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index fb5a549169ce..5a3ddec181a0 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2986,8 +2986,6 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
        struct huge_bootmem_page *m = NULL; /* initialize for clang */
        int nr_nodes, node;

-       if (nid != NUMA_NO_NODE && nid >= nr_online_nodes)
-               return 0;
        /* do node specific alloc */
        if (nid != NUMA_NO_NODE) {
                m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h),
@@ -3095,7 +3093,7 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h)
        }

        /* do node specific alloc */
-       for (i = 0; i < nr_online_nodes; i++) {
+       for_each_online_node(i) {
                if (h->max_huge_pages_node[i] > 0) {
                        hugetlb_hstate_alloc_pages_onenode(h, i);
                        node_specific_alloc = true;
@@ -4059,7 +4057,7 @@ static int __init hugetlb_init(void)
                        default_hstate.max_huge_pages =
                                default_hstate_max_huge_pages;

-                       for (i = 0; i < nr_online_nodes; i++)
+                       for_each_online_node(i)
default_hstate.max_huge_pages_node[i] =
default_hugepages_in_node[i];
                }
@@ -4168,15 +4166,15 @@ static int __init hugepages_setup(char *s)
                count = 0;
                if (sscanf(p, "%lu%n", &tmp, &count) != 1)
                        goto invalid;
+               if (tmp > MAX_NUMNODES || !node_online(tmp))
+                       goto invalid;
                /* Parameter is node format */
                if (p[count] == ':') {
                        if (!hugetlb_node_alloc_supported()) {
                                pr_warn("HugeTLB: architecture can't support node specific alloc, ignoring!\n");
                                return 0;
                        }
-                       if (tmp >= nr_online_nodes)
-                               goto invalid;
-                       node = array_index_nospec(tmp, nr_online_nodes);
+                       node = array_index_nospec(tmp, MAX_NUMNODES);
                        p += count + 1;
                        /* Parse hugepages */
                        if (sscanf(p, "%lu%n", &tmp, &count) != 1)
@@ -4304,7 +4302,7 @@ static int __init default_hugepagesz_setup(char *s)
         */
        if (default_hstate_max_huge_pages) {
                default_hstate.max_huge_pages = default_hstate_max_huge_pages;
-               for (i = 0; i < nr_online_nodes; i++)
+               for_each_online_node(i)
                        default_hstate.max_huge_pages_node[i] =
                                default_hugepages_in_node[i];
                if (hstate_is_gigantic(&default_hstate)) 


Yes, node_online is not a safe function which will cause panic if a very
big number nid is received. So, this patch needs to be modified.
Thanks.
--------------oZ3qP6BvvK3R5YZQkDsAlhVj--