From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA8C0C433FE for ; Wed, 29 Sep 2021 05:38:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F5A7613CD for ; Wed, 29 Sep 2021 05:38:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5F5A7613CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 86BC894000D; Wed, 29 Sep 2021 01:38:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F2F3940009; Wed, 29 Sep 2021 01:38:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 693B594000D; Wed, 29 Sep 2021 01:38:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id 56C48940009 for ; Wed, 29 Sep 2021 01:38:27 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 11E931801EE82 for ; Wed, 29 Sep 2021 05:38:27 +0000 (UTC) X-FDA: 78639505854.38.037B15D Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by imf03.hostedemail.com (Postfix) with ESMTP id B2FC230000A6 for ; Wed, 29 Sep 2021 05:38:26 +0000 (UTC) Received: by mail-yb1-f173.google.com with SMTP id h2so2706359ybi.13 for ; Tue, 28 Sep 2021 22:38:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5ztsiqAX3AYgWG18b943VFbtp7/SfntcI2880JjxeAU=; b=f1d6CzJbbhEpoUsytAYkt/ziq9L+2m2NcmNHs3VOKO09Hty1IGqtaIs4JsoS/kdCC3 wMQF5JWof7knpNS3wDzN+XuhNcKDHD/eqoULwFstloRUQerKzijmhjSqd8lh1bpQpYpO yTJCSWVyvMAq6Q6y/pAmaQyArgYHXUexSDX157lMQPDeexslCa9zFp1R++qrB2vMRERA rE4DfzR18mycN9q8Lr0sC4d/Pqk3R93uKjCg3Vz+Ga96ItbTlcjhUKHIfdE0GiW6HQWJ 33Nclt37+5g07AMXv/HzcehbgLcDLhCXDgQajCNJ96W1Tw/Hms4G9e0FgqBTh2Zs27Ea FL3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5ztsiqAX3AYgWG18b943VFbtp7/SfntcI2880JjxeAU=; b=7VVG41/tcWvnSCsdfT0jo1oA5p1H69F8+va9WNqqH6CRd+m4PxjLYolcl+ZRZs7VGb Bzid4qN/pCcHxrIPJQ2VOghi4i92POpf+PccwvDcuB8xsP+oQZTlb8ftQ67m9PNzyt3T UaqBN/VlFIy7A1W35t/GRl7xSnrz/Cr4sMd7kcUs8tT+XYhEhmYZb+2PpAe9xucwJt1J 1JtJI4r6Efp0M1aUjPKM1pTFXSt2dGP3rJJkZTbvF28tbewGrlbWiP399TBQBuMxugzf 5wywvn1HdyVXsoPCPC6c8CUd+HSlMy6M0F5Ar63oEthRHzP7Vs8TxCwldmFPmIBXG3Tj s0yQ== X-Gm-Message-State: AOAM532gwH0ISG9w7rdvyfV3s9V4jqtQf0wNlBGauq54n9cEas9ar6Nh nVt4xBP7nHmXieTJ2BvON3Qomvhl9aaqZ1vy7FU= X-Google-Smtp-Source: ABdhPJxe/ywDtg1YeCswW8swTkhx53zYX0oC7IN4j3bnhNTvMGH/NngSiQvuhMiTuZNEAr/cDBCGChTcWMI7jgwUnRc= X-Received: by 2002:a25:30c2:: with SMTP id w185mr11165444ybw.321.1632893906007; Tue, 28 Sep 2021 22:38:26 -0700 (PDT) MIME-Version: 1.0 References: <20210927104149.46884-1-yaozhenguo1@gmail.com> In-Reply-To: From: Zhenguo Yao Date: Wed, 29 Sep 2021 13:38:15 +0800 Message-ID: Subject: Re: [PATCH v7] hugetlbfs: Extend the definition of hugepages parameter to support node allocation To: Mike Rapoport Cc: Mike Kravetz , mpe@ellerman.id.au, benh@kernel.crashing.org, paulus@samba.org, corbet@lwn.net, Andrew Morton , yaozhenguo@jd.com, Matthew Wilcox , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Linux Memory Management List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B2FC230000A6 X-Stat-Signature: dmwrx73ht7jcrege33khgs7pgbizrmfh Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=f1d6CzJb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of yaozhenguo1@gmail.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=yaozhenguo1@gmail.com X-HE-Tag: 1632893906-525076 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Thanks for your advice. Mike Rapoport =E4=BA=8E2021=E5=B9=B49=E6=9C=8829=E6=97=A5= =E5=91=A8=E4=B8=89 =E4=B8=8A=E5=8D=8812:47=E5=86=99=E9=81=93=EF=BC=9A > > Hi, > > On Mon, Sep 27, 2021 at 06:41:49PM +0800, Zhenguo Yao wrote: > > We can specify the number of hugepages to allocate at boot. But the > > hugepages is balanced in all nodes at present. In some scenarios, > > we only need hugepages in one node. For example: DPDK needs hugepages > > which are in the same node as NIC. if DPDK needs four hugepages of 1G > > size in node1 and system has 16 numa nodes. We must reserve 64 hugepage= s > > in kernel cmdline. But, only four hugepages are used. The others should > > be free after boot. If the system memory is low(for example: 64G), it w= ill > > be an impossible task. So, Extending hugepages parameter to support > > specifying hugepages at a specific node. > > For example add following parameter: > > > > hugepagesz=3D1G hugepages=3D0:1,1:3 > > > > It will allocate 1 hugepage in node0 and 3 hugepages in node1. > > > > Signed-off-by: Zhenguo Yao > > --- > > ... > > > diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpag= e.c > > index 9a75ba078e1b..dd40ce6e7565 100644 > > --- a/arch/powerpc/mm/hugetlbpage.c > > +++ b/arch/powerpc/mm/hugetlbpage.c > > @@ -229,17 +229,22 @@ static int __init pseries_alloc_bootmem_huge_page= (struct hstate *hstate) > > m->hstate =3D hstate; > > return 1; > > } > > + > > +bool __init node_specific_alloc_support(void) > > I'd suggest to namespace this to hugetlb, e.g. > > hugetlb_node_alloc_supported() > Looks good to me. > > +{ > > + return false; > > +} > > #endif > > > > > > -int __init alloc_bootmem_huge_page(struct hstate *h) > > +int __init alloc_bootmem_huge_page(struct hstate *h, int nid) > > { > > > > #ifdef CONFIG_PPC_BOOK3S_64 > > if (firmware_has_feature(FW_FEATURE_LPAR) && !radix_enabled()) > > return pseries_alloc_bootmem_huge_page(h); > > #endif > > - return __alloc_bootmem_huge_page(h); > > + return __alloc_bootmem_huge_page(h, nid); > > } > > > > #ifndef CONFIG_PPC_BOOK3S_64 > > ... > > > @@ -2868,33 +2869,41 @@ struct page *alloc_huge_page(struct vm_area_str= uct *vma, > > return ERR_PTR(-ENOSPC); > > } > > > > -int alloc_bootmem_huge_page(struct hstate *h) > > +int alloc_bootmem_huge_page(struct hstate *h, int nid) > > __attribute__ ((weak, alias("__alloc_bootmem_huge_page"))); > > -int __alloc_bootmem_huge_page(struct hstate *h) > > +int __alloc_bootmem_huge_page(struct hstate *h, int nid) > > { > > struct huge_bootmem_page *m; > > int nr_nodes, node; > > > > + if (nid >=3D nr_online_nodes) > > + return 0; > > + /* do node specific alloc */ > > + if (nid !=3D NUMA_NO_NODE) { > > + m =3D memblock_alloc_try_nid_raw(huge_page_size(h), huge_= page_size(h), > > + 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid); > > + if (m) > > + goto found; > > + else > > + return 0; > > Nit: you could make it a bit simpler with > > if (!m) > return 0; > goto found; > Looks more regular. > > + } > > + /* do all node balanced alloc */ > > for_each_node_mask_to_alloc(h, nr_nodes, node, &node_states[N_MEM= ORY]) { > > - void *addr; > > - > > - addr =3D memblock_alloc_try_nid_raw( > > + m =3D memblock_alloc_try_nid_raw( > > huge_page_size(h), huge_page_size(h), > > 0, MEMBLOCK_ALLOC_ACCESSIBLE, node); > > - if (addr) { > > - /* > > - * Use the beginning of the huge page to store th= e > > - * huge_bootmem_page struct (until gather_bootmem > > - * puts them into the mem_map). > > - */ > > - m =3D addr; > > + /* > > + * Use the beginning of the huge page to store the > > + * huge_bootmem_page struct (until gather_bootmem > > + * puts them into the mem_map). > > + */ > > + if (m) > > goto found; > > - } > > + else > > + return 0; > > ditto > > > } > > - return 0; > > > > found: > > - BUG_ON(!IS_ALIGNED(virt_to_phys(m), huge_page_size(h))); > > /* Put them into a private list first because mem_map is not up y= et */ > > INIT_LIST_HEAD(&m->list); > > list_add(&m->list, &huge_boot_pages); > > -- > Sincerely yours, > Mike.