From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7679DC433E2 for ; Mon, 31 Aug 2020 21:44:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1574220E65 for ; Mon, 31 Aug 2020 21:44:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="xY+u2X8H" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1574220E65 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61D706B0003; Mon, 31 Aug 2020 17:44:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CE496B0037; Mon, 31 Aug 2020 17:44:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BD756B0055; Mon, 31 Aug 2020 17:44:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 1C7BF6B0003 for ; Mon, 31 Aug 2020 17:44:50 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A4BC03632 for ; Mon, 31 Aug 2020 21:44:49 +0000 (UTC) X-FDA: 77212193898.01.bone76_5f1556627093 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 64B8C1004B357 for ; Mon, 31 Aug 2020 21:44:49 +0000 (UTC) X-HE-Tag: bone76_5f1556627093 X-Filterd-Recvd-Size: 5980 Received: from userp2120.oracle.com (userp2120.oracle.com [156.151.31.85]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Mon, 31 Aug 2020 21:44:48 +0000 (UTC) Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 07VLdYpx005165; Mon, 31 Aug 2020 21:44:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=tVlNrj0ehWjnoj5BpwktClyWnlSQW859Sjeyx2LRjNc=; b=xY+u2X8Hh1uXjj/5EjE00n7GWYkGE4Zzts/a7g31LOpv1M0itJ4K+iJOXcJYm85OdA2H CmpK0TrDfwIpeYMhJxKhw4b6sUW1meEMluV0aI4YdxRCmhi7ETJdtTkL3sreejn/Y+ut lX7C4K9cDGRJ0nm2XdqGA6/tVGOOhTCuR1zm5Mc2TpnrC/nNiSSzV5NYUA19qx7QNMT6 iuvvIfaaJcZEQUJkQxiaaH1/nd7+Ee75TleStH98niofnyisbzzOxuGmXrV1Orfg8Y4w q9s3K+ufXAU03qP/xBHyhBKVX3rmLkqfWOZXxFK6AN6IF/e3TX1iKbo9kjQ+ahUGLgXy Yw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 337qrhft3r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 31 Aug 2020 21:44:44 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 07VLeaem151523; Mon, 31 Aug 2020 21:44:43 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3020.oracle.com with ESMTP id 3380x1d11c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 31 Aug 2020 21:44:43 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 07VLifUb005692; Mon, 31 Aug 2020 21:44:41 GMT Received: from [192.168.2.112] (/50.38.35.18) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 31 Aug 2020 14:44:41 -0700 Subject: Re: [PATCH] mm/hugetlb: try preferred node first when alloc gigantic page from cma To: Li Xinhai , linux-mm@kvack.org, Michal Hocko Cc: akpm@linux-foundation.org, Roman Gushchin References: <20200830140418.605627-1-lixinhai.lxh@gmail.com> From: Mike Kravetz Message-ID: <640ddf82-26b1-3e38-5245-df481bc0756e@oracle.com> Date: Mon, 31 Aug 2020 14:44:40 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <20200830140418.605627-1-lixinhai.lxh@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9730 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 phishscore=0 mlxlogscore=999 adultscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008310127 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9730 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 adultscore=0 mlxscore=0 lowpriorityscore=0 phishscore=0 clxscore=1011 suspectscore=0 priorityscore=1501 spamscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008310127 X-Rspamd-Queue-Id: 64B8C1004B357 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 8/30/20 7:04 AM, Li Xinhai wrote: > Since commit cf11e85fc08cc6a4 ("mm: hugetlb: optionally allocate gigantic > hugepages using cma"), the gigantic page would be allocated from node > which is not the preferred node, although there are pages available from > that node. The reason is that the nid parameter has been ignored in > alloc_gigantic_page(). > > After this patch, the preferred node is tried first before other allowed > nodes. Thank you! This is an issue that needs to be fixed. > Fixes: cf11e85fc08cc6a4 ("mm: hugetlb: optionally allocate gigantic hugepages using cma") > Cc: Roman Gushchin > Cc: Mike Kravetz > Cc: Michal Hocko > Signed-off-by: Li Xinhai > --- > mm/hugetlb.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index a301c2d672bf..4a28b8853d47 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1256,8 +1256,15 @@ static struct page *alloc_gigantic_page(struct hstate *h, gfp_t gfp_mask, > struct page *page; > int node; > > + if (hugetlb_cma[nid]) { > + page = cma_alloc(hugetlb_cma[nid], nr_pages, > + huge_page_order(h), true); > + if (page) > + return page; > + } > + When looking at your changes, I noticed that this code for allocation from CMA does not take gfp_mask into account. The 'normal' use case is to allocate pool pages with something similar to: echo 16 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages The routine alloc_pool_huge_page will try to interleave pages among nodes: ... gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; for_each_node_mask_to_alloc(h, nr_nodes, node, nodes_allowed) { ... which will eventually call alloc_gigantic_page. If __GFP_THISNODE is set we really do not want to execute the below for loop in alloc_gigantic_page. I think the convention in the mm code is that only the lowest level allocation routines should interpret the GFP flags. We may need to make an exception here and check for __GFP_THISNODE. Michal would be the best person to comment and perhaps make a recommendation. -- Mike Kravetz > for_each_node_mask(node, *nodemask) { > - if (!hugetlb_cma[node]) > + if (node == nid || !hugetlb_cma[node]) > continue; > > page = cma_alloc(hugetlb_cma[node], nr_pages, >