From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B74CEC48BF6 for ; Mon, 26 Feb 2024 03:35:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 18EEC6B0152; Sun, 25 Feb 2024 22:35:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1173E6B0154; Sun, 25 Feb 2024 22:35:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED2F56B0155; Sun, 25 Feb 2024 22:35:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DAE426B0152 for ; Sun, 25 Feb 2024 22:35:03 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3111BC06A0 for ; Mon, 26 Feb 2024 03:35:03 +0000 (UTC) X-FDA: 81832538886.15.6D1F3D2 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by imf25.hostedemail.com (Postfix) with ESMTP id B2917A0002 for ; Mon, 26 Feb 2024 03:34:59 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AbgkUi8s; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708918501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sF16PtfMksFLXAGoqg1Gy1LZK+T652VTSpOR/ck/ZMU=; b=AdUjTDxfvWMHx8IUQ3NFi4H4zadtcJpJ9Pxh+BpSI9o5+4+i8kNY4pOzUA9A6b3MT8vQTE c1iKzg+QHyDjRTlZ52N2wVTXe2YoS6Y3Ux7lXJa65dVcAynq21WyAoTLbeku/V7uENWZS4 2nV3wdQwS5lhHv17ZPyYk0pdjHx6+x8= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=AbgkUi8s; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708918501; a=rsa-sha256; cv=none; b=lzRsHqlUDT3ua+FoPD8xhpfHjQ0NtWOAoe5LQfCdApioF/LgcxRhU+f2zWEU3iIEcVvR7r oigHsCbJkvpAa3NZets+iN2AwCK0bEdLyUxYmbe9qSXMCNeE8V/SBl29DV4Qkq0wNekfJk njvxaNCa9Z/pey7eD94WJwZ5T5SFopY= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708918496; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=sF16PtfMksFLXAGoqg1Gy1LZK+T652VTSpOR/ck/ZMU=; b=AbgkUi8sRdMN6hJNxA4DykHoJJNp9Vdrvz9A9DcjQloW3jESCH7EtlBGAvjIfXNCP04uqeo0y56c8JaRDkGbNohg1T3ckuMxZFpNxsRMPAdcUm8cy3oR8PUAbcP1EYhI5bm3s1RIsiIyWOE8+3pUokr3Pc9xtV9EOfxMObWsjA4= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0W1AMo4-_1708918492; Received: from 30.97.56.44(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W1AMo4-_1708918492) by smtp.aliyun-inc.com; Mon, 26 Feb 2024 11:34:52 +0800 Message-ID: <9b49d2c8-4ace-4095-8610-8becf96eb023@linux.alibaba.com> Date: Mon, 26 Feb 2024 11:34:51 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 2/3] mm: hugetlb: make the hugetlb migration strategy consistent To: Oscar Salvador Cc: akpm@linux-foundation.org, muchun.song@linux.dev, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <0514e5139b17ecf3cd9e09d86c93e586c56688dc.1708507022.git.baolin.wang@linux.alibaba.com> <0a06dc7f-3a49-42ba-8221-0b4a3777ac0b@linux.alibaba.com> From: Baolin Wang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: p4orxrjowwh93jn899twwn7a8y6ccoem X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B2917A0002 X-HE-Tag: 1708918499-719542 X-HE-Meta: U2FsdGVkX1/idKNMBx5zX6PHBj1Bi/3TlrK0z/xPy6jkzMlqVr8VyZS2pESVSeCjPW1UoG1PC6V7wBbxzNXm23qYLArtoF9WXPhZlnnS48NV7SINchJhrRrc1QTSvrU5f0mOgqNSWQgKOiwWa/K20d39PAdBI7Dvah+VLGlolg5l/kjAblHyWI2eviA1vjrUvCzdy68lO2QAO9+XQi3sYLgHVZ78cQmzRSC+TDaGNpP1fjDh+Q1NwrWeD6uc5WLYDCctuiE3XedHMRJoVLHClIxNHcbe/3dKYYtiFU9PNSUSP1NX59P2k2p6kr4WgfbVJqzOUsFuodXz7BLMiUbsDvOv+01YUiqIx+5bwPnih892sikJxOu+d4lBRVJTFGFUHnfLBwysa6u81Ca/Bk6sXDIp+t2l8rUbUb9ZJAC5D9EFK16adgKjmXlvMJ6ccpdy7ZZFSkZtXeWtr+LAWZoEP8Y4GSDaOopN3P42giS8vIF0CG2hbpPPkhcunZAIex1j4hnHAohYPN8jAX3EiCl4RUnZ9r1WPcSubqo7IaFM6DzssZt3XpKuNIc9/gjOXw12zGqqpWQrpv2+waze1muaF6AgEcKqZgqBGpfU1BiV/EJ2vi6eAAAeBovqrzg7ft9tg8C4BC6kbasDNgjsywN7/5KmCarKsOWySBpvC279aNAw7rGKRrN8mrLLVt/WWLFVLq3DzyqF2eYMqsWvcLSVzBsaFPdhCYo58IXQm6CXeZP8RI0EjPjyy5OvLDnRtiif7lnRSXz/hFHjTJ/PCnsNU3CY1S0SJMlrFdDow+yHcUhgPZ91L7wV6OA+vjriZ/vIk2g5EOiS15Ax0W1YK/DlEVcw1QRiP22poe+IFfb/DxTh1SIDQJRMkgsyNiRfeGMcc8crd3iXtgO29yrb8llaREUT0SvKDoZTtrxy3UMmxc54fjoN1aNuhmKwKFwq0tS7/BNG9VKh9n9+fT0xpC3 k+i9L/G9 CjyagNVCqXoCw2bWtg4FnMdr1VfxCmSGVKhqiB/QK3x+u/1l88knfcpB3h2Ip/wAf1jicCLv4w5S8ZTPJpjHXXdH0wt6bGa9YtGS0DIyd/xnouROJo4F7V6LG2nOpkglevnbf0hi8VIuS5mS+KFZe67EaqpC4kpWa2yjNBxebYAIX6s4LNMpl+wwpYDfCYm4INmNjTrK3/2o6ACAHqnDSUHpCzyJwJuBecrL0DZbFqvcRTWhQP5OJtnU2VOg8uirWObJhBsJBjlxtbmGUrVg6KKFglg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/23 22:19, Oscar Salvador wrote: > On Fri, Feb 23, 2024 at 10:56:48AM +0800, Baolin Wang wrote: > >> I previously considered passing the MR_reason argument to the >> htlb_modify_alloc_mask(), which is only used by hugetlb migration. >> But in alloc_hugetlb_folio_nodemask(), if there are available hugetlb on >> other nodes, we should allow migrating, that will not break the per-node >> hugetlb pool. >> >> That's why I just change the gfp_mask for allocating a new hguetlb when >> migration, that can break the pool. > > Code-wise I think this is good, but I'm having some feelings > about where filter out the mask. > Ok, I'm trying to get my head around this. > It's been a while since I looked into hugetlb code, so here we go. > > You mentioned that the only reason not to fiddle with gfp_mask before calling > in alloc_hugetlb_folio_nodemask(), was that we might be able to find a hugetlb > page in another node, and that that's ok because since all nodes remain with > the same number of hugetlb pages, per-node pool doesn't get broken. > > Now, I see that dequeue_hugetlb_folio_nodemask() first tries to get the zonelist > of the preferred node, and AFAICS, if it has !GFP_THISNODE, it should also > get the zonelists of all other nodes, so we might fallback. Right. > In the hope of finding a way to be able to filter out in htlb_modify_alloc_mask(), > I was trying to see whether we could skip GFP_THISNODE in > dequeue_hugetlb_folio_nodemask() but no because we might end up dequeueing > a hugetlb which sits in another node, while we really specified __GFP_THISNODE. > > The only way might be to somehow decouple dequeue_hugetlb_folio_nodemask() > from alloc_hugetlb_folio_nodemask() and do some kind of gfp modification > between the two calls. IMO, I'm not sure whether it's appropriate to decouple dequeue_hugetlb_folio_nodemask() from alloc_hugetlb_folio_nodemask() into two separate functions for the users to call, because these details should be hidden within the hugetlb core implementation. Instead, I can move the gfp_mask fiddling into a new helper, and move the helper into alloc_migrate_hugetlb_folio(). Temporary hugetlb allocation has its own gfp strategy seems reasonable to me. > Another thing I dislike is the "-1" in alloc_hugetlb_folio_vma(). > I think at least it deserves a comment like "Passing -1 will make us stick > to GFP_THISNODE". Sure, will add some comments. > Although that is another thing, we will pass "-1" which forces GFP_THISNODE > when allocating a newly fresh hugetlb page, but in dequeue_hugetlb_folio_nodemask() > we might get a page from a different node. > That doesn't break per-node pool, but it is somehow odd? Only hugetlb_mfill_atomic_pte() will use -1, which is used to allocate a temporary hugetlb to hold the copied content that will be immediately released if uffd copy completes (see commmit 8cc5fcbb5be8). Therefore, here it is allowed to fallback to available hugetlb on other nodes, but it is not allowed to break the per-node pool.