From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E437C47DD9 for ; Wed, 28 Feb 2024 08:39:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BEEB46B02B0; Wed, 28 Feb 2024 03:39:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B9E586B02B2; Wed, 28 Feb 2024 03:39:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8DFB6B02B5; Wed, 28 Feb 2024 03:39:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 98F2B6B02B0 for ; Wed, 28 Feb 2024 03:39:54 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6862F80F45 for ; Wed, 28 Feb 2024 08:39:54 +0000 (UTC) X-FDA: 81840564708.07.C7148B8 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf02.hostedemail.com (Postfix) with ESMTP id 49F5580004 for ; Wed, 28 Feb 2024 08:39:52 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xf0qsAU1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="9rw8ia/v"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xf0qsAU1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="9rw8ia/v"; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709109592; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WyQQpRkr/dgNcbsO1eOjVkFSW+jjxrbD60UrCSG6L7A=; b=Q8KhWws675Nphu2kIEPz6Xf5gqOfGbhsSL2spW3tId0bILlgaUXmkAJ68WTygHj3KIAJoT 2S5g/vQVbgXmFF3VUDVLQWIPyYeUpRN980rbGYv7NNY6WlfPHBG3qZaHGF0lhtzhUZ4Kt9 3KGXVgIJYp18lFtksSzUeKeVle7iOCk= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xf0qsAU1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="9rw8ia/v"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xf0qsAU1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="9rw8ia/v"; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709109592; a=rsa-sha256; cv=none; b=Z07XYzd5S7J7kSg+hSG7fxNdqFUWuEF8+9dYrd2Pv41yMSknvz0Km3KPRICwjxkBkk1sW1 ikJqPdUx1XEv3vS93yT9xNMvGzaqpwMF3BjSPVKC0483yvN9daTkPgcl7PnpbLsDyOi47V c5t/bc5id19z1SDMX0HyKWWIizEtxdg= Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:98]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6CDA1226B4; Wed, 28 Feb 2024 08:39:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1709109590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WyQQpRkr/dgNcbsO1eOjVkFSW+jjxrbD60UrCSG6L7A=; b=xf0qsAU1igRFXb9tEENxIyAjwHPec+aS+fIcinAzSwIoDlTMt16ZSbPZU+4PdTKWUV5sZj 6cvnHDR5nZKIG2bIUEbDTr5JnhCafEYUef60DyQs2dR9QBrzTp2okoCyyPALYkVwBJye7k DORULyfPeTy7461r9CmH5bLRg9+Q22g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1709109590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WyQQpRkr/dgNcbsO1eOjVkFSW+jjxrbD60UrCSG6L7A=; b=9rw8ia/vXSxgrWnVtJr62ecULnaSFnU67U76L5iLr/T22hwwhy6PetNREl91eVn5L0CsVi aPah4C2nzab1upDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1709109590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WyQQpRkr/dgNcbsO1eOjVkFSW+jjxrbD60UrCSG6L7A=; b=xf0qsAU1igRFXb9tEENxIyAjwHPec+aS+fIcinAzSwIoDlTMt16ZSbPZU+4PdTKWUV5sZj 6cvnHDR5nZKIG2bIUEbDTr5JnhCafEYUef60DyQs2dR9QBrzTp2okoCyyPALYkVwBJye7k DORULyfPeTy7461r9CmH5bLRg9+Q22g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1709109590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WyQQpRkr/dgNcbsO1eOjVkFSW+jjxrbD60UrCSG6L7A=; b=9rw8ia/vXSxgrWnVtJr62ecULnaSFnU67U76L5iLr/T22hwwhy6PetNREl91eVn5L0CsVi aPah4C2nzab1upDQ== Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id C65B513A42; Wed, 28 Feb 2024 08:39:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([10.150.64.162]) by imap2.dmz-prg2.suse.org with ESMTPSA id C7eKLVXx3mXSRAAAn2gu4w (envelope-from ); Wed, 28 Feb 2024 08:39:49 +0000 Date: Wed, 28 Feb 2024 09:41:02 +0100 From: Oscar Salvador To: Baolin Wang Cc: akpm@linux-foundation.org, muchun.song@linux.dev, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/3] mm: hugetlb: make the hugetlb migration strategy consistent Message-ID: References: <8d35b8ae-b8d8-4237-bfcf-ed63c0bb4223@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8d35b8ae-b8d8-4237-bfcf-ed63c0bb4223@linux.alibaba.com> X-Rspamd-Queue-Id: 49F5580004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: qfdqpr1qb6cta1d1w7utcq99jd8i53zg X-HE-Tag: 1709109592-521344 X-HE-Meta: U2FsdGVkX199AEgtejYtzGAle78Ykw2lVCRR02VNMowGDz1ESyErweybA7EaW+p5r5SVM5zXQxUMyC+Tqd26Sax2QD54ZKf9SNOGWfJzgqlGcJ4Wxqb0dB23LMqqGL+uwm2eCeTIPROoaemHkDinMJmr2tCU8OWxOTUnxw0KCzhWAG8Y3yLCucQMlVymrxhL8a+oHxGuJ1xy+1ApdUl+pmREekSrcIFndwSy4eQZuPixj1yGphbWY0OA0MiS+kpK6w6370vKviPqrEyj/eTXgftdmf1j1ddEY+KnSSllO2EW862yk1o7AxGSieYN/Izs77kIrp4Fjw1pOporjl6CUpehj97d0JbG4lP7TpT4EX9kkgsSHGpsMU6xOuLBQOF+Op3v3HDMg3WAeCbMfiG7RYK0JbIUlrSCScF7FU/NlnH7cFTR+JicTRIDVql416raysCoqGUdBArXPZPPOBWjI7HRwWtrB8wR4nr412I1/22FucJ3zhoh4M6hhfEqc1DpjEAzO/02MmPrASeKzW3sGk2/OUzWIa//BzO8qB/1NcmZB14emPwjWAtrt4Nwjg1e1DpSwDNDEqzaZSrKUpBkowgN1uRYVMluCHe6b5VPiAavMHHJMGouTmB2K52f3EBMRrPFHXoieaNH1xtmYeBdwXHq8/Kv30QbqA6JhpjxNCISnU7s12SIxvAsh0r8Ip2DPvnXGS57W4AH/anjHezo/bsZsOiLLv+YkNzFYpNGN/XVTQl9d+K5qMA4pzdbWZqAbY/yTWJVqygBXBZ/nQddYl3Dj2Et/ISvUjli7VtxeoafxLNxbG8ULJeCS/tUSSsiShPTczBSFMlPAXwlprsoPl19sviJnutEfVme94rm9ryQu5DdnJcIIANF9WFrXvS4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 28, 2024 at 03:40:08PM +0800, Baolin Wang wrote: > > > On 2024/2/27 23:17, Oscar Salvador wrote: > > On Tue, Feb 27, 2024 at 09:52:26PM +0800, Baolin Wang wrote: > > > > > --- a/mm/hugetlb.c > > > +++ b/mm/hugetlb.c > > > @@ -2567,13 +2567,38 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, > > > } > > > static struct folio *alloc_migrate_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, > > > - int nid, nodemask_t *nmask) > > > + int nid, nodemask_t *nmask, int reason) > > > > I still dislike taking the reason argument this far, and I'd rather have > > this as a boolean specifing whether we allow fallback on other nodes. > > That would mean parsing the reason in alloc_migration_target(). > > If we don't add a new helper e.g: gfp_allow_fallback(), we can just do > > it right there an opencode it with a e.g: macro etc. > > > > Although doing it in an inline helper might help hiding these details. > > > > That's my take on this, but let's see what others have to say. > > Sure. I also expressed my preference for hiding these details within the > hugetlb core as much as possible. > > Muchun, what do you think? Thanks. JFTR: I'm talking about https://lore.kernel.org/linux-mm/ZdxXLTDZn8fD3pEn@localhost.localdomain/ or maybe something cleaner which doesn't need a new helper (we could if we want though): diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index c1ee640d87b1..ddd794e861e6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -73,6 +73,16 @@ struct resv_map { #endif }; +#define MIGRATE_MEMORY_HOTPLUG 1UL << MR_MEMORY_HOTPLUG +#define MIGRATE_MEMORY_FAILURE 1UL << MR_MEMORY_FAILURE +#define MIGRATE_SYSCALL 1UL << MR_SYSCALL +#define MIGRATE_MBIND 1UL << MR_MEMPOLICY_MBIND +#define HTLB_ALLOW_FALLBACK (MIGRATE_MEMORY_HOTPLUG| \ + MIGRATE_MEMORY_FAILURE| \ + MIGRATE_SYSCALL| \ + MIGRATE_MBIND) + + /* * Region tracking -- allows tracking of reservations and instantiated pages * across the pages in a mapping. diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ed1581b670d4..7e8d6b5885d6 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2619,7 +2619,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, /* folio migration callback function */ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool allow_fallback) { spin_lock_irq(&hugetlb_lock); if (available_huge_pages(h)) { @@ -2634,6 +2634,12 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, } spin_unlock_irq(&hugetlb_lock); + /* + * We cannot fallback to other nodes, as we could break the per-node pool + */ + if (!allow_fallback) + gfp_mask |= GFP_THISNODE; + return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask); } diff --git a/mm/migrate.c b/mm/migrate.c index cc9f2bcd73b4..c1f1d011629d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2016,10 +2016,15 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private) if (folio_test_hugetlb(src)) { struct hstate *h = folio_hstate(src); + bool allow_fallback = false; + + if ((1UL << reason) & HTLB_ALLOW_FALLBACK) + allow_fallback = true; gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); return alloc_hugetlb_folio_nodemask(h, nid, - mtc->nmask, gfp_mask); + mtc->nmask, gfp_mask, + allow_fallback); } if (folio_test_large(src)) { -- Oscar Salvador SUSE Labs