From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D93B4C5475B for ; Wed, 6 Mar 2024 03:55:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4ECE16B007D; Tue, 5 Mar 2024 22:55:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 475E86B0080; Tue, 5 Mar 2024 22:55:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C8446B0083; Tue, 5 Mar 2024 22:55:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 154046B007D for ; Tue, 5 Mar 2024 22:55:24 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4072F40493 for ; Wed, 6 Mar 2024 03:55:23 +0000 (UTC) X-FDA: 81865249326.02.3E36AA8 Received: from mail-ua1-f54.google.com (mail-ua1-f54.google.com [209.85.222.54]) by imf28.hostedemail.com (Postfix) with ESMTP id A39C6C0006 for ; Wed, 6 Mar 2024 03:55:19 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="jPNd7a/t"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf28.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709697320; a=rsa-sha256; cv=none; b=Q/DdXjYYsTDmrlisFgOw2Cv53D/kUqwZVudUgRdX84WHNU76he8YfKRUpjfmDkZCRCX1aw FaO+/tiX9yLc9avVnr/HpPyEj2UwJXuB7IYgsVBCwI0g3Y5WmaQ4jdTo885Jd9F81lUt6v iuWv4IOE7bPwC3qgRLow2DGvpHvLABQ= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="jPNd7a/t"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf28.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709697320; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uztHPh5m6/k/YQgSozU3xl+sBK6noaMrM17GF/+3NFs=; b=f1Jh3YZ85XywMt5jIrKXKM6FzVAciKCJzIw7osAsJd0ImMKmR9P31mgwij16XV8Iz5qY2f hMkqyr1mHiVZlPN0zPtF2Ee8AC6b7wPe7r0ESOOGTT4Dkn+lDhfP/xLLNh7jr7SE0aiqsT qXGnR8Bb3ptyEcKAytzriHj7/+RHAqU= Received: by mail-ua1-f54.google.com with SMTP id a1e0cc1a2514c-7da763255b7so2417615241.2 for ; Tue, 05 Mar 2024 19:55:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709697318; x=1710302118; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uztHPh5m6/k/YQgSozU3xl+sBK6noaMrM17GF/+3NFs=; b=jPNd7a/tECN4ig8Rmahcgbe4j7PmwRSyVg54ZuLus6vmERN4kTnd3PNXLrsy1I3I+A koBRMYkPhhfeJy2PBpuX18qU7PGgs56JihvOzOsYj3mOSvQ+ID5I4wupqp0sPwSsIaYI HeJSViVz5l+gP8ALWjDjkNyshm4o/gF1dLXV5IVK+w24gstpeGNPucL8KPkkHnYfilAW 1S5Jp9Te2PM3pE/eNGi1VTM98MCpi5gL4mIO1YtYUg2Zoy3FpmnliSTXkj2LIiGUdlwt SrW0mc21lcGpjr3kSQw7uQTYlYs06KPSYc9tS+mYMHrbDSkhldqd7slAN29IuNod1Xv7 NknA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709697318; x=1710302118; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uztHPh5m6/k/YQgSozU3xl+sBK6noaMrM17GF/+3NFs=; b=EAiZFXTAPdfbbtzn/mT9gMeCUc/vKSCl7OyUsd/h1H91MFJAn8Ayw/ICl2mMHqOnPm maItAsY16057nXtUu9KSi+eP7dpPIs2vft375kcbRdjKEaJHqeIs1IIyimKdTT03uZ7g CnWBUY0VHPKDlDAwNSls0hRxlPd4SNbz7O48maPboEqPr3MOazZiqzs45GYuhDQTRa8r jbO5NIAqcLGzj7OWmQm1Ab5CXABzQdymEQZq7GLLAeN6f65VL+vBalpS5tzRyImhX77e u07gUx726sH8rKGjr9+SVX9QDOXAET9LnE9dP4y/W0ZbGg+4uXpChZ2s3tDfHlk1wiMJ OwmQ== X-Forwarded-Encrypted: i=1; AJvYcCXp8Z0O4XhckDPyG86z/Of2rPsf2t/QiqQIPN2FhfxUOPT3XcUmbkmwGTi5ESqiqEmuQuIfl8u7Nrv5NiUNxcGbYSM= X-Gm-Message-State: AOJu0YwbIh5Ugmg6NxHzzlUEDnsyC8USxM6S1g65eH5rBPtwENxjHmnd ZCF/tB6MRmwXCZ+6Alh7H1qM+5whW7cMPg15lxva7MEK109B/n37VmOz1+Uy72zceVdu/yCO8gq 8i6tINz5bnhE6y0q1B4VU9ke9ITCSA5bbVdChIg== X-Google-Smtp-Source: AGHT+IFL4Fnwe2jyRS6YBPBmNcPht468oprH/h2dgRAKYEKhGBH97gw91noZEOItrRp8dylLlQ8oLiX9iDiaYDigGIw= X-Received: by 2002:a05:6102:3bcf:b0:472:eca1:46c with SMTP id a15-20020a0561023bcf00b00472eca1046cmr721701vsv.30.1709697318414; Tue, 05 Mar 2024 19:55:18 -0800 (PST) MIME-Version: 1.0 References: <20240301082248.3456086-1-horenchuang@bytedance.com> <20240301082248.3456086-2-horenchuang@bytedance.com> <87jzmibtyh.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edco85b4.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87edco85b4.fsf@yhuang6-desk2.ccr.corp.intel.com> From: "Ho-Ren (Jack) Chuang" Date: Tue, 5 Mar 2024 19:55:07 -0800 Message-ID: Subject: Re: [External] Re: [PATCH v1 1/1] memory tier: acpi/hmat: create CPUless memory tiers after obtaining HMAT info To: "Huang, Ying" Cc: Gregory Price , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, Eishan Mirakhur , Vinicius Tavares Petrucci , Ravis OpenSrc , Alistair Popple , "Rafael J. Wysocki" , Len Brown , Andrew Morton , Dave Jiang , Dan Williams , Jonathan Cameron , linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , linux-cxl@vger.kernel.org, qemu-devel@nongnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A39C6C0006 X-Stat-Signature: a7fm8a9cmg4mqwscampoib9fpz4i56f8 X-HE-Tag: 1709697319-334342 X-HE-Meta: U2FsdGVkX18cuUPv2o2aiBnjW7kY10mYsqJdGCNOuTv6SXFSeSvzKwySGkzdwi4EA4e/6d63PEBk28ur/Z7/Szn0sPb/aWSZX26xIdVHMAICb8wdQRDYXPZbacxCw0NO2ir+Uo/hpWD/6MAPAOWbAXaT7wPkCjaclq6nBo5ETEwItXvCr19ipJ0NOURcStk8f+h1XOSbcguaU2vkLIDddwk9fwPqMgG5V4fmDGE2PJCHHHGjeTpbl9s++xT03FNOkihjhno1+17s5ToyUMbAPMhDyiPQfzfmDjNwA8VBq627Od8DXTI46Uk4/v0ubCxYgY0zi97ZJbmUBN/irZZVu17H/Ghh0yqe3dshTJQ6vr3ovS5q2hr/gcOY8ynLPWq5vgoRnL4daiu9Ui+pcyG0pl+PN3EYBbQJyLzFE575LidA9wqb0KaTWj5t6QckqJh1LwcZBfjEdLzzoakbjwA04IfQoWRZcBLTA+MR/53w3qW8tmY4O5ebGlFYluCIw6khrhFBOuKo6QzMfDnt5NvMM7wbBos+LFFTWN/G4tcAhNZu2Xh0deZF2XcyTtzaawHJuvl06fiI85GBXqvVOgnTk2k1pRa0PW4UkEV/bv8462NOJIW4h3fDBPILUTPHsY8tyYxoCAxfTRL/GzX5Jtyr2motfzvuksyXAvVLjYQBTmB3PNRJKQKeHLgRAcAMwj3lLhqftGQTCfJQV6FUdjxaVJLF3hePWFXTyGXAGEHIqdT4mXpxGQhd00Am360bt90/d/pA1wdurKreihqt76SrZbq9ncQHn3wS15ARdt7MfCAYXYLfqg3uQRolM4h0JGPHgMNkjUCFtMe4DMQZrVjFRgx6DbfypOn2FlMW+DRTS/SWqGry6NbOfX0RZptF9Ds0gqzNNVKxw+FLAwoXjLLtzNrG4g+FNwKSQztJ/ZETAQAgRpgH1NJ0eDGklfc61BWW1Rg680G41DLkIc5MTNu Szv6w9ld sQp3qOngZAaXoZzsNGY07AmOpfRIj8eKx42FZAuopi3kRdq70PVD41+3ZpyTOYjQUGC7589izVe9xk7pYvYw6GLC00Z2PNhHTxAqksN7Z3hQheZDkdvUqzF5kLklpI3FM07Ns4KIp7kXF823h/Ci55jgwBnv7699ykR1JyThzvrmRWfJwqN3qM0HozPpPiWMrUamj+DYerzzxs8dyMyA58R+mFvRgwqTBP5gIoXg5i35Z/zdGHtI9pMYi8fhxBe7iNr/mrwknbIMw0ame/lEicaeysJHdDcgaZxlhl63iE+ysRuFo7wvpkVHXYX6pyyxREDDypKSXcwipzvb0PpQ9Ls2NuBrHmTmhafUgcDPid8O1R7bu0Epif8fAPCc9th7P3KiHwT9mCxbP0FI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Mar 5, 2024 at 6:27=E2=80=AFPM Huang, Ying w= rote: > > "Ho-Ren (Jack) Chuang" writes: > > > On Sun, Mar 3, 2024 at 6:42=E2=80=AFPM Huang, Ying wrote: > >> > >> Hi, Jack, > >> > >> "Ho-Ren (Jack) Chuang" writes: > >> > >> > * Introduce `mt_init_with_hmat()` > >> > We defer memory tier initialization for those CPUless NUMA nodes > >> > until acquiring HMAT info. `mt_init_with_hmat()` is introduced to > >> > post-create CPUless memory tiers after obtaining HMAT info. > >> > It iterates through each CPUless memory node, creating memory tiers = if > >> > necessary. Finally, it calculates demotion tables again at the end. > >> > > >> > * Introduce `hmat_find_alloc_memory_type()` > >> > Find or allocate a memory type in the `hmat_memory_types` list. > >> > > >> > * Make `set_node_memory_tier()` more generic > >> > This function can also be used for setting other memory types for a = node. > >> > To do so, a new argument is added to specify a memory type. > >> > > >> > * Handle cases where there is no HMAT when creating memory tiers > >> > If no HMAT is specified, it falls back to using `default_dram_type`. > >> > > >> > * Change adist calculation code to use another new lock, mt_perf_loc= k. > >> > Iterating through CPUlist nodes requires holding the `memory_tier_lo= ck`. > >> > However, `mt_calc_adistance()` will end up trying to acquire the sam= e lock, > >> > leading to a potential deadlock. Therefore, we propose introducing a > >> > standalone `mt_perf_lock` to protect `default_dram_perf`. This appro= ach not > >> > only avoids deadlock but also prevents holding a large lock simultan= eously. > >> > >> The patch description is used to described why we need the change, and > >> how we do that, but not what we do. People can tell what is done from > >> the code itself. > >> > > > > Got it. Thanks. Will rewrite it after the code is finalized. > > > >> > Signed-off-by: Ho-Ren (Jack) Chuang > >> > Signed-off-by: Hao Xiang > >> > --- > >> > drivers/acpi/numa/hmat.c | 3 ++ > >> > include/linux/memory-tiers.h | 6 +++ > >> > mm/memory-tiers.c | 76 ++++++++++++++++++++++++++++++++-= --- > >> > 3 files changed, 77 insertions(+), 8 deletions(-) > >> > > >> > diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c > >> > index d6b85f0f6082..9f57338b3cb5 100644 > >> > --- a/drivers/acpi/numa/hmat.c > >> > +++ b/drivers/acpi/numa/hmat.c > >> > @@ -1038,6 +1038,9 @@ static __init int hmat_init(void) > >> > if (!hmat_set_default_dram_perf()) > >> > register_mt_adistance_algorithm(&hmat_adist_nb); > >> > > >> > + /* Post-create CPUless memory tiers after getting HMAT info */ > >> > + mt_init_with_hmat(); > >> > + > >> > return 0; > >> > out_put: > >> > hmat_free_structures(); > >> > diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tie= rs.h > >> > index 69e781900082..2f845e90c033 100644 > >> > --- a/include/linux/memory-tiers.h > >> > +++ b/include/linux/memory-tiers.h > >> > @@ -48,6 +48,7 @@ int mt_calc_adistance(int node, int *adist); > >> > int mt_set_default_dram_perf(int nid, struct access_coordinate *per= f, > >> > const char *source); > >> > int mt_perf_to_adistance(struct access_coordinate *perf, int *adist= ); > >> > +void mt_init_with_hmat(void); > >> > >> HMAT isn't universally available. It's a driver in fact. So, don't p= ut > >> driver specific code in general code. > >> > > > > Please see below regarding "move code to hmat.c" > > > >> > #ifdef CONFIG_MIGRATION > >> > int next_demotion_node(int node); > >> > void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets= ); > >> > @@ -136,5 +137,10 @@ static inline int mt_perf_to_adistance(struct a= ccess_coordinate *perf, int *adis > >> > { > >> > return -EIO; > >> > } > >> > + > >> > +static inline void mt_init_with_hmat(void) > >> > +{ > >> > + > >> > +} > >> > #endif /* CONFIG_NUMA */ > >> > #endif /* _LINUX_MEMORY_TIERS_H */ > >> > diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c > >> > index 0537664620e5..7a0a579b3deb 100644 > >> > --- a/mm/memory-tiers.c > >> > +++ b/mm/memory-tiers.c > >> > @@ -35,7 +35,9 @@ struct node_memory_type_map { > >> > }; > >> > > >> > static DEFINE_MUTEX(memory_tier_lock); > >> > +static DEFINE_MUTEX(mt_perf_lock); > >> > static LIST_HEAD(memory_tiers); > >> > +static LIST_HEAD(hmat_memory_types); > >> > static struct node_memory_type_map node_memory_types[MAX_NUMNODES]; > >> > struct memory_dev_type *default_dram_type; > >> > > >> > @@ -502,7 +504,7 @@ static inline void __init_node_memory_type(int n= ode, struct memory_dev_type *mem > >> > } > >> > } > >> > > >> > -static struct memory_tier *set_node_memory_tier(int node) > >> > +static struct memory_tier *set_node_memory_tier(int node, struct me= mory_dev_type *new_memtype) > >> > >> No. memory_dev_type are passed to the function via node_memory_types[n= ode].memtype. > >> > > > > Got it. Will mimic the way kmem.c does. Thanks. > > > >> > { > >> > struct memory_tier *memtier; > >> > struct memory_dev_type *memtype; > >> > @@ -514,7 +516,7 @@ static struct memory_tier *set_node_memory_tier(= int node) > >> > if (!node_state(node, N_MEMORY)) > >> > return ERR_PTR(-EINVAL); > >> > > >> > - __init_node_memory_type(node, default_dram_type); > >> > + __init_node_memory_type(node, new_memtype); > >> > > >> > memtype =3D node_memory_types[node].memtype; > >> > node_set(node, memtype->nodes); > >> > @@ -623,6 +625,56 @@ void clear_node_memory_type(int node, struct me= mory_dev_type *memtype) > >> > } > >> > EXPORT_SYMBOL_GPL(clear_node_memory_type); > >> > > >> > +static struct memory_dev_type *hmat_find_alloc_memory_type(int adis= t) > >> > >> Similar function existed in drivers/dax/kmem.c. Please abstract them > >> and move them here. > >> > > > > Got it. Will try. Thanks. > > > >> > +{ > >> > + bool found =3D false; > >> > + struct memory_dev_type *mtype; > >> > + > >> > + list_for_each_entry(mtype, &hmat_memory_types, list) { > >> > + if (mtype->adistance =3D=3D adist) { > >> > + found =3D true; > >> > + break; > >> > + } > >> > + } > >> > + if (!found) { > >> > + mtype =3D alloc_memory_type(adist); > >> > + if (!IS_ERR(mtype)) > >> > + list_add(&mtype->list, &hmat_memory_types); > >> > + } > >> > + return mtype; > >> > +} > >> > + > >> > +static void mt_create_with_hmat(int node) > >> > +{ > >> > + struct memory_dev_type *mtype =3D NULL; > >> > + int adist =3D MEMTIER_ADISTANCE_DRAM; > >> > + > >> > + mt_calc_adistance(node, &adist); > >> > + if (adist !=3D MEMTIER_ADISTANCE_DRAM) { > >> > + mtype =3D hmat_find_alloc_memory_type(adist); > >> > + if (IS_ERR(mtype)) > >> > + pr_err("%s() failed to allocate a tier\n", __f= unc__); > >> > + } else { > >> > + mtype =3D default_dram_type; > >> > + } > >> > + > >> > + set_node_memory_tier(node, mtype); > >> > +} > >> > + > >> > +void mt_init_with_hmat(void) > >> > +{ > >> > + int nid; > >> > + > >> > + mutex_lock(&memory_tier_lock); > >> > + for_each_node_state(nid, N_MEMORY) > >> > + if (!node_state(nid, N_CPU)) > >> > + mt_create_with_hmat(nid); > >> > + > >> > + establish_demotion_targets(); > >> > + mutex_unlock(&memory_tier_lock); > >> > +} > >> > +EXPORT_SYMBOL_GPL(mt_init_with_hmat); > >> > + > >> > >> I guess that we can put most hmat related code above in hmat.c. > >> > > > > To put the heat-related code to hmat.c I will need to export some > > static functions in memory-tiers.c, like set_node_memory_tier() and > > establish_demotion_targets(). Is that ok? > > Think about this again. In fact, although there are "_hmat" in the name > of the above functions, the code has nothing to do with hmat. So, we > should rename these functions, but don't need to move them to hmat.c. > And, to set memory_tier for CPUless node on system without HMAT. We > should call mt_init_with_hmat() with late_initcall(). Where HMAT > information is expected to be available on system with HMAT. On system > without HMAT, default_dram_type will be used. > Sounds good and thank you! I'm working on v2 according to the feedback. > >> > static void dump_hmem_attrs(struct access_coordinate *coord, const = char *prefix) > >> > { > >> > pr_info( > >> > @@ -636,7 +688,7 @@ int mt_set_default_dram_perf(int nid, struct acc= ess_coordinate *perf, > >> > { > >> > int rc =3D 0; > >> > > >> > - mutex_lock(&memory_tier_lock); > >> > + mutex_lock(&mt_perf_lock); > >> > if (default_dram_perf_error) { > >> > rc =3D -EIO; > >> > goto out; > >> > @@ -684,7 +736,7 @@ int mt_set_default_dram_perf(int nid, struct acc= ess_coordinate *perf, > >> > } > >> > > >> > out: > >> > - mutex_unlock(&memory_tier_lock); > >> > + mutex_unlock(&mt_perf_lock); > >> > return rc; > >> > } > >> > > >> > @@ -700,7 +752,7 @@ int mt_perf_to_adistance(struct access_coordinat= e *perf, int *adist) > >> > perf->read_bandwidth + perf->write_bandwidth =3D=3D 0) > >> > return -EINVAL; > >> > > >> > - mutex_lock(&memory_tier_lock); > >> > + mutex_lock(&mt_perf_lock); > >> > /* > >> > * The abstract distance of a memory node is in direct proport= ion to > >> > * its memory latency (read + write) and inversely proportiona= l to its > >> > @@ -713,7 +765,7 @@ int mt_perf_to_adistance(struct access_coordinat= e *perf, int *adist) > >> > (default_dram_perf.read_latency + default_dram_perf.wr= ite_latency) * > >> > (default_dram_perf.read_bandwidth + default_dram_perf.= write_bandwidth) / > >> > (perf->read_bandwidth + perf->write_bandwidth); > >> > - mutex_unlock(&memory_tier_lock); > >> > + mutex_unlock(&mt_perf_lock); > >> > > >> > return 0; > >> > } > >> > @@ -797,7 +849,7 @@ static int __meminit memtier_hotplug_callback(st= ruct notifier_block *self, > >> > break; > >> > case MEM_ONLINE: > >> > mutex_lock(&memory_tier_lock); > >> > - memtier =3D set_node_memory_tier(arg->status_change_ni= d); > >> > + memtier =3D set_node_memory_tier(arg->status_change_ni= d, default_dram_type); > >> > if (!IS_ERR(memtier)) > >> > establish_demotion_targets(); > >> > mutex_unlock(&memory_tier_lock); > >> > @@ -836,7 +888,15 @@ static int __init memory_tier_init(void) > >> > * types assigned. > >> > */ > >> > for_each_node_state(node, N_MEMORY) { > >> > - memtier =3D set_node_memory_tier(node); > >> > + if (!node_state(node, N_CPU)) > >> > + /* > >> > + * Defer memory tier initialization on CPUless= numa nodes. > >> > + * These will be initialized when HMAT informa= tion is > >> > + * available. > >> > + */ > >> > + continue; > >> > + > >> > + memtier =3D set_node_memory_tier(node, default_dram_ty= pe); > >> > >> On system with HMAT, how to fall back CPU-less node to > >> default_dram_type? I found your description, but I don't find it in c= ode. > >> > > I assume you meant without HMAT, if so, > > because if no HMAT, adist will not be updated in mt_calc_adistance(): > > Sorry, typo. On system with HMAT, mt_init_with_hmat() will not be > called. I have a solution in above comment. > > -- > Best Regards, > Huang, Ying > > > + int adist =3D MEMTIER_ADISTANCE_DRAM; > > + > > + mt_calc_adistance(node, &adist); > > + if (adist !=3D MEMTIER_ADISTANCE_DRAM) { > > =E2=80=A6 > > + } else { > > + mtype =3D default_dram_type; > > + } > > + > > + set_node_memory_tier(node, mtype); > > > >> > if (IS_ERR(memtier)) > >> > /* > >> > * Continue with memtiers we are able to setup > >> > >> -- > >> Best Regards, > >> Huang, Ying --=20 Best regards, Ho-Ren (Jack) Chuang =E8=8E=8A=E8=B3=80=E4=BB=BB