From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCFAAC54E58 for ; Wed, 20 Mar 2024 23:28:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B992E6B007B; Wed, 20 Mar 2024 19:28:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B487C6B0082; Wed, 20 Mar 2024 19:28:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E9986B0083; Wed, 20 Mar 2024 19:28:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8D2896B007B for ; Wed, 20 Mar 2024 19:28:15 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 51DB614109D for ; Wed, 20 Mar 2024 23:28:15 +0000 (UTC) X-FDA: 81919008150.18.4AEDA71 Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) by imf26.hostedemail.com (Postfix) with ESMTP id 0446C140010 for ; Wed, 20 Mar 2024 23:28:11 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=J0+5FE4k; spf=pass (imf26.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710977292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zJzBq5JYuB2XAGJ0KALCMa2K2ac89AEZaVfd/fyT9P0=; b=XGRV+T0cJYv6YiNCtoKJWHrPJ+xUOuyY4/7J1Ia4JfOpMf/GTLatZXW9SXg+JTAaTRM6qe IGS156sc2bljd9sIwNm6wtQa3wCAUFyxK58urpbgn+bGLGSq/B6WlI1hlB39mRISnCX6wU 9vy9AanMPWC8KoelY6r8aAP1LVZlxVU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=J0+5FE4k; spf=pass (imf26.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710977292; a=rsa-sha256; cv=none; b=k81tv7JNqDRFJg1QYZKKayahv72SX2l1VV4TelCNQOllVAYB/bsCZAUWrXZaXPTXWc65L/ j8y3Xgzm4fEVZJmNtjnfY3ZjdmXmSjAo/do0+rUYTc+07sIYNl420bdcu1RF3hZ89oFKgb 5NZDxP1NWcJ+VNbDGymznOuFuHi2Qdk= Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-dc742543119so377151276.0 for ; Wed, 20 Mar 2024 16:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1710977291; x=1711582091; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zJzBq5JYuB2XAGJ0KALCMa2K2ac89AEZaVfd/fyT9P0=; b=J0+5FE4kubkc8pcgNDtLhIlSLc5JVnpkKaN/x2AKybruRzfc+pSaEmLahNloSFlrm2 KSRK8kBo+YjXLSJpFh98dih0q40duXbd37Nfx2umiHd6xKAssAp3cT1nszGFuE9u2mB5 anYPJHuhp6uoqruglMlAMSoueU7q6LktEgak/43rW+N4typGJon0aGeTLV+gyYYusMeV HufJM/9nWH+5abXTP9NxXn4Iz9fltP7JeL/NQV/X+OLZF6xqsNkN1q51eATicZjjXmH3 zVNvmDw2Kt6m81CdJwqRCI1k1AXKcGbOpsDQbT4Izaag99JQH9I4Y+pRWVN5Ektm1ssE Cb9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710977291; x=1711582091; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zJzBq5JYuB2XAGJ0KALCMa2K2ac89AEZaVfd/fyT9P0=; b=d3zISz3QGX/lzv0cjQOF7COnEBNVQoS8cM2xflQOlx5XGUTCPghwQfFhFXqGOVH0GR DTg8T6gH5tw//wJQCo8uIrfBQcu/wO9Ex/GwiTTpbzcEWKb55cHTL81GggfJCf8hLeBl 4uYNPdkeRmX17gah87b5iPuPxTo9/9CrW4PncHiV7dU56QZmdQXKpWmhZkgXgzk51yfG upMg47HVr2xuKTbef98B29m0EY4KvJb2szcy3NIIH+9i5VO5Hs0SJB6vBCP4qnKGv2US eL064yHLYWltIIi+mL/N/EYDVlpwvKmIIVEuS9y+v8ODHbE7t3qgVt1crXOKbRow5lhL vrOA== X-Forwarded-Encrypted: i=1; AJvYcCV1jtcoomN3BT1HIlW/ZoeAU+iZ0Zy8vJ9FhFAraWs5wkKcAbqEAcjztjDP3i2O/cP4tkpakltd+mJa4EfJYU9avR8= X-Gm-Message-State: AOJu0Yx0FE7f9Bur09LXXvkAOxEr3rp5FC7OAu74DnjanrT2hYuqXzIr xt6GG/UGFmI4QM4qd0GxLve5DiyR/lTRy44/fmJrD68aBn/E8vUd/F455UCPQAzTpvDWwxDmZmT 5P6sFFsWUQSNgXTlIM7vFteKMPr+fbBcjZcGK8w== X-Google-Smtp-Source: AGHT+IFGJliSrDOUQ4ErqYoYgEzAjB2StnHNHTXjgXve2MnBnRyAPXbxRVaPnc6fOZEVfVxO8SzIJ0MjV2hraKOtNuk= X-Received: by 2002:a25:fe03:0:b0:dd1:517b:571d with SMTP id k3-20020a25fe03000000b00dd1517b571dmr314816ybe.16.1710977290644; Wed, 20 Mar 2024 16:28:10 -0700 (PDT) MIME-Version: 1.0 References: <20240320061041.3246828-1-horenchuang@bytedance.com> <20240320061041.3246828-2-horenchuang@bytedance.com> <87edc5s7ea.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87edc5s7ea.fsf@yhuang6-desk2.ccr.corp.intel.com> From: "Ho-Ren (Jack) Chuang" Date: Wed, 20 Mar 2024 16:27:59 -0700 Message-ID: Subject: Re: [External] Re: [PATCH v3 1/2] memory tier: dax/kmem: create CPUless memory tiers after obtaining HMAT info To: "Huang, Ying" Cc: Gregory Price , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, Eishan Mirakhur , Vinicius Tavares Petrucci , Ravis OpenSrc , Alistair Popple , Srinivasulu Thanneeru , Dan Williams , Vishal Verma , Dave Jiang , Andrew Morton , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , qemu-devel@nongnu.org, Hao Xiang Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 0446C140010 X-Rspam-User: X-Stat-Signature: o9nmfaefbfgd79zuk545zd11wd4m7iuu X-Rspamd-Server: rspam01 X-HE-Tag: 1710977291-514461 X-HE-Meta: U2FsdGVkX19miei/Th7Zo7kw19LLV51yi6X38FqwSaZ2t37vi0HYbvS21wIhK2HWWvi7pDBu7qrXKk62rNVPSrgck6SchySin5lV+QIAXxEVnGEQ63ot0fqFDdRE1Ze4nvXI8m+fT6jCkaWvg1YGTNtdIEMAH8lxLfvUi7EdYdzNZGIlIbXrjHH4MyAF7yDSKslDn8ItqLffBcZgn2hrGq8ftqfPwpNjVjLSTOFWvLHIrdL2/jVlura9Jp5kMV7OHeF/986m64muJVq8ZjSuYWGLyGFLm8ZXUCMM/m8ho15QzJu7po99y0CZzl+C0f1oVXuuq3XbGZ5NXQn91VKhHEAcwFti5Cpno2w2FVoqvQAuJOiJtijvFoSfnLHUc3TRc2mZanX/dWHDs1aibM0watUzyUO89FnmWso6UUA7mbtrs0S7B0GItl3TBCAokbJ5myd/LTeCdtSkfxOBW5+Jmi90RQmVgOBg/ICXIqJSxQSMz80pqR8nXH06wWUhYbW5LMk7b/18DKi3Q+nbpryBFuESSfe18MJNCaVoZUyho47DGgRe/Q2SfawkvCTHhIvrwLymPJG7zGaQ7GQsfmo77lnfqA6wsLSTyxLb0sH19dK9yxuPg087yBz9rUyySifwu7T5CPA9l5J6Jv3kFBkXbbkWv77YpC1MZyukssUDMwqvsuMOgJqJWB5Ig18pLQF47CETmh/sZUKm23ShwHoaNyFPf+/NDEBLNKYU0YMVkD5tVP02MU5hVBBK0wlGDiSNJfNFdZmHS9Np1u8RhIp4xoGNAT2iRs2P9zkghEmTi9LJ3ec0VvfZkhBffNVyOkvtybq8tgAKpvs2xxXtp6iOSVb2V5PaB/VKf9Cj0/hgz4H+iw+WcaK4kezvMhKQgjChMIy2+XXI3M4qkRnARhAPjzgNGDsFKAij4kFWqifcmd6GAqRF9flC6J5Au4N3a0CYaQp85Nf40H8wih+i5q3 8FdGnD8y 2ZFj0wWuw4C0oW1CMQPGxwSSeAGmQ/s/NEhLvHdtq1rJzOTrdgTxJP2N6Rm9IH6No1wbXiT9RfvdjW4zVDeTzwpdsgxLZh+oqs54H5Dx8qplobwqu7joELhW7jlueijEss9DbasrFrnW1K/oU/is0ffNjD1qj6pP56au3tBKa8goZUjRbwBMvJbeqECfeYEC5sXwFVlYHZwN8FvimU20y9NxgtQSwpwwnZ1klJyQz+WwPdkfEtRkkHXmKu6pzKUlfX2dQOvgjZUqPgbMFE8yti2uSRAIm+/DlSh6HaEtIKwW1dUA2cyH9JgOMeppaoTtKQrZyobozS5H7hKrkHwAEuYek7PwKn5Dhu2jLU6vF1eF88QAhuVeF5KqChqoatT2rKbIYBIeozVnYoU4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 20, 2024 at 12:15=E2=80=AFAM Huang, Ying = wrote: > > "Ho-Ren (Jack) Chuang" writes: > > > The current implementation treats emulated memory devices, such as > > CXL1.1 type3 memory, as normal DRAM when they are emulated as normal me= mory > > (E820_TYPE_RAM). However, these emulated devices have different > > characteristics than traditional DRAM, making it important to > > distinguish them. Thus, we modify the tiered memory initialization proc= ess > > to introduce a delay specifically for CPUless NUMA nodes. This delay > > ensures that the memory tier initialization for these nodes is deferred > > until HMAT information is obtained during the boot process. Finally, > > demotion tables are recalculated at the end. > > > > More details: > > You have done several stuff in one patch. So you need "more details". > You may separate them into multiple patches. One for echo "*" below. > But I have no strong opinion on that. > > > * late_initcall(memory_tier_late_init); > > Some device drivers may have initialized memory tiers between > > `memory_tier_init()` and `memory_tier_late_init()`, potentially bringin= g > > online memory nodes and configuring memory tiers. They should be exclud= ed > > in the late init. > > > > * Abstract common functions into `mt_find_alloc_memory_type()` > > Since different memory devices require finding or allocating a memory t= ype, > > these common steps are abstracted into a single function, > > `mt_find_alloc_memory_type()`, enhancing code scalability and concisene= ss. > > > > * Handle cases where there is no HMAT when creating memory tiers > > There is a scenario where a CPUless node does not provide HMAT informat= ion. > > If no HMAT is specified, it falls back to using the default DRAM tier. > > > > * Change adist calculation code to use another new lock, `mt_perf_lock`= . > > In the current implementation, iterating through CPUlist nodes requires > > holding the `memory_tier_lock`. However, `mt_calc_adistance()` will end= up > > trying to acquire the same lock, leading to a potential deadlock. > > Therefore, we propose introducing a standalone `mt_perf_lock` to protec= t > > `default_dram_perf`. This approach not only avoids deadlock but also > > prevents holding a large lock simultaneously. > > > > * Upgrade `set_node_memory_tier` to support additional cases, including > > default DRAM, late CPUless, and hot-plugged initializations. > > To cover hot-plugged memory nodes, `mt_calc_adistance()` and > > `mt_find_alloc_memory_type()` are moved into `set_node_memory_tier()` t= o > > handle cases where memtype is not initialized and where HMAT informatio= n is > > available. > > > > * Introduce `default_memory_types` for those memory types that are not > > initialized by device drivers. > > Because late initialized memory and default DRAM memory need to be mana= ged, > > a default memory type is created for storing all memory types that are > > not initialized by device drivers and as a fallback. > > > > Signed-off-by: Ho-Ren (Jack) Chuang > > Signed-off-by: Hao Xiang > > --- > > drivers/dax/kmem.c | 13 +---- > > include/linux/memory-tiers.h | 7 +++ > > mm/memory-tiers.c | 94 +++++++++++++++++++++++++++++++++--- > > 3 files changed, 95 insertions(+), 19 deletions(-) > > > > diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c > > index 42ee360cf4e3..de1333aa7b3e 100644 > > --- a/drivers/dax/kmem.c > > +++ b/drivers/dax/kmem.c > > @@ -55,21 +55,10 @@ static LIST_HEAD(kmem_memory_types); > > > > static struct memory_dev_type *kmem_find_alloc_memory_type(int adist) > > { > > - bool found =3D false; > > struct memory_dev_type *mtype; > > > > mutex_lock(&kmem_memory_type_lock); > > - list_for_each_entry(mtype, &kmem_memory_types, list) { > > - if (mtype->adistance =3D=3D adist) { > > - found =3D true; > > - break; > > - } > > - } > > - if (!found) { > > - mtype =3D alloc_memory_type(adist); > > - if (!IS_ERR(mtype)) > > - list_add(&mtype->list, &kmem_memory_types); > > - } > > + mtype =3D mt_find_alloc_memory_type(adist, &kmem_memory_types); > > mutex_unlock(&kmem_memory_type_lock); > > > > return mtype; > > It seems that there's some miscommunication about my previous comments > about this. What I suggested is to create one separate patch, which > moves mt_find_alloc_memory_type() and mt_put_memory_types() into > memory-tiers.c. And make this patch the first one of the series. > I will make mt_find_alloc/mt_put_memory_type changes as a separate patch and the first of my patch series. Thanks. > > diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.= h > > index 69e781900082..b2135334ac18 100644 > > --- a/include/linux/memory-tiers.h > > +++ b/include/linux/memory-tiers.h > > @@ -48,6 +48,8 @@ int mt_calc_adistance(int node, int *adist); > > int mt_set_default_dram_perf(int nid, struct access_coordinate *perf, > > const char *source); > > int mt_perf_to_adistance(struct access_coordinate *perf, int *adist); > > +struct memory_dev_type *mt_find_alloc_memory_type(int adist, > > + struct list_head = *memory_types); > > #ifdef CONFIG_MIGRATION > > int next_demotion_node(int node); > > void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets); > > @@ -136,5 +138,10 @@ static inline int mt_perf_to_adistance(struct acce= ss_coordinate *perf, int *adis > > { > > return -EIO; > > } > > + > > +struct memory_dev_type *mt_find_alloc_memory_type(int adist, struct li= st_head *memory_types) > > +{ > > + return NULL; > > +} > > #endif /* CONFIG_NUMA */ > > #endif /* _LINUX_MEMORY_TIERS_H */ > > diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c > > index 0537664620e5..d9b96b21b65a 100644 > > --- a/mm/memory-tiers.c > > +++ b/mm/memory-tiers.c > > @@ -6,6 +6,7 @@ > > #include > > #include > > #include > > +#include > > We don't need this anymore. > Thanks. I will remove it. > > #include "internal.h" > > > > @@ -36,6 +37,11 @@ struct node_memory_type_map { > > > > static DEFINE_MUTEX(memory_tier_lock); > > static LIST_HEAD(memory_tiers); > > +/* > > + * The list is used to store all memory types that are not created > > + * by a device driver. > > + */ > > +static LIST_HEAD(default_memory_types); > > static struct node_memory_type_map node_memory_types[MAX_NUMNODES]; > > struct memory_dev_type *default_dram_type; > > > > @@ -505,7 +511,8 @@ static inline void __init_node_memory_type(int node= , struct memory_dev_type *mem > > static struct memory_tier *set_node_memory_tier(int node) > > { > > struct memory_tier *memtier; > > - struct memory_dev_type *memtype; > > + struct memory_dev_type *memtype, *mtype =3D NULL; > > It seems unnecessary to introduce another variable, just use memtype? > Yes, I will consolidate them. > > + int adist =3D MEMTIER_ADISTANCE_DRAM; > > pg_data_t *pgdat =3D NODE_DATA(node); > > > > > > @@ -514,7 +521,18 @@ static struct memory_tier *set_node_memory_tier(in= t node) > > if (!node_state(node, N_MEMORY)) > > return ERR_PTR(-EINVAL); > > > > - __init_node_memory_type(node, default_dram_type); > > + mt_calc_adistance(node, &adist); > > + if (adist !=3D MEMTIER_ADISTANCE_DRAM && > > + node_memory_types[node].memtype =3D=3D NULL) { > > + mtype =3D mt_find_alloc_memory_type(adist, &default_memor= y_types); > > + if (IS_ERR(mtype)) { > > + mtype =3D default_dram_type; > > + pr_info("Failed to allocate a memory type. Fall b= ack.\n"); > > + } > > + } else > > + mtype =3D default_dram_type; > > This can be simplified to > > mt_calc_adistance(node, &adist); > if (node_memory_types[node].memtype =3D=3D NULL) { > mtype =3D mt_find_alloc_memory_type(adist, &default_memor= y_types); > if (IS_ERR(mtype)) { > mtype =3D default_dram_type; > pr_info("Failed to allocate a memory type. Fall b= ack.\n"); > } > } > Sounds good! I will do. > > + __init_node_memory_type(node, mtype); > > > > memtype =3D node_memory_types[node].memtype; > > node_set(node, memtype->nodes); > > @@ -623,6 +641,55 @@ void clear_node_memory_type(int node, struct memor= y_dev_type *memtype) > > } > > EXPORT_SYMBOL_GPL(clear_node_memory_type); > > > > +struct memory_dev_type *mt_find_alloc_memory_type(int adist, struct li= st_head *memory_types) > > +{ > > + bool found =3D false; > > + struct memory_dev_type *mtype; > > + > > + list_for_each_entry(mtype, memory_types, list) { > > + if (mtype->adistance =3D=3D adist) { > > + found =3D true; > > + break; > > + } > > + } > > + if (!found) { > > + mtype =3D alloc_memory_type(adist); > > + if (!IS_ERR(mtype)) > > + list_add(&mtype->list, memory_types); > > + } > > + > > + return mtype; > > +} > > +EXPORT_SYMBOL_GPL(mt_find_alloc_memory_type); > > + > > +/* > > + * This is invoked via late_initcall() to create > > + * CPUless memory tiers after HMAT info is ready or > > + * when there is no HMAT. > > Better to avoid HMAT in general code. How about something as below? > > This is invoked via late_initcall() to initialize memory tiers for > CPU-less memory nodes after drivers initialization. Which is > expect to provide adistance algorithms. > Got it. Thanks. " This is invoked via `late_initcall()` to initialize memory tiers for CPU-less memory nodes after driver initialization, which is expected to provide `adistance` algorithms. " > > + */ > > +static int __init memory_tier_late_init(void) > > +{ > > + int nid; > > + > > + mutex_lock(&memory_tier_lock); > > + for_each_node_state(nid, N_MEMORY) > > + if (!node_state(nid, N_CPU) && > > + node_memory_types[nid].memtype =3D=3D NULL) > > + /* > > + * Some device drivers may have initialized memor= y tiers > > + * between `memory_tier_init()` and `memory_tier_= late_init()`, > > + * potentially bringing online memory nodes and > > + * configuring memory tiers. Exclude them here. > > + */ > > + set_node_memory_tier(nid); > > + > > + establish_demotion_targets(); > > + mutex_unlock(&memory_tier_lock); > > + > > + return 0; > > +} > > +late_initcall(memory_tier_late_init); > > + > > static void dump_hmem_attrs(struct access_coordinate *coord, const cha= r *prefix) > > { > > pr_info( > > @@ -631,12 +698,16 @@ static void dump_hmem_attrs(struct access_coordin= ate *coord, const char *prefix) > > coord->read_bandwidth, coord->write_bandwidth); > > } > > > > +/* > > + * The lock is used to protect the default_dram_perf. > > + */ > > +static DEFINE_MUTEX(mt_perf_lock); > > Miscommunication here too. Should be moved to near the > "default_dram_perf" definition. And it protects not only > default_dram_perf. > I will move it closer to default_dram_perf*. And change it to: +/* + * The lock is used to protect default_dram_perf*. + */ > > int mt_set_default_dram_perf(int nid, struct access_coordinate *perf, > > const char *source) > > { > > int rc =3D 0; > > > > - mutex_lock(&memory_tier_lock); > > + mutex_lock(&mt_perf_lock); > > if (default_dram_perf_error) { > > rc =3D -EIO; > > goto out; > > @@ -684,7 +755,7 @@ int mt_set_default_dram_perf(int nid, struct access= _coordinate *perf, > > } > > > > out: > > - mutex_unlock(&memory_tier_lock); > > + mutex_unlock(&mt_perf_lock); > > return rc; > > } > > > > @@ -700,7 +771,7 @@ int mt_perf_to_adistance(struct access_coordinate *= perf, int *adist) > > perf->read_bandwidth + perf->write_bandwidth =3D=3D 0) > > return -EINVAL; > > > > - mutex_lock(&memory_tier_lock); > > + mutex_lock(&mt_perf_lock); > > /* > > * The abstract distance of a memory node is in direct proportion= to > > * its memory latency (read + write) and inversely proportional t= o its > > @@ -713,7 +784,7 @@ int mt_perf_to_adistance(struct access_coordinate *= perf, int *adist) > > (default_dram_perf.read_latency + default_dram_perf.write= _latency) * > > (default_dram_perf.read_bandwidth + default_dram_perf.wri= te_bandwidth) / > > (perf->read_bandwidth + perf->write_bandwidth); > > - mutex_unlock(&memory_tier_lock); > > + mutex_unlock(&mt_perf_lock); > > > > return 0; > > } > > @@ -826,7 +897,8 @@ static int __init memory_tier_init(void) > > * For now we can have 4 faster memory tiers with smaller adistan= ce > > * than default DRAM tier. > > */ > > - default_dram_type =3D alloc_memory_type(MEMTIER_ADISTANCE_DRAM); > > + default_dram_type =3D mt_find_alloc_memory_type( > > + MEMTIER_ADISTANCE_DRAM, &default_= memory_types); > > if (IS_ERR(default_dram_type)) > > panic("%s() failed to allocate default DRAM tier\n", __fu= nc__); > > > > @@ -836,6 +908,14 @@ static int __init memory_tier_init(void) > > * types assigned. > > */ > > for_each_node_state(node, N_MEMORY) { > > + if (!node_state(node, N_CPU)) > > + /* > > + * Defer memory tier initialization on CPUless nu= ma nodes. > > + * These will be initialized after firmware and d= evices are > > + * initialized. > > + */ > > + continue; > > + > > memtier =3D set_node_memory_tier(node); > > if (IS_ERR(memtier)) > > /* > > -- > Best Regards, > Huang, Ying --=20 Best regards, Ho-Ren (Jack) Chuang =E8=8E=8A=E8=B3=80=E4=BB=BB