From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11337C47DD9 for ; Thu, 28 Mar 2024 01:37:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A34B6B0087; Wed, 27 Mar 2024 21:37:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 752EA6B0088; Wed, 27 Mar 2024 21:37:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61ABF6B0089; Wed, 27 Mar 2024 21:37:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 44F076B0087 for ; Wed, 27 Mar 2024 21:37:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 00129A144B for ; Thu, 28 Mar 2024 01:37:25 +0000 (UTC) X-FDA: 81944735292.30.1DA040A Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by imf21.hostedemail.com (Postfix) with ESMTP id 6F2BD1C000A for ; Thu, 28 Mar 2024 01:37:23 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NZd0OSvZ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.9 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711589844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FkY8tv+4S9UvcucZvcs83MzJ3+hwTeteY/5c1Mc4Z18=; b=TIuftumqL3Q/44oU0YyxxpHdufQnR9VPjwp4J0FJgh1+RMfj3mkmFFrIfda++qSqUqV/SC WD8XDAxHdLRpwNn2+HO2ftUOeuHu8oqVidp0sp//6EsnRASZ07vBU4y3ROP32LWAjHaXRK 9OsYsMrnobYs7Vj2mRQXeGGAZYp3ntM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NZd0OSvZ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.9 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711589844; a=rsa-sha256; cv=none; b=IwAauieR9XkwtMG4eqkwh8z5SWFbzPVUbHU+Ftr/WJ+MtmWK5AKWioqHZA5FrvDzlOx0Xi Og++rLC+vq+q/LPxrTnryy4Z0Hh5c8az+xtI78JtL8FW1+KGqjbNYHnm+WAHB+lA6gqHUL CKKEqkW0HIUWx5a5uRcmxIZfraaztm0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1711589843; x=1743125843; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=O1k80KbUBNPKN2Cz8jdjH7/l3Qy2e0v1DTF3Pf8/QLU=; b=NZd0OSvZhy4ZOkKJfJiGIsYqXpvmjAKB3u/wgcu3roB9S8e60yOAIdmU vknDlzEQ4pQlHHzYuXVBlH50mQzDO59p/Gn3gTuweIiryOFeA8ia/+cDx XYbWg6tc41OiPldeI5WW4ha217h37S4acu2KFHQOkqMqm4fuzJS1puqdW /9f0J9q4yW5MoBwqpCumNMu27MACYEtpQ/9IDaT6py+St4WN8mIWIlV72 vrbcXpsBahnT4wywO1zEr9gN5G600yQYKcFkSQlbbzGUZPwRKns2EHWLo 7g2NHNxQzQIpx3i4Z2A8mGVgizQytb9K8d7HcBeA9gGbkjqCCs4UHrtf4 Q==; X-CSE-ConnectionGUID: EFPr8xfkSj+fUyzKNx2TOw== X-CSE-MsgGUID: UE+IdQMeTMaxmb7A0Zn1Iw== X-IronPort-AV: E=McAfee;i="6600,9927,11026"; a="17453693" X-IronPort-AV: E=Sophos;i="6.07,160,1708416000"; d="scan'208";a="17453693" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Mar 2024 18:37:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,160,1708416000"; d="scan'208";a="21131863" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Mar 2024 18:37:16 -0700 From: "Huang, Ying" To: "Ho-Ren (Jack) Chuang" Cc: "Gregory Price" , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, "Eishan Mirakhur" , "Vinicius Tavares Petrucci" , "Ravis OpenSrc" , "Alistair Popple" , "Srinivasulu Thanneeru" , Dan Williams , Vishal Verma , Dave Jiang , Andrew Morton , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , qemu-devel@nongnu.org, Hao Xiang Subject: Re: [PATCH v6 2/2] memory tier: create CPUless memory tiers after obtaining HMAT info In-Reply-To: <20240327072729.3381685-3-horenchuang@bytedance.com> (Ho-Ren Chuang's message of "Wed, 27 Mar 2024 07:27:28 +0000") References: <20240327072729.3381685-1-horenchuang@bytedance.com> <20240327072729.3381685-3-horenchuang@bytedance.com> Date: Thu, 28 Mar 2024 09:35:22 +0800 Message-ID: <87v857kujp.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6F2BD1C000A X-Stat-Signature: on8qzqas5ai81mijutjsf4gairjox5dz X-HE-Tag: 1711589843-176870 X-HE-Meta: U2FsdGVkX1+xVLGjuxxD6aIIPWGmMT6rT2cmLOk0rm/qYdCRG2jbptX2Ek8POkZ0HWe+BlhMdvwOzEyPGjLy5Xmt6rxN8q67PkENxjXHS0UW6tlFzf4t/4tam5Ynj2sLIHM2tSxKq+XbK8mIia5iqNsTRdz4+KzNhTGx0P20Gf7xrW3TTPbrV6vegTCWEpJPlRKV9f58y96MMldu7dKUEYr5zoNoyOVyy4fYHBlRqmSBa4i39yyNJuMgZYiB+oTJ57LzJCnHUm4M1DP5Qp3nnrnJiQoOhwKjsZ332iyVPNMZSqIT+G5hQ/wTRcQgrWH3dC4MYgOCPfDtN9IQKKtSBZ1SwCwAj2JJSdPorhhYbRCZYjMpqRy9MO2oiFQ+qPHwhhzeVPmeUH799vh8UNpx7SXb8Q4pd8cwxb+0phiziEyAmzxtbfk7tpggyzzqhalp/GwnpFbfI8qgAzpJSy1Ad6vRBU+wywVCvDqXxjrCY7Vr9MMAZUcPuXY7dnU4eCQYa6sZvEejfrFvKqFgAXQy+xHkvlmxe0Cmm2Bfd2cLf9F26Dx+qH324H7jyQZspI6KIXpw1vBH7x7Cquzf1MzriZle0WVGcUxproJmej8SztldFE4jREnbpQfqyeTzUaqEWcheWcO51aU0MxaFx/TdBMoEoJejfGjTEhl09EcCOYZqlFd5Hb8YmIAU0GQHlmCYFgIs8XEYRPXliK9NqZLcDrUfF3TUBqWEccEd/N/Zo0QWSGdF+9fMlgEqAFN3Zw1uPQZLtG870FA3I43GdMKdMzU92xjQH3ZGpzlLtlCAVCj3oSZ7txQ0pA0nmdtnYB1K7VA4lNcqKMgDzwIq+Kh9mCerlq71JnX5WBv4nk+QuTx2QMvE/L9N0xdbv+SoJbYUSiHLzcNHga6iAR1eohDULGMS7vh5H02T4JDtf+DGch3UAgBSNsFXwyAYN9fkTMh4qzBSUmcrbpm+Ti9Zsrq XwDXfkl8 WVDi2VAYItkzH3JxRRUFlwIuTvG2WHUk82NkHcOPRg7eCXzlMCr2iQ9LQ7GXeqeViFYTJZrnHzco3HcGbZOlICiZPrMlNrXmsM/rsa2txFQ1/XuSn+iuH80vdrZy4TcLJngNuOda+fP8bk4mcnUjICqozLKdiBtKdeU6vlAchIbqvNg2EJBI5n0K9TZtFTeOh1KMRc4mnYekF5uY8ZAJwzXpZmn8GXjzslViiM049Otn5p11GV5GaKINvFOMJ/Hw2YU+pXQ8t59r9zTU+nJBlHOpLf1esypVpXOjmXm/ODjXx8UQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: "Ho-Ren (Jack) Chuang" writes: [snip] > @@ -655,6 +672,34 @@ void mt_put_memory_types(struct list_head *memory_types) > } > EXPORT_SYMBOL_GPL(mt_put_memory_types); > > +/* > + * This is invoked via `late_initcall()` to initialize memory tiers for > + * CPU-less memory nodes after driver initialization, which is > + * expected to provide `adistance` algorithms. > + */ > +static int __init memory_tier_late_init(void) > +{ > + int nid; > + > + mutex_lock(&memory_tier_lock); > + for_each_node_state(nid, N_MEMORY) > + if (!node_state(nid, N_CPU) && > + node_memory_types[nid].memtype == NULL) Think about this again. It seems that it is better to check "node_memory_types[nid].memtype == NULL" only here. Because for all node with N_CPU in memory_tier_init(), "node_memory_types[nid].memtype" will be !NULL. And it's possible (in theory) that some nodes becomes "node_state(nid, N_CPU) == true" between memory_tier_init() and memory_tier_late_init(). Otherwise, Looks good to me. Feel free to add Reviewed-by: "Huang, Ying" in the future version. > + /* > + * Some device drivers may have initialized memory tiers > + * between `memory_tier_init()` and `memory_tier_late_init()`, > + * potentially bringing online memory nodes and > + * configuring memory tiers. Exclude them here. > + */ > + set_node_memory_tier(nid); > + > + establish_demotion_targets(); > + mutex_unlock(&memory_tier_lock); > + > + return 0; > +} > +late_initcall(memory_tier_late_init); > + [snip] -- Best Regards, Huang, Ying