From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76F60C3DA6E for ; Wed, 3 Jan 2024 08:31:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06B3E6B015E; Wed, 3 Jan 2024 03:31:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 01D716B0160; Wed, 3 Jan 2024 03:31:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFDFE6B0162; Wed, 3 Jan 2024 03:31:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CE7D86B015E for ; Wed, 3 Jan 2024 03:31:50 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B307A1A0311 for ; Wed, 3 Jan 2024 08:31:50 +0000 (UTC) X-FDA: 81637331580.17.CCAE7DD Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by imf06.hostedemail.com (Postfix) with ESMTP id 436A1180016 for ; Wed, 3 Jan 2024 08:31:47 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=F+M33c0z; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704270708; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qpIYVnB26ar0XABqyNrJGq8kfepQ8Hu+WAIsLKikk7s=; b=kSWztLX23rTAo82DGQM5vDL7i6CnwxN94msQipfMcF6akxutcUHBWwBkaU6RLEoVM9QC41 vDviMJrlo98+7JxqlcUS14oglD4GQKzLjqC83l+ch7EyvIoBGJH38zXKfvv/HW0uaeSaQ3 ZbwPpCKzDv/4SD6ebc68vVTChiba2HE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=F+M33c0z; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704270708; a=rsa-sha256; cv=none; b=s5x0P8e71f3tSGZD4gZBN9kCUvOuouOPbi8kURIBgN4uJqh2QitzG5zn8WybaKSwVazh6v e1Bt+fm7WBwaoa+P18leUmHz+MjWTf81q8Fi+m54WQMJwd8tTUMBROrZg1ZvLKULVEqx6x CJd5ZQQak8Xry2SWvNDGir6Sdupi9Ac= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704270707; x=1735806707; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=K2D+DOH/3k8UCY1rTsgGy6wghC2CdFVcWHWsDmKE3ec=; b=F+M33c0zwehKM1jcyM0A4qzd1MrDlv5antYWt3DYRXbgob2ND0ZrBdT9 LlYRA/Y6en3MPtuZlcuCNRkA5V7iLuhZN5Zlk7FLvhNEZeaqqdV/LF8sU /ZfpYbJgk6IKup6UN4vs5j96A4Zv62EwmKEvhi2ufwR29leJ7p6R60qRh nA7bv88OE9f1jlmy6gNr1tWHUOqoOUhi7GhUp/fi1KEKlUCU9nZDij86j sODD1jO2skaQM/knHudiysXMbJbFk9SPu+kW3L675/yJmVkWPzQ/jH5Zq hwwmAV/e+4ZSsBDrOG613IHAADlmb/R1Sirn5MFbMn/eUH9d0xCsrdDDa A==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="395854475" X-IronPort-AV: E=Sophos;i="6.04,327,1695711600"; d="scan'208";a="395854475" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2024 00:31:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="773079904" X-IronPort-AV: E=Sophos;i="6.04,327,1695711600"; d="scan'208";a="773079904" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2024 00:31:41 -0800 From: "Huang, Ying" To: Srinivasulu Thanneeru Cc: gregory.price , Srinivasulu Opensrc , "linux-cxl@vger.kernel.org" , "linux-mm@kvack.org" , "aneesh.kumar@linux.ibm.com" , "dan.j.williams@intel.com" , "mhocko@suse.com" , "tj@kernel.org" , "john@jagalactic.com" , Eishan Mirakhur , "Vinicius Tavares Petrucci" , Ravis OpenSrc , "Jonathan.Cameron@huawei.com" , "linux-kernel@vger.kernel.org" , Johannes Weiner , Wei Xu Subject: Re: [EXT] Re: [RFC PATCH v2 0/2] Node migration between memory tiers In-Reply-To: (Srinivasulu Thanneeru's message of "Wed, 3 Jan 2024 07:56:42 +0000") References: <20231213175329.594-1-sthanneeru.opensrc@micron.com> <87cyv8qcqk.fsf@yhuang6-desk2.ccr.corp.intel.com> <87fs00njft.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edezc5l1.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 03 Jan 2024 16:29:42 +0800 Message-ID: <87a5pmddl5.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 436A1180016 X-Stat-Signature: exjf5ge9dhoejmz8basw6ajigij53734 X-HE-Tag: 1704270707-415468 X-HE-Meta: U2FsdGVkX1/c4Vugl9VhLrPwTfGK0CgcgC+g91cPAlnjTyLi6VTY1P/oo2PlK3/WW3ggZMzCWy31AdTEDG8TK0/Q+n6/1caWcV4Kitj6H1r7Z5ja2AqYMnWFHaKt2HStq4CeQfH79fQJUQ/wbabzstVTSG8S/6spX8WgCtRV6smPElFrfdF/fRNwGHs24RNcwkl02OKp5lGBmHPHguLlrB8CqJnFwDcLbE7dWEbIYn4xInlfxErOiQpfQRZBbSrtcgP04SsFiN4B5/VrbH5Gl2KsqNupDJKySmfQGYZ7s92j8IX1vvilUdIZ5xzxpPwa74fQOOSji8hmjn1phOLPXpidNDOLBTojE2GTs+LxZTD//9ej5FqxJy2ljtJxUhjC7bfrQkmqavazE76O6PCsy5taC1I6QCRvUdtWfsI2bdF96r+LqEFnk68/y8ICDEEzNDr41TZBQ2DlfiO7TLBuW7LuhWafxG06azyTBmwTzOEUPuG2JkbbWl54/uRDLsxJLt9dEmKE0w4bLfOF0bBsXXEXwG3WkoIccSpqBq4vAFNJKWIc8+BYLz/rhlUrAEPtiVJYsE7lo94GvQ5RF5biIqzwRm4Py5BTQ0ZlLSGi0LDj+hoOaKV84nKwxH6q2nxT9RIYfwrz0P9cpc3Zeu0vTWAiw2ZtxcKqAPN/iA2Az/Oam3hlqLqGDh0t6kpMVuKLAXAJDPMvAdP3xprBSyIL9LEjLO4wV9d2/s9bEoqT/sRWH8jJx+nS2F3bNEpKy+Hu4ImIx+bqMHrVzbOtI5vaCWpInQ0SBORtummRGtfzXvN/FrisovLz0y8tLuOC++8Z4+ZAY9hXDxUlGor+dky4lf4FF8DZ193CIUk638bvMzidB5+DTfPBLr7sK1gkKzPm+8+c7+6//LPzaHZYTI2dbJlDY2C0gHgamfXAvYd4VFQHX3/St3tHV9YNEa5H/jhlt3Z3xpvx8MsgbkEwqpr MFOKXKP/ uzitzRqmZcxbkMpe8F4XbdNNnwYVHZyY7nwv8IjO9dC2aED/GxkvFp+QcAcDRxkCJVTiBudS29mk9FOUEF+pvy0W5uu3NSKW2j+hvbKN/YKxXgfwoopYVcuwlFUMBw1nnJcz8Dd4SOEO2qp340/jPPHYzE6jpv6YUcyLbA2vpi52O8MkPvBHueSSCb5xtO+o2jibUCETa+f4ss6h9+tMaQzEAWXUuR6IoYpJZqubLhuZfMjv/ZcduJrPHZP269XPLqnKXbkmObQV7krkBdpiyQjJs6r3qMWbli0UulMykBmg2B4XflnyMc69+NL8TAA5CLJnl8MBtCBvL9FJyn+p6e3TvgMDuB5H0lQLQV4Wwo8kSIwgJ0xaUfSki3FTPPn+d9BfndJDbP3HiUk1zomBklvDw6iMOj1NtquUSQqbutnOQQC9Jw3TXkpS5944nC1ByQ418fatYIsnT6I3Fyb72Q7Hlrv8wqrW25t8FDfl2hy1eC/xxdsmaT4VjWIH6EQK56jXD4jJZ1M7w0fTdNHWqM2F74W+pBwkkn82r721igttzMBmtMfR4487M/3jOnmQCf5VZMts4RNuH4YOhjUjL3qUn7wpoAhlusvq7E09I9yfRFfOv1G4koUUbReMlrKxfdB2O9Lvur4o9eZo4egtZb/DJdr5WbOvq6oA2bfVM+k4NDV5H1GOn3A2j5LcbsCr5dcVFe5fVMY6R/ucOghYbkllYw7qii6eVjfRge6g5O/MhtU9a9gzyh8t1bFAHIrt2Ry6b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Srinivasulu Thanneeru writes: > Micron Confidential > > > > Micron Confidential >> -----Original Message----- >> From: Huang, Ying >> Sent: Wednesday, January 3, 2024 11:38 AM >> To: Srinivasulu Thanneeru >> Cc: gregory.price ; Srinivasulu Opensrc >> ; linux-cxl@vger.kernel.org; linux- >> mm@kvack.org; aneesh.kumar@linux.ibm.com; dan.j.williams@intel.com; >> mhocko@suse.com; tj@kernel.org; john@jagalactic.com; Eishan Mirakhur >> ; Vinicius Tavares Petrucci >> ; Ravis OpenSrc ; >> Jonathan.Cameron@huawei.com; linux-kernel@vger.kernel.org; Johannes >> Weiner ; Wei Xu >> Subject: Re: [EXT] Re: [RFC PATCH v2 0/2] Node migration between memory >> tiers >> >> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless >> you recognize the sender and were expecting this message. >> >> >> Srinivasulu Thanneeru writes: >> >> > Micron Confidential >> > >> > Hi Huang, Ying, >> > >> > My apologies for wrong mail reply format, my mail client settings got >> changed on my PC. >> > Please find comments bellow inline. >> > >> > Regards, >> > Srini >> > >> > >> > Micron Confidential >> >> -----Original Message----- >> >> From: Huang, Ying >> >> Sent: Monday, December 18, 2023 11:26 AM >> >> To: gregory.price >> >> Cc: Srinivasulu Opensrc ; linux- >> >> cxl@vger.kernel.org; linux-mm@kvack.org; Srinivasulu Thanneeru >> >> ; aneesh.kumar@linux.ibm.com; >> >> dan.j.williams@intel.com; mhocko@suse.com; tj@kernel.org; >> >> john@jagalactic.com; Eishan Mirakhur ; Vinicius >> >> Tavares Petrucci ; Ravis OpenSrc >> >> ; Jonathan.Cameron@huawei.com; linux- >> >> kernel@vger.kernel.org; Johannes Weiner ; Wei Xu >> >> >> >> Subject: [EXT] Re: [RFC PATCH v2 0/2] Node migration between memory >> tiers >> >> >> >> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless >> >> you recognize the sender and were expecting this message. >> >> >> >> >> >> Gregory Price writes: >> >> >> >> > On Fri, Dec 15, 2023 at 01:02:59PM +0800, Huang, Ying wrote: >> >> >> writes: >> >> >> >> >> >> > ============= >> >> >> > Version Notes: >> >> >> > >> >> >> > V2 : Changed interface to memtier_override from adistance_offset. >> >> >> > memtier_override was recommended by >> >> >> > 1. John Groves >> >> >> > 2. Ravi Shankar >> >> >> > 3. Brice Goglin >> >> >> >> >> >> It appears that you ignored my comments for V1 as follows ... >> >> >> >> >> >> >> >> >> https://lore.k/ >> %2F&data=05%7C02%7Csthanneeru%40micron.com%7C3e5d38eb47be463c2 >> 95c08dc0c229d22%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C63 >> 8398590664228240%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD >> AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C >> &sdata=7fPxb1YYR2tZ0v2FB1vlXnMJFcI%2Fr9HT2%2BUD1MNUd%2FI%3D&re >> served=0 >> >> ernel.org%2Flkml%2F87o7f62vur.fsf%40yhuang6- >> >> >> desk2.ccr.corp.intel.com%2F&data=05%7C02%7Csthanneeru%40micron.com >> >> >> %7C5e614e5f028342b6b59c08dbff8e3e37%7Cf38a5ecd28134862b11bac1d56 >> >> >> 3c806f%7C0%7C0%7C638384758666895965%7CUnknown%7CTWFpbGZsb3d >> >> >> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3 >> >> >> D%7C3000%7C%7C%7C&sdata=OpMkYCar%2Fv8uHb7AvXbmaNltnXeTvcNUTi >> >> bLhwV12Fg%3D&reserved=0 >> > >> > Thank you, Huang, Ying for pointing to this. >> > >> https://lpc.ev/ >> ents%2Fevent%2F16%2Fcontributions%2F1209%2Fattachments%2F1042%2F1 >> 995%2FLive%2520In%2520a%2520World%2520With%2520Multiple%2520Me >> mory%2520Types.pdf&data=05%7C02%7Csthanneeru%40micron.com%7C3e >> 5d38eb47be463c295c08dc0c229d22%7Cf38a5ecd28134862b11bac1d563c806 >> f%7C0%7C0%7C638398590664228240%7CUnknown%7CTWFpbGZsb3d8eyJW >> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3 >> 000%7C%7C%7C&sdata=1fGraxff7%2F1hNaE0an0xEudSKSUvaF3HgClMkmdC7 >> n8%3D&reserved=0 >> > >> > In the presentation above, the adistance_offsets are per memtype. >> > We believe that adistance_offset per node is more suitable and flexible. >> > since we can change it per node. If we keep adistance_offset per memtype, >> > then we cannot change it for a specific node of a given memtype. >> > >> >> >> >> >> >> https://lore.k/ >> %2F&data=05%7C02%7Csthanneeru%40micron.com%7C3e5d38eb47be463c2 >> 95c08dc0c229d22%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C63 >> 8398590664228240%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD >> AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C >> &sdata=7fPxb1YYR2tZ0v2FB1vlXnMJFcI%2Fr9HT2%2BUD1MNUd%2FI%3D&re >> served=0 >> >> ernel.org%2Flkml%2F87jzpt2ft5.fsf%40yhuang6- >> >> >> desk2.ccr.corp.intel.com%2F&data=05%7C02%7Csthanneeru%40micron.com >> >> >> %7C5e614e5f028342b6b59c08dbff8e3e37%7Cf38a5ecd28134862b11bac1d56 >> >> >> 3c806f%7C0%7C0%7C638384758666895965%7CUnknown%7CTWFpbGZsb3d >> >> >> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3 >> >> >> D%7C3000%7C%7C%7C&sdata=O0%2B6T%2FgU0TicCEYBac%2FAyjOLwAeouh >> >> D%2BcMI%2BflOsI1M%3D&reserved=0 >> > >> > Yes, memory_type would be grouping the related memories together as >> single tier. >> > We should also have a flexibility to move nodes between tiers, to address >> the issues. >> > described in use cases above. >> >> We don't pursue absolute flexibility. We add necessary flexibility >> only. Why do you need this kind of flexibility? Can you provide some >> use cases where memory_type based "adistance_offset" doesn't work? > > - /sys/devices/virtual/memory_type/memory_type/ adistance_offset > memory_type based "adistance_offset will provide a way to move all nodes of same memory_type (e.g. all cxl nodes) > to different tier. We will not put the CXL nodes with different performance metrics in one memory_type. If so, do you still need to move one of them? > Whereas /sys/devices/system/node/node2/memtier_override provide a way migrate a node from one tier to another. > Considering a case where we would like to move two cxl nodes into two different tiers in future. > So, I thought it would be good to have flexibility at node level instead of at memory_type. -- Best Regards, Huang, Ying