From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED551C35274 for ; Mon, 18 Dec 2023 05:57:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C7A26B006E; Mon, 18 Dec 2023 00:57:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 677C36B007E; Mon, 18 Dec 2023 00:57:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 517C96B0081; Mon, 18 Dec 2023 00:57:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3ADD06B007E for ; Mon, 18 Dec 2023 00:57:42 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0AD501608DB for ; Mon, 18 Dec 2023 05:57:42 +0000 (UTC) X-FDA: 81578882364.21.2E79D91 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by imf10.hostedemail.com (Postfix) with ESMTP id 01E8AC000A for ; Mon, 18 Dec 2023 05:57:38 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bcPaq7Pl; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702879060; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YEKoRm7ZRxY9Jv8ulG1cMX3YMYOHxGg/UgywBNuJOeM=; b=DkiJentp9kFKdgQrNXOZVPWx53aEBRsDXjn6QQx37XzphCWoyp+9NnA2Gv/nf7puTRyjUK +2ds78AeoiDKIHKnUUiWjx/25BgqpErSKeja1slwGjQfICO41TD6sVNTC2Kf8rc2szf/LY pDGLh7/ExATnougREbNZq8Lkhy7VvQE= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bcPaq7Pl; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702879060; a=rsa-sha256; cv=none; b=bku9Va3bRxIXxIFhEPuh3YRyJRU9e1of6ofqXks/BbBoWxck+dRQmt5udytUCQZcF5gglq zjjQ2Yq+VaTUfgCYegZ/47wYfJSXKxs5yvpz53KQeVHtILIRxk7GyRN5ZieDt29fxzGA80 3AaDxKEg6t2aPrOeoJtVPBiWtJtSmX4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702879059; x=1734415059; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=EUwgnCLnOCjvSoWerIdZN9qy6Wju/5BfP/1nGT2V6Qc=; b=bcPaq7PlJwNgOZtfAKQZytstiGTbDXG1+kgEijDIRrcP2Mm+m7ewno1o wlU+lBh/wL5Kr8QCaDst3D8TPptGbe6S38fuGrNaEEAH6++qIpiIS5Ol+ HyzJ4l5DuNmxA2ugWi6DVdVoh+p6xpaZ7xedM83BvvaWpzuy6HGqIAjvU JHg5T+emiqa0XKfz6txKs5qx8JgmBRueT2Uwrma4AQCCk1aFm/fNyl0a0 yAA9StlWWcrkpxS0b4pV9FIt/YcxVhT+yOw3Er4PcF7uYkkpH8PJ9zdGr d4nKfN61pa6WsDET+Y6F7cWMpNzOGjy8212MAG1L4d9YIdBqWahkTCieq w==; X-IronPort-AV: E=McAfee;i="6600,9927,10927"; a="481645809" X-IronPort-AV: E=Sophos;i="6.04,284,1695711600"; d="scan'208";a="481645809" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Dec 2023 21:57:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,284,1695711600"; d="scan'208";a="17803674" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Dec 2023 21:57:33 -0800 From: "Huang, Ying" To: Gregory Price Cc: , , , , , , , , , , , , , , Johannes Weiner , Wei Xu Subject: Re: [RFC PATCH v2 0/2] Node migration between memory tiers In-Reply-To: (Gregory Price's message of "Fri, 15 Dec 2023 12:42:56 -0500") References: <20231213175329.594-1-sthanneeru.opensrc@micron.com> <87cyv8qcqk.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Mon, 18 Dec 2023 13:55:34 +0800 Message-ID: <87fs00njft.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Stat-Signature: weysdodiy16548gsyquygepyb6gng1ni X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 01E8AC000A X-HE-Tag: 1702879058-191589 X-HE-Meta: U2FsdGVkX18epT6sE/hwF9IzYbmKGk4z3ws1PR7UiB/R6ArysvyEZED46XSllocFpRtOG/ux7hsgayWvyg216PRh6XY3EA/6KoY7tv8TRLT+P38WVL8CR/+eHtp4LUTstYTAC+FRlRL+dcmvqGe+Y9TCz5TGMi8GoFprwhRocKMY7/LKN/xS3HWEREqeOcXrrPzMUxtSNZbtbqSMqWswphleV8cRlYrVHOAFtk63vneMBRkBkzUZvxNvVlL/6LzNpImBelFfL7YMcLXs7vLTs9mbOCPSQYZy/2EahbHObsozuXAGCY0Vyk4iKrrmNSFttfa/iFW3T91WTC9KPJZYjor9zORY+K26LXoNuqe+wTRPQkbAO7LW7zACbMBaKr9LbETTcEgjam5Aw6CTCZ1kttP99SLzK3JMlj/9x8WaYH7dv+rNXr0aYIhp30D/S/p/W6LFUMZHX6+5L8mW4XnsvmdsBtzsYb91XWkpjLNb9rRI+ic4DIZ+4fO6oTjsNAWZ3zNC9kuEMx8M1w3K/hnXqSfnv2T3Gm7/r59qowhroPAMlg/C78s6djYRXg4Qg3MsigHpHL/8WfMGaSgUJEe8ltJwy6fYFpgQ07l7ZnXk/SMh2nk4WPLQb7OMxMFUxmmJnsHFs+AZlOprva+YZCZejVK4t9ZaERUMAfKjCJ8PjJfz3GLUwdVkN+6baf3z1+InfOEE/zkFlMOHEiCG6zdhXozoH/mbLWesg5PRfLCLPRSmsJja6Chgfg0UI4eeju/tUv2VN2MLOAlicJ8FtLd/Mp1jkUcUh4ksITYkdtsH5BVGo6lKh2BjRNvMYlzrQctfdavqc6lKBf3U6xKSExoAJdcr9cyVlEjapVQMBgZmCCVFHeCxVJh5DBf9RVpw7ssGPPPq4Nn2NOfGbXLsDBx/RzybAFKOLUFNyhpDPywnLg0mg5s19XUqpTWvix4rtHHu27PC10IRuKapAuI9MEc 1cQJKNHx kj9mK/g+3kqvg2dqmdzDzvXn/1YSaMvV5b6oph6+6nwjv06XhcMznBo5/TYTtOFuQ5hYFFYsxqcU4K8U1jadGkhbheax6gNKqW9+KkOfg+I1stWmAkRpwy/Z7PhfbQmj6HiKeD1DSKK32nZLbXZSIMaX8zZDh1kq2JCdI+lLekqGxo0b6xC3o76JB1w5apDpNOHSCl5e30OO/hW7mOojE+3y2ELV3TX6vWy9k3otABIqxqyJBa6BG8/0vZPdmq8oAoyQ1O8KH/Do3tyKaAbnAvViEk0IvW3ZNmR1s33lxQM4AjhjVuYkACBe3yVVNUxMdRnIA5FKMEFrErNL7bMYxHUWHmqNzO51Qemb6ZvwjbyFXl25jXCk7cCNgWM3wO1+MbJJ3qVZfGxPoFenYHuCc1QYHBg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Gregory Price writes: > On Fri, Dec 15, 2023 at 01:02:59PM +0800, Huang, Ying wrote: >> writes: >> >> > ============= >> > Version Notes: >> > >> > V2 : Changed interface to memtier_override from adistance_offset. >> > memtier_override was recommended by >> > 1. John Groves >> > 2. Ravi Shankar >> > 3. Brice Goglin >> >> It appears that you ignored my comments for V1 as follows ... >> >> https://lore.kernel.org/lkml/87o7f62vur.fsf@yhuang6-desk2.ccr.corp.intel.com/ >> https://lore.kernel.org/lkml/87jzpt2ft5.fsf@yhuang6-desk2.ccr.corp.intel.com/ >> https://lore.kernel.org/lkml/87a5qp2et0.fsf@yhuang6-desk2.ccr.corp.intel.com/ >> > > Not speaking for the group, just chiming in because i'd discussed it > with them. > > "Memory Type" is a bit nebulous. Is a Micron Type-3 with performance X > and an SK Hynix Type-3 with performance Y a "Different type", or are > they the "Same Type" given that they're both Type 3 backed by some form > of DDR? Is socket placement of those devices relevant for determining > "Type"? Is whether they are behind a switch relevant for determining > "Type"? "Type" is frustrating when everything we're talking about > managing is "Type-3" with difference performance. > > A concrete example: > To the system, a Multi-Headed Single Logical Device (MH-SLD) looks > exactly the same as an standard SLD. I may want to have some > combination of local memory expansion devices on the majority of my > expansion slots, but reserve 1 slot on each socket for a connection to > the MH-SLD. As of right now: There is no good way to differentiate the > devices in terms of "Type" - and even if you had that, the tiering > system would still lump them together. > > Similarly, an initial run of switches may or may not allow enumeration > of devices behind it (depends on the configuration), so you may end up > with a static numa node that "looks like" another SLD - despite it being > some definition of "GFAM". Do number of hops matter in determining > "Type"? In the original design, the memory devices of same memory type are managed by the same device driver, linked with system in same way (including switches), built with same media. So, the performance is same too. And, same as memory tiers, memory types are orthogonal to sockets. Do you think the definition itself is clear enough? I admit "memory type" is a confusing name. Do you have some better suggestion? > So I really don't think "Type" is useful for determining tier placement. > > As of right now, the system lumps DRAM nodes as one tier, and pretty > much everything else as "the other tier". To me, this patch set is an > initial pass meant to allow user-control over tier composition while > the internal mechanism is sussed out and the environment develops. The patchset to identify the performance of memory devices and put them in proper "memory types" and memory tiers via HMAT has been merged by v6.7-rc1. 07a8bdd4120c (memory tiering: add abstract distance calculation algorithms management, 2023-09-26) d0376aac59a1 (acpi, hmat: refactor hmat_register_target_initiators(), 2023-09-26) 3718c02dbd4c (acpi, hmat: calculate abstract distance with HMAT, 2023-09-26) 6bc2cfdf82d5 (dax, kmem: calculate abstract distance with general interface, 2023-09-26) > In general, a release valve that lets you redefine tiers is very welcome > for testing and validation of different setups while the industry evolves. > > Just my two cents. -- Best Regards, Huang, Ying