From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5ECEC43334 for ; Mon, 25 Jul 2022 06:03:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F10D94000A; Mon, 25 Jul 2022 02:03:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A1668E0001; Mon, 25 Jul 2022 02:03:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E850C94000A; Mon, 25 Jul 2022 02:03:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DA1058E0001 for ; Mon, 25 Jul 2022 02:03:00 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B604B40147 for ; Mon, 25 Jul 2022 06:03:00 +0000 (UTC) X-FDA: 79724578920.16.2343D3A Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf13.hostedemail.com (Postfix) with ESMTP id CF692200BB for ; Mon, 25 Jul 2022 06:02:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658728978; x=1690264978; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=xMvvaH+s6vqY1Iy3PvmDnSThXUYdc1gpUxDXwIleWbw=; b=LOHkjiImw+2ee7bW/T6e8CHqAITy+sq31s6C4Mwsb3Wxfut50sTuBDB/ bpyllDSWRs/HH/QxcDb3aPjTvVk9Y0JRA40nNqarIzOQspl9xkgplpLBj xLz4mYh+1COMNdBaJzgnENbF8fP4igPFo6Pk7+Qb0WDkgmfLSiosve1UA J5lTMVpoqsL51iW+Zo5nvNcil4dCwELPFihzy1yUKOj1fJSrC3GSPoXXU wYvovU8kG/iQoayGbjBZOqeOge1spodLtiJccFPE1glcsrtF1F5qJ0RtJ IwcbExlrFrmXP+hI4VlUf2d4D4lZHSGF+2s4G/p4pkGFBmL5NP7mCrZ+K w==; X-IronPort-AV: E=McAfee;i="6400,9594,10418"; a="286379568" X-IronPort-AV: E=Sophos;i="5.93,192,1654585200"; d="scan'208";a="286379568" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jul 2022 23:02:57 -0700 X-IronPort-AV: E=Sophos;i="5.93,192,1654585200"; d="scan'208";a="574913607" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.239.13.94]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jul 2022 23:02:53 -0700 From: "Huang, Ying" To: Jonathan Cameron Cc: Wei Xu , Aneesh Kumar K V , Johannes Weiner , "Linux MM" , Andrew Morton , "Yang Shi" , Davidlohr Bueso , Tim C Chen , Michal Hocko , "Linux Kernel Mailing List" , Hesham Almatary , Dave Hansen , "Alistair Popple" , Dan Williams , Subject: Re: [PATCH v8 00/12] mm/demotion: Memory tiers and demotion References: <20220704070612.299585-1-aneesh.kumar@linux.ibm.com> <87r130b2rh.fsf@yhuang6-desk2.ccr.corp.intel.com> <60e97fa2-0b89-cf42-5307-5a57c956f741@linux.ibm.com> <87r12r5dwu.fsf@yhuang6-desk2.ccr.corp.intel.com> <0a55e48a-b4b7-4477-a72f-73644b5fc4cb@linux.ibm.com> <87mtde6cla.fsf@yhuang6-desk2.ccr.corp.intel.com> <87ilo267jl.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edyp67m1.fsf@yhuang6-desk2.ccr.corp.intel.com> <875yk15svi.fsf@yhuang6-desk2.ccr.corp.intel.com> <20220719150032.000066a6@Huawei.com> Date: Mon, 25 Jul 2022 14:02:49 +0800 In-Reply-To: <20220719150032.000066a6@Huawei.com> (Jonathan Cameron's message of "Tue, 19 Jul 2022 15:00:32 +0100") Message-ID: <878rohzq46.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658728980; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VJV4z8daCeJS5CiVQgXJ3h/rKSHaxrz8VU2t9VzJknc=; b=1GY0++4oaudLYcWrIbCzx8Khreas7uAv6SnOOHM/gDATjPvCtoPQqfeZ76X2UUaSXIuk2W Gvrc9roIJzLImFlh7aLI9buiy22otifT1hJrkFA5WaydkpQiJG6I7u5Du99NqufF+EEgFn shpWP+4NIwpwUDPSMERNfrin4ngPFcY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658728980; a=rsa-sha256; cv=none; b=da359yuIoMyHiHqbGnibGtU8M6HHi/A2rH92eqwHBvwuZQ5TSH4g5jSGMUN7YWbNy3wMln dj9RsUDW1NIK8YfUDwDvuo8f2t1FIW8y6QyilVdmSHKy3G5annTsNU1owo34PVXCCnaVjR 2RXtLvEBZq04kcON+DX2lId450u6KSw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LOHkjiIm; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf13.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 192.55.52.120) smtp.mailfrom=ying.huang@intel.com X-Stat-Signature: edq5p7uipcqxhupygf4yq7pchdo74ajw X-Rspamd-Queue-Id: CF692200BB Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LOHkjiIm; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf13.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 192.55.52.120) smtp.mailfrom=ying.huang@intel.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1658728978-340514 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, Jonathan, Jonathan Cameron writes: > On Wed, 13 Jul 2022 16:17:21 +0800 > "Huang, Ying" wrote: > >> Wei Xu writes: [snip] >> > >> > Th "memory type" and "abstract distance" concepts sound to me similar >> > to the memory tier "rank" idea. >> >> Yes. "abstract distance" is similar as "rank". >> >> > We can have some well-defined type/distance/rank values, e.g. HBM, >> > DRAM, CXL_DRAM, PMEM, CXL_PMEM, which a device can register with. The >> > memory tiers will build from these values. It can be configurable to >> > whether/how to collapse several values into a single tier. >> >> The memory types are registered by drivers (such as kmem_dax). And the >> distances can come from SLIT, HMAT, and other firmware or driver >> specific information sources. >> >> Per my understanding, this solution may make memory tier IDs more >> unstable. For example, the memory ID of a node may be changed after the >> user override the distance of a memory type. Although I think the >> overriding should be a rare operations, will it be a real issue for your >> use cases? > > Not sure how common it is, but I'm aware of systems that have dynamic > access characteristics. i.e. the bandwidth and latency of a access > to a given memory device will change dynamically at runtime (typically > due to something like hardware degradation / power saving etc). Potentially > leading to memory in use needing to move in 'demotion order'. We could > handle that with a per device tier and rank that changes... > > Just thought I'd throw that out there to add to the complexity ;) > I don't consider it important to support initially but just wanted to > point out this will only get more complex over time. > Thanks for your information! If we make the mapping from the abstract distance range to the memory tier ID stable at some degree, the memory tier ID can be stable at some degree, e.g., abstract distance range memory tier ID 1 -100 0 101-200 1 201-300 2 301-400 3 401-500 4 500- 5 Then if the abstract distance of a memory device changes at run time, its memory tier ID will change. But the memory tier ID of other memory devices can be unchanged. If so, the memory tier IDs are unstable mainly when we change the mapping from the abstract distance range to memory tier ID. Best Regards, Huang, Ying