From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFD47C35FF1 for ; Fri, 14 Mar 2025 14:24:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D0DE7280002; Fri, 14 Mar 2025 10:24:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C970E280001; Fri, 14 Mar 2025 10:24:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B108E280002; Fri, 14 Mar 2025 10:24:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8F801280001 for ; Fri, 14 Mar 2025 10:24:22 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1351DA92D0 for ; Fri, 14 Mar 2025 14:24:23 +0000 (UTC) X-FDA: 83220376806.11.CF53872 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf09.hostedemail.com (Postfix) with ESMTP id 1A9B6140017 for ; Fri, 14 Mar 2025 14:24:20 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741962261; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eQI4SfCXYMwy/Xv3Yh4NYErQJYT1drS369IBnJsnAzc=; b=pNg4w2bLZTNDV5l8uIKXJfEYJt628gnhbW3uobKjRU7HA2y/zhVUX2CUQc4afdFdRtIVVH xKtYgRZzt/doMLgcJu2qbW/UNP1wkhI7P5PZWMr1ikDp5yn3wHaY63XG9HQ/NA/zwyAPo/ yPHz6jnU62kXtJ/eIzInXLq6DRzgQY0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741962261; a=rsa-sha256; cv=none; b=NbAmeVrAteXpP6nM+USJaNn7gIH4cqd/87zSfTIFy0ZamnS9JU9OENnVLLq4ChITiuviOW AoG6hlb2W3IraaTEkgWJVxQ3a+FI3SD5Exr++82zK4uLy8/uptyQCrgidByHiv32Ex0Nni Sz8CXd45VDPBjBeHCddPP7HQjmoHjaM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZDml235Zjz6J9YF; Fri, 14 Mar 2025 22:21:34 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 8B9D9140155; Fri, 14 Mar 2025 22:24:15 +0800 (CST) Received: from localhost (10.203.177.66) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 14 Mar 2025 15:24:13 +0100 Date: Fri, 14 Mar 2025 14:24:12 +0000 From: Jonathan Cameron To: "Huang, Ying" CC: Raghavendra K T , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [LSF/MM/BPF TOPIC] Unifying sources of page temperature information - what info is actually wanted? Message-ID: <20250314142412.00001689@huawei.com> In-Reply-To: <87h64u2xkh.fsf@DESKTOP-5N7EMDA> References: <20250123105721.424117-1-raghavendra.kt@amd.com> <20250131122803.000031aa@huawei.com> <20250131130901.00000dd1@huawei.com> <87h64u2xkh.fsf@DESKTOP-5N7EMDA> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.203.177.66] X-ClientProxiedBy: lhrpeml100010.china.huawei.com (7.191.174.197) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1A9B6140017 X-Stat-Signature: hncyzu7pk99dksx8kdsdb3snffpazjro X-HE-Tag: 1741962260-804958 X-HE-Meta: U2FsdGVkX1+NorPEoQOt/X9UGjqWE+Y8Lc0VoWNzhB6TDG3qfp1hJ295F3i5KMNWWEzfRDL6lN4zmc2IsX3G5z5MhSEAArFGla6x7mM8ix9E1Xdn8J1s2ugAm1uHW/qO/eBY0FPRiIDSeqmNVLtp2AltgB0rnAbqsvg+pXBWGBoC0L+KwD5tqlSH1a4SauQA3suo/icj50RYdNOKjJDli5sPBZaGx+TFxaoeKhhQF4OT1JMFvEboSEdv0njzjtbm3bGH0OYA9RQqt0/9rvUHj8AgUIwzh3JTPx1PcxyJ5d2TdvciZv0voe3QfZmw79RRiyeED5RPMKiZ3eYeNkJshaK95mzoxx8FB1hN18gcILfmn/bRLVT0ir0wLRAL9Wsb8w6T/gAnmBOmGzmAxpO5+fh5C5WscHcsaOL6iD14muDafviGCQcjCSSJfiDWpToBlIaifJRdhwqnVSO+JkyzY6bBLh1aOtwKOhAy/EvAjx5lAtJRXhX+RUk/x0gwbfHPEoa+Qgq6JlgyeDwCMNTjqstuW9UpSFW2GnykDlvMGvS1GY971jB2gmMvz7V30Qqjz+8Wxy89aIN41FPemciylTulXqLQGJUoe46dfdjkhZPt95pvqwWwJFHc84qmmMt8FNmKgRQgJBODHNQJ8iJxPjc2YR8PnOzAsnnQMuyN4Szfly7qoKgdYxu3gV4+cFUo0iNusIIc3sdkUKx87HTmjx/xNrYT/WYSoRhLR3SLo0a5Y2o1sd1uABpSJrthFgyCRLbbEhGF94JRhPKD51488lBkrMegg7cD3ez/0D+hadlTJ6M6C013AGz6WxMD+zB3LifJsA044jyd8yqUyJKmvSqED/dRAv7P9k7+IJxYFYs7Lgyw5m3rjT+LegWPEMDIvRhptWZ1BVBa8Q49ncmZzHZZOby/ODZk8wQgBpx1A8PUQQYiAOYGP8+Ld4jxOQu9EwUwx5BVafwZvG5dT6Y EOk4Y9mZ YYyInN2thfjE83vowi3QMfU35Go3+cESYiL9H/MMVdWp8NWBMN79r0Og6irpnenxJOK9x2cAoBf8+fleW20bsQ/4lRHaBnIASXPq9nobyKk9Edmk9I6DlMdrcS8bLJxghoFuOsK/Tml9V3ln3/VhQCZbwDbNDG8wS4VtO3rdGHX7pkK7PluybW6rP5xRpP/TaD72Eino05/sx6FfYo493hIG0FgksdzneMkTJ80p6uVe35woMtrHE4YWUCN7tZhYIhwGTqiLAjshjygTMEx5yWrMcxvp+Rpb0TBIpcJrU0Hpw1wZYlvk4snb9mGGI0QY4RTkcXD5WzSA994PdApvo3+fkVg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000405, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, 16 Feb 2025 14:49:50 +0800 "Huang, Ying" wrote: > Hi, Jonathan, > > Sorry for late reply. Sorry for even later reply! > > Jonathan Cameron writes: > > > On Fri, 31 Jan 2025 12:28:03 +0000 > > Jonathan Cameron wrote: > > > >> > Here is the list of potential discussion points: > >> ... > >> > >> > 2. Possibility of maintaining single source of truth for page hotness that would > >> > maintain hot page information from multiple sources and let other sub-systems > >> > use that info. > >> Hi, > >> > >> I was thinking of proposing a separate topic on a single source of hotness, > >> but this question covers it so I'll add some thoughts here instead. > >> I think we are very early, but sharing some experience and thoughts in a > >> session may be useful. > > > > Thinking more on this over lunch, I think it is worth calling this out as a > > potential session topic in it's own right rather than trying to find > > time within other sessions. Hence the title change. > > > > I think a session would start with a brief listing of the temperature sources > > we have and those on the horizon to motivate what we are unifying, then > > discussion to focus on need for such a unification + requirements > > (maybe with a straw man). > > > >> > >> What do the other subsystems that want to use a single source of page hotness > >> want to be able to find out? (subject to filters like memory range, process etc) > >> > >> A) How hot is page X? > >> - Is this useful, or too much data? What would use it? > >> * Application optimization maybe. Very handy for developing algorithms > >> to do the rest of the options here as an Oracle! > >> - Provides both the cold and hot end of the scale, but maybe measurement > >> techniques vary and can not be easily combined. Hard in general to combine > >> multiple sources of truth if aiming for an absolute number. > >> > >> B) Which pages are super hot? > >> - Probably these that make the most difference if they are in a slower memory tier. > >> > >> C) Some pages are hot enough to consider moving? > >> - This may be good enough to get the key data into the fast memory over time. > >> - Can combine sources of info as being able to compare precise numbers doesn't matter. > >> > >> D) Which pages are fairly cold? > >> - Likewise maybe good enough over time. > >> > >> E) Which pages are very cold? > >> - Ideal case for tiering. Swap these with the super hot ones. > >> - Maybe extra signal for swap / zswap etc > >> > >> F) Did these hot pages remain hot (and same for cold) > >> - This is needed to know when to back off doing things as we have unstable > >> hotness (two phase applications are a pain for this), sampling a few > >> pages may be fine. > >> > >> Messy corners: > >> > >> Temporal aspects. > >> - If only providing lists of hottest / coldest in last second, very hard > >> to find those that are of a stable temperature. We end up moving > >> very hot data (which is disruptive) and it doesn't stay hot. > >> - Can reduce that affect by long sampling windows on some measurement approaches > >> (on hardware trackers that can trash accuracy due to resource exhaustion > >> and other subtle effects). > >> - bistable / phase based applications are a pain but perhaps up to higher > >> levels to back off. > >> > >> My main interest is migrating in tiered systems but good to look at what > >> else would use a common layer. > >> > >> Mostly I want to know something that is useful to move, and assume convergence > >> over the long term with the best things to move so to me the ideal layer has > >> following interface (strawman so shoot holes in it!): > >> > >> 1) Give me up to X hotish pages from a slow tier (greater than a specific measure > >> of temperature) > > Because the hot pages may be available upon page accessing (such PROT_NONE > page fault), the interface may be "push" style instead of "pull" style, > e.g., Absolutely agree that might be the approach, but with some form of back pressure as for at least some approaches it is much cheaper to find a find a few hot pages than to find lots of them. More complex if you want a few of the very hottest or just hotter than X. > > int register_hot_page_handler(void (*handler)(struct page *hot_page, int temperature)); > > >> 2) Give me X coldish pages a faster tier. > >> 3) I expect to ask again in X seconds so please have some info ready for me! > >> 4) (a path to get an idea of 'unhelpful moves' from earlier iterations - this > >> is bleeding the tiering application into a shared interface though). > > In addition to get a list hot/cold pages, it's also useful to get > hot/cold statistics of a memory device (NUMA node), e.g., something like > below, > > Access frequency percent > > 1000 HZ 10% > 600-1000 HZ 20% > 200- 600 HZ 50% > 1- 200 HZ 15% > < 1 HZ 5% > > Compared with hot/cold pages list, this may be gotten with lower > overhead and can be useful to tune the promotion/demotion alrogithm. At > the same time, a sampled (incomplete) list of hot/cold page list may be > available too. I agree it's useful info and 'might' be cheaper to get. Depends on the tracking solution and impacts of sampling approaches. > > >> If we have multiple subsystems using the data we will need to resolve their > >> conflicting demands to generate good enough data with appropriate overhead. > >> > >> I'd also like a virtualized solution for case of hardware PA trackers (what > >> I have with CXL Hotness Monitoring Units) and classic memory pool / stranding > >> avoidance case where the VM is the right entity to make migration decisions. > >> Making that interface convey what the kernel is going to use would be an > >> efficient option. I'd like to hide how the sausage was made from the VM. > > --- > Best Regards, > Huang, Ying