From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49FC4C02198 for ; Sun, 16 Feb 2025 06:50:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3793D280019; Sun, 16 Feb 2025 01:50:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 32A89280016; Sun, 16 Feb 2025 01:50:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F174280019; Sun, 16 Feb 2025 01:50:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 01406280016 for ; Sun, 16 Feb 2025 01:50:01 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6CB191C9203 for ; Sun, 16 Feb 2025 06:50:01 +0000 (UTC) X-FDA: 83124883002.09.4A5B153 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) by imf13.hostedemail.com (Postfix) with ESMTP id B0BBF20005 for ; Sun, 16 Feb 2025 06:49:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=BMn+0kHg; spf=pass (imf13.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739688600; a=rsa-sha256; cv=none; b=uDD4qFdSsTv9QdMAm4wfuKryiy3d0NeI1gBmWUJEHmo0aDxJ9PWhrht2yVgtIMTVsOySMZ YIy96Esk4Jzn28ACaJT3SNSYlNxZAGc/LU5xmEJR3GGr+WmW8kO5znEriixaUf+MdZKl5Z ug1jjp0zrHhnn/DngVKNDhBXL3PO9qc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=BMn+0kHg; spf=pass (imf13.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.119 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739688600; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A4i12Hq2xcwxxA1hyCxMdNiAIgNeSCHhJ94p41qg+Yc=; b=iRUcYqli+xAxypdh7aqQV37rYw7po7UriJEdBNVst3ndexXSuaTW/2s2AdfGHTeMrO9Puo IoeXOSV9Kp0QiI1G3+3ghCMc3jc+5iI74oS4AggRBtdFTBQJXvkEhwXQ6Hyq/nqOBgwxae px/EA4dg6VFvk7Hrc2WcGXE1VuDDQnI= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1739688595; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; bh=A4i12Hq2xcwxxA1hyCxMdNiAIgNeSCHhJ94p41qg+Yc=; b=BMn+0kHgoiMOl/YO0XyBSNCYp4nNiKcQJd8LvnAmRjOacdjx1CbeVyziL1EJ+oL8hwUx/HK83Jf+Az/lPcftcpdPsXhvOHDZY3R/KaSGwo/Sh8LUYbfAZxNkJDy7AwKyU8bIImNfjfSWp2/MtFisF6zoNZHKMD/CyoGUgpmlL28= Received: from DESKTOP-5N7EMDA(mailfrom:ying.huang@linux.alibaba.com fp:SMTPD_---0WPVzX6N_1739688591 cluster:ay36) by smtp.aliyun-inc.com; Sun, 16 Feb 2025 14:49:53 +0800 From: "Huang, Ying" To: Jonathan Cameron Cc: Raghavendra K T , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [LSF/MM/BPF TOPIC] Unifying sources of page temperature information - what info is actually wanted? In-Reply-To: <20250131130901.00000dd1@huawei.com> (Jonathan Cameron's message of "Fri, 31 Jan 2025 13:09:01 +0000") References: <20250123105721.424117-1-raghavendra.kt@amd.com> <20250131122803.000031aa@huawei.com> <20250131130901.00000dd1@huawei.com> Date: Sun, 16 Feb 2025 14:49:50 +0800 Message-ID: <87h64u2xkh.fsf@DESKTOP-5N7EMDA> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Stat-Signature: hgjcghi1qnxmz5kjsgdfqkjmdqnjegj5 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B0BBF20005 X-Rspam-User: X-HE-Tag: 1739688598-832004 X-HE-Meta: U2FsdGVkX1/vtXTqpH7aleGyZz6swiNIZT/ix1DbbStuZoNx9/FH1tRXJwFNOPHNJwzmMvexFpEkMU3wN/kBabb/4YpO9ZVYHlYXAz2OC1svxhXFAirSC65dZKOhqQ4AeDoAUn9wrCduK+bygVZ2NO/GgFP6GjBOkSqeXzKuNeJPTO6J4vUNLmJJ5yhtpY51Pymz8jNFFx6AR6PcP9FHuqKjx7SF3kIDr0BvnmHcqRjh8goPir6Y+hnebMBTDKt/yAx7UrrxfpPnbjjrLxSYufBp/y9SuKnsuZq6DZEzXVmTvTJcKORVPcazoUaDPRmB9ark0INVspmnsLfMg2bG5sK7IrBQBA+v95crLr2vfGj9NRLBFZ3W8IqYKf+rYmovTuDnSbTs5dnO9tEUez/AvE4XblRL7yPqcksLcRJwDuVc8zxjICc0S9wIiX9O3GunInwt3UcVDBV2j2CcdOEBmOtj4NIRiSBUWhOCVIj9Uhr15q7ZeR6UgZRU7Ll+ztRxVRYBYCDigrm6o848aoHA8RxBunSYoFzpo0qy+MGL3GLUROLg7naYqhmH0PTSb4zmxEfW94F0T4ch9kb/cDzYvmYz4GIzMnNeivGuDn2Stak8ifQlimCm7RNKg8OhiBbbHLTaZVNYWZJGvv0V1RqvtdHcbj8/EM+4pwt+N8b1OAjJLevQhCz5jZ5npVTTr2cOOccl5qwgSdIFvf8KyPGdAaQ5jaPmo4Tg0iIhrISrXmYekgpD3nGYsOYa1txC0rkRCWwxxDdGvoBF6FCddP4czGKWLq16UOyzQ5Nnw22QDIg2TkUogwJTI5uX1XvGcqgwJbYMxjA6Y3iTKvK8/Obkwqo5/WRGIhuqBJIeyMPPJ1ste7CdxZQcB3JNrYKlbA6CtdXcjyhuMHQL7Ok/SpzHvjvQRWnszYs2kqzQ52BCaeovlDng1wCExM5IK45zhIEQPVXDHJjY3Wvnmpmtdcq NKBViauY Zs0MQQI70VP9MssLWErPuS/Lq19n6GCc2qvrOapgewa/8K0NyNvXYY/75jpE29QscVMHVqTIp7dH1L56yp73FpcIG/UWhJRGtj9LtGviTQwVzJ39bTaF32LVyGu4MbsEaWsQ0i9Rv2mxnpzY9KBQVQlEw1EpS6vJGfaPA1LsTF02aMo1xvmocKeRbLNcI6PQvd/K/zdvFp7O4ZQoF9wiQGVctyFssUUIK4UY7FZFWq+J7WVyDOQbRvq6b3JsrK6+3JhC9iY3fqXyC/gO/S9e8pydx8A3BfGYsKb9Eh+tPOQAyb0wlDOFNc9FLyLm0fLsWpMyVk2cYEq6c4WsDGDfiE5LEmzaM3m/P4e0E X-Bogosity: Ham, tests=bogofilter, spamicity=0.024445, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, Jonathan, Sorry for late reply. Jonathan Cameron writes: > On Fri, 31 Jan 2025 12:28:03 +0000 > Jonathan Cameron wrote: > >> > Here is the list of potential discussion points: >> ... >> >> > 2. Possibility of maintaining single source of truth for page hotness that would >> > maintain hot page information from multiple sources and let other sub-systems >> > use that info. >> Hi, >> >> I was thinking of proposing a separate topic on a single source of hotness, >> but this question covers it so I'll add some thoughts here instead. >> I think we are very early, but sharing some experience and thoughts in a >> session may be useful. > > Thinking more on this over lunch, I think it is worth calling this out as a > potential session topic in it's own right rather than trying to find > time within other sessions. Hence the title change. > > I think a session would start with a brief listing of the temperature sources > we have and those on the horizon to motivate what we are unifying, then > discussion to focus on need for such a unification + requirements > (maybe with a straw man). > >> >> What do the other subsystems that want to use a single source of page hotness >> want to be able to find out? (subject to filters like memory range, process etc) >> >> A) How hot is page X? >> - Is this useful, or too much data? What would use it? >> * Application optimization maybe. Very handy for developing algorithms >> to do the rest of the options here as an Oracle! >> - Provides both the cold and hot end of the scale, but maybe measurement >> techniques vary and can not be easily combined. Hard in general to combine >> multiple sources of truth if aiming for an absolute number. >> >> B) Which pages are super hot? >> - Probably these that make the most difference if they are in a slower memory tier. >> >> C) Some pages are hot enough to consider moving? >> - This may be good enough to get the key data into the fast memory over time. >> - Can combine sources of info as being able to compare precise numbers doesn't matter. >> >> D) Which pages are fairly cold? >> - Likewise maybe good enough over time. >> >> E) Which pages are very cold? >> - Ideal case for tiering. Swap these with the super hot ones. >> - Maybe extra signal for swap / zswap etc >> >> F) Did these hot pages remain hot (and same for cold) >> - This is needed to know when to back off doing things as we have unstable >> hotness (two phase applications are a pain for this), sampling a few >> pages may be fine. >> >> Messy corners: >> >> Temporal aspects. >> - If only providing lists of hottest / coldest in last second, very hard >> to find those that are of a stable temperature. We end up moving >> very hot data (which is disruptive) and it doesn't stay hot. >> - Can reduce that affect by long sampling windows on some measurement approaches >> (on hardware trackers that can trash accuracy due to resource exhaustion >> and other subtle effects). >> - bistable / phase based applications are a pain but perhaps up to higher >> levels to back off. >> >> My main interest is migrating in tiered systems but good to look at what >> else would use a common layer. >> >> Mostly I want to know something that is useful to move, and assume convergence >> over the long term with the best things to move so to me the ideal layer has >> following interface (strawman so shoot holes in it!): >> >> 1) Give me up to X hotish pages from a slow tier (greater than a specific measure >> of temperature) Because the hot pages may be available upon page accessing (such PROT_NONE page fault), the interface may be "push" style instead of "pull" style, e.g., int register_hot_page_handler(void (*handler)(struct page *hot_page, int temperature)); >> 2) Give me X coldish pages a faster tier. >> 3) I expect to ask again in X seconds so please have some info ready for me! >> 4) (a path to get an idea of 'unhelpful moves' from earlier iterations - this >> is bleeding the tiering application into a shared interface though). In addition to get a list hot/cold pages, it's also useful to get hot/cold statistics of a memory device (NUMA node), e.g., something like below, Access frequency percent > 1000 HZ 10% 600-1000 HZ 20% 200- 600 HZ 50% 1- 200 HZ 15% < 1 HZ 5% Compared with hot/cold pages list, this may be gotten with lower overhead and can be useful to tune the promotion/demotion alrogithm. At the same time, a sampled (incomplete) list of hot/cold page list may be available too. >> If we have multiple subsystems using the data we will need to resolve their >> conflicting demands to generate good enough data with appropriate overhead. >> >> I'd also like a virtualized solution for case of hardware PA trackers (what >> I have with CXL Hotness Monitoring Units) and classic memory pool / stranding >> avoidance case where the VM is the right entity to make migration decisions. >> Making that interface convey what the kernel is going to use would be an >> efficient option. I'd like to hide how the sausage was made from the VM. --- Best Regards, Huang, Ying