From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43E7EC02181 for ; Fri, 24 Jan 2025 18:05:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A97E0280086; Fri, 24 Jan 2025 13:05:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A220D280079; Fri, 24 Jan 2025 13:05:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C184280086; Fri, 24 Jan 2025 13:05:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6BFF3280079 for ; Fri, 24 Jan 2025 13:05:16 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E6831A1749 for ; Fri, 24 Jan 2025 18:05:15 +0000 (UTC) X-FDA: 83043122190.28.64D5566 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf23.hostedemail.com (Postfix) with ESMTP id AAA43140015 for ; Fri, 24 Jan 2025 18:05:13 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737741914; a=rsa-sha256; cv=none; b=ug+yUFT90gWVSX/3wNt4xb2+i4HKaDWqHr+muCCkp6ui/rrd+XDoj8Jr6ksuPBDMuiwNAi 37bR86UQdyaNbwTskksMb/NbvVFztg6crZmyK6REXLRuh8P88991JcBtDolV5deJetJ7+l +jHo2NEilTkYvf791QN5xEt0OvsN2io= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737741914; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p+3p4ZXCOG75KlhBBWdwBHZNgrtGpr4N8xycjOL5j70=; b=iMlkgwhhoLYjioeIXJBwolMr7ww5zufiLs9T+1Oj0ZCaU2xmgmK7YFNgVHTEqzhWKt8/uI DEt4X7pww44X1kJbYO5ugDBvrsgW8sm++9faYdiGbB1D0Bi2H0IFqEIUEGcb5uI/Z5CoV8 FpV1+GqxwIyblHFLO3ImoeJc6xRfLfA= Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4YflzL3VYyz6M4Y0; Sat, 25 Jan 2025 02:03:10 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 61E681402DB; Sat, 25 Jan 2025 02:05:09 +0800 (CST) Received: from localhost (10.203.177.66) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 24 Jan 2025 19:05:07 +0100 Date: Fri, 24 Jan 2025 18:05:06 +0000 From: Jonathan Cameron To: Raghavendra K T CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [LSF/MM/BPF TOPIC] Overhauling hot page detection and promotion based on PTE A bit scanning Message-ID: <20250124180506.00002936@huawei.com> In-Reply-To: References: <20250123182050.53941-1-sj@kernel.org> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.203.177.66] X-ClientProxiedBy: lhrpeml100012.china.huawei.com (7.191.174.184) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspam-User: X-Rspamd-Queue-Id: AAA43140015 X-Rspamd-Server: rspam10 X-Stat-Signature: fgpsq6txaz99z7razdhbtit7k88tg5pd X-HE-Tag: 1737741913-538402 X-HE-Meta: U2FsdGVkX19xedsSnJ8BvhR2ABVHPA7xPWZReu7WoqkipTIOdrieLmBSLafRgUXp1fIKgCuUt4SOtLJa4CgBAswB0Z3DP9rSEfSHOhhYZ0eQz+YaivetKiMrcq1tTVGvlaDXFyS+PiQfbI/9520i6xrq4dJvjfySwF6Uv/TnJ0U+tf3uxOjeCX+yWuL5Z/Y6+wfN29ry+hjrC29V/MadzgHHoQWFRP9vYoTl3kkDPZ2UmHeQJI1pla74hHRtm3aS9tHLipEi6jKs7Gk2cSidVeYU8NI/NRtI7+PFvVGqqyNGvGanTQM1HP7TIhKfX4CaWF8WpVpvdYBrSTF3jIaq8QGRivlzqIEgI57gWI8nzD7nfvnQ7q4kxNsFClPurjl3cqwgCNaxZPHMxI90o4K3yJQEszFT0slxPDzgWjxIyprlIrHg+gvsE7dFQcRestvmvF+HaSY0t0NLsxJ5wywOgMQp36TXe++DSBG309WwgZT10jgJiCXqNDZia2BMwqJlPyW0aB6yBxjNQExvtZkDd7yJ8WgYMSWjpvlOKoM13wxiET3HGqKGfLoXXcmQk1WHr7g1TxutSQUKnyw059O4lwjv6rjfR9jupBkAWyMtdEm3Xo8stNXmRAjkoHGSWDdKW8t9nxzGWLCd9n1pe/h0SIQjWr+r30EwDT+PG1RffmZSShDM+lFWv21Ycfp2QJCuTDQeOLQXMvJTVWPLw+sSXgEdYbWEnBJATK+i9t53UaTCM/sUXHmAg3IokIPdlfvbqBrpWTA+fvDRy1hJ5X3wgNkAuAq6Jkjd7t8Ac6Tepy/Lj6Ql9TtwPSwuk80E+hqAav+n47wW1F8N6w/R/LCXFr5ReSVN5Oixmxw6Yc5VB0wi6zFO6vpQTHEAwxWndVBmfIe8Pmp9TuONOJTNu4a9jZKQO1wNr6rMhxnWtYlXdjf8Z/c7rEkXPf/nRXLT8tXaK0gel3zFUJnWW7JHBHa 5rL3vi21 vAHdMgkLYeqQZMKnhO8eTDJ3RG5BpV+m0bGq1XbT1QHKrGKsX1go1rEqge0DH5OfN/xMjqUUKAkZ6kKHVGj/mQtOQDK+2ke2IgkRdDwN/MIwDxkNFqykfhUANQcELZ2tl9QJ+NSpcBIc8FhchDtrGt9bVl/R+mIRQdMZYUg20M9J3PwYOF+VHU0MH1BYP4MmT2w/JAOgS380XJS+jyV+xlD7xzpZxWiw7OK3zhFkEKdKnvbPBKUyBJl9COvQxuTcxUJ6NNNqGGzt9I4KJWafzsXr9yL2gViuAnvP1xT5nSQEMrY6fYy2PUd21qFKgC2JMI4q4qedwz0oLq9HN8sFPlRQ/dGZ2eiat4aXu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 24 Jan 2025 14:24:40 +0530 Raghavendra K T wrote: > On 1/23/2025 11:50 PM, SeongJae Park wrote: > > Hi Raghavendra, > > > > On Thu, 23 Jan 2025 10:57:21 +0000 Raghavendra K T wrote: > > > >> Bharata and I would like to propose the following topic for LSFMM. > >> > >> Topic: Overhauling hot page detection and promotion based on PTE A bit scanning. > > > > Thank you for proposing this. I'm interested in this! > > > > Thank you. > > [...] > > >> virtual vs physical address based scanning, > > > > DAMON supports both virtual and physical address spaces monitoring. DAMON's > > pages migration is currently not supported for virtual address spaces, though I > > believe adding the support is not difficult. > > > > Will check this. > > [...] > > >> > >> 3. Discuss how hardware provided hotness info (like AMD IBS) can further aid > >> promotion. Bharata had posted an RFC [4] on this a while back. > > > > Maybe CXL Hotness Monitoring Unit could also be an interesting thing to discuss > > together. > > > > Definitely. Thanks for the shout out SJ. I'm definitely interested in this topic from the angle of the hardware hotness monitoring units (roughly speaking ones that give you a list of hot pages - typically by PA). Making sure any solution works for those is perhaps key for the longer term. Not entirely clear to me that we are ready yet for data aggregation solution, or mixed techniques but definitely interesting to brainstorm. Until now my main focus has been on getting infrastructure in place to work out the lower levels of using a hardware hotness monitoring unit (using QEMU for now with TCG plugins to get the access data). In general, not stuff I suspect anyone will want to discuss at LSF/MM, but perhaps providing insights into how good data we might get could be. Unless the hardware units people build are very capable (and expensive) the chances are we will have to deal with accuracy limitations that I suspect the users of this data for migration etc do not want to explicitly deal with. If our tracking is coming from multiple sources we need to deal with differences in, and potentially estimation of accuracy. Anything efficient is going to have some accuracy issues (regions for Damon, access bit scanning frequency for your technique, sampling for page fault techniques, data in wrong place - access bit's will tell you to promote stuff that is always in cache - arguably a waste of time etc) I've no idea yet how painful this is going to be. Using the different sources to overcome limitations on each one is interesting but likely to be complex and tricky to generalize. Maybe access bit scanning to detect hotish large scale regions, then a hardware tracker to separate out 'hot' from 'warm'. Sounds fun, but far form general! Lots of problems to solve in this space. And when we have done that, there is paravirtualizing hardware trackers / other methods, application specific usage of the data (some apps will know better than the kernel and will want this data, security / side channels etc). For stretch goals there is even the fun question of hotness monitoring down stream of interleave, particularly when it's scrambled and not a power of 2 ways. Again, maybe not a general problem but will affect data biases. How much of that we want to hide down in implementations below some general 'give me hot stuff' is an open question (I'm guessing hide almost everything beyond controls on data bandwidth). Jonathan > > >> > >> 4. Overlap with DAMON and potential reuse. > > > > I confess that it seems some of the works might overlap with DAMON to my biased > > eyes. I'm looking forward to attend this session, to make it less biased and > > more aligned with people :) > > > > Yes. Agree. > > >> > >> Links: > >> > >> [1] https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@amd.com/ > >> [2] https://lore.kernel.org/linux-mm/20241226012833.rmmbkws4wdhzdht6@ed.ac.uk/T/ > >> [3] https://lore.kernel.org/lkml/Z4XUoWlU-UgRik18@gourry-fedora-PF4VCD3F/T/ > >> [4] https://lore.kernel.org/lkml/20230208073533.715-2-bharata@amd.com/ > > > > Again, thank you for proposing this topic, and I wish to see you at Montreal! > > > > Same here .. Thank you :) > > - Raghu > >