From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1040CC0218D for ; Thu, 23 Jan 2025 10:57:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BDBE6B007B; Thu, 23 Jan 2025 05:57:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36CB36B0082; Thu, 23 Jan 2025 05:57:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20F376B0083; Thu, 23 Jan 2025 05:57:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 01A896B007B for ; Thu, 23 Jan 2025 05:57:51 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9B1411C6DC7 for ; Thu, 23 Jan 2025 10:57:51 +0000 (UTC) X-FDA: 83038416342.14.1327239 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2078.outbound.protection.outlook.com [40.107.237.78]) by imf28.hostedemail.com (Postfix) with ESMTP id 8B9ADC0017 for ; Thu, 23 Jan 2025 10:57:48 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=doh+FO9I; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf28.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.237.78 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737629868; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=U4OfIM7g8nqw+RHN+y1hZGKIwQyjMtSAYWHHPiGqC0M=; b=fLaYSimyg5FxcG7v7EFBdd57QzIkiWiBo41TLfITqpJ7fXHE7QCorYUFWf77dozdP1bxIz RFC1QfTW63LKpKcposeNlWgC1Qv/7+G43sN+bKy8XH0l0nxr5fYsqWC60DLMJxCrH9tuM9 weVk17H0Rlji3g//p0Zipcxs5oovqgA= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1737629868; a=rsa-sha256; cv=pass; b=qzO9qjJVs4cMB2nyG6agM1suA4QpoWEpr5AQpKBRMjba9YKi6uyc2/k3DlVokGP5f+aP6b p8Ng8yVOZfvQVSZJZDVsw/SMhKd9L7o2fIU0wj85DGw6MeDulpDpO6kBAXK2DsZ1Ebc4Nv oCU+1eStuz+oE9EuhUrdx0E5cAalYVA= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=doh+FO9I; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf28.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.237.78 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HWppPlHAlfHbQkkjF4ucBJNAE7h3rKcvL+M7Tu8S1YpSntp5vIvvOBCqf+kcPJjIptUGwNurIowL9tLA9LX54mXWZIvoaX9mB64ZCpOUEzFpERmQ5AdJIXyFr+EGttoZnTttSTL2OOLyxOWZGRoykdpU9jVujt6m4PQMFIuFgAix3Y0BGbKWByjGR4hM9hyTJHTheVd9eUZIMIvrjFgZJXhUxrirZGtXxOq6+bmV9cTg2CJuZuBCaTecymLz4jCirV61DIH/wBMkMa4JfhLe/jPWFG4vwx4REVsuGkCGAGEAkMffmjld7Gw8hw5+YFmqrH6rljckPAPytGy6se7wOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U4OfIM7g8nqw+RHN+y1hZGKIwQyjMtSAYWHHPiGqC0M=; b=hmvSMCSbpJaGPDcH0ndZ9a1NYvRRl925CZItV+24LsBVFB39Yeaqpj0cY0dG5EFjcnVrZogeB1idsXKQYtMG6v1yYYU4reTHwa83hXfI4a9+ru+LekYVMvW94gUc2QyOIOnDS50jz0LBZ9UQ0Rz3ijfowXMuZHFdj7jz0asYqGBOAuVl3c3oNGzUc4Yzk/hbbHsyk1/mEopitt9vWOLp17y8Agt2NH2y/lvVtxODSg4GZRnst5BYVLonvnjhvRa/qX81kN1uoCLwcZIhIGXKfrdDwGiUo12mWy3CzkUITtLVn6rfNyOZgukvOkFkQLzKkJO9eUu6GZoEcelrxGUSgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kvack.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U4OfIM7g8nqw+RHN+y1hZGKIwQyjMtSAYWHHPiGqC0M=; b=doh+FO9ISjbL67hDGfkQPHXmVaDNxKnuqaEh4cZzEpAaj6DiDMclnGvrZuLnp/qmJ0QfjvQgNtyIib7VxRZBQdrqFIteM0U0sau1UiOgH0Go1F7XTQSJ72E1lE34SeeECJEfuTzJ2sImSm/4mkpuIqFBUaY4TIRqlzUKYc44Oxc= Received: from BL0PR02CA0039.namprd02.prod.outlook.com (2603:10b6:207:3d::16) by CYXPR12MB9340.namprd12.prod.outlook.com (2603:10b6:930:e4::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8377.17; Thu, 23 Jan 2025 10:57:44 +0000 Received: from BL6PEPF0001AB4B.namprd04.prod.outlook.com (2603:10b6:207:3d:cafe::35) by BL0PR02CA0039.outlook.office365.com (2603:10b6:207:3d::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8356.21 via Frontend Transport; Thu, 23 Jan 2025 10:57:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4B.mail.protection.outlook.com (10.167.242.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8377.8 via Frontend Transport; Thu, 23 Jan 2025 10:57:43 +0000 Received: from tunga.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 23 Jan 2025 04:57:33 -0600 From: Raghavendra K T To: , , , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [LSF/MM/BPF TOPIC] Overhauling hot page detection and promotion based on PTE A bit scanning Date: Thu, 23 Jan 2025 10:57:21 +0000 Message-ID: <20250123105721.424117-1-raghavendra.kt@amd.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4B:EE_|CYXPR12MB9340:EE_ X-MS-Office365-Filtering-Correlation-Id: cdd0ce97-b5ca-48a9-b947-08dd3b9cc368 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|7416014|82310400026|376014|1800799024|13003099007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?0ZGfJfg7bu6dR+DuvN54Dcn8nGgHQ8rhm/r+uQFKYRmJb8ckW0HZWySouOmi?= =?us-ascii?Q?/kRjUh6lYnWChdrorHrJMTF2WvefJZYyf2tfbDLSK9TNoqVuLhAc/Np+IbxO?= =?us-ascii?Q?WoWqWu0f66I6gmYgjIxkJd4rrFOWMXIg687ioAB3r5MxapBSCuIoT1svVeVn?= =?us-ascii?Q?NYW8Nc5opoVwIpTGonUQg/rVFf+5hUE2XJTY4uKST1E2bf+JVI6alkZmaJy1?= =?us-ascii?Q?zEIRkg06QTUIrP9QX2hggcqOoWeYpLtWABB34i24mZXzm1nBuE+d+XbODcFv?= =?us-ascii?Q?GXcVSfVWOPk/BCctTOVGckcglCPj/CU7CJM/EGpk8VHxbiuJM1y9J8WcNSWk?= =?us-ascii?Q?8fRPkzXTureXPBWDsIOD27Ph3EAD/qgU++M7fhPVgBu2lpC319iOOYT18g5D?= =?us-ascii?Q?ArJfDV8NKlLlS7det2V1ad2HlDJD+zc1Qw9KMruTtFggYJtY0uSWrd0Iky4d?= =?us-ascii?Q?bJ13MblE1OQEdW4CcbeG0wTvaXXrMXuA2Mkh4SfOiV0VkFu0jOfGahNOoPAR?= =?us-ascii?Q?+4p4hxFWe8ShDN1OLNGd1QKjE3mUzXgHbKF8EJIavEjR5xc0cONTy4Vk7NPi?= =?us-ascii?Q?oVtksUGOerItKDoUp4eY8ErbBa48s+DqAZuwJl9WIO4PWtpY7cig1p3PEbKp?= =?us-ascii?Q?5qJXc+uZOEf48x9Nvnew0p4up2/+3YMBb/4O3In0yIDF4VUoCxLX7P4fE9D1?= =?us-ascii?Q?R7PUiib60Lj4lgNnNBF3M+VsKDKPJCcqn05ndYKQaYX60TWsyoAHL93jFAlZ?= =?us-ascii?Q?KdXgn1HB5RiIZHcodnCa989hflLDJLWLNgEpR9PMuX+KEKS66Yg4WxJEmbsb?= =?us-ascii?Q?+NnQBdjtQmBgIzgoKwxfrMbJPvrTD1yiK3v7XaThBfaPAQN6MpQJr4y7X52b?= =?us-ascii?Q?UWy8NDS9iHBiHMcJ1rEL2M1liiZZqCVeQjzF3bYpRFN5YewFG3MwE+BEr3aT?= =?us-ascii?Q?bAqhADuLywhR8KAnv5WisW3oFTc1Sffh2RuqLmXWSgSNAqdok9au+F6HEzSK?= =?us-ascii?Q?H2E0tT5vF9gzf2udWTC5PSG9miPp9VxhBvubeKgMu18aPbGDdSKRuhgIZ/AE?= =?us-ascii?Q?SmEplvT8r28JiN7ryl5yQfoIDGf3lQPCgu3Rf08PVa1qcE1r9XN3K06XIxjE?= =?us-ascii?Q?DgHqDpb/JiMvAvH0wDE1L3khoB8XBFhzK3vsE6/kFM5ybEHgMxpIirdjyXUk?= =?us-ascii?Q?3wUBgilhpiZ5xmAwLIRK4aaQh3+O3AXlhmqraFLfuDqc8H0oF1Mp/9gmQfNT?= =?us-ascii?Q?IuRnO3+PnYlN1Kf0o6XZZhupbSj6Ifi164N796TkV6Os2FgP4DEdAACfKiAH?= =?us-ascii?Q?un17GPYiLr0kpI1QdhRXd5u1/zQ/NqNffKTLKczuZsNycyrkusOJhLrQxM9S?= =?us-ascii?Q?rpebTTpfkKRfrbIYCzNSDBl9IA71yrHAqoGzMDpY+LHbeKhRK0zyl2jk40Fj?= =?us-ascii?Q?Bw+eVWbaM3Ax3dsrH8wnYy1tqdSLgQyh?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(7416014)(82310400026)(376014)(1800799024)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2025 10:57:43.5210 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cdd0ce97-b5ca-48a9-b947-08dd3b9cc368 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR12MB9340 X-Stat-Signature: rkyzgu164q3qn593gk97dqduwa7jnaxr X-Rspamd-Queue-Id: 8B9ADC0017 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1737629868-583774 X-HE-Meta: U2FsdGVkX1/FSx8OG9eZp7TWHWLZwFHL82URaHqzYYXxNs4bHi/zHfaX5ASuk+7EsmGl7vTDAGZoJohvBYbywtGcJF4Ee84nQnxA1Ws2zrs8bq83/YL130xWbRRMeljaQvg/KXeuOKPj7AR6T0W/ZWJXrt8/HbS5w80r7UUks5PRVt0V0SH8eARQJzjGuyGgL2RUg6WlidDTq6botkk2TDqfM3U+iCX01vBimKBkte7gqJ77/lsOm2FEoethQgnfKbkit3ZtZcJzTxroz25odGfIdQiCWpsxM+XuBnYzpaJwVr/2qqT0dtsOflhWyY84DR/dA4ElOVF5BEnJithP1LErtPPmofLNoIE7TFpAKnhg5W00EO27GGpg1VzJY3uNLGUoEeLj0UjHL6i3Tj2MmoZmIYErHus6fxiFkU5lSDthfzCUWvlDIh99YawvCGRIbogiQxohgAAJ+GXzFHGOzwqUApbd6KgglfzC5HFMkPtDN9oXWJBLHGrDfV7FigwIeidWCe4r3gU3pmgB+BO1CFFymZQslHK+H0e/HZ7VYrk9jdu+tsfuMs7Z+/BMmgHRC4G2eEi0Ic6S7ESqdytRBIeFMhLENC64LAKiJ8G97BCzpy/VxWnWxaI1yRS2OmqaejTaMlQ5cTteCrBw04nOq0xq7QyZBCZ/71rL6XWM8x368U6AofbbapiY45RreHZBlpjE28pZCxkxrKkxwKr/VbXNx52hAPHcWUxZNFI/ZgkH18iiTrlG4KIWq3XmPkeFeLclToOCsMF3o3eVw2wWcodyg8/TKnlJZOa9YHm1kkclsjwRHT6H70+/ZLMxmzv65X4AwVavifyKvCIWqwJCBx5ATNGC06AU5K/8aAzNrB9tI+KiPjJrjZMQ4gENkmUQvmYVfS1wZi6AGbKCd1PKy/nXH4+c2SrT5e63LKdTF3qaBX49JV0X2Uq8ohb5f32UNaipaRUF7lC0yPd7MZG enQLIv0W mfIpHO71qP0SMTNUGb3H6IB+ABmTJvDfar49h56JTtschUfYAwdIdpuMI4dRLcMvLxKM4v5r/Z8aBYTVwdQtYpMVR+/JPp+OxDsQJ/6C4JCDXnl7UstF6nCgRoDWJP49vwHyLuLIv9SZvf0qig8R0nePoXx9Khob4vf5LLRb3CPFPgXLesxOtd4WteSGv++f0RDEcVAOpI63EfpjGxl7wAvRA6e9qbQNQZ5hue8QF86/w1j2DT6jzJBy3kXqQhAuh3TjxuLt2Fdnzg1XGuK+cdrH0/vu/w/p5JkC+sR0Cy03H2MwPbkuXEMmAiDJrzVMstMx69wLgw2IQhdUzSje1doF5wc5OLMAjqBeAOAxbNJxm8sNnGCDU9NFY5VbMkUSUbekwgqjofriRCYoLn7ixDTLOOBuqkLQZrOstdODGulY8+BKi1dmqDuU534G24yGMLciuWM6JeESGt37HtACuYtdTIRZFDSSko1JIWeza8zJfjf+QpyIJSBifRdUU7gejVqbf3sxgmclUxHn0WDCpvw7QNNocBVgVX7ytQYHCClAg6EP1zwOZolc5EHIpuQHLN7K3ORSQYIPaWOQElOoIn2W8fzdrxPfbR1D3xfOhq8eCFspjrhQVNFMLZVPnfLbSAHA0GQa68ZAk3UknMKZ5hnBqViIzSbggV+Ejz4X8Ln8bcdnWyfrPr//44ym1JX/uc7zmE6O0EkW6GhILJCq+/eRltrXS6KiTWzLGoDzIuecNDDAfpjLEJx37qko8NP4K5aDUA9YhHiECrjnlcSOOoHP1NB5uH+IgvubrnA/Ze8k8lyuC/E7CEpaamLgkQJQ1NKd+fDfajd1gXNZB2Ccf+mb8Fg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000162, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Bharata and I would like to propose the following topic for LSFMM. Topic: Overhauling hot page detection and promotion based on PTE A bit scanning. In the Linux kernel, hot page information can potentially be obtained from multiple sources: a. PROT_NONE faults (NUMA balancing) b. PTE Access bit (LRU scanning) c. Hardware provided page hotness info (like AMD IBS) This information is further used to migrate (or promote) pages from slow memory tier to top tier to increase performance. In the current hot page promotion mechanism, all the activities including the process address space scanning, NUMA hint fault handling and page migration are performed in the process context. i.e., scanning overhead is borne by the applications. I had recently posted a patch [1] to improve this in the context of slow-tier page promotion. Here, Scanning is done by a global kernel thread which routinely scans all the processes' address spaces and checks for accesses by reading the PTE A bit. The hot pages thus identified are maintained in list and subsequently are promoted to a default top-tier node. Thus, the approach pushes overhead of scanning, NUMA hint faults and migrations off from process context. The topic was presented in the MM alignment session hosted by David Rientjes [2]. The topic also finds a mention in S J Park's LSFMM proposal [3]. Here is the list of potential discussion points: 1. Other improvements and enhancements to PTE A bit scanning approach. Use of multiple kernel threads, throttling improvements, promotion policies, per-process opt-in via prctl, virtual vs physical address based scanning, tuning hot page detection algorithm etc. 2. Possibility of maintaining single source of truth for page hotness that would maintain hot page information from multiple sources and let other sub-systems use that info. 3. Discuss how hardware provided hotness info (like AMD IBS) can further aid promotion. Bharata had posted an RFC [4] on this a while back. 4. Overlap with DAMON and potential reuse. Links: [1] https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@amd.com/ [2] https://lore.kernel.org/linux-mm/20241226012833.rmmbkws4wdhzdht6@ed.ac.uk/T/ [3] https://lore.kernel.org/lkml/Z4XUoWlU-UgRik18@gourry-fedora-PF4VCD3F/T/ [4] https://lore.kernel.org/lkml/20230208073533.715-2-bharata@amd.com/