From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AD78C6FD1F for ; Fri, 29 Mar 2024 17:28:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96C556B0085; Fri, 29 Mar 2024 13:28:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 91C3E6B0087; Fri, 29 Mar 2024 13:28:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 795A36B0088; Fri, 29 Mar 2024 13:28:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5C2956B0085 for ; Fri, 29 Mar 2024 13:28:26 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CF9B880343 for ; Fri, 29 Mar 2024 17:28:25 +0000 (UTC) X-FDA: 81950760570.11.F45C114 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2118.outbound.protection.outlook.com [40.107.220.118]) by imf02.hostedemail.com (Postfix) with ESMTP id E856680008 for ; Fri, 29 Mar 2024 17:28:22 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=mSIt+BXW; dmarc=pass (policy=none) header.from=memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf02.hostedemail.com: domain of gregory.price@memverge.com designates 40.107.220.118 as permitted sender) smtp.mailfrom=gregory.price@memverge.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711733303; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5FmNv599+nh86396q8j8OJAB8UeEzwCNMcoM6fgFtX0=; b=0tVEXFk3euBq5zlpph6dcgyFgzFMU1DwX5GiyEmK7Cn/qJurBnbPTDhJOOIjG7ns6t25Zp sCndNtNvi0WxWGbBRK2yCmGJbR4uVdMzi90rqnuP9+ETSFxvb4E7/SFPpiTxppYOp/d9nn +gYq1Eh2djgzi6liywV8kkuUjiDstwM= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=mSIt+BXW; dmarc=pass (policy=none) header.from=memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf02.hostedemail.com: domain of gregory.price@memverge.com designates 40.107.220.118 as permitted sender) smtp.mailfrom=gregory.price@memverge.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1711733303; a=rsa-sha256; cv=pass; b=uz1/nUR2UPXbOi4rfCXtZ7301bgjLwtNGranjJl0F1ttferHSLGwS1AdO5KRsQUTwe7qMU h8Rd4zChsj8T0QrSx7Sr/dDWufkw6LQErWruoGQ3It0IDetUAPbrx9qiYEAChpu7SlLsTw pt/q8AsVJNI659jt9OrSTGhoeRI4EVU= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YEvSK0jIhcwrjKLKyJ3CnWZ9eAAoLf8QUiod4PP5VJR+QGv7qhUDqCahDTuBiTipE5Bqmw502i3Gy3cqJcOH/PuqG7j7UuQbVvCYANmbnDotXuepWTHVCnZxSbd97ig8qT0N/cMG9NEi7GydnEG9tToSoYv7uGN8EiNWMK7IcilFuAVJrxf1k/e52XolPHWt0TFPzeE53oFeVel8Khq4G3Wi1+ds+0VMMP/ewIPIyflDtYym3v8HzBlTjAk1mQS5LigkAzJ4v5WFzj5DQ73iz8tvVt4LKoYIpNZFz+3axk7hU8uCt57qUM3jPKay8QkIqXJrV7GxmDk5bwHdAmJ4IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5FmNv599+nh86396q8j8OJAB8UeEzwCNMcoM6fgFtX0=; b=YX+NqJ7kiBcgCU4WG3SLeukDxphuMq+ZOBSFnkyLxFmLOYnU1pdmPUsJsR9ohGEuBErTSZkpMd7lGA5Vkysgh/Oe4mIWsrmht2b5m613jdokpVya1T2sW09kW1PLzrUBhbxaw78b+HZZLURZ7op2IXwgRbd+HACvIE8pLanhs7BeTfhAYEGVhKQFX0Cm1foGPC1xDNigeaEf3zdzqrcZ2PvOsRZlMaqWd5C3pT4sPMY5dWcIe64F+qKOVJBuO1sEbQdEzAsxPFpTB4Q6vDdNy97a524pj0e4Kzy3o5E8cqBpU1Ub2f9BtdXimuxJeuDXlfw7gT/7mMVCe6HS7E1gOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5FmNv599+nh86396q8j8OJAB8UeEzwCNMcoM6fgFtX0=; b=mSIt+BXWj8HqoMUvd1NgeHHb3Bl7+CLmIV6e87qSW7W0vMi8WRWyQxatEItKFUizZs/AOP0WjLxbsHG9mF8YLycpUcJN5S65lSLL6NvPZ6vlTjHo4A2hgtwmcKpCVFj4NWJ1nuP9fZOpaAqZD19jl85/vt9fhsPaHjRryTh5ksc= Received: from SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) by IA1PR17MB6213.namprd17.prod.outlook.com (2603:10b6:208:41b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.37; Fri, 29 Mar 2024 17:28:18 +0000 Received: from SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::6657:814f:5df0:bb5b]) by SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::6657:814f:5df0:bb5b%5]) with mapi id 15.20.7409.039; Fri, 29 Mar 2024 17:28:18 +0000 Date: Fri, 29 Mar 2024 13:28:13 -0400 From: Gregory Price To: Yuanchu Xie Cc: David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Huang Ying , Wei Xu , David Rientjes , Greg Kroah-Hartman , "Rafael J. Wysocki" , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Shuah Khan , Yosry Ahmed , Matthew Wilcox , Sudarshan Rajagopalan , Kairui Song , "Michael S. Tsirkin" , Vasily Averin , Nhat Pham , Miaohe Lin , Qi Zheng , Abel Wu , "Vishal Moola (Oracle)" , Kefeng Wang , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: Re: [RFC PATCH v3 0/8] mm: workingset reporting Message-ID: References: <20240327213108.2384666-1-yuanchu@google.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: BY3PR03CA0013.namprd03.prod.outlook.com (2603:10b6:a03:39a::18) To SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR17MB5512:EE_|IA1PR17MB6213:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d8BDYxitSD2of22mrgkzudUDRsPsi3J4QRIX1aN6Hn47flVUu6/IEo8vXlu0VHBkIbZ8ZZwn4cEsn16gqx6qC24jYJoF/0lNLtz0HT8OAhnW93ZTS/5ykL3gvFn+Iwr6vR6mFuZPegtgUUM7vfrvJ8R8HhOSQJ7adpSbsha0gRnxNdiYbbZRp6F0qNMfW9IH6jEddIlPePMErn7j+H8l4HpdODZqhVq/+FVigrd3dl3txx0EszWm5MD8ql/Xpo4crUn7OHq6d/rnGKX6Qal3iJOeOh43ignbXv7LF8ErhkGlX4ygha8NZ+c0YuE30EAGjtU68ZftoVw/3hn0k9BHXf1fyHQcta5FB5r253BNO6SEZ1A+IxFSDwtpNSTMQXBTO+gGFrsPeo3jSraPARnDaOLUV1x0KJX385UKYAiohQjtYqQu4gpLlK+KnAnF0Mpk/+jA5YBtU3UgOgZVb7HFz2W6B6g0+/JjRaZNpMAefL4ntlLg+LlIGLmSNjrsvhI46/qp0ih2Pbk0orodHs7pymuwMU1dNHvEkNoavtX1JRjRPwd1xRId8UJ51Vrmu7jHNkbVRn3usVRoU8hJtPXnUenIKqogU1YflJXlH8kj6AeSiNBEj08O/YlUvnszHK5J3nkXaLvamXb6u7sdHGAeNYzStHO6OzOVt8v2+Ix3xos= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR17MB5512.namprd17.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(7416005);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?alY5QzNYOUFSOUNXUTVrVzYrNElGeGZJZVFKVURNdkpHZUdURnd4Q2dzK2N1?= =?utf-8?B?QnlvMG41aXJQcnlCTjNIMkdvbVZoNkVQY2QzNmNVQTk1YkRsekFIYTMvSjlY?= =?utf-8?B?dmpQVzQ2RWFsOXMxTERYWTdOaGlNNHBSSkZMc0lNSmUwbS9HdENtRGFIMi90?= =?utf-8?B?QVN3ekZsaG51QlljeUsxNk9pSG9ucVpYM3R5SFJFNWtsWGloUW9zSnZhWE5m?= =?utf-8?B?Y0VZaHAwTENQemJiUGRZWmN5b1oyaDZSU1hWK3I2VjNRWG1aTm5CelpuaG1s?= =?utf-8?B?T3lobXpDQ0xJYUtlSEt5a04xVzZ3WWNLZjZSZUp6MUlvVDBtTlVmbVZET1c4?= =?utf-8?B?NDdmZ3BXdjJRNTBLakJ3N2R4SERuUDJjR0RwUGkwS1BzL0U0SjhDU1Z2d2ZX?= =?utf-8?B?RXN0akdxeWFtS3A4M3FUbHhqQ1IvNXR0aTNQSk9qcU9ud25CZjIyOVlHVVlp?= =?utf-8?B?emhXcW5zK3pWMWZqSzFYNXYvUmJvTkFIK2tjUmY2UTJsV2hNb1FvUkdXNEQ4?= =?utf-8?B?RUtvZ0VGK1NtL2tBdzNobnk5ak92NTVBL1RZeWVWcFlSUTNKK3FMREFqWnBC?= =?utf-8?B?UEdlMFIvZ3RqWWhZd3ZMMk9acDFMTWtBaExvc3AreGRSRTIvMDM3RUI5OHhI?= =?utf-8?B?NTEvNyt4VkxhWlV3THRoM0dGVlVzVCt5cW4wNSsrL3lMTUszTmF3M0oyNGpk?= =?utf-8?B?eUFhMlNtZGF1bUFPcWZETHZOeG5iS0YvUHlDWGtuS2NQSHltSG8zUGlqdG5B?= =?utf-8?B?SXVQM2R1dHhSa2w5RW4rZWdrMDY1S1FjS0h0ZFNBbVN4dGtMbWdNTXJTSDhs?= =?utf-8?B?WUVmUkd2Y3FXaDRCRldBT0lPZEhXUVRmemtUT1BJdnpCWnVqQ2JqTitROTRr?= =?utf-8?B?SEVXenMxTzdDaVVNdTBNN1N3NWprSVZnZ25DYW1YQWpzVDNSZW5xcFVINDNH?= =?utf-8?B?NHVvTVdXbDl1UEt6Zm50U09nSGRXcFVtbVd1L0k0QjEyMFgzd0dXTGdNdkt5?= =?utf-8?B?RU45YUo2TkZPOVpocnA2ZFhTUjVNZGRDNGJhUEcvUlhtNmNnTmNJeTRFam1M?= =?utf-8?B?a2Zpblh5TmhxSVV0NUxRbDdxcElydDMwMnVpSWFLRjN1b3VqKzJ5RjhvaFF5?= =?utf-8?B?Y0h2dGNPUUduMXRpRUxJUit1ZEtIMnRTZXNsWW0zZVRYUEdzd1RGSzl3Rkpv?= =?utf-8?B?Vjl4empISWJISDNibmN3Qnlpem5ydWVIRUVlYjhGV2lpMDNyUEJ6eVcrMkNF?= =?utf-8?B?alBUSjZZbmR0aDBqVFRsS1hjUytVU3JCQnY2U1NJR1J5RmozTWdoaWdZQXZR?= =?utf-8?B?OVFzbUg4dkNFZDVOREErK0ovR29BS0hjK2ZmRXNCZVEvQUY3QzJKM0pXM0VK?= =?utf-8?B?eFR4eWRpYWVDZ2VPalVxN1JUUzg1cG1DN1FKTXRXanhNdlVpQW9LcWxoUVkw?= =?utf-8?B?b3BsekZXeXRBanlUQkpIWS9zbWFQOVBET0hnZ25IcURKS1h2T25rcHg4bEx4?= =?utf-8?B?LzR5UERNNUxSa0FMM1hBWnJUcXJxWWpFcG9EWHNYZEJrc3lnS09uSHNKbkZj?= =?utf-8?B?RmJocVp4ODRIN2liSlBoSU9NSVJuZHdncnhrSDdUUjc5aDZnRlJTRXplVFla?= =?utf-8?B?eUxDWjBsem5ncEM2THBKcUZXb2VzbEladzZFakJZbFBXbDRmcWFaelFWL2Nw?= =?utf-8?B?WERwNjRpaHQybTM1cU9CQmRpSTlkTUcwZkliMlVmamNpK2xMMGZ4RDdLSjNZ?= =?utf-8?B?Wk5tbWhrQWpabFF0STVmR05zUTdSN1RsM1h6SGFHYUQ1ZGtSYlRib09HRk5T?= =?utf-8?B?SjRRbXhvVTVxTXhuTGxEcnVPVmFGeGFCUnZOUlBKL2VrbkFoNkUrdEhFK2tS?= =?utf-8?B?RXFiR2hmWnVUdFFaZFoxWVcwcnVHUzRqcTRjM1djdWxhQ3VJcU9RaWNwbG40?= =?utf-8?B?d2dodkRtbEZFdkZtZE83N2dNOTNoRUhGbEVHK21KeFBEY25YazBaK2djV1BI?= =?utf-8?B?Snpza2pHdkNodUNQVXpXekFzeHhXbTN5OXlHNWlZQmUzZnpyZk1wR2RsYjVl?= =?utf-8?B?T3VaUG9IL005UFFzTzBSTGRhQmJXaWh2b1pVR1NvRlJwTkVLemR6QjRIR3lY?= =?utf-8?B?Ui9XcWprZEowZFRXbnQxM3QwOEpxbm0yZ0VVY0tuWldYbnRxc2M5dncyZjFa?= =?utf-8?B?aXc9PQ==?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: f767b94f-d74b-4396-d193-08dc50159f9c X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Mar 2024 17:28:18.3368 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: P3vWrgl5ZKYm1c/aLqaIIdgVn6LkhTeVw2POOIzyQ+jtfNALeW/4EVp613jBNaTQCUKwNhYCoh1YrMdheUoCg3vxP00bGz+m1Hb9RfWwIUU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR17MB6213 X-Rspamd-Queue-Id: E856680008 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: fox5ith6xop7s5zi9gzkb9k4r7sioht5 X-HE-Tag: 1711733302-510094 X-HE-Meta: U2FsdGVkX197YQBUnp+2R4S99f7Ert7JSLuRdUoKG8l4L8e/VR/8YTlVhhtXEZg/+NSifSqoLvWhe0Re0/kdrftnsmgfZSD/YMfz0OeYfbjtxymoT6GlWoh51XODIBDTffzgD2E2pkswhsAQZf/J+xeWsUnYKNzQuFgxWsJrso89yWZ0mTZyS6jPCyq4LPgzJtLs6xPmTrCgKmquOv6d7KxuI4zVzdvzaXp2WeEfmR+WbnKyzck5OlVprM07eM0tX9tdOrAegsnmMSI0/3kWA0lzIKpcEioFBfzVH2LLck9mKFgUN/SVSMpKouVyQQwWr6LQ0JFcRetx57bZFF4hsCs/hiayvGxvYy+vyr/4749TUsdfYfhoSkLk2lxlTJDZlWvvnKTAI2jZZLZP+jY/JPKmyhqLFTdaQYyDk9RB9rmL2Fza8IeAVh47qGbGQqxCG2Dlf3DRWw7j92Fy3RUvCSLj6j5avlB6D8uYiFB11bnaKcV0RrTkV7a6hjg2UxFMJwKvCEsx9M8anhBCoADr7cdqHNFqhkUHWhCbno+iY9EV2S5K+nqUsPiSdVKwh/1A6dhjrI3wA9CQ+q8FQcGPfheuXYBmeURdfyFmsBI/3Ns0QY+HlmZpi9N3pcnI9laiVlteLrT8hMMZxl/Upu3eVlmIQY4r2uEbGeg36Yp8NuPUsBYS0vXCviT1scIQUanI+wJ0TBn2kDz9tbLfc0hn/nC+8UjWYICAkwz9UMJxRq9XtWjU3zCX8MA5+9rUixgy3wa0yn6El17yFw5by/L03W5NEMqPMo+enELeB7LPTEB+uH8gFp6soGXytx9gACcdFAXeOSJRaOk9mQVfrdHhAtSXXbyxarEkTHE0t7e2hMZR7CY+GRdtwht3swBk6t0z+soUc8Ta3i3q/traW9u3s38iaj9W0u5WTcZ9Z2cryKMr1piSEwUWHLUxueMX4cXmHMjoNP9Ggj84eC/gPox RFKZgucy dPVsl18k7QvSDBxtN9tWgfBpw0fz8uJJPrljtTvlpTSQtDAusiKV+qPAljR7B88EQ+JrYEq7raZrajGCjMekPSoCTReADeNPLp3pTRzFicwLnO9ZqWJKb6umWwwz5dmGYALsZCRmEUZR118Yvh1BGAt8GMN6hTbNSX/EWbSf8oDoDuQ5edOAb/4KrOYii688bQJK10yqb2uckQMr/YivH45zLwFRwqZ6WK8zDsjhkF+xeub9irM++qDK6CCFHXWDuY97eR4lOv5rtxH/iJSXukZYlmItl9Vm4/GkwRxaTOubd6b3KRAvkVB6cc2YhwFTyhYRdElI+PUu7RXxysxNVe5ednyg5nsWybhtyGKp8D0Qt8ZNwZkMCT3Km5A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 27, 2024 at 03:53:39PM -0700, Yuanchu Xie wrote: > On Wed, Mar 27, 2024 at 2:44 PM Gregory Price > wrote: > > > > Please note that this proposed interface (move_phys_pages) is very > > unlikely to be received upstream due to side channel concerns. Instead, > > it's more likely that the tiering component will expose a "promote X > > pages from tier A to tier B", and the kernel component would then > > use/consume hotness information to determine which pages to promote. > > I see that mm/memory-tiers.c only has support for demotion. What kind > of hotness information do devices typically provide? The OCP proposal > is not very specific about this. > A list of hot pages with configurable threshold? > Access frequency for all pages at configured granularity? > Is there a way to tell which NUMA node is accessing them, for page promotion? (caveat: i'm not a memory-tiers maintainer, you may want to poke at them directly for more information, this is simply spitballing an idea) I don't know of any public proposals of explicit hotness information provided by hardware yet, just the general proposal. For the sake of simplicity, I would make the assumption that you have the least information possible - a simple list of "hot addresses" in Host Physcal Address format. I.e. there's some driver function that amounts to: uint32_t device_get_hot_addresses(uint64_t *addresses, uint32_t buf_max); Where the return value is number of addresses the device returned, and the buf_max is the number of addresses that can be read. Drives providing this functionality would then register this as a callback when its memory becomes a member of some numa node. Re: source node - Devices have no real way of determining upstream source information. > > > > (Just as one example, there are many more realistic designs) > > > > So if there is a way to expose workingset data to the mm/memory_tiers.c > > component instead of via sysfs/cgroup - that is preferable. > > Appreciate the feedback. The data in its current form might be useful > to inform demotion decisions, but for promotion, are you aware of any > recent developments? I would like to encode hotness as workingset data > as well. There were some recent patches to DAMON about promotion/demotion. You might look there. ~Gregory