From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E614FE77188 for ; Wed, 18 Dec 2024 15:22:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DBFB6B00A2; Wed, 18 Dec 2024 10:22:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78B9F6B00A4; Wed, 18 Dec 2024 10:22:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 652BA6B00A6; Wed, 18 Dec 2024 10:22:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 48C466B00A2 for ; Wed, 18 Dec 2024 10:22:21 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8EE82120DED for ; Wed, 18 Dec 2024 15:22:20 +0000 (UTC) X-FDA: 82908445410.04.1F320F1 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf22.hostedemail.com (Postfix) with ESMTP id EA2C2C0013 for ; Wed, 18 Dec 2024 15:21:44 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jQYeKiCz; spf=pass (imf22.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734535324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6kF4dvPrafLxwImEDBQg0SUXX4ryPi0IaRgqXDHFQfk=; b=gLXVCSjG590HDjGCoE150QgI02Ujp5y2HqtoQF6somxI6kfNBdJM6YKaNSbvUtj4AwD72Z K6iocQjJGJA/U4FtllrlX6+qSRMD/11zuw9OfajM9B3/QdVpArVAL+GkzeDL+mTTgK3Qru UZSx+S4zNE0TdFLJZ1PTsIXsGSWslP4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jQYeKiCz; spf=pass (imf22.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734535324; a=rsa-sha256; cv=none; b=Pp5uS0/ONFYE52p5PlDxQeBQFk5OPv3Br1/MizxgC4/ngsCLg64s7SLTeA/jQcH70j0lI/ 31djqaluS2eQyh2U7LIXsnTORPkU+7CSj4fhL4cNkJszmlIfAL+iEYTqYsKLAURLcPf93X kDLQk7HHIh54xdaXCmbrPFE1wVNvPI0= Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-aa689a37dd4so1251093166b.3 for ; Wed, 18 Dec 2024 07:22:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734535337; x=1735140137; darn=kvack.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=6kF4dvPrafLxwImEDBQg0SUXX4ryPi0IaRgqXDHFQfk=; b=jQYeKiCzsEnlik2LH4d6GNyi0tKyCP3q96UXVhSk2bTt1N0KalxQ5X8HzptysrVrKS lkSuxQ+o0eKoiVFCeizqoyvOnKxIwNpJdMYX4rNFWnG5hL+sit5I5udY4tUEnGOZA7D1 6qprddd+6y2Camc/FXiUjExZMLZnwtTrBO1ucQsEiQlt1Sxtvb+QXKnFMDS1GczHF4Hn SN2qviasyOGQqXlGeYfiworYZw96D4Kw3bys4dWueSmACDsCydciypSV8lf8V32IZ+dN XFYZ6/+A/KyTS3vLA8P6V1Y4/2A5jLV2G80LJy+0wexfkssPPekbQrxrjUszpFsbRoyP MSDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734535337; x=1735140137; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6kF4dvPrafLxwImEDBQg0SUXX4ryPi0IaRgqXDHFQfk=; b=jsiBzGcXbj5KO6IIF3d1L+UTX/UE05e4UMlwa+MNVZqF+gNCrZkWKaYsZpaxPfi117 Gk089fC1sZJLR9MKJwuc6GAN1srO4FuvezcqKNp0rwMGVjdDen0tdGjmU1vWwJwJMDoz bE0WX0Mlg36z7puH/E0HcTQaHy+G9s/nopS3z3a1unvQztOcyfMsB06rQc3oLpWP3OVO iOfRRfTEuEc9cLkTNHhx4Z8/Bzfg6JSIwmlu+78WWgW4K0qEBaXBAiaIIiRx/LT/YlyT lEndUuE75QTlfbh3yGMBbSR+///Xpt90nIyuqzVBLxELin/n/4podAeyLJyB9a5ENXga hyVA== X-Forwarded-Encrypted: i=1; AJvYcCXJwJXsA1vfcVld9dUJ7cdkn2e5xUcSFYbJu2mBGpoc5Ye+hZtscWFxUIXouuYFMB30pycxCIBIzg==@kvack.org X-Gm-Message-State: AOJu0YxfilrLkhYfKWX3pUpBPb3lxKFs6GJl9dXHIN2RWHUvlCXeREVi 7ZchX4H7gtCmv3Yj/SQ5p05rbWvoemh70yVLZ+P7mE7i7rReoQuF X-Gm-Gg: ASbGncvKh+Pvug9hpfXE/QTD8uRcng8IhPrdYirKrrmIV4EEHdmMheHAt1gsU5ROGBB wk1a8UTFIip9tvVsNAc8+3PQX/gNJpbxm5ggi9mTW4xWUdk1FhQ61463aAw/81TWD5FF9mTmiAg 6jdw/9VkDmjCF+VkAz1XbYFZf0NKoUHkcwUj13uduyk5VkVw8ZRezpkRZTBL7J5ZaSSkBO66mqe PsO+5+iNbfq41cTlHYBpOHQcmWeaLz/snMJxtJMNTOOXlbH6H7DXMD0fGyH+1jItXC33jz3yDw= X-Google-Smtp-Source: AGHT+IFOfiXIL9z+RpDnGJ0hYOrrSgR47iGjCSN1hF/VPFrlrogud9L7c3x2vlzmIv3UKwmRGxgo0g== X-Received: by 2002:a17:907:720c:b0:aa6:25c6:d94f with SMTP id a640c23a62f3a-aabf47baa99mr274106466b.31.1734535336329; Wed, 18 Dec 2024 07:22:16 -0800 (PST) Received: from smtpclient.apple ([132.69.242.133]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab96359831sm564252166b.95.2024.12.18.07.22.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2024 07:22:15 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\)) Subject: Re: Slow-tier Page Promotion discussion recap and open questions From: Nadav Amit In-Reply-To: <6d582bb6-3ba5-1768-92f2-6025340a3cd4@google.com> Date: Wed, 18 Dec 2024 17:21:59 +0200 Cc: Aneesh Kumar , David Hildenbrand , John Hubbard , Kirill Shutemov , Matthew Wilcox , Mel Gorman , "Rao, Bharata Bhasker" , Rik van Riel , RaghavendraKT , Wei Xu , Suyeon Lee , Lei Chen , "Shukla, Santosh" , "Grimm, Jon" , sj@kernel.org, shy828301@gmail.com, Zi Yan , "Liam R . Howlett" , Gregory Price , "open list:MEMORY MANAGEMENT" Content-Transfer-Encoding: quoted-printable Message-Id: <806F5002-00BB-47E9-A265-5CBF2C4CB8A7@gmail.com> References: <6d582bb6-3ba5-1768-92f2-6025340a3cd4@google.com> To: David Rientjes X-Mailer: Apple Mail (2.3826.300.87.4.3) X-Rspamd-Queue-Id: EA2C2C0013 X-Stat-Signature: 1r4hhnc87xucmse68czigrn8stzc7k6b X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734535304-572364 X-HE-Meta: U2FsdGVkX1/bSEPeKuSDee+fRtkrCN2Iugppd7yHgmWsxt2ObWupE9cMA/sbFb3EyW0GI6vSG5Soaxm+EMbtcPWjl+rQ1NcyU8FjaM2rfmtAbN6sXapxO/5qm9GLCwPVQuMtqOhn2NQXKZJEhHrWOSIBqY2BTAnHozWrDllRVLfu5FSxg527DYeXdrtsJHojYTC311nl6IfKTZEe9VDYc9CWTHY+LMFKDsGrj7i5LR43qVg2an+PhDiNdRJmiDaPRtt/sEdz4/YBggYSGciqsq4mOZ99Daaip2KIWgFUP6bTwV+dcHYaZxetA5C2756QUN5coEFFke0mr4ElO7htRJe59ccJ8n1iYQNF5nskO4tQ51AzD+5ef0WE/9JpwJZVXzxb5CgXuu5kqP8kjqR2LxGhJZ6aUDlt3AbB3rnBjAvAs1+c4htz87uguZ81Xb4FJiXLVPANEpbaDjgNCEJZDi0QJUblaSPgIDMuwMm57aRy82kE+uCx2sEBQRHy7EaYBpW6XqG9biReY9ZY7TVxGuKqj29KldCGPKeP3Uoe1Hvr4aSSXO1SD3uNlGeqIYqH8VZZvAe5yKq1OYcN8BZXFmTw9q4w8XdvAMOUYJE4I8OxyQ82PPQoHN9lSuZdg46X6fCVMLHE0V+FXzuCg4PvZ6Zzei9y8xEnDwwK3wq6HA4Av/NeqqUFbQBzCx0POC/R6N+vIMZk7sThmekqKE+5nTLXZeiVXZg7Ye2QZ7zaCjLtKL/LnW3JlCK36H+VjNjVFI1lN/y6fpArivQzh9yFpl5cERNYGJaXETJT+5WvSLlDokQAmj8PLkz9EJaNhDL/iICUIHKOqijDsdsxwWFTyzPcA5RqSHmrWdRUGhdjliNtzLROr/U2GWg7tWokx5bUjrb+NRUecLZX26Mr/rPHo4MIGjwO9Pj1dhpGJYC3p7OhWU8unWbrEbSysDhHN+nE86Oyu+hSAxgGE2WU/hJ A+Q145Zp gdgIeMN75s1PKux5yyAGaghYTujfyS+5hdN93S+W0c70e5k0DOWdxKakKI+EI4Ng8SZbJ2APw/ZL8sNoepO5AdnQ2D8W5mGxkO8R+mohZ+HbfzwRhV/b10Cr+W7FFYJ75HcRJfjZeZ0Zyge3B7ZjxzaBZ9+rehHIdimETv1MBETjw/w21fPk3Abquy7TeNSDI0Em30ignR4GYthRJzeotx2/73pj6oN+MPz7wn7E9IdVOHQYL1YReZIc0zMTCvgMixUZlig+d9R7AeyZwM02ZZkBi+2sga12+iRpsP1PtR8KF9zOVa8i1Oa7Is1WbgmOYuhIm2JmsbiIfGuNoZ54IOm0h7fJ2OulhMVoWY5Cr8NGr7Q3n817Gtf+/sD5tq/dAwQvpI6ewKnuKK3pg4BJR7A56FPoDQjcDmKWA+jAPf6lKYzMII9AiSvNQBi4NeMpmrpKRe7Un1dmceqJG+BZ6SwEErAK8Gai2lbOsrPambIYgXIFYX+3Hkr2tNntdawC0l/3AyKRF1nwKdbwG6880Thk29jsIp2JeroAL X-Bogosity: Ham, tests=bogofilter, spamicity=0.273871, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On 18 Dec 2024, at 6:19, David Rientjes wrote: >=20 > Hi everybody, >=20 > We had a very interactive discussion last week led by RaghavendraKT on > slow-tier page promotion intended for memory tiering platforms, thank > you! Thanks as well to everybody who attended and provided great > questions, suggestions, and feedback. >=20 > The RFC patch series "mm: slowtier page promotion based on PTE A = bit"[1] > is a proposal to allow for asynchronous page promotion based on memory > accesses as an alternative to NUMA Balancing based promotions. There = was > widespread interest in this topic and the discussion surfaced multiple > use cases and requirements, very focused on CXL use cases. >=20 Just sharing my 2 cents. IIUC, the suggested approach has two benefits: 1. Fewer/no page-faults (as A-bit is used to detect usage) 2. Batching While (2) seems like a win that might be added un top of AutoNUMA, (1) is more delicate. As indicated in the patch-set, the "exact = destination=E2=80=9D is lost. At the same time, the last time I checked, the A-bit setting wasn=E2=80=99t free and cost something like 550 cycles (others saw = similar results [1]). So considering empty page-fault is ~1050 cycles (2014 number Linus measured [2]), there is a question how big of a win it is... [1] https://lore.kernel.org/all/20160620000606.GB3194@blaptop/ [2] Google+ post RIP=