From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66BC9C27C53 for ; Wed, 12 Jun 2024 17:00:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF8036B0085; Wed, 12 Jun 2024 13:00:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D80096B0096; Wed, 12 Jun 2024 13:00:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BFA236B0099; Wed, 12 Jun 2024 13:00:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9E1CF6B0085 for ; Wed, 12 Jun 2024 13:00:20 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3753DA17F8 for ; Wed, 12 Jun 2024 17:00:20 +0000 (UTC) X-FDA: 82222849800.18.A642D79 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by imf23.hostedemail.com (Postfix) with ESMTP id 3BD32140004 for ; Wed, 12 Jun 2024 17:00:17 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UeZ4PoJn; spf=pass (imf23.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718211618; a=rsa-sha256; cv=none; b=pWHAlHIkisSIaVm40N2pifvZS+TyB5QgezK+wJDc5PkgxBkfNzOEgM+EnPCtge/JXJqy0z HufUUNU0eByNltjIgg6g4xz6TFdLPnciOLBg017vt8MQVPs7A+Ctw1THCbGVSU/eeeWRh4 WpYWiof+IScIJu++YOW1cxm9AC9RuQI= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UeZ4PoJn; spf=pass (imf23.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718211618; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HsvWeW5eyBF6ROygK9LGppgGGqUZLW2+K1PQvnJ6/gk=; b=pcOO+k5II0ey26Jx22fb/m+I8eiz/BD1OU+U9P6Mc3teTGluP5dzNDCiMPx9GyXzvjMPqx F0yuw/IG/ujiHZVJtrny/y/7hgTH5dEoEklo2F6QY9NgiesD+zJr8aanupkp3Q0EHIP7pW e+j32X8Exf2fIlvQ5rOkAXAsIPwjLzc= Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-42171fa0a32so2585e9.0 for ; Wed, 12 Jun 2024 10:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718211617; x=1718816417; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=HsvWeW5eyBF6ROygK9LGppgGGqUZLW2+K1PQvnJ6/gk=; b=UeZ4PoJnfbivlVboJcpNyhJMQwMmbR09lXdOUXiZOH90VLEG6GK5JPbbO7tyW2D8nt tyl8a8gtfpdvOvhKHjAgu7JP73+Oqfqn01haQh0JPTP/VIlFQe2EHXH2cko0TRrIvS8T 2G23vmHUr2QSXzlFqh8PCUbvjCJMkxtUfpIwMQey8HaOGMbp7EE9z3uRRcAzmsIsgsmQ yTwprmKsyN/IhYwDLveiVAWt8Cgg1fNxrQuwsGfpd+0AK6Lqw7wEL0sMPzDEfa5VZs6U cieGeH1IVBdl6SMVIClZAk0IYdRlXcFK1eIGP10ixLCeTLetZGkx+UbxGFE/O3a0QtFE uRxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718211617; x=1718816417; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HsvWeW5eyBF6ROygK9LGppgGGqUZLW2+K1PQvnJ6/gk=; b=wdmeyLmXSStdbYLy3+9zsIXC+Xj7YzBsQr4+8NJRXes1ON8XUMY6iVgpRTZv23nZqV B6oBxAZgHuu9H302LHc/7PpKXS3y6iuS6ccIeRz10tuQy32WMVCq2hJj82aSwVFp7ZP4 zYsHgwJ7Wq0H3ojERbe7QZWjCa7eGiBeJdT3GIoS4Cmo9W3TU5hu4Gko/HhKxdLlMegf a4R/l3PJK/tiLPPJG1qC7exGQdLwaQdkHB4kl4vORJx7BxGSxGtMx9wlY405hVi5XikN uhcGytD/O3OnQoP4daKS0IhTAWIDo+zKRKxWWjBQp6pSq19SzwYPzRHYRalARrhpWwPf LloA== X-Forwarded-Encrypted: i=1; AJvYcCW+tLgmLh80ieHSqiLEaJm/QiwQANa+n0AXROIIjcS+QnF8gHjfezanG4MboJ9+55T0uE4YjT71OS8LKEnVpzmupuI= X-Gm-Message-State: AOJu0Yz2FIUcYaDwNeuI0EYXKQbune7kw9pUdvH/dIcZ/M+hHbe6xLs2 Nx9yZuY6LxRoRxC3UPdyauZiqZvt9e2DbqUIJwncIn4l2R+6oxmCqVTecyRVuuJYZr+/CkyfnwW o/vrT/VEzDIUhw7iENeSpTbA9r6r/OJMjOe5v X-Google-Smtp-Source: AGHT+IGkC8R6gs8FWiWHpUMtbGyq+bA7RbAETq6ID+j0qnlJP6qfOP8ypwWfjR3KUBqpX/w72oFg8by83yIZWemEZcA= X-Received: by 2002:a05:600c:501e:b0:421:7195:43e with SMTP id 5b1f17b1804b1-42280dae244mr2301425e9.0.1718211616345; Wed, 12 Jun 2024 10:00:16 -0700 (PDT) MIME-Version: 1.0 References: <20240611002145.2078921-1-jthoughton@google.com> <20240611002145.2078921-9-jthoughton@google.com> In-Reply-To: From: Yu Zhao Date: Wed, 12 Jun 2024 10:59:38 -0600 Message-ID: Subject: Re: [PATCH v5 8/9] mm: multi-gen LRU: Have secondary MMUs participate in aging To: Sean Christopherson Cc: James Houghton , Andrew Morton , Paolo Bonzini , Ankit Agrawal , Axel Rasmussen , Catalin Marinas , David Matlack , David Rientjes , James Morse , Jonathan Corbet , Marc Zyngier , Oliver Upton , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Suzuki K Poulose , Wei Xu , Will Deacon , Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 71h9zisdooyj3irhoyepfx8we3n6n377 X-Rspamd-Queue-Id: 3BD32140004 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1718211617-194942 X-HE-Meta: U2FsdGVkX1/IQzZL/MCYK20lU24VizRClxTkasvsJLL4gkSoYATLBjaYebqJvc46r/s0qWcGy6rqRDfxxn4Y/6vYWypt2tryQqVjF+qhFZKt9ptnZYwxL3fKiV4e7D59RYAHQpO/DM+Khh75sw+rhWstsD1mBXG24tYCKO/4IXz9yvL44vvvvjEWcJDFV5TrVtR2XLX6BTAnSPKx4FiYCYp7jHUUaqHfozamZJiYW7yhuqFHd+R90BchbFWVS5/ABHecwrkk+yWXKft8//55xFBK7i/JOpWA9THyQyr1NeCEmj9ZL+TnWTeCSNNLM830WTxcE1y24OixAAugY4skx13C3ptWPYzFsGcEcyEoVSt7ZwdhN65Vaul6197q8Va3S0waQIwfMr2lvnEHcfmXQ1/DCxNlfHOSHs7hN+rBC5X0r8YkhDLz/wru5p799qOh2X+hWN/emLlsoYegeAS3wPzCkulD7chtPFpuqDo4ifjwz4YwHJ+oUF0YAiR8XsuxYj3aqgoYwDAjQG3bf8gug2QyCqig2zCYG8BJY5w++GuFhxqezZqAyA8tWfQfwBDnH/cJPQwul9wRrQmp9QujdbIA+tJiRL2cqKJJgmFkq33Ree0+9z9/rIV0HE3EwntQdJ/0GvG0nzfJjO+FU//wjGi9xJkn40bFNxzCS1kBDCMc5Qq7/DLhK6DdmsdhivbzWro6u8px4kFR9TEgRSka4iLsX1Yr7h3mn2+pyB9LFDFVqLXHISN0QZbUGUDegXFUAjj11LOMSZwgYBCeXpOHpeVbob+cgd7hAB1OCY1d2zxAPKQ0Dk1yDpHojZDbbm8Pszz8wjc2HZUic08mqNP092CVW1b58LTP/2/FfsPPUjDxgqpaDJ/WztbjN3N4eR3qk39n65txU079JqwVnVzy2DXDrv5FKhT18/YwYelhhuQoaI8k9mZMNRIHxmhV1To8oqc2qujfNhchJUrvJqy QGSi8HS/ mkc5VjQaMB7/sm6BTZHnjLeSOe3F1ZuQuTYxGpDP/GvnUXnsq4HeUw6ZE+A+735zTsh5jC4dpZ2SdI4HehJm7AqyrOTj/aN6X3QIkA0OiaoNNcZ9lj9yQQcvaSRkpyWElFdYl5NtydHFhDoVY/nzJbPL8UCbepHvNwISTQAnzJ/6xz6dx4XmQvdewBIyRk0zO2nBVeldnX2eDAG2Pz+7VVYj9AQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024 at 10:02=E2=80=AFAM Sean Christopherson wrote: > > On Tue, Jun 11, 2024, James Houghton wrote: > > diff --git a/mm/rmap.c b/mm/rmap.c > > index e8fc5ecb59b2..24a3ff639919 100644 > > --- a/mm/rmap.c > > +++ b/mm/rmap.c > > @@ -870,13 +870,10 @@ static bool folio_referenced_one(struct folio *fo= lio, > > continue; > > } > > > > - if (pvmw.pte) { > > - if (lru_gen_enabled() && > > - pte_young(ptep_get(pvmw.pte))) { > > - lru_gen_look_around(&pvmw); > > + if (lru_gen_enabled() && pvmw.pte) { > > + if (lru_gen_look_around(&pvmw)) > > referenced++; > > - } > > - > > + } else if (pvmw.pte) { > > if (ptep_clear_flush_young_notify(vma, address, > > pvmw.pte)) > > referenced++; > > Random question not really related to KVM/secondary MMU participation. A= FAICT, > the MGLRU approach doesn't flush TLBs after aging pages. How does MGLRU = mitigate > false negatives on pxx_young() due to the CPU not setting Accessed bits b= ecause > of stale TLB entries? I do think there can be false negatives but we have not been able to measure their practical impacts since we disabled the flush on some host MMUs long ago (NOT by MGLRU), e.g., on x86 and ppc, ptep_clear_flush_young() is just ptep_test_andclear_young(). The theoretical basis is that, given the TLB coverage trend (Figure 1 in [1]), when a system is running out of memory, it's unlikely to have many long-lived entries in its TLB. IOW, if that system had a stable working set (hot memory) that can fit into its TLB, it wouldn't hit page reclaim. Again, this is based on the theory (proposition) that for most systems, their TLB coverages are much smaller than their memory sizes. If/when the above proposition doesn't hold, the next step in the page reclaim path, which is to unmap the PTE, will cause a page fault. The fault can be minor or major (requires IO), depending on the race between the reclaiming and accessing threads. In this case, the tradeoff, in a steady state, is between the PF cost of pages we shouldn't reclaim and the flush cost of pages we scan. The PF cost is higher than the flush cost per page. But we scan many pages and only reclaim a few of them; pages we shouldn't reclaim are a (small) portion of the latter. [1] https://www.usenix.org/legacy/events/osdi02/tech/full_papers/navarro/na= varro.pdf