From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5AEEC25B75 for ; Wed, 29 May 2024 22:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46DC46B009E; Wed, 29 May 2024 18:22:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41E136B00A1; Wed, 29 May 2024 18:22:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E53C6B00A4; Wed, 29 May 2024 18:22:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0A6CE6B009E for ; Wed, 29 May 2024 18:22:07 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AB2F1A07E7 for ; Wed, 29 May 2024 22:22:06 +0000 (UTC) X-FDA: 82172857452.14.1DEA293 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf01.hostedemail.com (Postfix) with ESMTP id CE52F40002 for ; Wed, 29 May 2024 22:22:04 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qxAwYFSF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717021324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dXNFpCWaiUyGAgeRu6GIx1Hrfse/W7At5lQK/blkjT8=; b=UUEwQ1k1XBJR+Y15Bu3Z5s/GHMJ2QTLHnyDaeRiZlOu7TQqRx+yY+55pirt1rXUPpIFPs6 UvgDRBsHDT8jIAoNCGToOZyuZEUfCpx4/l5VbygVJhDCE3pFiMbu4m2t85wup37QNzJHfR figmJNMeEojFENtaRMLdpx3Mb4vuIzk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qxAwYFSF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717021324; a=rsa-sha256; cv=none; b=uloYdkEC16yMplzzF6Pn0hXLR5lazIf65BsbV9cE3IDgkoOT9u18Rcg75nSm3NBdwm55fJ 7BcgfFGLvXmziB5Q0PGbYvIC8GZ7IJEKbZTwCHD0VoZdabDkTiiBt3rD0EqoNDTWg4t+xY E29oD832Y9IdEfdn0ZwJvVJP0iTFEL0= Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-41fef5dda72so8055e9.1 for ; Wed, 29 May 2024 15:22:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717021323; x=1717626123; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=dXNFpCWaiUyGAgeRu6GIx1Hrfse/W7At5lQK/blkjT8=; b=qxAwYFSFULcvnVkcQxXY+CF+6/aqaCgQIfSV55jBWyz/OUY01BKKitU5BN2Gagwcnm TA0nkv1AeAFYHv/MAqIBFPPSTE8ezR6NAwXLN6LzycJsOC2WbtwFQYO293nP0kH2uI5y oO4nfdxeqV+6nXHXZuT5vdfpdScsmWv1/Rrgw0hXmSPNTACr11fAnDFPNgw1e+hChWiL YSMWnjnMpy21o7eyG/frCkfPVSvrvWZU16VFyWYk3CDrUwtRUGzyi3fi7HhoYh13QGa2 8Xc8Yqpt683R2oFb6udKT8cJEuo58XY3n5vE5Ksd5uw5DMDCs8JtgRKEgnDXlYlJgLwH ibgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717021323; x=1717626123; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dXNFpCWaiUyGAgeRu6GIx1Hrfse/W7At5lQK/blkjT8=; b=kDm5Po/T8S64dY/jzhm3k+P7Fx5H/EyE1klmLJIrIcGPoyKkkAdR49Rz0PXMsPyaR/ lRjE8Ybx3PhZ/a9rSQnZB5BVZRIRx9yyS1oEtrZiBdG1d5inJrOzGffUUMMFn8qdH+5A ZAhXa/nuoIQHDjykT2dBN0qjyV69eo2bDCRSC5wgFgRrguIaZwlskcScT58h3G+CBm08 zqtp67KuCHxZ3HFcbomB9pZG6ZqyNxfua4u74whwp+WuPyWLhtuQO5pefkcMROvyggNo FWn1Kjf0OZfjtEgEBKhQGD+sCYHSfecpda4LpqxZfYKVTE86CMtua/sTLkUWf95isOsE c1Rw== X-Forwarded-Encrypted: i=1; AJvYcCVcasyAWAPhOTqrCI+IrIkEaMpgEv7JOTczVmRlglM0ncWcOLYmAdk1B4zfB+owoTD2DHa3SLPvMPUfpycoHgl3sMQ= X-Gm-Message-State: AOJu0Yw6EEPExyhwTsNkJoaYWlo2Fl5Nl9nXtgAoN8SHQniaqkMffzdW T8rC5AVAZIkSjxhVPyXd593BS77gEZ5let2Pzs+TyQTej4FR/FSUaD4PZrzk0VNIej1Eov471JM CqGqonmrXIhcV95fqdKFcnLMqIlTpLOiN5hmt X-Google-Smtp-Source: AGHT+IFqO7VFkypiYSn9vKfdPJnK37iD9z235VYJwF+JX9IAWq7RbNTOv7CYeuUojcdBQo9JI7fHcGh5LBFn8OI6V7U= X-Received: by 2002:a05:600c:2146:b0:41b:8715:1158 with SMTP id 5b1f17b1804b1-4212761becfmr670735e9.6.1717021322997; Wed, 29 May 2024 15:22:02 -0700 (PDT) MIME-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> In-Reply-To: From: Yu Zhao Date: Wed, 29 May 2024 16:21:24 -0600 Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging To: Sean Christopherson Cc: James Houghton , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CE52F40002 X-Stat-Signature: hsieyish74o6dy7kse371wi53331efaq X-HE-Tag: 1717021324-451430 X-HE-Meta: U2FsdGVkX18mhxmAXrW2wJKH9Hz1ez14oV1fXUS9JKi+qKuub4iQEmnut/4s9ZVVCCdCWIZm6pvt6EHLdvNi3f/IP26Y08aDO0uHq4oQ/tA5+5dxk1vPikmPBRNnOeRgCrflWXTfsOdRzrG9npC6N5+JFb/AmEbpyaW5ySvK2F2UgRMhPLYI5S8xbmVVzw1LAjGeN9WHPKsRSy8py8EJ2NhWf84ZqxWzmNoOwcs8zCbpSPqGRMAjmgqXVgsloPKOK2b3MVhh51c/NvMT78oVeLHsSW5oxRY5fQLzu2JASH7Br2ASwQafi+G+VFGds5QJlrkH2PqCMSE0b/L9onlrt2npL+Si3KWuut21Xq5hgisKOwTxIChLN0Q3Uy28qHCFtMLGqKzaFawrD2+h8YFKESU5WJp55An3gFWiFzzdJU1E+USnlMLUs8gz3C3wyuSuVpqS1WR1sMPIXivMIeVJNR0ZuRIO+759YTFKch2DSjq76BOWL0Tes027+jvkTcN5zBYyMbZVlUIrtVV3lLUZAbSYlDZ6LSP4ZddYhNt9w1LiGQZn7O0J0kJ4Gcf2DzpOjHpLmoFXGAT8MjLiD8Ibn5ODZvAqhPBe42RBh7Yj2CYgDpopTHMT5+KC4BCoiAWdz3gdTuzRlh9D1beE6TAzxnLCS5YxQ+YClY3UZELKXASyuSynAcvw8jS12Z7BwOK91x1RTy4bh3llph/ggkSeU40fa3wpDrEC9r3YEG8ZXbwHIOsK8HJts/FMdF/mng5BN4499wo96XH/FYSky6krJHRtHp53iM19/k70jQqHhmfwVFLak92K5dppDhjcpjyLuwiwdQmqyMg6fqwq9+yfj0Be1m4M5Z6Dj7Eca1uSceFrWuQbBxq2fPQpkHQmAZnkG70oUQS1fgnB4ouJ0O30TuDjEFXj33sKDmj+Zw7ddwmZGxf35hMh9kamH3SmOQghg5HA8JAU1qe2jgqZa83 m4dY1v+o tUMin7MqrsLKFhEYfxK+paAzOIPyTyGXuI3N9bxzLhPH4iE2a8d6XdSL+FUpHdTGd14GbWJdLMr+oKX48zGW6cZsCNZL4JDZNlb+h5TvKjJHQXjXMuMWoOWVrlycFw/CX3QyfJtB8tDgQXcdm4/4cLaOJY0aqFCfVizIO8zEJLrVx82xpr5xXKg85R0rFOS3u8ttjrXhw4dP6M6u53mx8ysR1UUSF7id9KjbTtYxvC59GsUDtcFuxdOiAeg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, May 29, 2024 at 3:59=E2=80=AFPM Sean Christopherson wrote: > > On Wed, May 29, 2024, Yu Zhao wrote: > > On Wed, May 29, 2024 at 12:05=E2=80=AFPM James Houghton wrote: > > > > > > Secondary MMUs are currently consulted for access/age information at > > > eviction time, but before then, we don't get accurate age information= . > > > That is, pages that are mostly accessed through a secondary MMU (like > > > guest memory, used by KVM) will always just proceed down to the oldes= t > > > generation, and then at eviction time, if KVM reports the page to be > > > young, the page will be activated/promoted back to the youngest > > > generation. > > > > Correct, and as I explained offline, this is the only reasonable > > behavior if we can't locklessly walk secondary MMUs. > > > > Just for the record, the (crude) analogy I used was: > > Imagine a large room with many bills ($1, $5, $10, ...) on the floor, > > but you are only allowed to pick up 10 of them (and put them in your > > pocket). A smart move would be to survey the room *first and then* > > pick up the largest ones. But if you are carrying a 500 lbs backpack, > > you would just want to pick up whichever that's in front of you rather > > than walk the entire room. > > > > MGLRU should only scan (or lookaround) secondary MMUs if it can be > > done lockless. Otherwise, it should just fall back to the existing > > approach, which existed in previous versions but is removed in this > > version. > > IIUC, by "existing approach" you mean completely ignore secondary MMUs th= at don't > implement a lockless walk? No, the existing approach only checks secondary MMUs for LRU folios, i.e., those at the end of the LRU list. It might not find the best candidates (the coldest ones) on the entire list, but it doesn't pay as much for the locking. MGLRU can *optionally* scan MMUs (secondary included) to find the best candidates, but it can only be a win if the scanning incurs a relatively low overhead, e.g., done locklessly for the secondary MMU. IOW, this is a balance between the cost of reclaiming not-so-cold (warm) folios and that of finding the coldest folios. Scanning host MMUs is likely to be a win because 1) there is usually access locality 2) there is no coarsed locking. If neither holds, scanning secondary MMUs would likely be a loss. And 1) is generally weaker for secondary MMUs, since it's about (guest) physical address space.