From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89833C3271E for ; Mon, 8 Jul 2024 17:31:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F043C6B0095; Mon, 8 Jul 2024 13:31:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB39E6B0096; Mon, 8 Jul 2024 13:31:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D54A26B0098; Mon, 8 Jul 2024 13:31:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B65E16B0095 for ; Mon, 8 Jul 2024 13:31:31 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 468891C2DEB for ; Mon, 8 Jul 2024 17:31:31 +0000 (UTC) X-FDA: 82317277182.30.A372A97 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf12.hostedemail.com (Postfix) with ESMTP id 7B68F40008 for ; Mon, 8 Jul 2024 17:31:29 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KCkrR45+; spf=pass (imf12.hostedemail.com: domain of jthoughton@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=jthoughton@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720459875; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=h5OvAknooFUouRttjkgdHPkXhVCuYFumFzAfg7hgFY0=; b=sglrNLOM1F3YAeBNIzXcUXmnSVnFFXQKUc2Z+XMlxvOmV+2RRxOv18nmcrofQDhmNztMcQ 1irBaoiFSq9pnHMQHeCNSv/MdUBPaIMGlXymZW0l9JtAkMSVjX7TwWXZ3dblQdIRFy1H7o nnm6UPgSNs1yN8R4QlBDrIJ3k8ibujg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KCkrR45+; spf=pass (imf12.hostedemail.com: domain of jthoughton@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=jthoughton@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720459875; a=rsa-sha256; cv=none; b=7ni1aAXYk/ABMO9o/OXGOYxatR3Jp8awSVk+C1wzqsQI+1xHNKo6n1FAQU5hyqb5EtCtR8 EyHHW9PrDX1BApRs1SFcljh4KJQFbuZPdafbzR4I8414n8Ni0dK34f3Lp+zdh4kmAZs7k0 ROBUdUzfHW/ycnOg86tTBNw15M7/Yho= Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-447df43324fso18371cf.1 for ; Mon, 08 Jul 2024 10:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720459888; x=1721064688; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=h5OvAknooFUouRttjkgdHPkXhVCuYFumFzAfg7hgFY0=; b=KCkrR45+nAAvldEenwMTaHFfHsMihBBTfjKc9phmiHXyytzBbLNj/L1y/H+Z6WthLr ZqqKv2BRXxDbTOngxS1O+tMNy3iCdSI8tE/ut/sye02307CjwdamAFi3JO7yobJquEsX gEskn2lwvch45z9L7AQ8LnL+J6OKlOMSIijy2DTErqXgX5DXdIVhAiDycuKwOGQdYvN5 Z3FirGgdgvbSp2fBatFjLFSy422qxtWBFiRrsnLi12+oOx7fAQuPzW9PkyucxP68o4kl LcU5HVzj7sMy9h0fvUfsMxU8nqU3umCjAW3cS0dY7tOpogD8SAoiuPwJ3BXfWGKV0d83 SDYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720459888; x=1721064688; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h5OvAknooFUouRttjkgdHPkXhVCuYFumFzAfg7hgFY0=; b=RAWoEc5UAqexceZhyccf6LL5B23yssHSRTxFfPj27cIB0fxxdnKR7sCdY4Rh10lQIi FDW8E6viNw4awRoPDZOM2/Fd5wH1Q4S563wUkorS/jeXtanPIzTom08EtMAejkM6Vxz2 QWh65AVcVLumKW8OQXMkLsh+mXcZql0hfHYqGZqH6gtOmxg5OTiKkXQKaz6TkXCGEeiK /JOqjbqr1wIoKP0LHaMtRr9V19bJd+QhanJTzDWgmMzZAKD/4CEFLsmBr3LG/8gMHzBv gQx6p6wbTCexOxH0M1hoqCUZhpAG2AmbmkPf45hazcxehPMwON4ixvwtw0IIFqbZBzzD cjhg== X-Forwarded-Encrypted: i=1; AJvYcCXphsc4cZBGxNXal7NyV93rDUO1p8p56AtuMdpwCuXmnBYcKKhQCqIPBwo7EEBLLSaDAwxqGTGphrTafDGZIk1brmY= X-Gm-Message-State: AOJu0YzF5OlVtGNG5JnErrs1HCw6D9aEu0HWhlpifNIGzUyt7kV4kke+ DxS5k06oysBbbwZoqSkgq1OgdrelwfvP4AyfPEMQbAx1vnSipw6eYZUjjApCSTDDMztsmyhEZrr Jh7bS9ntwkEr0yVrcyl1WRS2RelTJn4NkB4xI X-Google-Smtp-Source: AGHT+IH8n9BydNeekHTCqWhs8VlQPK1NNwxivhVTaYxOA+WSOHKqEde44rSbHv1zl8p/7A2Rz9umCfoVyqc4houcpcU= X-Received: by 2002:ac8:554f:0:b0:447:f44d:d0ed with SMTP id d75a77b69052e-447fbc7904bmr5181cf.1.1720459888197; Mon, 08 Jul 2024 10:31:28 -0700 (PDT) MIME-Version: 1.0 References: <20240611002145.2078921-1-jthoughton@google.com> <20240611002145.2078921-9-jthoughton@google.com> In-Reply-To: From: James Houghton Date: Mon, 8 Jul 2024 10:30:51 -0700 Message-ID: Subject: Re: [PATCH v5 8/9] mm: multi-gen LRU: Have secondary MMUs participate in aging To: Yu Zhao Cc: Andrew Morton , Paolo Bonzini , Ankit Agrawal , Axel Rasmussen , Catalin Marinas , David Matlack , David Rientjes , James Morse , Jonathan Corbet , Marc Zyngier , Oliver Upton , Raghavendra Rao Ananta , Ryan Roberts , Sean Christopherson , Shaoqin Huang , Suzuki K Poulose , Wei Xu , Will Deacon , Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 7B68F40008 X-Stat-Signature: 59royq64g3ahmyq39t4ay149q15bisoj X-HE-Tag: 1720459889-885743 X-HE-Meta: U2FsdGVkX1+wubo2HKZyCMfqQMt+c4LIyHnpOHyQ6c/2rG4hTgW0kkCkxVijvKiWiB81bLB2WFVfdEolpjIudzCX5oHvvnDs3+F1MlV3q9qIBnkv0Ej4Ie82cR7Hs6x24qWoKquOXtPzinRclXPmqdW6uMSLAtymC4y+0D8l0sFKkqe99O3ghWhaVQT5PDcq/P6TevOOsDGFlqCw+CfsU2tK9Nnvf94ycl/v48CENKwVAEjHZxqQzquseUDT0q97ufs58jqfFN8oChu0wIEXj3LtFJfHNjrR8kTXfgax4eaPcBkkdDJ/du/CNATw/GXgynzjKVfjly5NIi04HAiVLWgF+TnrmPTF1NE44pRe+TYosIE0O3uXJDrAgD/B8wGGu8plnNHOrRm8XLLxlKK/ZnihOM3qn7lB0FPJW8rLD4WJolYcdOtAAX7BncdVvjNHMXyNQ+0BFKd6rhK5bkKhHJKMGlDWqvYro1CkP5EPCSfwsSiJ+vRrrfpcPyDXAO3qe7ag1hRzMvfkLMNzPsaBh5CrfqtyyAd95WYeZrAhbLZ5PImTRWh2Gz6l+PvQM01asx40JVUzXIA8Kr2wR8tPqC0iljX42b9M96P9czERWDlpOBTHcA3ts+KSew0z5BzBYJLR4H9Fgl1VnwSxjAj1nPcVmKcLtOJ0r8m43wrwlw3irFPaLEKbrJDsyeIoaH2anJlAziVqLy1jBR8qAN6gCknQmjC9r1VRKZVKrpvthLR+CbCzUZ3JmMvcjO/lQm7FMAVpkKXw2IHtuOjmLFqpZpV/Kr40+mhWO2DrcOgpeBDFaQl0Ze9TgrqmPpSrWSbfzuaAxdiYpsYgu/YdvmuiGxsrC36z/e9x9h3gaoZOoOYyKGD8R7Cx2ADtC7JRUXD5tvLO477xOXOAe3cmBOne4ekNUyjzlgqWUucK2Zl4g+PuFOEbxSDMk0N/R3bgOf7GNb65liQVUXSZYkcUJ1e tFsIrpsY iS8Z4Tbo4+MMs6gSVd5GBhkN/F64Gf4eESS6D2qKNpGdAEA2KGuOF5TmCTGIx3FJ3hWbRr1JdnWgt4QrpZCF8wbklLVLUPhBx2dxA9jMZ7380Q+owlmyIkH0U2SrosOyFdv0HmWDMCv0m22styK/VhFgGToNl0uCkc0lofemUBYBZVOlhc8mOS0hc8K/WGwedOT9SPPGogz9DfqlgdG750YL4FMjvarh+5rY58hbSVsS3uTncfWiKnI69cGOsmDSa0AUoQqmORictXPE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jul 5, 2024 at 11:36=E2=80=AFAM Yu Zhao wrote: > > On Mon, Jun 10, 2024 at 6:22=E2=80=AFPM James Houghton wrote: > > > > Secondary MMUs are currently consulted for access/age information at > > eviction time, but before then, we don't get accurate age information. > > That is, pages that are mostly accessed through a secondary MMU (like > > guest memory, used by KVM) will always just proceed down to the oldest > > generation, and then at eviction time, if KVM reports the page to be > > young, the page will be activated/promoted back to the youngest > > generation. > > > > The added feature bit (0x8), if disabled, will make MGLRU behave as if > > there are no secondary MMUs subscribed to MMU notifiers except at > > eviction time. > > > > Implement aging with the new mmu_notifier_test_clear_young_fast_only() > > notifier. For architectures that do not support this notifier, this > > becomes a no-op. For architectures that do implement it, it should be > > fast enough to make aging worth it. > > > > Suggested-by: Yu Zhao > > Signed-off-by: James Houghton > > --- > > > > Notes: > > should_look_around() can sometimes use two notifiers now instead of= one. > > > > This simply comes from restricting myself from not changing > > mmu_notifier_clear_young() to return more than just "young or not". > > > > I could change mmu_notifier_clear_young() (and > > mmu_notifier_test_young()) to return if it was fast or not. At that > > point, I could just as well combine all the notifiers into one noti= fier, > > like what was in v2 and v3. > > > > Documentation/admin-guide/mm/multigen_lru.rst | 6 +- > > include/linux/mmzone.h | 6 +- > > mm/rmap.c | 9 +- > > mm/vmscan.c | 185 ++++++++++++++---- > > 4 files changed, 164 insertions(+), 42 deletions(-) > > ... > > > static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned l= ong end, > > struct mm_walk *args) > > { > > @@ -3357,8 +3416,9 @@ static bool walk_pte_range(pmd_t *pmd, unsigned l= ong start, unsigned long end, > > struct pglist_data *pgdat =3D lruvec_pgdat(walk->lruvec); > > DEFINE_MAX_SEQ(walk->lruvec); > > int old_gen, new_gen =3D lru_gen_from_seq(max_seq); > > + struct mm_struct *mm =3D args->mm; > > > > - pte =3D pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, = &ptl); > > + pte =3D pte_offset_map_nolock(mm, pmd, start & PMD_MASK, &ptl); > > if (!pte) > > return false; > > if (!spin_trylock(ptl)) { > > @@ -3376,11 +3436,12 @@ static bool walk_pte_range(pmd_t *pmd, unsigned= long start, unsigned long end, > > total++; > > walk->mm_stats[MM_LEAF_TOTAL]++; > > > > - pfn =3D get_pte_pfn(ptent, args->vma, addr); > > + pfn =3D get_pte_pfn(ptent, args->vma, addr, pgdat); > > if (pfn =3D=3D -1) > > continue; > > > > - if (!pte_young(ptent)) { > > + if (!pte_young(ptent) && > > + !lru_gen_notifier_test_young(mm, addr)) { > > walk->mm_stats[MM_LEAF_OLD]++; > > continue; > > } > > @@ -3389,8 +3450,9 @@ static bool walk_pte_range(pmd_t *pmd, unsigned l= ong start, unsigned long end, > > if (!folio) > > continue; > > > > - if (!ptep_test_and_clear_young(args->vma, addr, pte + i= )) > > - VM_WARN_ON_ONCE(true); > > + lru_gen_notifier_clear_young(mm, addr, addr + PAGE_SIZE= ); > > + if (pte_young(ptent)) > > + ptep_test_and_clear_young(args->vma, addr, pte = + i); > > > > young++; > > walk->mm_stats[MM_LEAF_YOUNG]++; > > > There are two ways to structure the test conditions in walk_pte_range(): > 1. a single pass into the MMU notifier (combine test/clear) which > causes a cache miss from get_pfn_page() if the page is NOT young. > 2. two passes into the MMU notifier (separate test/clear) if the page > is young, which does NOT cause a cache miss if the page is NOT young. > > v2 can batch up to 64 PTEs, i.e., it only goes into the MMU notifier > twice every 64 PTEs, and therefore the second option is a clear win. > > But you are doing twice per PTE. So what's the rationale behind going > with the second option? Was the first option considered? Hi Yu, I didn't consider changing this from your v2[1]. Thanks for bringing it up. The only real change I have made is that I reordered the (!test_spte_young() && !pte_young()) to what it is now (!pte_young() && !lru_gen_notifier_test_young()) because pte_young() can be evaluated much faster. I am happy to change the initial test_young() notifier to a clear_young() (and drop the later clear_young(). In fact, I think I should. Making the condition (!pte_young() && !lru_gen_notifier_clear_young()) makes sense to me. This returns the same result as if it were !lru_gen_notifier_test_young() instead, there is no need for a second clear_young(), and we don't call get_pfn_folio() on pages that are not young. WDYT? Have I misunderstood your comment? Also, I take it your comment was not just about walk_pte_range() but about the similar bits in lru_gen_look_around() as well, so I'll make whatever changes we agree on there too (or maybe factor out the common bits). [1]: https://lore.kernel.org/kvmarm/20230526234435.662652-11-yuzhao@google.= com/ > In addition, what about the non-lockless cases? Would this change make > them worse by grabbing the MMU lock twice per PTE? That's a good point. Yes I think calling the notifier twice here would indeed exacerbate problems with a non-lockless notifier. Thanks!