From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 386C7C4345F for ; Fri, 12 Apr 2024 18:45:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86B466B0092; Fri, 12 Apr 2024 14:45:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81A456B0093; Fri, 12 Apr 2024 14:45:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BB7C6B0095; Fri, 12 Apr 2024 14:45:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 501946B0092 for ; Fri, 12 Apr 2024 14:45:42 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 12AD240F97 for ; Fri, 12 Apr 2024 18:45:42 +0000 (UTC) X-FDA: 82001758524.29.B4A59FD Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf27.hostedemail.com (Postfix) with ESMTP id 3A4884000C for ; Fri, 12 Apr 2024 18:45:40 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nq9PX9xJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of dmatlack@google.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=dmatlack@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712947540; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ISLMFGOGyzbAhcth9c+vgcaA14m4ctbEbkD9TmYI1+8=; b=6JfYeBSPS+GXxU7rBpHOT7NqnU3e+ggtHOn1pnOYdeuTiudgl4ER+gtt0oyyjv25fxWbLC JdNh32sLFKooWzdQbganFNtksK8mNLxSjyv5HNttpz8TYh8VcC0sFQnrkZYiF3tf46I/FW vnu06GPNH2gMJCtNNjPYQ9s248vretE= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nq9PX9xJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of dmatlack@google.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=dmatlack@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712947540; a=rsa-sha256; cv=none; b=1u4/g/EJITw7TSSeCjiwd7E58VrlJR1lqMhurusCeAPH9AGP0Bv3GX6PUCx70KwuCvU42c UciNMjqSKk9a6DduXfxUcDy0xq4jcZV9WeM2HdTj6xSJQd93niErkOppDlJSydEVa0lOBF IQYbnWk+LuFHOSAJGGDZosMPHmWTo5Q= Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2a68a2b3747so817051a91.3 for ; Fri, 12 Apr 2024 11:45:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712947539; x=1713552339; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ISLMFGOGyzbAhcth9c+vgcaA14m4ctbEbkD9TmYI1+8=; b=nq9PX9xJSt5fT5Gi9sFJk33ejhYD1mzETMbSQNGRKuKzRlyBPs1ILj7tzCYDgI86sN c+Hk8Bz3g97hzbddssbEUuyPNX4Xd8k3rwMXMjLh49eql7+KaqIyjDsXQP6lskIeX9Xy h8Pak8zYf4q1aU9Kd+vXAI486uMkkfun6d/J83qkFflblVWgn8CgZAUjeUJcjwEpxRC8 jsTJJG7tavjB7252CJC6XFLlSKE2K9kQPQP4tpwRapwk/qScz2vHtLG5iHsC5hB4ppEz 2KWGB0aEG08MlKGM1pL+YDCDqC4pU2tNVceCubqM7+ygMtLuUUwfBpNPeSla3tSun9im Am5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712947539; x=1713552339; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ISLMFGOGyzbAhcth9c+vgcaA14m4ctbEbkD9TmYI1+8=; b=aeZ2U0QhuwyygyMRV/ITvhTT6B41xbB6zrIaj/a9ewkqTfa7DRdtDC1yxEiNZjpHKd cAvE16gwJ6jJPK8g3C8e3rUgzARfgEvTKTK1+N/ill8oc/73X9iKPJ+WmbPmx2eIjw4o 7vYrli/0vm0oiEg/vuz33/E/nbZI/6dGIaxE3zp4a3+yAWLdvLEzC3gSCwEWvqAcvhgE Ibb/HKznOlKyvBxDD8AwrpZtqxkrLx1Lq37yU5bh/NnJLHoS2HU6kJSKC5nEk8laCG7b UgW66a4nN1e276sU/uFjUHh0o7iR0sVWbmWR3Mbqcq5isKF/o9y9fBGeR+rsyDb37MuQ 0+dg== X-Forwarded-Encrypted: i=1; AJvYcCWciSeBL49rXzfTADbeFccgyiyQ2m76gUqg+UGIjdEBkaZBN3654QuTabtwJkj2GQ2v5uTePHF2Jt75Q2+EsNbd4FM= X-Gm-Message-State: AOJu0YwAr7uXUP+dJZaPhxqUBp056zAuLVCxPlahvFObRzsu00Q/ZUfi x9yJMX/rJfpM7S5arSKuZnGWpODt2oRyr7eDcykuOvYuChfX0XqdgkNw06O+eA== X-Google-Smtp-Source: AGHT+IErdx6N/Za0LUKOOdr+VzWwjNC6bZuRp56Svh5aVZ0iXA0UlC0kIs2SDcy0//cfbrlbvHooqg== X-Received: by 2002:a17:90a:9af:b0:2a5:3e4e:29a0 with SMTP id 44-20020a17090a09af00b002a53e4e29a0mr3344779pjo.6.1712947538843; Fri, 12 Apr 2024 11:45:38 -0700 (PDT) Received: from google.com (210.73.125.34.bc.googleusercontent.com. [34.125.73.210]) by smtp.gmail.com with ESMTPSA id i20-20020a632214000000b005cd835182c5sm3009589pgi.79.2024.04.12.11.45.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 11:45:38 -0700 (PDT) Date: Fri, 12 Apr 2024 11:45:33 -0700 From: David Matlack To: James Houghton Cc: Andrew Morton , Paolo Bonzini , Yu Zhao , Marc Zyngier , Oliver Upton , Sean Christopherson , Jonathan Corbet , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Shaoqin Huang , Gavin Shan , Ricardo Koller , Raghavendra Rao Ananta , Ryan Roberts , David Rientjes , Axel Rasmussen , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: Re: [PATCH v3 1/7] mm: Add a bitmap into mmu_notifier_{clear,test}_young Message-ID: References: <20240401232946.1837665-1-jthoughton@google.com> <20240401232946.1837665-2-jthoughton@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240401232946.1837665-2-jthoughton@google.com> X-Rspamd-Queue-Id: 3A4884000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: rimcksnscbrbm431reqyjzfixfbnztct X-HE-Tag: 1712947540-56938 X-HE-Meta: U2FsdGVkX1+cAYNwMFJrGhvOtxvxVANNGtsHrrQooYwtiaI1hl2sf0EUbW5W5uHW/tYRblW3f5+MU70YbINVoogFjTaNWKtcwdRqZRiRd7VPGdeu5Resl7be3YNZ6iE1WBlHJa9ZeGuOuKQpWUwlmAMY1wQ8cYmK2xCYYnPeqNgphDvX/D4EoazDDfCkMlySSxk0ts9JGseIODc3uM5B1LHX25Qm6VmiAOYiV0bgl8RHmiTXBHDHSAD+Au1i7MpP2bWQ3BduuuNRgpR9U77otzgSOrMeSi65qV1rDvy0GngQJDyf4RUdQENf7tpvwr8iF4B7G3SX+jrmDLGiPIbaH0YOkVk0s0yt13IZO7if2l87rVNtaFIgev6K4LC0GKA+NSRL7MzbnJbQzGD9MrghogoR4MZ3ideOgeCym17z6sOiObazWfG+NCRZ0cbQCtNgd2RoLXuJWJWMlQZ/lwrZYrWvKvzFo8rcp5s+fBeQIr6vO7IWsiuRgnVVTRfGIt0NUFu58kl5QmuOqmwpKPp5b5ZkzxrbmxG1hSluEfBdArUO1Jjxrm0L+O7IDyZ74wHzp1442nDMor/rJvYQkdrrJZradEU7QAIHKuKF+zLAqsWEdpQMPXRI6kInk2kJ88jb6uOIaneROJ2hCf6kg+N7kz30UOsh5mwZ0eH+E3ZZf3Cxg9tmZbIzGwZXHIw71v0izK5hwOgubhUbMt/EO+kW328r0KBaZ62t020ACznJWRH6zcg8h8KBPu7me/fuDzVLOWRJQdu6IvhdVRo91csEst8GqCflK/4BpsPiHyDzYc3HpJW9AXy3yqUlQD1+V8q6LjI3cOt9gX24QqzjAuEegC2mAgfmMJ7zznq5tV9ALGzATGKBcs+ghp/HVwqvcN2IZRu8DGFpv4xU+jk0+V7JecUauiK/h0oTSlDvdtrDd7LerxWmNjPX0eAql4odJ9bCgCbm4E2G/JrRK68V7kF TMWF8L8l zsKe64oNyg/lEQ6/P24Z6ntXvn74Gx3e2QSiElBYzDgavFCt6Tcvz0/dyC53dXYztMaq5YiGOyZYq/lsMPmGGCNjNG1wa9UbPyobmhJvOouISiyNvp4IWaUMjseR+WX3PwP9KAOSgFgEOkgpMKDnfhoSFaNdzz/eaQmKD5dkX2jJKQ8mR/VM7lp6YyEUq63nxFqAzbStM0Q2/bhydDzOjQ8dv9y7DtLlzpFxugdUt1+Stj9W6i75kw5aMx5OqFVO8kvxWI3aOAZbTZCPtrv1fI2qWC+G9gVmGyo21E6IFuknmU+l+uK/W67dWrGfDzrERSKZB1WNs5jQ4RDkeDE9KvGadsZd59moYrJtm2Feezo34mnvqhz/XgQqtDI5ZFLIgWNtPYsTOmeYZ9c0WXXqT2VVPhpvPm3MNFIofcj39vT0BvYPicFH/erOy+FUYYn2O/TZdd/p9SIXhgrTUshnlWaSKoKa1tyP/vfzTmfgMWlFbOMs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024-04-01 11:29 PM, James Houghton wrote: > The bitmap is provided for secondary MMUs to use if they support it. For > test_young(), after it returns, the bitmap represents the pages that > were young in the interval [start, end). For clear_young, it represents > the pages that we wish the secondary MMU to clear the accessed/young bit > for. > > If a bitmap is not provided, the mmu_notifier_{test,clear}_young() API > should be unchanged except that if young PTEs are found and the > architecture supports passing in a bitmap, instead of returning 1, > MMU_NOTIFIER_YOUNG_FAST is returned. > > This allows MGLRU's look-around logic to work faster, resulting in a 4% > improvement in real workloads[1]. Also introduce MMU_NOTIFIER_YOUNG_FAST > to indicate to main mm that doing look-around is likely to be > beneficial. > > If the secondary MMU doesn't support the bitmap, it must return > an int that contains MMU_NOTIFIER_YOUNG_BITMAP_UNRELIABLE. > > [1]: https://lore.kernel.org/all/20230609005935.42390-1-yuzhao@google.com/ > > Suggested-by: Yu Zhao > Signed-off-by: James Houghton > --- > include/linux/mmu_notifier.h | 93 +++++++++++++++++++++++++++++++++--- > include/trace/events/kvm.h | 13 +++-- > mm/mmu_notifier.c | 20 +++++--- > virt/kvm/kvm_main.c | 19 ++++++-- > 4 files changed, 123 insertions(+), 22 deletions(-) > > diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h > index f349e08a9dfe..daaa9db625d3 100644 > --- a/include/linux/mmu_notifier.h > +++ b/include/linux/mmu_notifier.h > @@ -61,6 +61,10 @@ enum mmu_notifier_event { > > #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) > > +#define MMU_NOTIFIER_YOUNG (1 << 0) > +#define MMU_NOTIFIER_YOUNG_BITMAP_UNRELIABLE (1 << 1) MMU_NOTIFIER_YOUNG_BITMAP_UNRELIABLE appears to be unused by all callers of test/clear_young(). I would vote to remove it. > +#define MMU_NOTIFIER_YOUNG_FAST (1 << 2) Instead of MMU_NOTIFIER_YOUNG_FAST, how about MMU_NOTIFIER_YOUNG_LOOK_AROUND? i.e. The secondary MMU is returning saying it recommends doing a look-around and passing in a bitmap? That would avoid the whole "what does FAST really mean" confusion. > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > index fb49c2a60200..ca4b1ef9dfc2 100644 > --- a/virt/kvm/kvm_main.c > +++ b/virt/kvm/kvm_main.c > @@ -917,10 +917,15 @@ static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn, > static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn, > struct mm_struct *mm, > unsigned long start, > - unsigned long end) > + unsigned long end, > + unsigned long *bitmap) > { > trace_kvm_age_hva(start, end); > > + /* We don't support bitmaps. Don't test or clear anything. */ > + if (bitmap) > + return MMU_NOTIFIER_YOUNG_BITMAP_UNRELIABLE; Wouldn't it be a bug to get a bitmap here? The main MM is only suppost to pass in a bitmap if the secondary MMU returns MMU_NOTIFIER_YOUNG_FAST, which KVM does not do at this point. Put another way, this check seems unneccessary. > + > /* > * Even though we do not flush TLB, this will still adversely > * affect performance on pre-Haswell Intel EPT, where there is > @@ -939,11 +944,17 @@ static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn, > > static int kvm_mmu_notifier_test_young(struct mmu_notifier *mn, > struct mm_struct *mm, > - unsigned long address) > + unsigned long start, > + unsigned long end, > + unsigned long *bitmap) > { > - trace_kvm_test_age_hva(address); > + trace_kvm_test_age_hva(start, end); > + > + /* We don't support bitmaps. Don't test or clear anything. */ > + if (bitmap) > + return MMU_NOTIFIER_YOUNG_BITMAP_UNRELIABLE; Same thing here.