From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABA96C433F5 for ; Tue, 4 Oct 2022 18:52:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C82856B0071; Tue, 4 Oct 2022 14:52:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C324B6B0073; Tue, 4 Oct 2022 14:52:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFB208E0001; Tue, 4 Oct 2022 14:52:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 986AE6B0071 for ; Tue, 4 Oct 2022 14:52:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 535CA140503 for ; Tue, 4 Oct 2022 18:52:38 +0000 (UTC) X-FDA: 79984163196.15.C9AE10E Received: from mail-il1-f173.google.com (mail-il1-f173.google.com [209.85.166.173]) by imf21.hostedemail.com (Postfix) with ESMTP id 0D23B1C000F for ; Tue, 4 Oct 2022 18:52:37 +0000 (UTC) Received: by mail-il1-f173.google.com with SMTP id i9so7019360ilv.9 for ; Tue, 04 Oct 2022 11:52:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=OIgPyGrMPjz+gDLAg9kEoqdqEvTP4OkVilCCr8OCKSk=; b=m6VUfw3ZZzy6RhysyIB1Hgg5/KlyWRGF31b91A6T0mWaTZMmO0RLaiIiI8YldeQXik ezqy4m0OTJj9Bi/bWfXIMaMDbf679eqqbZiGig+salzLLX6dFEvxSdmSAlbo7SfMMuCd EalkGfEk5qcpH7kjkd6n1jDjnWlc+s1ivMhnpctdeZP9MS+5HXzfd3HDQZi/fBr2EqGl sPlBbi36vLjcFXveD6mQLUir5RyJM1PZzbCCKu91NpUcYV1vrDk+pOMmPKsnghzMOs11 SxXY99ezyJeV8sjLcTB6jxEB3tOdF1MVptcbPi9h/PDWIn8r6XeJP59ELC+FClswjrjz THAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=OIgPyGrMPjz+gDLAg9kEoqdqEvTP4OkVilCCr8OCKSk=; b=uAEMjqzHezzx8nkXMAnxNG93+49IixtCX5ARV5rYldkdisD3uvlWpBSzDSkeP5L+BS eGw9Cd9V6YY8HiQUHpRY5hvTsseEZI8187Q+9peTD7awXsmr5Mh+mPY74bK7wL90PFQG iV0ZHqFQHOZalIKN9/+Wr14/jn88hZ0dRo1u9eIqtE5Cn1tg+VbKivRd0rFTT0XFpFbo x88ZRobkhGjkd4odIfrdiueIc2VrPCpmd3pNhBhIUEp49b6dx848AySi1Wt4vVDizoHJ cLNgLPP4l4DjdaaNqUBmAllyjPIm4CSAsbXIG2wFEHnoB+RM8f2GbO123iZMjyAWOvds oHbQ== X-Gm-Message-State: ACrzQf1cRlo/ZuAS33A5oeOhUDGN8V0Vo7/QOBvzkEsNuwKuGD2JMmR6 OvGZ7Ys+8XtcPyF3gg+IqTNakycs243/qu7Nh8WOTQ== X-Google-Smtp-Source: AMsMyM4d8lAYdsJB32d8lIXQfbqETNRdu+zpA/9R3+8n3icR7P2QIdvZCoIKD6q4TqfoIMSTwTHt6COCrIzKNDiID0Q= X-Received: by 2002:a05:6e02:1bc6:b0:2f9:98c3:62e2 with SMTP id x6-20020a056e021bc600b002f998c362e2mr6898105ilv.240.1664909557191; Tue, 04 Oct 2022 11:52:37 -0700 (PDT) MIME-Version: 1.0 References: <50dfe81bf95db91e6148b421740490c35c33233e.camel@redhat.com> In-Reply-To: From: Mingwei Zhang Date: Tue, 4 Oct 2022 11:52:26 -0700 Message-ID: Subject: Re: The root cause of failure of access_tracking_perf_test in a nested guest To: Emanuele Giuseppe Esposito Cc: David Matlack , Jim Mattson , Maxim Levitsky , "kvm@vger.kernel.org" , Paolo Bonzini , Vladimir Davydov , linux-mm@kvack.org, Sean Christopherson Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664909558; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OIgPyGrMPjz+gDLAg9kEoqdqEvTP4OkVilCCr8OCKSk=; b=axGqiK5wG66lHLrNh4eYGLZonJpwzCnFzlBOBSX503EPOVSWbJ1ek5WXMqTojNFiDElMMK L3GMP1bhwhX7kVOhjp3GEtYSeS9MLBlRB2tQdEXapsWU2QTavzS2/MwlwWVTyWkillSBaU CkWeMZyJvs+9L6A02X9d+f22m/HMCNQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=m6VUfw3Z; spf=pass (imf21.hostedemail.com: domain of mizhang@google.com designates 209.85.166.173 as permitted sender) smtp.mailfrom=mizhang@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664909558; a=rsa-sha256; cv=none; b=m/H0Hd2f7c2nlRAC3jcrdWS2sfeP0p0D24QvWVu8OXdnB4qTVP47CjopRIwMUUyAt5xd5y wI/YazJ1bDByD4wfgPCzKwVCDBtmVDph7/i2xSp1kx7EY7akX8ZH44QFt7eib1dt1CWVFA 4Jk/cvhbF1MYHWIcc5wFw/ELMvzgoPM= X-Rspamd-Queue-Id: 0D23B1C000F Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=m6VUfw3Z; spf=pass (imf21.hostedemail.com: domain of mizhang@google.com designates 209.85.166.173 as permitted sender) smtp.mailfrom=mizhang@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: qdcfx4yun5pjfpubabx8qitpdonea9r5 X-HE-Tag: 1664909557-811511 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 26, 2022 at 1:50 AM Emanuele Giuseppe Esposito wrote: > > > > Am 23/09/2022 um 22:28 schrieb David Matlack: > > On Fri, Sep 23, 2022 at 12:25:00PM -0700, Jim Mattson wrote: > >> On Fri, Sep 23, 2022 at 3:16 AM Maxim Levitsky wrote: > >>> > >>> Because of this, when the guest clears the accessed bit in its nested EPT entries, KVM doesn't > >>> notice/intercept it and corresponding EPT sptes remain the same, thus later the guest access to > >>> the memory is not intercepted and because of this doesn't turn back > >>> the accessed bit in the guest EPT tables. > >> > >> Does the guest execute an INVEPT after clearing the accessed bit? > > > > No, that's the problem. In L1, access_tracking_perf_test is using > > page_idle to mark guest memory as idle, which results in clear_young() > > notifiers being sent to KVM clear access bits. clear_young() is > > explicitly allowed to omit flushes, so KVM happily obliges. > > > > /* > > * clear_young is a lightweight version of clear_flush_young. Like the > > * latter, it is supposed to test-and-clear the young/accessed bitflag > > * in the secondary pte, but it may omit flushing the secondary tlb. > > */ > > int (*clear_young)(struct mmu_notifier *subscription, > > struct mm_struct *mm, > > unsigned long start, > > unsigned long end); > > > > We could modify page_idle so that KVM performs TLB flushes. For example, > > add a mechanism for userspace to trigger a TLB flush. Or change > > page_idle to use clear_flush_young() (although that would be incredibly > > expensive since page_idle only allows clearing one pfn at a time). But > > I'm not sure creating a new userspace API just for this test is really > > worth it, especially with multigen LRU coming soon. Can we add an operation that causes KVM to flush guest TLB explicitly? For instance, we can use any operation that causes a change in EPT/NPT, which would invoke an explicit TLB flush. E.g., enabling dirty logging will do the job. Alternatively, adding a memslot for the guest, letting the guest touch it and then removing it at host level will also flush the TLB. I believe the both should be architecturally neutral and the latter seems more stable. In any case, would an explicit TLB suffice in this case? I think this will cause the zapping of PTEs in L0 EPT/NPT. Thanks. -Mingwei