From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73D20C4332F for ; Wed, 12 Oct 2022 22:53:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06EFA6B0071; Wed, 12 Oct 2022 18:53:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F39D86B0073; Wed, 12 Oct 2022 18:53:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8C416B0074; Wed, 12 Oct 2022 18:53:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C0B286B0071 for ; Wed, 12 Oct 2022 18:53:55 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 97C03C05E2 for ; Wed, 12 Oct 2022 22:53:55 +0000 (UTC) X-FDA: 80013801630.09.DF7D9B6 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) by imf24.hostedemail.com (Postfix) with ESMTP id 3E0C3180025 for ; Wed, 12 Oct 2022 22:53:55 +0000 (UTC) Received: by mail-io1-f44.google.com with SMTP id 187so13645iov.10 for ; Wed, 12 Oct 2022 15:53:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=RQNjMU9F7vPSKS5uWf+TJ+P/AZGC+XAnlHRyO/MXA08=; b=rEFHKVqXjRvwQxwzGNNKKFp9aOGsLl53zJS6B4HC04ArBiJtVtMayr7GLgt65O+xtM XSii2F5kbX/kegvMstg+rbvPHPeaMkSzCFYq6w70RrfeBBgGGoaM4ENO9LCmsIWUWwDE eT2JK8bZsylPfKGQ7s4U+Cb/JN39xCy4jiycSKOf9k8zvlQzvZkLieDkLuJQoOGN6pMW 0YpmVTJWofQHNkSH9CJ5f0t4QaVOBbQk7q0xtASQHjdEhZuCd8xQATwsDwqRpwIOesq1 YBzAX5ooidsB6GNtasb8F4BlSnmBBl31994gI/IcgXqb1zyWn8D0pHaeHb0Bm97dxjfs E18w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RQNjMU9F7vPSKS5uWf+TJ+P/AZGC+XAnlHRyO/MXA08=; b=6WP25BRLEUOMQ2MCHEtk8UnXWsdZIZGE3INyU555/xbCq6C6Klz1SJ6H++11nhrIal ezNwzTbeeMvFD5VIw0q6crn1L1WaU5Oxm7gYSGvgZP4EOqGECLbEkpUIFTW4W5EN4WKt 7hnPRTEeADZ6GFfU6ZS1j40rZB7Ff1T1Em1wltUJ755Mx9xwOCuy4jDrL7aHn97Jgm2u 3meQVzyS8tcxaTM547FDl4cxTx1FH5VDltG6FV7Hf68CPmR82GLaAJL+vb3vwb9avUWf O1jRGPntYeU9s6POMiPgnP6EyRVN33r/zfzkkY9j5i+o2N7orhGA92vkzBIYLW3X6FJP YcIg== X-Gm-Message-State: ACrzQf1H8jW6EcSA9z/JOlP7egp/fQaO9/rJ9eRRyYBQO0MMi3vmw1k0 CMP7+Q0CR73mDgt/LVa5gvYsDjekoYzXw0XFU8uN9Q== X-Google-Smtp-Source: AMsMyM7e+//UCnfRh+nTLZE0owcwmsn4kCNm6rKOkfT2PnUs3F5HM7g0xf1cXM3+hs8+Yq0l9mPms5F8A4MHIDVW4Zs= X-Received: by 2002:a5e:d506:0:b0:6a4:b8b3:a6f0 with SMTP id e6-20020a5ed506000000b006a4b8b3a6f0mr14689600iom.138.1665615234351; Wed, 12 Oct 2022 15:53:54 -0700 (PDT) MIME-Version: 1.0 References: <318682c1-34a5-44e3-a15e-ef71067d4fd7@amd.com> In-Reply-To: <318682c1-34a5-44e3-a15e-ef71067d4fd7@amd.com> From: Alper Gun Date: Wed, 12 Oct 2022 15:53:43 -0700 Message-ID: Subject: Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault To: "Kalra, Ashish" Cc: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-coco@lists.linux.dev, linux-mm@kvack.org, linux-crypto@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, jroedel@suse.de, thomas.lendacky@amd.com, hpa@zytor.com, ardb@kernel.org, pbonzini@redhat.com, seanjc@google.com, vkuznets@redhat.com, jmattson@google.com, luto@kernel.org, dave.hansen@linux.intel.com, slp@redhat.com, pgonda@google.com, peterz@infradead.org, srinivas.pandruvada@linux.intel.com, rientjes@google.com, dovmurik@linux.ibm.com, tobin@ibm.com, bp@alien8.de, michael.roth@amd.com, vbabka@suse.cz, kirill@shutemov.name, ak@linux.intel.com, tony.luck@intel.com, marcorr@google.com, sathyanarayanan.kuppuswamy@linux.intel.com, dgilbert@redhat.com, jarkko@kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665615235; a=rsa-sha256; cv=none; b=r65XgECMterM7iqvZOIZ8vmBPzWPl0+2bWfZYkSZrFM5fE89aDOWWtiPdW5e+7uacp5hOn Tb/U6OAk2/YgxmXcaFPB8Q84kEk9yy5w/LIzjHKyXVX4K3LAwvvwL29b+LFW2ZIO+NLiDt cbSl3XYlaoEr6B8/hU327mMHs8DytSI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rEFHKVqX; spf=pass (imf24.hostedemail.com: domain of alpergun@google.com designates 209.85.166.44 as permitted sender) smtp.mailfrom=alpergun@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665615235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RQNjMU9F7vPSKS5uWf+TJ+P/AZGC+XAnlHRyO/MXA08=; b=ojS7cXeXWifQAn4US34qIWj546D6YJXj+kuZcU/3hgZeTTNOktrJlxc7s1jArT2Rs/xaoT jSISuCoIXgQbCvgNnlTY2Tj/dKn5kf4TxMDZ7/2cKcB2VBnUuGJaUyZEgunvyy1OLvC5b7 waS2WLaT0vZEb09AYUtrhoIngTjZ6Vk= X-Stat-Signature: 4osuecdtw7awmbmzajc5943bu6coydp1 X-Rspamd-Queue-Id: 3E0C3180025 X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rEFHKVqX; spf=pass (imf24.hostedemail.com: domain of alpergun@google.com designates 209.85.166.44 as permitted sender) smtp.mailfrom=alpergun@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam06 X-HE-Tag: 1665615235-821909 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 10, 2022 at 7:32 PM Kalra, Ashish wrote: > > Hello Alper, > > On 10/10/2022 5:03 PM, Alper Gun wrote: > > On Mon, Jun 20, 2022 at 4:13 PM Ashish Kalra wrote: > >> > >> From: Brijesh Singh > >> > >> When SEV-SNP is enabled in the guest, the hardware places restrictions on > >> all memory accesses based on the contents of the RMP table. When hardware > >> encounters RMP check failure caused by the guest memory access it raises > >> the #NPF. The error code contains additional information on the access > >> type. See the APM volume 2 for additional information. > >> > >> Signed-off-by: Brijesh Singh > >> --- > >> arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++ > >> arch/x86/kvm/svm/svm.c | 14 +++++--- > >> 2 files changed, 86 insertions(+), 4 deletions(-) > >> > >> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c > >> index 4ed90331bca0..7fc0fad87054 100644 > >> --- a/arch/x86/kvm/svm/sev.c > >> +++ b/arch/x86/kvm/svm/sev.c > >> @@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) > >> > >> spin_unlock(&sev->psc_lock); > >> } > >> + > >> +void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) > >> +{ > >> + int rmp_level, npt_level, rc, assigned; > >> + struct kvm *kvm = vcpu->kvm; > >> + gfn_t gfn = gpa_to_gfn(gpa); > >> + bool need_psc = false; > >> + enum psc_op psc_op; > >> + kvm_pfn_t pfn; > >> + bool private; > >> + > >> + write_lock(&kvm->mmu_lock); > >> + > >> + if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level))) > >> + goto unlock; > >> + > >> + assigned = snp_lookup_rmpentry(pfn, &rmp_level); > >> + if (unlikely(assigned < 0)) > >> + goto unlock; > >> + > >> + private = !!(error_code & PFERR_GUEST_ENC_MASK); > >> + > >> + /* > >> + * If the fault was due to size mismatch, or NPT and RMP page level's > >> + * are not in sync, then use PSMASH to split the RMP entry into 4K. > >> + */ > >> + if ((error_code & PFERR_GUEST_SIZEM_MASK) || > >> + (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) { > >> + rc = snp_rmptable_psmash(kvm, pfn); > > > > > > Regarding this case: > > RMP level is 4K > > Page table level is 2M > > > > Does this also cause a page fault with size mismatch? If so, we > > shouldn't try psmash because the rmp entry is already 4K. > > > > I see these errors in our tests and I think it may be happening > > because rmp size is already 4K. > > > > [ 1848.752952] psmash failed, gpa 0x191560000 pfn 0x536cd60 rc 7 > > [ 2922.879635] psmash failed, gpa 0x102830000 pfn 0x37c8230 rc 7 > > [ 3010.983090] psmash failed, gpa 0x104220000 pfn 0x6cf1e20 rc 7 > > [ 3170.792050] psmash failed, gpa 0x108a80000 pfn 0x20e0080 rc 7 > > [ 3345.955147] psmash failed, gpa 0x11b480000 pfn 0x1545e480 rc 7 > > > > Shouldn't we use AND instead of OR in the if statement? > > > > I believe this we can't do, looking at the typical usage case below : > > [ 37.243969] #VMEXIT (NPF) - SIZEM, err 0xc80000005 npt_level 2, > rmp_level 2, private 1 > [ 37.243973] trying psmash gpa 0x7f790000 pfn 0x1f5d90 > > This is typically the case with #VMEXIT(NPF) with SIZEM error code, when > the guest tries to do PVALIDATE on 4K GHCB pages, in this case both the > RMP table and NPT will be optimally setup to 2M hugepage as can be seen. > > Is it possible to investigate in more depth, when is the this case being > observed: Yes, I added more logs and I can see that these errors happen when RMP level is 4K and NPT level is 2M. psmash fails as expected. I think it is just a log, there is no real issue but the best is not trying psmash if rmp level is 4K. > RMP level is 4K > Page table level is 2M > We shouldn't try psmash because the rmp entry is already 4K. > > Thanks, > Ashish > > > if ((error_code & PFERR_GUEST_SIZEM_MASK) && ... > >