From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2685BC61D96 for ; Tue, 21 Nov 2023 21:20:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7C976B0498; Tue, 21 Nov 2023 16:20:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2A296B049B; Tue, 21 Nov 2023 16:20:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E2986B049D; Tue, 21 Nov 2023 16:20:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6A9826B0498 for ; Tue, 21 Nov 2023 16:20:43 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 45916B5D05 for ; Tue, 21 Nov 2023 21:20:43 +0000 (UTC) X-FDA: 81483230766.27.4D8D5A9 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 0880FC000F for ; Tue, 21 Nov 2023 21:20:38 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TD9d0mAH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700601639; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Nt8kvnxFSl+/wVVyFPR7xfVyeuKxVIIu+IJHgsCJP0Y=; b=7sUmZaDUSmXxxVRrE/1yS3Jzp9s4GSAMpMe1aIjytf1++MczvLj7+sbA3n2J6Avr7EiOhm EmryJ1bLNFiR55PlkXzk2bzRtHvcU1qOzFdQUuLUrU4/7p0Uj5BwNej8O09MtUISUwW59F yI/J4QM3ptGua9ERs3KLa91+RBJDYQE= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TD9d0mAH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700601639; a=rsa-sha256; cv=none; b=feSAj8130ZtugUtDnudWVNX7UItsdSsg+6mR7cryz6WYgF7QA5Tq8RItF5AXr0/IIPdS3q W/gb3i9PSVMJ5iynE/k9ttKp8vC4S2iIMmlugJfpIXAcZG3YXGUH5azoqnOOlIrj3c3ltH yxc4ixReiUEW9XIt5qWjDJQTHTeQLTc= Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1ce95f96edcso25963485ad.0 for ; Tue, 21 Nov 2023 13:20:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700601638; x=1701206438; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Nt8kvnxFSl+/wVVyFPR7xfVyeuKxVIIu+IJHgsCJP0Y=; b=TD9d0mAHqFoGftKlhz/LH8CrH5a6PdSr03VjURt3aX0LQslm8CZRaQ1tsM7Bvw1nDL euGQ+vtb1rfttzOY/wIIWoEdiKgBHwmhiraf6nuFiy2tJ8WOBSufTmn97wvXTHvzvjzW LeH9eFC5bfJMPgS06AR8shTZXBEpC/SPehlbnsaPk/PKl/xwZCoMTe7H7ONCFu+RX07r rcDvYfwO0AELN1LpjZET2+H5grLRNZYAAVmOcAFfkQjCldoM5neVBdZ5wm2URSfyzQb7 u/ZFzb8nqI0Ah4OEGJcXQznTKr2LldCOiH3eRbA4tPNCEUFWrEpe7yvQNB3co7uDBw7E XQUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700601638; x=1701206438; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Nt8kvnxFSl+/wVVyFPR7xfVyeuKxVIIu+IJHgsCJP0Y=; b=NvWN/uFLhnq2cr5fJ8rG6QlQSMAD2meXhlTLWJBIMjh01NuAuf4UG4oTsvm7mxTriX 6A+rAStzrvK7Pa/4b9hLlXAGdNHQur7MRKJRnZVKn52wzpdz38dV3/hQ48NvDcr7IL76 jNj3LOkM5Lgnm1aipBT/XEAk3elSBUlLIRxEHSKJtpNhca+oyL++XMjZPnlEvaSCU5Hs hSVOkU7PaIjKexPmBy3RCMpESjGOckRbOu5/WvPmaj9fn3YTCjEYbgmmsgHllHZ+GCE2 8uMJwOVkD/XWvdwGmEtO25j2i7SCIYwHSQRijjPtMK62Wnk5mmcg2Tbp/zXabi8dXjh6 Jv5g== X-Gm-Message-State: AOJu0Yy2LXoI7tk7UNbSOYDP/x8DolCbdPB3pgyieJOPQkMLCC1Me0HA YLDLZorCQa7iR1N7ze+E6gI= X-Google-Smtp-Source: AGHT+IEPwy5ASKSSy89dfpOvuqw7AqdqnJir15Ua8HeuGGJBuSGULdcSSfuzaTCr2on+/k5ZZyh5Vg== X-Received: by 2002:a17:902:f687:b0:1cc:3bfc:69b1 with SMTP id l7-20020a170902f68700b001cc3bfc69b1mr443828plg.24.1700601637908; Tue, 21 Nov 2023 13:20:37 -0800 (PST) Received: from localhost.localdomain (c-73-254-87-52.hsd1.wa.comcast.net. [73.254.87.52]) by smtp.gmail.com with ESMTPSA id j2-20020a170902758200b001bf52834696sm8281924pll.207.2023.11.21.13.20.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 13:20:37 -0800 (PST) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, kirill.shutemov@linux.intel.com, kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, luto@kernel.org, peterz@infradead.org, akpm@linux-foundation.org, urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, thomas.lendacky@amd.com, ardb@kernel.org, jroedel@suse.de, seanjc@google.com, rick.p.edgecombe@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, linux-hyperv@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 4/8] x86/sev: Enable PVALIDATE for PFNs without a valid virtual address Date: Tue, 21 Nov 2023 13:20:12 -0800 Message-Id: <20231121212016.1154303-5-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231121212016.1154303-1-mhklinux@outlook.com> References: <20231121212016.1154303-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0880FC000F X-Stat-Signature: 7n3ezypt4d7qmyh47fkiu9z57b38qj35 X-Rspam-User: X-HE-Tag: 1700601638-934188 X-HE-Meta: U2FsdGVkX1+oxouRiJifGD+JxL6kPFue+ccAu3V8uec4HiyyFRr74RzcBiFSxwuchEK/tuyx9l0qaKvcLugrN2MOKkgVJNwq4IzbPXv+q0uzJ2MfTMDOBnW6Np8QYT2RxRqP9y3g2FGtRC1ARAaJvydd5ER0v5U+LC6jQGyElrW3hCwuMQl4QVfJIcmerR/czPRqOvyDb/H2Hyyh0hmsKGvZkJc5wuzv9pgZDkZQw+rI3ed4paTQr9Df5jGLxnoz/FvLWqbwyX9ou6VnisOhqMwca+8zFEnoLJiiHOomEQhY4ShfaOoRbamELfCTAOlib0G9JsG3ahELLpQGGlsyIRJhxBsLGTw7nxRJ0wRIJzhHRVuwcw962OXQ5fRB7Tba5cuLHkWCpn3LyBr62iynsXaM1VF2rUjA+GhvIvdeHANIjhNLJsFtLh3h6zDLky+ehTg0xFTg5bMxyw95YFms+aDE7tqqueYiyqLgb2ZvJ0xKUwSypW0DN9cPZsJ/wuBAQDiwMLuM5Oh16TEMxbVO9SyJyNewyugTdB6I5n99k3048LTgRiIdRdiiyLO/2eL24j9HpS3h1KGFbhe/NgK9oH+U9PmATvsCmKc5VG+UHGF2gzv11VrALjc2W8P52CbvpuFnK7pitwWyqpehcqM8CaT2aRcqPY/bxhTnDx1DJwdvtly+cGCeWuiZ9lkO2EoBSPkJGfS2ztT6NmoWnIZlF7f9CVLx50H0cD879Jo4vfBQafonlp6oEa5Vg3E3kTzy10QIMBDKIhE4IJleqbmYfMT6SOV1HA/8T+pqc1T8zSiJZZRwMs/7BZJnSc6Z4RNNRCexraxIDH70HASb0TX6gDdlF1lzvSA7zKuo5BOsbv/VfKuA1Jub0Oe3Yla5jNt43aPP4i1Zdu0w9L67ridIDJsfWGrmNOS9QlrKJcUzChtFcc31IKaVUa2q4RGuA2MIse28ca/sLGoupBhNf6v XD1x+9Gv 1R+piaTmB71UlSsRjO/dfHty5UKUexUmbCkNSp/HvJUQJUeCFAIBl8n0V1AZNSs+pnP1CDHiixwGgNiLReTpt3teVB9OF9h3vQ7UMLKOyq7XQlbrjh4nD87tSw2cAvAIDibIofWcVykDgXGLmUVPy+DU9jjebo2f3MeVE6lhvi1Bm88XmGXZsBqVd3LuuPIhMNOwxAVRubfIWpqVx3h2TLZC2AWhT/gGIHKLvB1oTK3fqxM7c5X3+76TQ+ftwxm04oINp4iowLCJgpQ5fbF1kmnsQzU2AmRN12wkpI7rJw67jLTT8uSSLT3mmB9u39N+VP/MK1bluqsZCw5efaSfcSuk1qXdWs3V4gcpduOMEASzJQoAgaHKO2hRJaPlWgKU+1WBDNFm3S7MBE3fh/9+NVCmdfnmnqfm/qcv7Gq8mvIGRQFP89fgAJoLZqMtwwyB0o9koHu6VCGS9Y4uQOkN/BaZ343RVahT85OEpOGFWd+64DveIOMjVyJRzG1gNOoZcWRJj20YgpcU+oqY4EyqL4TEUZD3zFZK+dczj3gdHKRWNOUNjArind4gPvUR3RPR3lqUojYnCopLASQ8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Michael Kelley For SEV-SNP, the PVALIDATE instruction requires a valid virtual address that it translates to the PFN that it operates on. Per the spec, it translates the virtual address as if it were doing a single byte read. In transitioning a page between encrypted and decrypted, the direct map virtual address of the page may be temporarily marked invalid (i.e., PRESENT is cleared in the PTE) to prevent interference from load_unaligned_zeropad(). In such a case, the PVALIDATE that is required for the encrypted<->decrypted transition fails due to an invalid virtual address. Fix this by providing a temporary virtual address that is mapped to the target PFN just before executing PVALIDATE. Have PVALIDATE use this temp virtual address instead of the direct map virtual address. Unmap the temp virtual address after PVALIDATE completes. The temp virtual address must be aligned on a 2 Mbyte boundary to meet PVALIDATE requirements for operating on 2 Meg large pages, though the temp mapping need only be a 4K mapping. Also, the temp virtual address must be preceded by a 4K invalid page so it can't be accessed by load_unaligned_zeropad(). This mechanism is used only for pages transitioning between encrypted and decrypted. When PVALIDATE is done for initial page acceptance, a temp virtual address is not provided, and PVALIDATE uses the direct map virtual address. Signed-off-by: Michael Kelley --- arch/x86/boot/compressed/sev.c | 2 +- arch/x86/kernel/sev-shared.c | 57 +++++++++++++++++++++++++++------- arch/x86/kernel/sev.c | 32 ++++++++++++------- 3 files changed, 67 insertions(+), 24 deletions(-) diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 454acd7a2daf..4d4a3fc0b725 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -224,7 +224,7 @@ static phys_addr_t __snp_accept_memory(struct snp_psc_desc *desc, if (vmgexit_psc(boot_ghcb, desc)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); - pvalidate_pages(desc); + pvalidate_pages(desc, 0); return pa; } diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index ccb0915e84e1..fc45fdcf3892 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -1071,35 +1071,70 @@ static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info) } } -static void pvalidate_pages(struct snp_psc_desc *desc) +#ifdef __BOOT_COMPRESSED +static int pvalidate_pfn(unsigned long vaddr, unsigned int size, + unsigned long pfn, bool validate, int *rc2) +{ + return 0; +} +#else +static int pvalidate_pfn(unsigned long vaddr, unsigned int size, + unsigned long pfn, bool validate, int *rc2) +{ + int rc; + struct page *page = pfn_to_page(pfn); + + *rc2 = vmap_pages_range(vaddr, vaddr + PAGE_SIZE, + PAGE_KERNEL, &page, PAGE_SHIFT); + rc = pvalidate(vaddr, size, validate); + vunmap_range(vaddr, vaddr + PAGE_SIZE); + + return rc; +} +#endif + +static void pvalidate_pages(struct snp_psc_desc *desc, unsigned long vaddr) { struct psc_entry *e; - unsigned long vaddr; + unsigned long pfn; unsigned int size; unsigned int i; bool validate; - int rc; + int rc, rc2 = 0; for (i = 0; i <= desc->hdr.end_entry; i++) { e = &desc->entries[i]; - vaddr = (unsigned long)pfn_to_kaddr(e->gfn); - size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; + size = e->pagesize; validate = e->operation == SNP_PAGE_STATE_PRIVATE; + pfn = e->gfn; - rc = pvalidate(vaddr, size, validate); - if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { - unsigned long vaddr_end = vaddr + PMD_SIZE; + if (vaddr) { + rc = pvalidate_pfn(vaddr, size, pfn, validate, &rc2); + } else { + vaddr = (unsigned long)pfn_to_kaddr(pfn); + rc = pvalidate(vaddr, size, validate); + } - for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { - rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { + unsigned long last_pfn = pfn + PTRS_PER_PMD - 1; + + for (; pfn <= last_pfn; pfn++) { + if (vaddr) { + rc = pvalidate_pfn(vaddr, RMP_PG_SIZE_4K, + pfn, validate, &rc2); + } else { + vaddr = (unsigned long)pfn_to_kaddr(pfn); + rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + } if (rc) break; } } if (rc) { - WARN(1, "Failed to validate address 0x%lx ret %d", vaddr, rc); + WARN(1, "Failed to validate address 0x%lx ret %d ret2 %d", + vaddr, rc, rc2); sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); } } diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 7eac92c07a58..08b2e2a0d67d 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -790,7 +790,7 @@ void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op } static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, - unsigned long vaddr_end, int op) + unsigned long vaddr_end, int op, unsigned long temp_vaddr) { struct ghcb_state state; bool use_large_entry; @@ -842,7 +842,7 @@ static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long /* Page validation must be rescinded before changing to shared */ if (op == SNP_PAGE_STATE_SHARED) - pvalidate_pages(data); + pvalidate_pages(data, temp_vaddr); local_irq_save(flags); @@ -862,12 +862,13 @@ static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long /* Page validation must be performed after changing to private */ if (op == SNP_PAGE_STATE_PRIVATE) - pvalidate_pages(data); + pvalidate_pages(data, temp_vaddr); return vaddr; } -static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) +static void set_pages_state(unsigned long vaddr, unsigned long npages, + int op, unsigned long temp_vaddr) { struct snp_psc_desc desc; unsigned long vaddr_end; @@ -880,23 +881,30 @@ static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) vaddr_end = vaddr + (npages << PAGE_SHIFT); while (vaddr < vaddr_end) - vaddr = __set_pages_state(&desc, vaddr, vaddr_end, op); + vaddr = __set_pages_state(&desc, vaddr, vaddr_end, + op, temp_vaddr); } void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) { - if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) - return; + struct vm_struct *area; + unsigned long temp_vaddr; - set_pages_state(vaddr, npages, SNP_PAGE_STATE_SHARED); + area = get_vm_area(PAGE_SIZE * (PTRS_PER_PMD + 1), 0); + temp_vaddr = ALIGN((unsigned long)(area->addr + PAGE_SIZE), PMD_SIZE); + set_pages_state(vaddr, npages, SNP_PAGE_STATE_SHARED, temp_vaddr); + free_vm_area(area); } void snp_set_memory_private(unsigned long vaddr, unsigned long npages) { - if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) - return; + struct vm_struct *area; + unsigned long temp_vaddr; - set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); + area = get_vm_area(PAGE_SIZE * (PTRS_PER_PMD + 1), 0); + temp_vaddr = ALIGN((unsigned long)(area->addr + PAGE_SIZE), PMD_SIZE); + set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE, temp_vaddr); + free_vm_area(area); } void snp_accept_memory(phys_addr_t start, phys_addr_t end) @@ -909,7 +917,7 @@ void snp_accept_memory(phys_addr_t start, phys_addr_t end) vaddr = (unsigned long)__va(start); npages = (end - start) >> PAGE_SHIFT; - set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); + set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE, 0); } static int snp_set_vmsa(void *va, bool vmsa) -- 2.25.1