From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF85BC4707B for ; Tue, 16 Jan 2024 02:20:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 310656B007D; Mon, 15 Jan 2024 21:20:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 299576B0080; Mon, 15 Jan 2024 21:20:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1BA06B0081; Mon, 15 Jan 2024 21:20:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DBF3C6B007D for ; Mon, 15 Jan 2024 21:20:35 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B1D1D140468 for ; Tue, 16 Jan 2024 02:20:35 +0000 (UTC) X-FDA: 81683570430.30.D999861 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf01.hostedemail.com (Postfix) with ESMTP id BE7AD40005 for ; Tue, 16 Jan 2024 02:20:33 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mDi6lq+j; spf=pass (imf01.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705371633; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bse0/qXZQoiuVJq8kHe7VoVGITL3RsI555OnNeb2YB0=; b=EKKF+ScKVWLG5Lu+LTilA8+SNyEmp5ENRSOCKTJACAlj7BUh86zRWEHa4rEAprSE6XwuSL 7spAzVYf1FuFieqiaAG78G5HxZztzXTVZYlL+OsKEOVHBg2KKlrk8q7O0l/A17yY7qpgFJ hbDt8453Nj4dKJtIMjftzrhdjKf9zsI= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mDi6lq+j; spf=pass (imf01.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705371633; a=rsa-sha256; cv=none; b=LZ+JsrW1aepEGqIHoRw5gM+s/1LFB+dBRCkoMlGVQ7XjdPf0RpwP/GmgueZIpod30JkIcT /YpNvMRCSWWI3REoVh1H9BEo3hZbDZASizsZ6sz0paK7DB5zO9crdvYEBQFhWY/6IPnbFd 2/CKYjwypIWP8coBG8c70Z6y+RJPH/k= Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6d9b050e88cso4999336b3a.0 for ; Mon, 15 Jan 2024 18:20:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705371633; x=1705976433; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bse0/qXZQoiuVJq8kHe7VoVGITL3RsI555OnNeb2YB0=; b=mDi6lq+jNBOqyviqSP68LkcAZNQNCHU4kGSRM8ZnS+DtmXFnkhBzeJDdwXSW0xlz24 FoMZ6Eurz5QwPCqMjVEDfJMZYo+gf/Z41qNkFDWj27tCGGB4kT4M2AboOJ3UcizBzMsj kk16S8xtwlmfSaVsSR9KPD9MDsGMbXCVFhlwwJZLIf2Q01GS97Fd+g2YrpWM6AtM6E6l tkxIyS6eWx2RUJTcw2Wc1v9tFvem56o2eawp4i1Jv1IJMLY5Udw6ahgn6h4oItMhQc17 /WSvnYCeyQmrvL3j2YjGXTnw/jRkUWqey1dEL1R8oUW06OxVxXI4q5rxK5YSe90V76Dl nNoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705371633; x=1705976433; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bse0/qXZQoiuVJq8kHe7VoVGITL3RsI555OnNeb2YB0=; b=YgeezzU2CmogZCSkmjJSxm6OpNQYoNtUJ67z+R77ahF/ix6bEdJlUPXy3g3FsGBHrk rsNvAeVFqHOmsJ1ye6Bfs8O9VnTenNoNhd8zMnlNH5lS/hnEmFBok0knuMFSsviy90W5 68/oXiW/+qfOQ8HEPxk9e2XGKgizkLOaIw1b8x/866DVpSPsGqc/q3/MMEyCqKzzmH36 k4x1gbjeSaXzES6cNHC8OsIg4gAxbgb7TwoCeg4r9yJuzbGA4UgoTjW2i/nD31JZ6EWu KRCjK6dcI4bFix0ajdY5zE0Wub3LU1nDJEkQlk8Fm9dWKctKaWC9Je5YD+V4Z8pKQuvE xiAQ== X-Gm-Message-State: AOJu0YzhzW61PxRNJpxOvXRg784WpwsfrV4ISzYKdgJIdhaiuHtQPZA8 49Hk9bwstG8cxkFcbiy6kDc= X-Google-Smtp-Source: AGHT+IEr2+CDYGS0V8/X7I1MGszR2ZNlclrOwLdx0/MeFP/gOuYxw/HddIq+C+XRl375qrBivD1cYQ== X-Received: by 2002:a05:6a20:e618:b0:19b:1da3:bafc with SMTP id my24-20020a056a20e61800b0019b1da3bafcmr760682pzb.4.1705371632710; Mon, 15 Jan 2024 18:20:32 -0800 (PST) Received: from localhost.localdomain (c-73-254-87-52.hsd1.wa.comcast.net. [73.254.87.52]) by smtp.gmail.com with ESMTPSA id kn14-20020a170903078e00b001d1d1ef8be5sm8193379plb.173.2024.01.15.18.20.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jan 2024 18:20:32 -0800 (PST) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, kirill.shutemov@linux.intel.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, luto@kernel.org, peterz@infradead.org, akpm@linux-foundation.org, urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, thomas.lendacky@amd.com, ardb@kernel.org, jroedel@suse.de, seanjc@google.com, rick.p.edgecombe@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, linux-hyperv@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v4 3/3] x86/hyperv: Make encrypted/decrypted changes safe for load_unaligned_zeropad() Date: Mon, 15 Jan 2024 18:20:08 -0800 Message-Id: <20240116022008.1023398-4-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240116022008.1023398-1-mhklinux@outlook.com> References: <20240116022008.1023398-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: BE7AD40005 X-Rspam-User: X-Stat-Signature: xr9nany1zrr7qhmpnw5scdx8zqb3bm3c X-Rspamd-Server: rspam01 X-HE-Tag: 1705371633-809390 X-HE-Meta: U2FsdGVkX187ewGZtn+Q2d5FEb6ZN5jGLBKJi5R5w7FXvdhlXChGv/Fkl//fbz4doX78gO1N77OSAMq8dSZEUGIf5gcyNZYi3MBXxkDfhMdTTpmSNAiis+m14JELx58w0qKmIic5bYbwe9UJNlIk+Nms/aXJ+ISvtJ97FTUQua7xz3tZMXfuxJQe0mIAU5JVtz5m++76Wl58VJcWPp+9S1SGyOEDiC85Atz7osrdgbMMR9SNxRy6xiBcjkWg8tHcvtzBqZR47jo32fkBEzn0BPg/1HoreKVmHxeV/H5V2P4ZIaQCih3dvycBf3FvZQQseC1kqHyY3K8C3ADSWq2bsn1bjQm3nEAG2O77a0sAt5meQyJ6dbZuFMSJojtsJyaBElFooCfy191WXF+o/LDRxRaaSxMlTmxgfDK9FXTx+xxxP0vwbK8s+ej3LqLihS5SbqcpekhfQX/HXMO8LJz24E8WVcO3zSvm0PQSgiJPZjpx9iKIHco4OgFDganeUX7iRWbjo3ekBmVBA+HLvSAC0Jil98EcCK7uzOtSnb5PuwvrKlCiaXhsJJXycxZPLR1rlmHcNr7TnvcwYdtrbQqeqtjaMSzIS3kju15rv+BG4Sj05gqI3lq/F27/+R7c1DbZk+/9uIFiSXwLHyf7jNVpke5miM9jTdbPl/1m/VQQkuJXaFKTvtIanTqqtGsDX4WqOPLenQEFXMHsc9wkUkaL8yMjHhPiIGC7+uzjidSue6sZhwwgPXRizbApXbxUERworStFLQUAjZUHNKbj+05/9o2VuPgapkSGqZm7v7fTy231OHJOo83wElsxnuYpTu4uVVCmOizgVS8DbK+AyzT7cknqSqqvT/Bu6WYFqSlrDEEoDiIwNAiVbcmKocD++U2DfL53LCkHPrM8myLi7poQHOyzhz9N9dnU3EZUebwdDWeTyPOcHv1/INQLQegZEoFt73QKS1NwgyiYvjCwuPs fQm6Qb7l xu4fjswd6K3458uzqplJ5fP3Xn5LwsKKjR4nnahD0iOhSxfx7sGc9zSpDYJ5EnLBjh+fQuFLnc/YytjTLi2CllaybBA3CKNX3Aet1W7JAqRaWSpIPXtzUK0UFrJStXAIYKULa2kzeTm9TJ2lq0weqtUzA7Myxbv6xxYhuaWcRWKBnhVjFDnZ2owU1vJoMPa8ihz/Pw+wSMwEIPayJbtfPs4vRf2WfPON5bcLs7U8A/GlRDWGv6Hm93XxFL3guUeDNrHJ3iLMzi5neqdzo/qtg3ekgpE0UVya/lwvxtpHlhwiB4fp/tsQQ/dEW+XpP8hFv1YzHMU3uA2VAccevTpdB3Du3zo8TCeTVFCrMcF4B7okBCMKTcHYiGwHI/2v47Gj4CgsQeXa7XL1pRUDn17kyUmVjZgItOJDq9NbXFEXykts3XNP5N24OnTDHdXRFdTHzpHYC8coFqcpWmQC3wtOH+H9doXz3qEDRfZ1t+Kfm4v7Kld7hzErjJ7hnPTjntk18CBOTMU3OQcJAyuuJaIkwpEVRp99VnQRdmePRI2LIkIJ2hn1kal20Qb/9SjlHVsCk+3LaFF6RopxmDRQUjezAJvRPXfgnm8XwomqfeL3/EVQ6NY3KQz3RiU12E7EcnyyEpGm35bUDx3WA1FQKtPtEH1by8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Michael Kelley In a CoCo VM, when transitioning memory from encrypted to decrypted, or vice versa, the caller of set_memory_encrypted() or set_memory_decrypted() is responsible for ensuring the memory isn't in use and isn't referenced while the transition is in progress. The transition has multiple steps, and the memory is in an inconsistent state until all steps are complete. A reference while the state is inconsistent could result in an exception that can't be cleanly fixed up. However, the kernel load_unaligned_zeropad() mechanism could cause a stray reference that can't be prevented by the caller of set_memory_encrypted() or set_memory_decrypted(), so there's specific code to handle this case. But a CoCo VM running on Hyper-V may be configured to run with a paravisor, with the #VC or #VE exception routed to the paravisor. There's no architectural way to forward the exceptions back to the guest kernel, and in such a case, the load_unaligned_zeropad() specific code doesn't work. To avoid this problem, mark pages as "not present" while a transition is in progress. If load_unaligned_zeropad() causes a stray reference, a normal page fault is generated instead of #VC or #VE, and the page-fault-based fixup handlers for load_unaligned_zeropad() resolve the reference. When the encrypted/decrypted transition is complete, mark the pages as "present" again. Signed-off-by: Michael Kelley Reviewed-by: Kuppuswamy Sathyanarayanan --- arch/x86/hyperv/ivm.c | 53 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 49 insertions(+), 4 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 851107c77f4d..95036feb95e7 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -502,6 +503,31 @@ static int hv_mark_gpa_visibility(u16 count, const u64 pfn[], return -EFAULT; } +/* + * When transitioning memory between encrypted and decrypted, the caller + * of set_memory_encrypted() or set_memory_decrypted() is responsible for + * ensuring that the memory isn't in use and isn't referenced while the + * transition is in progress. The transition has multiple steps, and the + * memory is in an inconsistent state until all steps are complete. A + * reference while the state is inconsistent could result in an exception + * that can't be cleanly fixed up. + * + * But the Linux kernel load_unaligned_zeropad() mechanism could cause a + * stray reference that can't be prevented by the caller, so Linux has + * specific code to handle this case. But when the #VC and #VE exceptions + * routed to a paravisor, the specific code doesn't work. To avoid this + * problem, mark the pages as "not present" while the transition is in + * progress. If load_unaligned_zeropad() causes a stray reference, a normal + * page fault is generated instead of #VC or #VE, and the page-fault-based + * handlers for load_unaligned_zeropad() resolve the reference. When the + * transition is complete, hv_vtom_set_host_visibility() marks the pages + * as "present" again. + */ +static bool hv_vtom_clear_present(unsigned long kbuffer, int pagecount, bool enc) +{ + return !set_memory_np(kbuffer, pagecount); +} + /* * hv_vtom_set_host_visibility - Set specified memory visible to host. * @@ -522,8 +548,10 @@ static bool hv_vtom_set_host_visibility(unsigned long kbuffer, int pagecount, bo int i, pfn; pfn_array = kmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL); - if (!pfn_array) - return false; + if (!pfn_array) { + result = false; + goto err_set_memory_p; + } for (i = 0, pfn = 0; i < pagecount; i++) { /* @@ -548,14 +576,30 @@ static bool hv_vtom_set_host_visibility(unsigned long kbuffer, int pagecount, bo } } - err_free_pfn_array: +err_free_pfn_array: kfree(pfn_array); + +err_set_memory_p: + /* + * Set the PTE PRESENT bits again to revert what hv_vtom_clear_present() + * did. Do this even if there is an error earlier in this function in + * order to avoid leaving the memory range in a "broken" state. Setting + * the PRESENT bits shouldn't fail, but return an error if it does. + */ + if (set_memory_p(kbuffer, pagecount)) + result = false; + return result; } static bool hv_vtom_tlb_flush_required(bool private) { - return true; + /* + * Since hv_vtom_clear_present() marks the PTEs as "not present" + * and flushes the TLB, they can't be in the TLB. That makes the + * flush controlled by this function redundant, so return "false". + */ + return false; } static bool hv_vtom_cache_flush_required(void) @@ -618,6 +662,7 @@ void __init hv_vtom_init(void) x86_platform.hyper.is_private_mmio = hv_is_private_mmio; x86_platform.guest.enc_cache_flush_required = hv_vtom_cache_flush_required; x86_platform.guest.enc_tlb_flush_required = hv_vtom_tlb_flush_required; + x86_platform.guest.enc_status_change_prepare = hv_vtom_clear_present; x86_platform.guest.enc_status_change_finish = hv_vtom_set_host_visibility; /* Set WB as the default cache mode. */ -- 2.25.1