From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D569C77B60 for ; Thu, 23 Mar 2023 17:03:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B773C6B0072; Thu, 23 Mar 2023 13:03:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B27E66B0074; Thu, 23 Mar 2023 13:03:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C7E86B0075; Thu, 23 Mar 2023 13:03:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 89E4A6B0072 for ; Thu, 23 Mar 2023 13:03:07 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2F67B140767 for ; Thu, 23 Mar 2023 17:03:07 +0000 (UTC) X-FDA: 80600783214.17.F389C1E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id A5A444002F for ; Thu, 23 Mar 2023 17:03:04 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=C5XfOnrB; spf=pass (imf17.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679590984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cglotdbZ9kvOmjiiA3xfBy0x7OKBSKTg0p62tpcUq34=; b=1ae+MLXhfSUYFuDWXMqY92UUhXb21vhXaxfirmrYJQVU/Foyx/o6ckTngT8CeJfsF5Aelm b0Ew9ch1fPihPyoLp3cSh+rG70crrpPt7dtxYB6BmUYQhdli4z0vZIKeGj1y7lKeioraRv VTnfIjMy1it3+TPPArq8+b/ydU2Krmw= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=C5XfOnrB; spf=pass (imf17.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679590984; a=rsa-sha256; cv=none; b=7Akl8eGRRIXTJ8PqVmDOxys0L8Usi/+3Rf5HQP8SdyP1xkebw4lmYYy0BtcZP8XV0PBVza J8BjOPrqcp6fXxY4tmj8AJXvUpakY0aprW1oQ6sc2/ObnMstmUkFn0PHzY3Rk0XiMNfIPR 5KFJ+k6xbFgOGxULoXEHDX+bWl1K8To= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679590983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cglotdbZ9kvOmjiiA3xfBy0x7OKBSKTg0p62tpcUq34=; b=C5XfOnrBRaRihbPdqXjNNu3ObIaHHvLfImtsXjeslnZKnPd9lSpbx3MGAGdnAZWK+hCaH9 ipbfEgfwLZPREAEl3V5dMtDpd9rZsj8wECZNMVL+Wrz3uyXgd4B62r6KDKscVT5gMV5qlc mmOABX+NvqpYkwcfQu5r8h6FsVtSxmg= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-345-6o7JXgCZMBao33Hh6VPKXw-1; Thu, 23 Mar 2023 13:03:00 -0400 X-MC-Unique: 6o7JXgCZMBao33Hh6VPKXw-1 Received: by mail-wm1-f71.google.com with SMTP id j27-20020a05600c1c1b00b003edd2023418so1307380wms.4 for ; Thu, 23 Mar 2023 10:02:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679590979; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cglotdbZ9kvOmjiiA3xfBy0x7OKBSKTg0p62tpcUq34=; b=Sg3I9+Z69VQ6YYQkzq9Na46zAgpl5RxNV8WLH4XBYCm8q+s5cmBQ5gzrO/QwMKIyRP yTdAYh8waWbNapzHllhfg6tYi2hwJZdPwsR9yOLUUvr5H7pAxlTC5Pl8YYzYnbpsCrB/ E/nj89MEnCVMSXmZwB8/H2rO7tYlBHopscZjN9hH1JyXf0N9qDeJjFpHnY+mhKrxMbAC 0eQNdGKI2H4TZ4lTeAniZnNAcEPWezMqh88+beDLxv1xPvyJOh9gKp/Ho+bTV1FvYdYe 1VNg80Lq9Ui4NAlxeQ0FHkGg3SVQDGwGM75fQTBia8e3X6P9gJB50Yfmt5/Ro7gm/oNa 3Qyg== X-Gm-Message-State: AO0yUKVfp05rTCFoxnfksGn4ZM3je4QxDAO1XCTXf4/ljtA1EKxjPxFi FD8Dg6Xxr76fBFq4riReuRtnKGQkhbzPu1t0djjQVdufpnEH4BspHo5gT+GFHlY5e8xRlMx6i6r B1sD0hRw+QNc= X-Received: by 2002:a1c:7406:0:b0:3ed:320a:3721 with SMTP id p6-20020a1c7406000000b003ed320a3721mr281090wmc.22.1679590978974; Thu, 23 Mar 2023 10:02:58 -0700 (PDT) X-Google-Smtp-Source: AK7set/2X0vV6gTyBSJOJJGL1Fdh0QWcbOfAS0eHPNeVmmhuLkrzvxlO/MiBZ4ftHK9wZ9NuLSr74Q== X-Received: by 2002:a1c:7406:0:b0:3ed:320a:3721 with SMTP id p6-20020a1c7406000000b003ed320a3721mr281045wmc.22.1679590978561; Thu, 23 Mar 2023 10:02:58 -0700 (PDT) Received: from ?IPV6:2003:cb:c704:e500:5cdf:c280:4b31:4016? (p200300cbc704e5005cdfc2804b314016.dip0.t-ipconnect.de. [2003:cb:c704:e500:5cdf:c280:4b31:4016]) by smtp.gmail.com with ESMTPSA id fk6-20020a05600c0cc600b003ee7169d57dsm2353857wmb.40.2023.03.23.10.02.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 23 Mar 2023 10:02:57 -0700 (PDT) Message-ID: <3dd0e43d-36f7-3325-7680-33779e9b0a55@redhat.com> Date: Thu, 23 Mar 2023 18:02:56 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [PATCH v10 02/16] x86/virt/tdx: Detect TDX during kernel boot To: "Huang, Kai" , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" Cc: "Hansen, Dave" , "Luck, Tony" , "bagasdotme@gmail.com" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "kirill.shutemov@linux.intel.com" , "Christopherson,, Sean" , "Chatre, Reinette" , "pbonzini@redhat.com" , "tglx@linutronix.de" , "Yamahata, Isaku" , "linux-mm@kvack.org" , "peterz@infradead.org" , "Shahar, Sagi" , "imammedo@redhat.com" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" References: <35a2421ca97d9e8dd938dcd744674602f4faa617.1678111292.git.kai.huang@intel.com> <90f6a15c-0dec-4a19-7a21-b18b73932a21@redhat.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A5A444002F X-Stat-Signature: xyoh58zda769d1z49pweeorkinhtn5my X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1679590984-749126 X-HE-Meta: U2FsdGVkX18kJkzo+hd0ddW6COC5CLqbi0LCfUmH8Zl52GDQU2n4Ai29yf7XKXsFqR0U7u/8QNrlT4ECo+aVqWtRhWhmYSc6E6Rffwzt72G5NrGLc5fWD65JwySRbehndaKrHdOIVMCNG0OBqcAY6tvWxE98aCETMyB2uUwJkAOUYuBzuh6w/A2vsJXWt1+McqCgo852UUBjzYlZq+1FUd14LWrYXxilepoisXme0olXjUUVmtpVkvTpJI2DRP+VTOw98RmIUoZ55oCrZljg9ZnQB8wJ2OOLdKzSNLNhR04k6AGd1UQbWjIJp/0ILXQEACSDgPMjkmh41M9IXDSan888AJW7DX6uwSYbw0/910WH7KAn9wE2F1DeWCgvpSkkPaCAcBuQEqhqL/S52YKU2bJaAvKcXqwtuqF04DITlJwaMe4bwQwvxM1IcEtBchaLLt4KEtGMX+4W7HdyvLbDHvt+7UXb3L7a2SE1EqyN6MyJmnOUA+NkcuBVqbHX4skJ56/cCKtX666iYotMxVUiRhdqPH9WSjmEwmt/V1XCuI5v/AcgIweMIE7epTnWyl4hCcmCJM8VV7gKIJUUvnxJwNPrwY3QdDvlXsYnOPNIK3AAu/8i4pMUB3KjcuZvQo7G62y0FhSuOJQX6QfYJyPmks8INQGf55PPTK/ARm6v9iJe8XneJ/nNgTCYZknwspNbzmKLU/5hryAGzBtcP+GJ0btjKb2m50/vvTHW9HV8/BXaQM5HTLyu9ErXwWRLyErnYkB2SJd8veiPYTchHN+Krrd4l0HNSWTEcCau+n19iJo4M+g4lZXqLtwTPiDOGBV7gZ77wvpkUrYI2CxmsRrGtV2jEibBBHp26qOTI4o7A1o4WDsaYNwqXMSvCQiteMAvkUh8xqCfD1lIJodqo1KnryDNLpRldV1yMZ1rUZF28SJjtd5NFYcd4TWWUT9XeQTp3hSLIVF3EJ0JhF64WDx cGXbV1QJ fcCOAsR2VZ9c95odxXIMHIYm2f8Sb6UTbuGhBOm1hr3RK3x0q+VXFbZYxOnVwrRKxih9zS95IouutAsGAR58rsougoZKXIGDZcqyLVYY71bDL+jGnTqjWGQqOu4vhCFaSK4oSepMwqwkxFdwKROdxxM8M/bwHecTbiFB76qh4e+f0DRkthjOFw2Z6eOeDFaBNkrULT6QT9SVQ4ZCvpcdGnplh8cnO8sYfCgncm3C+hNV61rw0vE3oRcsg3Kj3XHUhToRhvQfDtokU8m73esGO9jQE+cFZuXFG/b3NveD+kLXMagOXHw66/L0CJx8bL+YPtzs1fRlX2/h03a9dvaetk6E2tQ/iWoTaL/7ytA31hAp0vX/+5l5jyNzbrtSt+cm2V2qOPNM1GR0AqJNCeKkEyV9FKzHyEkLxpoLHla2Pdnw6oEv7UmvaXKcoPF5cMZU9s6AbiTCTuL2L+rabOz4qefTOfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 16.03.23 23:37, Huang, Kai wrote: > On Thu, 2023-03-16 at 13:48 +0100, David Hildenbrand wrote: >> On 06.03.23 15:13, Kai Huang wrote: >>> Intel Trust Domain Extensions (TDX) protects guest VMs from malicious >>> host and certain physical attacks. A CPU-attested software module >>> called 'the TDX module' runs inside a new isolated memory range as a >>> trusted hypervisor to manage and run protected VMs. >>> >>> Pre-TDX Intel hardware has support for a memory encryption architecture >>> called MKTME. The memory encryption hardware underpinning MKTME is also >>> used for Intel TDX. TDX ends up "stealing" some of the physical address >>> space from the MKTME architecture for crypto-protection to VMs. The >>> BIOS is responsible for partitioning the "KeyID" space between legacy >>> MKTME and TDX. The KeyIDs reserved for TDX are called 'TDX private >>> KeyIDs' or 'TDX KeyIDs' for short. >>> >>> TDX doesn't trust the BIOS. During machine boot, TDX verifies the TDX >>> private KeyIDs are consistently and correctly programmed by the BIOS >>> across all CPU packages before it enables TDX on any CPU core. A valid >>> TDX private KeyID range on BSP indicates TDX has been enabled by the >>> BIOS, otherwise the BIOS is buggy. >> Sorry for the late reply! >> So we don't trust the BIOS, but trust the BIOS that it won't hot-remove >> physical memory or hotplug physical CPUS (if I understood the cover >> letter correctly)? :) > > The "trust" in this context means security, but not functionality. BIOS needs > to do the right thing in order to make things work correctly in terms of > functionality. > > For physical memory hotplug or CPU hotplug, we don't have patch to _explicitly_ > distinguish them (from logical memory hotplug and logical cpu online/offline), > but actually they are kinda also handled: For memory hotplug, and hot-added > memory is rejected to go online (because they cannot be in TDX's convertible > memory ranges). For CPU hotplug, we have a function to do per-cpu > initialization (tdx_cpu_enable() in patch 5), and it will return error for hot- > added physical cpu. Make sense, thanks! > >> >>> >>> The TDX module is expected to be loaded by the BIOS when it enables TDX, >>> but the kernel needs to properly initialize it before it can be used to >>> create and run any TDX guests. The TDX module will be initialized by >>> the KVM subsystem when KVM wants to use TDX. >>> >>> Add a new early_initcall(tdx_init) to detect the TDX by detecting TDX >>> private KeyIDs. Also add a function to report whether TDX is enabled by >>> the BIOS. Similar to AMD SME, kexec() will use it to determine whether >>> cache flush is needed. >>> >>> The TDX module itself requires one TDX KeyID as the 'TDX global KeyID' >>> to protect its metadata. Each TDX guest also needs a TDX KeyID for its >>> own protection. Just use the first TDX KeyID as the global KeyID and >>> leave the rest for TDX guests. If no TDX KeyID is left for TDX guests, >>> disable TDX as initializing the TDX module alone is useless. >> >> Does that really happen in practice that we care about that at all? >> Seems weird and rather like a broken firmware or sth like that ... > > No it doesn't happen in practice, because the BIOS is sane enough. > > But since the public spec doesn't explicitly say it is guaranteed this doesn't > happen when TDX is enabled, I just added this sanity check. Okay! > >> >>> >>> To start to support TDX, create a new arch/x86/virt/vmx/tdx/tdx.c for >>> TDX host kernel support. Add a new Kconfig option CONFIG_INTEL_TDX_HOST >>> to opt-in TDX host kernel support (to distinguish with TDX guest kernel >>> support). So far only KVM uses TDX. Make the new config option depend >>> on KVM_INTEL. >>> >>> Signed-off-by: Kai Huang >>> Reviewed-by: Kirill A. Shutemov >> >> >> [...] >> >>> --- >>> arch/x86/Kconfig | 12 ++++ >>> arch/x86/Makefile | 2 + >>> arch/x86/include/asm/msr-index.h | 3 + >>> arch/x86/include/asm/tdx.h | 7 +++ >>> arch/x86/virt/Makefile | 2 + >>> arch/x86/virt/vmx/Makefile | 2 + >>> arch/x86/virt/vmx/tdx/Makefile | 2 + >>> arch/x86/virt/vmx/tdx/tdx.c | 105 +++++++++++++++++++++++++++++++ >>> 8 files changed, 135 insertions(+) >>> create mode 100644 arch/x86/virt/Makefile >>> create mode 100644 arch/x86/virt/vmx/Makefile >>> create mode 100644 arch/x86/virt/vmx/tdx/Makefile >>> create mode 100644 arch/x86/virt/vmx/tdx/tdx.c >>> >>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig >>> index 3604074a878b..fc010973a6ff 100644 >>> --- a/arch/x86/Kconfig >>> +++ b/arch/x86/Kconfig >>> @@ -1952,6 +1952,18 @@ config X86_SGX >>> >>> If unsure, say N. >>> >>> +config INTEL_TDX_HOST >>> + bool "Intel Trust Domain Extensions (TDX) host support" >>> + depends on CPU_SUP_INTEL >>> + depends on X86_64 >>> + depends on KVM_INTEL >>> + help >>> + Intel Trust Domain Extensions (TDX) protects guest VMs from malicious >>> + host and certain physical attacks. This option enables necessary TDX >>> + support in host kernel to run protected VMs. >> >> s/in host/in the host/ ? > > Sure. > >> >> Also, is "protected VMs" the right term to use here? "Encrypted VMs", >> "Confidential VMs" ... ? > > "Encrypted VM" perhaps is not a good choice, because there are more things than > encryption. I am also OK with "Confidential VMs", but "protected VMs" is also > used in the KVM series (not upstreamed yet), and also used by s390 by looking at > the git log. > > So both "protected VM" and "confidential VM" work for me. > > Not sure anyone else wants to comment? I'm fine as long as it's used consistently. "Protected VM" would have been the one out of the 3 alternatives that I have heard least frequently. > >> > [...] > >>> +static u32 tdx_global_keyid __ro_after_init; >>> +static u32 tdx_guest_keyid_start __ro_after_init; >>> +static u32 tdx_nr_guest_keyids __ro_after_init; >>> + >>> +/* >>> + * Use tdx_global_keyid to indicate that TDX is uninitialized. >>> + * This is used in TDX initialization error paths to take it from >>> + * initialized -> uninitialized. >>> + */ >>> +static void __init clear_tdx(void) >>> +{ >>> + tdx_global_keyid = 0; >>> +} >> >> Why not set "tdx_global_keyid" last, such that you don't have to clear >> when anything goes wrong before that? Seems more straight forward. > > My thinking was by reserving the global keyid and taking it out first, I can > check the remaining keyids for TDX guests easily: > > > + if (!nr_tdx_keyids) { > + pr_info("initialization failed: too few private KeyIDs > available.\n"); > + goto no_tdx; > + } > > Otherwise need to do: > > if (nr_tdx_keyids < 2) { > ... > } > > Also, in the later patch to handle memory hotplug we will add an additional step > to register_memory_notifier() which can also fail, so I just introduced > clear_tdx() here. > > But nothing is big deal, and yes we can set the global keyid at last and remove > clear_tdx(). Good, that simplifies things, thanks! -- Thanks, David / dhildenb