From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64604C7EE25 for ; Wed, 7 Jun 2023 20:08:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9AEC6B0072; Wed, 7 Jun 2023 16:08:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4A986B0074; Wed, 7 Jun 2023 16:08:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B12A98E0001; Wed, 7 Jun 2023 16:08:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9F4D26B0072 for ; Wed, 7 Jun 2023 16:08:43 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6A50DAF3E7 for ; Wed, 7 Jun 2023 20:08:43 +0000 (UTC) X-FDA: 80877039726.28.7A68821 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf10.hostedemail.com (Postfix) with ESMTP id 9C769C000A for ; Wed, 7 Jun 2023 20:08:41 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=GkWNBRV6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3yOOAZAYKCL0vhdqmfjrrjoh.frpolqx0-ppnydfn.ruj@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3yOOAZAYKCL0vhdqmfjrrjoh.frpolqx0-ppnydfn.ruj@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686168521; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jgJbQ4s3E0ptV0J61xFQqdeD0UJeIxytYchnod3vyfo=; b=CB0pyD/a1NGwP6gzg/3RY5Op/VcEzo9npkJG/RifszXmzg8/gSQUgK+m08DMnXSlkT2oMa N8BxCBhN+8oy0p9CJipgfyrzDVpjExbXVYbVVe/JRpXKX93aQvI2+qQrW3RTb4OZoPfcE7 wItLOw/B2Ai9j2IqLh5XAXL32pBR620= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=GkWNBRV6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3yOOAZAYKCL0vhdqmfjrrjoh.frpolqx0-ppnydfn.ruj@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3yOOAZAYKCL0vhdqmfjrrjoh.frpolqx0-ppnydfn.ruj@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686168521; a=rsa-sha256; cv=none; b=k4TO3JsaFYIEpgGkYCjwRsZlyEZoH4p70HlmJvz6hwPGES6Z4ntVYA9KToeQ4VtEddHSaw nq/mpI52tdFSs5lhVQOqkNK4h1qysDiCnxQt8rFud6u4QSOZJrLj9RZJvKB84XAon4zx0t Ka3n3mK7WGeAcCcijyAlUN30eB/5eDc= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-bb24045f986so8346808276.2 for ; Wed, 07 Jun 2023 13:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686168520; x=1688760520; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jgJbQ4s3E0ptV0J61xFQqdeD0UJeIxytYchnod3vyfo=; b=GkWNBRV6t1YZMx4xdqZJQJwX7kJ0WBw5UzCt6VQht83tzs9v99hEKa28gsZVLS8Lpc rmtNOhdK5FKaqiHM6l92mTz/IjTB+gqmHgR7QUsBQEZiMumo4PhSNY5s+K+O1aqZipjJ qipL5YLQNjEEBgbTfkVKiPO8ks5b59YTER+ai+7epmaIEVQFH1FSm/vGPw+FAohfugWb k3aBtk9WMfkoUi+eugsbMre6SbpIUJzW1upG/p3vPwZuN3haloXRHYwNNNdVk4p0bKq5 /5XusrILJ1buGjmFr/lmCCjqDHSGnJvsRGRUSsdwOW1FDwexSFscY125DLQjVSoiC2Uu Porg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686168520; x=1688760520; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jgJbQ4s3E0ptV0J61xFQqdeD0UJeIxytYchnod3vyfo=; b=j1pZu3jjaBYAQgivg6G3qV9bnrvsF5FLHG8cAk/QTUS4D5AZ6sX1gXmX5WtichVLCB oZktrH4D7K17vyUCTmomoVgshdmdFq24HGQI61QS3jcglZ52xc4e3RM0XRdVP7mvNjrJ d0LAdgeBu1AK5vOqvO5HLGWI83aVpkVQ2u4mHF5qLk54vbSr9f92ToSUWez9AJYavPar i+JSUC/xBPHKhMXBs5qc4eoAn0zvFNsxgEhuIqYy6pFL7xyLvBKBbAwzIk2wlDYE+imJ bbeMvh+QgcNQy1dxDNw2DFaKKu5fQyTaoWO4aVhdIgrpoBugtoIoVREiJGFUWZUEzOJm 3khw== X-Gm-Message-State: AC+VfDw7V8MCUREDv87P1DarJOUc3M6ymrxQ/oSFyQETvz47LMEC+AOu gEUocF8V0Ky/FsOj8v9ZJoF8j87cQ4Q= X-Google-Smtp-Source: ACHHUZ4CWqZ7S6LmS14VmBgENlfpWqKaIe0/ag4iqXmtNl7mS/uCOoepaJdD5hQkkWrltaj+S1OMb1y3lbk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:12cc:b0:ba7:8099:c5f2 with SMTP id j12-20020a05690212cc00b00ba78099c5f2mr2419940ybu.8.1686168520674; Wed, 07 Jun 2023 13:08:40 -0700 (PDT) Date: Wed, 7 Jun 2023 13:08:39 -0700 In-Reply-To: <20230607194721.GI2244082@ls.amr.corp.intel.com> Mime-Version: 1.0 References: <92e19d74-447f-19e0-d9ec-8a3f12f04927@intel.com> <20230607185355.GH2244082@ls.amr.corp.intel.com> <20230607194721.GI2244082@ls.amr.corp.intel.com> Message-ID: Subject: Re: [PATCH v11 05/20] x86/virt/tdx: Add SEAMCALL infrastructure From: Sean Christopherson To: Isaku Yamahata Cc: Dave Hansen , Kai Huang , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, kirill.shutemov@linux.intel.com, tony.luck@intel.com, peterz@infradead.org, tglx@linutronix.de, pbonzini@redhat.com, david@redhat.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, ying.huang@intel.com, reinette.chatre@intel.com, len.brown@intel.com, ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com Content-Type: text/plain; charset="us-ascii" X-Rspam-User: X-Stat-Signature: k9pctixfttsykh36eb4d3rss1qfb4sm3 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9C769C000A X-HE-Tag: 1686168521-480306 X-HE-Meta: U2FsdGVkX18nl3O4qIhU42Bt2b6fl5WDgTTcJlQ41ME49ov7I0/C6XKJjQBCosgWnktCdwFTHUsGIuiKNhmlDjuPHMDfyNOkalqEUpExpU+zZCU9F/6+Wn3HZsFZ3iatfRPX2XS/K32EUTewZyI5JNXJGHfgVlV54EPwc0qriL90CcwJTNu/PiG+g7hvjGZbosJGiwNi3XCQE+9ZcCgltH/SQBY5NbMBLJ1PubOdWAVYbuKfcNuOOwOCwYXnU0WbufQN3ZOQZz+ocwH1mEOYnbDwx3KWZXjoBtVZ+QgZFAkyVNV2PwAE8675ptB4FiNy32kU7svxwPHpyVQsm6LV6zBb6dc1Gc5SOI7oMdKzlgGFXn57g6KiLQVnuM1dWxUSd1oIEPooJKdfuU4H6BlCq4QYFNkeozdE4LAUJH5EMX0ZUyQo+6BUUV0eLI2ZFIfGLYcVg4oRsV2Ftv5FoNVz4HuvhgmTIiRw6bivFc3TNE7Ut1XyRnnouiGGs8Hkyu2IFVzgOTu5hJfua7g4z9qoilaZjWj16UbTbuw2YPNkbvtJwpSVxWWrhMdR2uM5vJt4uROc3ohEAxfG0n9MhUc34FIpfCmexADtdnpxb8XNKzpZ10mWOSbp537ZINqxwG+sGk7lbjywRp8fsYO73wEpdjfQ6+BGTot3hFUXwV88vYCFIHfM6ukSzIgCoya+JbhHJrnWdRpDAr0MEi/Ho1UX93Hz8XS8SNZBkSW/Q4rFQ2vfiDocJVYy1NCdyKMjjtvvkfgxcJZgmAZhIfsAb0Cf3hhqWuY11V+JftWHtVcW0WhNgMea9MQmVgjVhXwI/jfjHdbU3znabC5G3842kUp5N8qvHxaqgOuJZl7ex95uVM5LEAyZmKP5pxqREjSgRFaxW2wH1j7JKfJSEbyyhA4r+LyLUdfjrCfSXI6YV4W2shv+WxQUmtCCXo0y3Jhn4RlahNaTvv/Srgnj36g1Hyb DsguMWVQ k8WvNk8j4gs98FX++o31TSDV/vt3P5MIz7QspGJEhgyD/oGLQsBr8rXQ44Ed8Lu+GLT1IloadiDVzcNjzlfioJ5ew9+Bz08l/thiuX5Do6Ew/7gUVnBGQ4NPQH1Cqx/hRCFqY71a/tEOMBurnnl4iO/rVmAHO9wzX1xfEC5qPL3K5CxwBDKrOza0/LL6MPM4fECWFzsF8CoJIS2XyL5k6QVvg4U/bhlGWZjKyWoW8O1alA/FSqLyiwUyWUNAWt+Iwh5ri7qm0FTu/hffNM6zpdnIwuPpnzg1M2g2rXXGHkWQF1sHPvfLykSA97KkEfYt95220HfnD/jDxYqPhdz+EFBe274R/z8WU/T3FPxkxasOgIz4sBJQFVENv5Xct8iFPvN6MQW0RrzdBbD83FUt6cyS9mAikIJftiHAcWj/Er3VqFudbyKuTvnCkYKu02EODBIC19PvK1+Wvij4UpHYD1/gV9epyzRVr53z8XCGZuWTNApSDYpRgMeVpWxYgPK16NlSivm1jpGnmXqhH9fqp3rXRJIVEP0EzxLf4rUKZ3AzQPMeIouq7ktlhuIASfRnOg6BdVn/559tlwJFaFX979aSiO8brBhn7HYaB04bhQGeMYewseKEKGSsexDMTuQpsRm0zL4T323bzEnt8Tqpt5+DRzXsIQmL5vCD0X0MT4i2IINA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jun 07, 2023, Isaku Yamahata wrote: > On Wed, Jun 07, 2023 at 12:27:33PM -0700, > Dave Hansen wrote: > > > On 6/7/23 11:53, Isaku Yamahata wrote: > > >>> VMX enabling, and KVM is the only user of TDX. This implementation > > >>> chooses to make KVM itself responsible for enabling VMX before using > > >>> TDX and let the rest of the kernel stay blissfully unaware of VMX. > > >>> > > >>> The current TDX_MODULE_CALL macro handles neither #GP nor #UD. The > > >>> kernel would hit Oops if SEAMCALL were mistakenly made w/o enabling VMX > > >>> first. Architecturally, there is no CPU flag to check whether the CPU > > >>> is in VMX operation. Also, if a BIOS were buggy, it could still report > > >>> valid TDX private KeyIDs when TDX actually couldn't be enabled. > > >> I'm not sure this is a great justification. If the BIOS is lying to the > > >> OS, we _should_ oops. > > >> > > >> How else can this happen other than silly kernel bugs. It's OK to oops > > >> in the face of silly kernel bugs. > > > TDX KVM + reboot can hit #UD. On reboot, VMX is disabled (VMXOFF) via > > > syscore.shutdown callback. However, guest TD can be still running to issue > > > SEAMCALL resulting in #UD. > > > > > > Or we can postpone the change and make the TDX KVM patch series carry a patch > > > for it. > > > > How does the existing KVM use of VMLAUNCH/VMRESUME avoid that problem? > > extable. From arch/x86/kvm/vmx/vmenter.S > > .Lvmresume: > vmresume > jmp .Lvmfail > > .Lvmlaunch: > vmlaunch > jmp .Lvmfail > > _ASM_EXTABLE(.Lvmresume, .Lfixup) > _ASM_EXTABLE(.Lvmlaunch, .Lfixup) More specifically, KVM eats faults on VMX and SVM instructions that occur after KVM forcefully disables VMX/SVM. E.g. with reboot -f, this will be reached without first stopping VMs: static void kvm_shutdown(void) { /* * Disable hardware virtualization and set kvm_rebooting to indicate * that KVM has asynchronously disabled hardware virtualization, i.e. * that relevant errors and exceptions aren't entirely unexpected. * Some flavors of hardware virtualization need to be disabled before * transferring control to firmware (to perform shutdown/reboot), e.g. * on x86, virtualization can block INIT interrupts, which are used by * firmware to pull APs back under firmware control. Note, this path * is used for both shutdown and reboot scenarios, i.e. neither name is * 100% comprehensive. */ pr_info("kvm: exiting hardware virtualization\n"); kvm_rebooting = true; on_each_cpu(hardware_disable_nolock, NULL, 1); } which KVM x86 (VMX and SVM) then queries when deciding what to do with a spurious fault on a VMX/SVM instruction /* * Handle a fault on a hardware virtualization (VMX or SVM) instruction. * * Hardware virtualization extension instructions may fault if a reboot turns * off virtualization while processes are running. Usually after catching the * fault we just panic; during reboot instead the instruction is ignored. */ noinstr void kvm_spurious_fault(void) { /* Fault while not rebooting. We want the trace. */ BUG_ON(!kvm_rebooting); } EXPORT_SYMBOL_GPL(kvm_spurious_fault);