From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63C13EB64D7 for ; Fri, 30 Jun 2023 10:07:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D079B8E0007; Fri, 30 Jun 2023 06:07:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB74A8D0001; Fri, 30 Jun 2023 06:07:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B808C8E0007; Fri, 30 Jun 2023 06:07:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A8F618D0001 for ; Fri, 30 Jun 2023 06:07:19 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6E41D40EC9 for ; Fri, 30 Jun 2023 10:07:19 +0000 (UTC) X-FDA: 80958986598.26.603CC8E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf30.hostedemail.com (Postfix) with ESMTP id BD0FF80018 for ; Fri, 30 Jun 2023 10:07:16 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RAtEFZtn; dmarc=none; spf=none (imf30.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688119637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3y/9Jj3obwNlU1SHCuDEqbkfMRtUW049ByAh+VjrS7s=; b=cM1BAx/FaWMqN087zW8S9RLbY0DyfIlyxqJ0RJDR5gRRRPWbVyWMIROI5mkjrupzzqFMKj WVSrOBhfM+tDHiyHvgM84DbzR6EZPJHl/9aZmQaMHN/bfhII+pWH+l1HqCIY+0JaOd0SQM i01bmrGNuH3MkBsC5Px8864GTpz4/JM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RAtEFZtn; dmarc=none; spf=none (imf30.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688119637; a=rsa-sha256; cv=none; b=aHAaBEEZODVnpalTU4bSd7KrBP7J6T40V/faJHq2Zr5/23v/oTCgoD/3WOa8LbPzD00GQu bQTSeRgtYxfocDeUJwVKtExeh+VNGRwmPZHHRKGChBIjfeZNVfdYOYJq6a0YykWGcBg9YH 7wrc4ASD42jDFW4l6X99uNn7VvYk8q4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=3y/9Jj3obwNlU1SHCuDEqbkfMRtUW049ByAh+VjrS7s=; b=RAtEFZtnK5tETG0+gXxsyxFmRT 3JrM9TmL1/M9nWvYz886V0MIHUZF+C5yK24DZWqJ3jR/XalWnP/FlHGAw8YcDlK1PfIjPsgM9zzXD u6jIoIq2LSI3yibgHYi6bf18sITxHZ01MXkMQnXrrP5eieLS1Dh3ULf9oj2ovRBw0z73S/w+8BWld LyVwdPgOcoq38WnJeA6H017NQwGeTGqLh7GSIk5PdqVguyENId1Ilp9zLfDWfrfnHOjAbsn1CFWPX TG2GpTNHM4+yeAuT9f9GbV18BUZGcyTBKN1C2mvSP17gC3WPMibuXnIBWOvVjOpHLjJAEC7SByHYN PLg/NPhw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qFB1q-005e1n-19; Fri, 30 Jun 2023 10:07:02 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 195DE300118; Fri, 30 Jun 2023 12:07:00 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 0204224819479; Fri, 30 Jun 2023 12:06:59 +0200 (CEST) Date: Fri, 30 Jun 2023 12:06:59 +0200 From: Peter Zijlstra To: "Huang, Kai" Cc: "kvm@vger.kernel.org" , "Raj, Ashok" , "Hansen, Dave" , "david@redhat.com" , "bagasdotme@gmail.com" , "Luck, Tony" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "linux-kernel@vger.kernel.org" , "Christopherson,, Sean" , "mingo@redhat.com" , "pbonzini@redhat.com" , "linux-mm@kvack.org" , "tglx@linutronix.de" , "kirill.shutemov@linux.intel.com" , "Chatre, Reinette" , "Yamahata, Isaku" , "nik.borisov@suse.com" , "hpa@zytor.com" , "Shahar, Sagi" , "imammedo@redhat.com" , "bp@alien8.de" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" , "x86@kernel.org" Subject: Re: [PATCH v12 20/22] x86/virt/tdx: Allow SEAMCALL to handle #UD and #GP Message-ID: <20230630100659.GF2533791@hirez.programming.kicks-ass.net> References: <20230628152900.GI2438817@hirez.programming.kicks-ass.net> <20230628203823.GR38236@hirez.programming.kicks-ass.net> <42e13ccf7f27a68c0dd64640eed378c38ef40967.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42e13ccf7f27a68c0dd64640eed378c38ef40967.camel@intel.com> X-Rspamd-Queue-Id: BD0FF80018 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: d4hj5hcp6tcsmefpwpamdbsudhuqwfbu X-HE-Tag: 1688119636-895392 X-HE-Meta: U2FsdGVkX191N7imBpbJl7GsdBzPdfElvy+DBbYVemBF6I5+pVE2acWsK8vaUcPWfGiz0wYdfrR8HLhwgyonoorgqAGeu/ixmvBaNNQh+jSrQLjDxkkIXt0+FiR0wrH2ZcCOLdRmpa4dSgFQ0SIIxE95KjIzLXlHO9DCzaQzc62q71CUuILMvEIBrL96sdGDReUwT1Xp7u5O3crvNPzZHfS2cmVqiZM8azxN/nPO7vdZrfwuQEEo813lbk5cDaeTyfpp0IFld7IaE4auNDrpfVUMAT9D0kaP2ty6Lqh40gE+4NgLLW4bnJ+E6EZ3LucDV6fbG6yy4GKjAUOPqafOReUjsqJUu2ukYPhsj+3o4Ury9uRzyQS9okIEplPPrELLxvOf76mab7+EXsXZSe8j/Je84QXgWilhesghGi2LsQ64c4FNQe07nSgOQ6k0t7yT6RzqUy9U5xS2/JXak6+3VKLUYw7yj7dUljO9iAgxvcYrrqX3FRJowexQCrY27R0SApQ7n52YzShtJkrTcWO3wK9emawt0pW/737Wk2R3AIFwXGv6oFe8DKO2n2ir7bYXdg7IflpUiuUnvREVFtIgj2ZUfs3zUcrq5VeF9Wy8jkptPKfPHwGE0SJtMYJarKk0cJnvpmrGZi1L/bM9eLVuHFUJcih/vbj81Az/SiWQ+bJ4/K2sGMe42BOLGSlCLxRIc8C3yFRsuKyxAwzsLzZi8E/OGNQ9b5jfMhTcDG2CrSgMbgG4IxF+eVBtTwT9GjUQpjapgI8o+j9Fn04I/MH7XlwPRYlVh1kjgBjmrnFX9fvPBaxhligiuHoMcZvOO1TGCgteqcrIBFZ2X6tGaZiyxHzC8yz6jyuCAWknebvr+5IYaeUcJB7nq0FKFONR1y/wEyhD8dF2sWj9qHNJ/0NUUNXnC53yP+JaMmgw4TBD8TOoJwWSNEA3s7VDWnM6I7XLUTR7nvKw0y8QeMP/3eL mTruvITW wDzouPMwEV39kaBX8yLfBf3C5S0ZSx3tvL+eqwOgEgDmp8QPk04UJHG/OMy8AlefctaHvNylYxbCJ8/PQ/Aa2toyPZ8hzMhbTEp67yKJ7quZsWtLok57MmrVoCEuYlsHKL9l3LLRrtXM3VWkmTei3G6JBPrleUp++T3qg8M87AG3kUjR29T1uLA9jZGIadJNPGfWiUMWGjUkizSs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000022, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jun 29, 2023 at 10:33:38AM +0000, Huang, Kai wrote: > On Wed, 2023-06-28 at 22:38 +0200, Peter Zijlstra wrote: > > On Wed, Jun 28, 2023 at 05:29:01PM +0200, Peter Zijlstra wrote: > > > On Tue, Jun 27, 2023 at 02:12:50AM +1200, Kai Huang wrote: > > > > diff --git a/arch/x86/virt/vmx/tdx/tdxcall.S b/arch/x86/virt/vmx/tdx/tdxcall.S > > > > index 49a54356ae99..757b0c34be10 100644 > > > > --- a/arch/x86/virt/vmx/tdx/tdxcall.S > > > > +++ b/arch/x86/virt/vmx/tdx/tdxcall.S > > > > @@ -1,6 +1,7 @@ > > > > /* SPDX-License-Identifier: GPL-2.0 */ > > > > #include > > > > #include > > > > +#include > > > > > > > > /* > > > > * TDCALL and SEAMCALL are supported in Binutils >= 2.36. > > > > @@ -45,6 +46,7 @@ > > > > /* Leave input param 2 in RDX */ > > > > > > > > .if \host > > > > +1: > > > > seamcall > > > > > > So what registers are actually clobbered by SEAMCALL ? There's a > > > distinct lack of it in SDM Vol.2 instruction list :-( > > > > With the exception of the abomination that is TDH.VP.ENTER all SEAMCALLs > > seem to be limited to the set presented here (c,d,8,9,10,11) and all > > other registers should be available. > > RAX is also used as SEAMCALL return code. > > Looking at the later versions of TDX spec (with TD live migration, etc), it > seems they are already using R12-R13 as SEAMCALL output: > > https://cdrdv2.intel.com/v1/dl/getContent/733579 Urgh.. I think I read an older versio because I got bleeding eyes from all this colour coded crap. All this red is unreadable :-( Have they been told about the glories of TeX and diff ? > E.g., 6.3.15. NEW: TDH.IMPORT.MEM Leaf > > It uses R12 and R13 as input. 12 and 14. They skipped 13 for some mysterious raisin. But also, 10,11 are frequently used as input with this new stuff, which already suggests the setup from your patches is not tenable. > > Can we please make that a hard requirement, SEAMCALL must not use > > registers outside this? We can hardly program to random future > > extentions; we need hard ABI guarantees here. > > > I believe all other GPRs are just saved/restored in SEAMCALL/SEAMRET, so in > practice all other GPRs not used as input/output should not be clobbered. But I > will confirm with TDX module guys. And even it's true in practice it's better > to document it. > > But I think we also want to ask them to stop adding more registers as > input/output. > > I'll talk to TDX module team on this. Please, because 12,14 are callee-saved, which means we need to go add push/pop to preserve them :-( Then you end up with something like this... /* * TDX_MODULE_CALL - common helper macro for both * TDCALL and SEAMCALL instructions. * * TDCALL - used by TDX guests to make requests to the * TDX module and hypercalls to the VMM. * SEAMCALL - used by TDX hosts to make requests to the * TDX module. * *------------------------------------------------------------------------- * TDCALL/SEAMCALL ABI: *------------------------------------------------------------------------- * Input Registers: * * RAX - TDCALL Leaf number. * RCX,RDX,R8-R11 - TDCALL Leaf specific input registers. * * Output Registers: * * RAX - TDCALL instruction error code. * RCX,RDX,R8-R11 - TDCALL Leaf specific output registers. * R12-R14 - extra output registers * *------------------------------------------------------------------------- * * __tdx_module_call() function ABI: * * @fn (RDI) - TDCALL Leaf ID, moved to RAX * @regs (RSI) - struct tdx_regs pointer * * Return status of TDCALL via RAX. */ .macro TDX_MODULE_CALL host:req ret:req extra:0 FRAME_BEGIN movq %rdi, %rax movq $TDX_SEAMCALL_VMFAILINVALID, %rdi movq TDX_MODULE_rcx(%rsi), %rcx movq TDX_MODULE_rdx(%rsi), %rdx movq TDX_MODULE_r8(%rsi), %r8 movq TDX_MODULE_r9(%rsi), %r9 movq TDX_MODULE_r10(%rsi), %r10 movq TDX_MODULE_r11(%rsi), %r11 .if \extra pushq r12 pushq r13 pushq r14 // movq TDX_MODULE_r12(%rsi), %r12 // movq TDX_MODULE_r13(%rsi), %r13 // movq TDX_MODULE_r14(%rsi), %r14 .endif .if \host 1: seamcall /* * SEAMCALL instruction is essentially a VMExit from VMX root * mode to SEAM VMX root mode. VMfailInvalid (CF=1) indicates * that the targeted SEAM firmware is not loaded or disabled, * or P-SEAMLDR is busy with another SEAMCALL. %rax is not * changed in this case. * * Set %rax to TDX_SEAMCALL_VMFAILINVALID for VMfailInvalid. * This value will never be used as actual SEAMCALL error code as * it is from the Reserved status code class. */ cmovc %rdi, %rax 2: .else tdcall .endif .if \ret movq %rcx, TDX_MODULE_rcx(%rsi) movq %rdx, TDX_MODULE_rdx(%rsi) movq %r8, TDX_MODULE_r8(%rsi) movq %r9, TDX_MODULE_r9(%rsi) movq %r10, TDX_MODULE_r10(%rsi) movq %r11, TDX_MODULE_r11(%rsi) .endif .if \extra movq %r12, TDX_MODULE_r12(%rsi) movq %r13, TDX_MODULE_r13(%rsi) movq %r14, TDX_MODULE_r14(%rsi) popq %r14 popq %r13 popq %r12 .endif FRAME_END RET .if \host 3: mov $TDX_SW_ERROR, %rdi or %rdi, %rax jmp 2b _ASM_EXTABLE_FAULT(1b, 3b) .endif .endm