From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97585EB64D7 for ; Fri, 30 Jun 2023 12:07:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D29E98E000E; Fri, 30 Jun 2023 08:07:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD9AC8D0001; Fri, 30 Jun 2023 08:07:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B7AF88E000E; Fri, 30 Jun 2023 08:07:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A5BDB8D0001 for ; Fri, 30 Jun 2023 08:07:14 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4471F1407A1 for ; Fri, 30 Jun 2023 12:07:14 +0000 (UTC) X-FDA: 80959288788.13.1466933 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf18.hostedemail.com (Postfix) with ESMTP id 6511A1C0005 for ; Fri, 30 Jun 2023 12:07:09 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b="KC/TbbzT"; spf=none (imf18.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688126830; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aQIbBeHHxL8b1EjNDcJ4bA1/I8pw3VBqIBojsRFOLbw=; b=GYgQ/hUeL45WhaHHS+3fnLNT9miW9UCmvHDK280amdYpzlw6D0c3714g7SnaXAt+fK5JrV LvE3Ke7Q29oRl3EVW5oxDGx9ptCyYMIKn5xtuXY6WNPeKVb1wmRhZvsvsPpgMFqv1Rr6ww T7rv0K6cItBs8AD3y1BfCKHZ1xWWTmM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b="KC/TbbzT"; spf=none (imf18.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688126831; a=rsa-sha256; cv=none; b=OP+NGOTV4cIuQPuIBZA+aMKYU+PLjysHm5ISZJqv3wvWDlN3fi100+q0LXDk0qE4sPyl4T 3TZwjssPqY+IxJDZ+iN15kmpeIAwe7QpDikNCdStdKBCbJjfBNYGGNlyygtzMbZ+4FyPIM kWsCm5TN0PSdHyQNclS3gmFB6mExz+Y= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=aQIbBeHHxL8b1EjNDcJ4bA1/I8pw3VBqIBojsRFOLbw=; b=KC/TbbzT8bzIA0JpUlRQYGkWSu X36S4/OI2xzOEihSBR6z3wSl1KCcHUBx4VX7XNxsLT9r0RDfusT+qDwypgxa2N2qTmYDJZbwKmPHo afaRLdxGeSE1dFf81lP+9jSfsYVKnLeTp9Xlz58gos5YrnsB+H3ln/H4v+d2+LynRreSEDtIViXqF EVweNvkVBisoB9UnnDIQu1Gl/yksQNfy3ois/kPe3S6Zu323pM5taGFZ4WpIJh+LiT5Xt5TNsGaIZ wjUCmtD54dZSTarHds4fj4AMRwlh28BlrQovnx+z1An9E1XKvSSpvsAXfl9OZahC0gHLlGhH6sXlE wGpkDy/w==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qFCto-007yu1-06; Fri, 30 Jun 2023 12:06:53 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 41827300338; Fri, 30 Jun 2023 14:06:50 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 28EC024806A2C; Fri, 30 Jun 2023 14:06:50 +0200 (CEST) Date: Fri, 30 Jun 2023 14:06:50 +0200 From: Peter Zijlstra To: "Huang, Kai" Cc: "kvm@vger.kernel.org" , "Raj, Ashok" , "Hansen, Dave" , "david@redhat.com" , "bagasdotme@gmail.com" , "Luck, Tony" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "linux-kernel@vger.kernel.org" , "Christopherson,, Sean" , "mingo@redhat.com" , "pbonzini@redhat.com" , "linux-mm@kvack.org" , "tglx@linutronix.de" , "kirill.shutemov@linux.intel.com" , "Chatre, Reinette" , "Yamahata, Isaku" , "nik.borisov@suse.com" , "hpa@zytor.com" , "Shahar, Sagi" , "imammedo@redhat.com" , "bp@alien8.de" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" , "x86@kernel.org" Subject: Re: [PATCH v12 20/22] x86/virt/tdx: Allow SEAMCALL to handle #UD and #GP Message-ID: <20230630120650.GB2534364@hirez.programming.kicks-ass.net> References: <20230628152900.GI2438817@hirez.programming.kicks-ass.net> <20230628203823.GR38236@hirez.programming.kicks-ass.net> <42e13ccf7f27a68c0dd64640eed378c38ef40967.camel@intel.com> <20230630100659.GF2533791@hirez.programming.kicks-ass.net> <20230630102141.GA2534364@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230630102141.GA2534364@hirez.programming.kicks-ass.net> X-Rspamd-Queue-Id: 6511A1C0005 X-Rspam-User: X-Stat-Signature: bbehrmcisam7kfcojxjpicggegqooo3y X-Rspamd-Server: rspam01 X-HE-Tag: 1688126829-436512 X-HE-Meta: U2FsdGVkX1+2UYMcqQjDQTvv1EBuSz+QWCwUd8Rg0Qjh3Fnkx8R0u03vKSjvEXEW2kyenu0psCjazRSb1KtqoA0zlSmPi2gbECyzV1G1NRfC8DBdXnY9j/e5GLq/HaTl31yLlxglMpREH06/dcUZV74TavU+wPH011tnPBTSZ1hdZl1AFY19O8hogerbmxgWC8JhA2JCQFYzaTKDWDWGF2bgvLdkFYHF3911eR2lYlXRbSC2jzitkUYglxVHmaVw+RN7dVXhNTAzy0tgIW4Yq3rs8Rh5iODds6zfBgvIcEUYfa4Q+asobF2wAdLPkyidnGJmbuh4/2V97J5nffCOcZSMPSxDkitOJjaH6vQlT028oKdQgBPmv5PQX1nOnEKGGywXhWazcTgJSNmPvye590bYQnL9xCZ5ZrX2zI/+7uZ5e94Oh6W23DoSqAVajw1YnU+REbKB+MVC4KmEnTZMd8/+gWnv7M24Nrc/DAYLxA8KYcxPkvbQIRSwbMqOd3s6Qgw3Op8iCdN1M7nKxtH3FEREmVpuUmCrYGZab0Lg8hweK49b0/XsskTZy3Pls+fUIUg+lwCQlD5XDtTprJH6/SZDurpN2DxE/OXrBSqDdu8UOqaRmE5J6LLOMiQSCKjXEl0nbxHuMzQxTh/IGzZxqS9rJopemivw7+hYPrjekE+EW5bIFYHgL3ndsHzPvUI2xMkhh7dmj/t/sjp/IdunZa2eHCxGlOvbNBlFlA2Dr7FhEsukr8qskzKCfJ6XS1L7byHQIzBH7JhpEUbMEZXvKlPQPlLiCsKvxaYHEb2O6jCyRN/gNGB81WosaKOxuIeY0mQDyzc7bfmynkbF2hbP4isNNWM/7LHDvy6KIHGbc8HFR5YSHU4f2ama1q2xyJkbeOYAR8E/rUNyCCJgwexK/5FLs5OPH1BF5xUkyOVWPjHWMjocXEjUZtRq4lD2svAum04A/WiPR/SIaWZ6RKR CSKRoNpB j/HkwTHyQuiRaoxF1hD5mynmfPvkGm6JqWaelLfZ77v0rO2imqVgrdohMK7Ifs5B64C8c++wXQ/AVxasypfv5epuE4pIDlHnOrwNod2yYNqH1h14/r35YVTTz63Q34s3sDyqlufa4DJ9BSz/hzfJ0k8cmeKPU+M2CWD8KgqBMLdH1BGJcu5LPPFBF+fVIe9JwSyznSUpcXO7MyLM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 30, 2023 at 12:21:41PM +0200, Peter Zijlstra wrote: > On Fri, Jun 30, 2023 at 12:07:00PM +0200, Peter Zijlstra wrote: > > On Thu, Jun 29, 2023 at 10:33:38AM +0000, Huang, Kai wrote: > > > > Looking at the later versions of TDX spec (with TD live migration, etc), it > > > seems they are already using R12-R13 as SEAMCALL output: > > > > > > https://cdrdv2.intel.com/v1/dl/getContent/733579 > > > > Urgh.. I think I read an older versio because I got bleeding eyes from > > all this colour coded crap. > > > > All this red is unreadable :-( Have they been told about the glories of > > TeX and diff ? > > > > > E.g., 6.3.15. NEW: TDH.IMPORT.MEM Leaf > > > > > > It uses R12 and R13 as input. > > > > 12 and 14. They skipped 13 for some mysterious raisin. > > Things like TDH.SERVTD.BIND do use R13. > > > But also, 10,11 are frequently used as input with this new stuff, which > > already suggests the setup from your patches is not tenable. > > > TDG.SERVTD.RD *why* can't they pass that TD_UUID as a pointer? Using *4* > registers like that is quite insane. > > TDG.VP.ENTER :-(((( that has b,15,si,di as additional output. > > That means there's not a single register left unused. Can we still get > this changed, please?!? Can't :/, VP.ENTER mirrors VP.VMCALL, so we need to deal with both. So I think the below deals with everything and unifies __tdx_hypercall() and __tdx_module_call(), since both sides needs to deal with exactly the same trainwreck. /* * Used for input/output registers values of the TDCALL and SEAMCALL * instructions when requesting services from the TDX module. * * This is a software only structure and not part of the TDX module/VMM ABI. */ struct tdx_module_args { /* callee-clobbered */ u64 rdx; u64 rcx; u64 r8; u64 r9; /* extra callee-clobbered */ u64 r10; u64 r11; /* callee-saved + rdi/rsi */ u64 rdi; u64 rsi; u64 rbx; u64 r12; u64 r13; u64 r14; u64 r15; }; /* * TDX_MODULE_CALL - common helper macro for both * TDCALL and SEAMCALL instructions. * * TDCALL - used by TDX guests to make requests to the * TDX module and hypercalls to the VMM. * * SEAMCALL - used by TDX hosts to make requests to the * TDX module. * *------------------------------------------------------------------------- * TDCALL/SEAMCALL ABI: *------------------------------------------------------------------------- * Input Registers: * * RAX - Leaf number. * RCX,RDX,R8-R11 - Leaf specific input registers. * RDI,RSI,RBX,R11-R15 - VP.VMCALL VP.ENTER * * Output Registers: * * RAX - instruction error code. * RCX,RDX,R8-R11 - Leaf specific output registers. * RDI,RSI,RBX,R12-R15 - VP.VMCALL VP.ENTER * *------------------------------------------------------------------------- * * So while the common core (RAX,RCX,RDX,R8-R11) fits nicely in the * callee-clobbered registers and even leaves RDI,RSI free to act as a base * pointer some rare leafs (VP.VMCALL, VP.ENTER) make a giant mess of things. * * For simplicity, assume that anything that needs the callee-saved regs also * tramples on RDI,RSI. This isn't strictly true, see for example EXPORT.MEM. */ .macro TDX_MODULE_CALL host:req ret:req saved:0 FRAME_BEGIN movq %rdi, %rax movq TDX_MODULE_rcx(%rsi), %rcx movq TDX_MODULE_rdx(%rsi), %rdx movq TDX_MODULE_r8(%rsi), %r8 movq TDX_MODULE_r9(%rsi), %r9 movq TDX_MODULE_r10(%rsi), %r10 movq TDX_MODULE_r11(%rsi), %r11 .if \saved pushq rbx pushq r12 pushq r13 pushq r14 pushq r15 movq TDX_MODULE_rbx(%rsi), %rbx movq TDX_MODULE_r12(%rsi), %r12 movq TDX_MODULE_r13(%rsi), %r13 movq TDX_MODULE_r14(%rsi), %r14 movq TDX_MODULE_r15(%rsi), %r15 /* VP.VMCALL and VP.ENTER */ .if \ret pushq %rsi .endif movq TDX_MODULE_rdi(%rsi), %rdi movq TDX_MODULE_rsi(%rsi), %rsi .endif .Lcall: .if \host seamcall /* * SEAMCALL instruction is essentially a VMExit from VMX root * mode to SEAM VMX root mode. VMfailInvalid (CF=1) indicates * that the targeted SEAM firmware is not loaded or disabled, * or P-SEAMLDR is busy with another SEAMCALL. RAX is not * changed in this case. */ jc .Lseamfail .if \saved && \ret /* * VP.ENTER clears RSI on output, use it to restore state. */ popq %rsi xor %edi,%edi movq %rdi, TDX_MODULE_rdi(%rsi) movq %rdi, TDX_MODULE_rsi(%rsi) .endif .else tdcall /* * RAX!=0 indicates a failure, assume no return values. */ testq %rax, %rax jne .Lerror .if \saved && \ret /* * Since RAX==0, it can be used as a scratch register to restore state. * * [ assumes \saved implies \ret ] */ popq %rax movq %rdi, TDX_MODULE_rdi(%rax) movq %rsi, TDX_MODULE_rsi(%rax) movq %rax, %rsi xor %eax, %eax; .endif .endif // \host .if \ret /* RSI is restored */ movq %rcx, TDX_MODULE_rcx(%rsi) movq %rdx, TDX_MODULE_rdx(%rsi) movq %r8, TDX_MODULE_r8(%rsi) movq %r9, TDX_MODULE_r9(%rsi) movq %r10, TDX_MODULE_r10(%rsi) movq %r11, TDX_MODULE_r11(%rsi) .if \saved movq %rbx, TDX_MODULE_rbx(%rsi) movq %r12, TDX_MODULE_r12(%rsi) movq %r13, TDX_MODULE_r13(%rsi) movq %r14, TDX_MODULE_r14(%rsi) movq %r15, TDX_MODULE_r15(%rsi) .endif .endif // \ret .Lout: .if \saved popq %r15 popq %r14 popq %r13 popq %r12 popq %rbx .endif FRAME_END RET /* * Error and exception handling at .Lcall. Ignore \ret on failure. */ .Lerror: .if \saved && \ret popq %rsi .endif jmp .Lout .if \host .Lseamfail: /* * Set RAX to TDX_SEAMCALL_VMFAILINVALID for VMfailInvalid. * This value will never be used as actual SEAMCALL error code as * it is from the Reserved status code class. */ movq $TDX_SEAMCALL_VMFAILINVALID, %rax jmp .Lerror .Lfault: /* * SEAMCALL caused #GP or #UD. Per _ASM_EXTABLE_FAULT() RAX * contains the trap number, convert to a TDX error code by * setting the high word to TDX_SW_ERROR. */ mov $TDX_SW_ERROR, %rdi or %rdi, %rax jmp .Lerror _ASM_EXTABLE_FAULT(.Lcall, .Lfault) .endif .endm