From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AE5AEB64DD for ; Wed, 5 Jul 2023 10:21:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E387C6B0072; Wed, 5 Jul 2023 06:21:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE72E8D0002; Wed, 5 Jul 2023 06:21:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C87AE8D0001; Wed, 5 Jul 2023 06:21:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B907E6B0072 for ; Wed, 5 Jul 2023 06:21:57 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C9B580B3A for ; Wed, 5 Jul 2023 10:21:57 +0000 (UTC) X-FDA: 80977167474.04.0FA1F5C Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 20EE120021 for ; Wed, 5 Jul 2023 10:21:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TkLfIfSO; spf=none (imf13.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688552515; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4Ry2jwT0YbMwWbjx+3d3lF7amM0BUUKmYAyIeJr5PH0=; b=WGgB1wqwyx+tHo22PsbZaKK+Ns8s4adw8LGofZlQRhP5SmesWMI6sUSjtNQuJX5ft5c3JT bfbrcbD58eKwUiphqgugc+QbY+ODnHXTypc681Y62ATCd1IJ0H4/8GDbMGrZlBS3LC8wRm 7LDBHDTi/i75QP5mkIIft4G69uuCSts= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688552515; a=rsa-sha256; cv=none; b=WtWOEmFAAGGoEzrl+ayCsDQQjLQWxulW9i9N3jBekcsSG7Pcd55uv1O2A0rZxGqiB6Bgrj w1aLPxJNBlWui1pFSxxQKVx4DRVWFRgUw5ws/Bo8mrdmXts+f/p9q/jn9OiFcCaAUBqszO /ycr9GEvKzUszebDW43milAl6B/Uiis= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TkLfIfSO; spf=none (imf13.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=4Ry2jwT0YbMwWbjx+3d3lF7amM0BUUKmYAyIeJr5PH0=; b=TkLfIfSOKiy03+vFG2OK6XsHNN Iq/mWTSZwNCg6/mZQN71Of8QJbi5+cWYhWfQs7Bg4NO1fX6WIuakygNQYhMZCOyeZBV/fVZJcp4+m JaHFPtOq0506oqa9eJxIq3H1ZQlw/Xgm/Sy6TUK5IRLCoKiLI4pzqvQ93rUo5XLnJbVhPlg7hbmrd +GdtrTHYNPvLalFg8XnTjE3HI89O7Lhkz+M3iW5BX3ZQIYNQfBnDHchympbxWrlXeIEFDnNibnAdw YLit9T+us3bY6+IX/Kpgx/R/E2ggxQ1gpsSUQ/Se+5PiF54l+IifukRhkO7I8VfhslIyaLjqT3SIt oxlzWqyA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qGzdj-009yvy-Tr; Wed, 05 Jul 2023 10:21:39 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 02A2F30005E; Wed, 5 Jul 2023 12:21:38 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id DD7032028F056; Wed, 5 Jul 2023 12:21:37 +0200 (CEST) Date: Wed, 5 Jul 2023 12:21:37 +0200 From: Peter Zijlstra To: "Huang, Kai" Cc: "kvm@vger.kernel.org" , "Raj, Ashok" , "Luck, Tony" , "david@redhat.com" , "bagasdotme@gmail.com" , "Hansen, Dave" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "kirill.shutemov@linux.intel.com" , "Chatre, Reinette" , "Christopherson,, Sean" , "pbonzini@redhat.com" , "mingo@redhat.com" , "tglx@linutronix.de" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "Yamahata, Isaku" , "nik.borisov@suse.com" , "hpa@zytor.com" , "Shahar, Sagi" , "imammedo@redhat.com" , "bp@alien8.de" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" , "x86@kernel.org" Subject: Re: [PATCH v12 20/22] x86/virt/tdx: Allow SEAMCALL to handle #UD and #GP Message-ID: <20230705102137.GX4253@hirez.programming.kicks-ass.net> References: <20230628152900.GI2438817@hirez.programming.kicks-ass.net> <20230628203823.GR38236@hirez.programming.kicks-ass.net> <42e13ccf7f27a68c0dd64640eed378c38ef40967.camel@intel.com> <20230630100659.GF2533791@hirez.programming.kicks-ass.net> <20230630102141.GA2534364@hirez.programming.kicks-ass.net> <20230630120650.GB2534364@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: 7ba7xoxw713dtn3gt1ewioz9d64errrs X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 20EE120021 X-Rspam-User: X-HE-Tag: 1688552513-592516 X-HE-Meta: U2FsdGVkX1/tZB9c6ifvbgmOq4RvP6jLezfiPJIopO7IronmBnTvhi4pB2D9VOgtpvZm5sQY39UKB54eSQYCBPAj33qLKxai6UO4Lsn/+13nmbJM/inBYiVfj+HscxOr8gwtoVQyGB2WP1+bh7WVeYkGS5Iclbm/bKwHzaNEbb/41YVuhgRKp5A605p9bYvNPwmQ+0aVEkIEI5bFw175zbV7kznpzxJKcvuhRwtvCqzumH23sxnVYfF1f/XMPiHn8IE7tycmKkss45R109nFGP7Gz35Y2j5o1G/vZlVf8D8ThkVy7GbJaT0pM+G1Ou7RcdBpdueOK64Hm6cjv/Ih8EB8yAbmV2vdJQBZymTWlBCD1hz598Cuk0c3YP2EgF1GBoanWYrvQCoHRCzBihygqVa7vwpJ8bXrajKrKoZ0u2h0cXDleul0fTx12m11Py6/eSaSHTvwhRY3zfx2wSV5HllRrwVlWSbRx5BpTveWvgTOLFqB2MFMIIrbLU7rNuFnmy25Jd0+KUY8b342/tUIGEwcG7SuXwUcUsEXCBEOx2p5FMO/5AMDB5Qh1LOFQtaTawAzHSLZmo6VV6I3ewMInzbh/i4Ey2JwdS1Psneq6uJ1+MbWUFQwURZ5xdNahrlM03lSCjPLzeypcH5+rNlLnDC4vuFE6VyVac891AjpuROM+PFTAvsep2aHHguoOumlRKWSybKOC9lpvQSlm6LEQt7YwEOYg2dgvdJsqOFtUvE5qSLJk9wqTa3VmMNfpvWrkaEdLf80EIFLqyGtGuNVCpVTT6MbJAuyi7/zZoTlq7KwcM83m/33vLEq02WlLZ776cV4t/+B41tLG5GcENfuGj6R47+NBZpxksKh049wKLiBWlVbv922m6/H3u0WLWGacW4jQ5nVLoVBcA0g/nwPOM5juFbGdsNMwYhvNuJHDnwKPOmXRiDmCwS+aL8qPJqpJuCb4nAyq1sYy+a5EIh /p30C6nW arCxHQvZA+t+fixTiDaxwvvk7P9Lu6gwuPStLKLnI8tOhSOI7KnKxOxqDJX2TFZ53WA7FLPePlbk3Ume9unipZlGBD+nuEpV7rSsBi2fe/GatGNHUrvK5NLxaZuBfdHV1R3eGpZRjQXc5egY4Hf2Ef7z48eG03dqRNnFYqmk1aaBi9PTB24RTNaOaMw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 03, 2023 at 12:15:13PM +0000, Huang, Kai wrote: > > > > > So I think the below deals with everything and unifies __tdx_hypercall() > > and __tdx_module_call(), since both sides needs to deal with exactly the > > same trainwreck. > > Hi Peter, > > Just want to make sure I understand you correctly: > > You want to make __tdx_module_call() look like __tdx_hypercall(), but not to > unify them into one assembly (at least for now), right? Well, given the horrendous trainwreck this is all turning into, I through it prudent to have it all in a single place. The moment you go play games with callee-saved registers you're really close to what hypercall does so then they might as well be the same. > I am confused you mentioned VP.VMCALL below, which is handled by > __tdx_hypercall(). But why? It really isn't *that* special if you consider the other calls that are using callee-saved regs, yes it has the rdi/rsi extra, but meh, it really just is tdcall-0. > > *------------------------------------------------------------------------- > > * TDCALL/SEAMCALL ABI: > > *------------------------------------------------------------------------- > > * Input Registers: > > * > > * RAX - Leaf number. > > * RCX,RDX,R8-R11 - Leaf specific input registers. > > * RDI,RSI,RBX,R11-R15 - VP.VMCALL VP.ENTER > > * > > * Output Registers: > > * > > * RAX - instruction error code. > > * RCX,RDX,R8-R11 - Leaf specific output registers. > > * RDI,RSI,RBX,R12-R15 - VP.VMCALL VP.ENTER > > As mentioned above, VP.VMCALL is handled by __tdx_hypercall(). Also, VP.ENTER > will be handled by KVM's own assembly. They both are not handled in this > TDX_MODULE_CALL assembly. I don't think they should be special, they're really just yet another leaf call. Yes, they have a shit calling convention, and yes VP.ENTER is terminally broken for unconditionally clobbering BP :-( That really *must* be fixed. > > .Lcall: > > .if \host > > seamcall > > /* > > * SEAMCALL instruction is essentially a VMExit from VMX root > > * mode to SEAM VMX root mode. VMfailInvalid (CF=1) indicates > > * that the targeted SEAM firmware is not loaded or disabled, > > * or P-SEAMLDR is busy with another SEAMCALL. RAX is not > > * changed in this case. > > */ > > jc .Lseamfail > > > > .if \saved && \ret > > /* > > * VP.ENTER clears RSI on output, use it to restore state. > > */ > > popq %rsi > > xor %edi,%edi > > movq %rdi, TDX_MODULE_rdi(%rsi) > > movq %rdi, TDX_MODULE_rsi(%rsi) > > .endif > > .else > > tdcall > > > > /* > > * RAX!=0 indicates a failure, assume no return values. > > */ > > testq %rax, %rax > > jne .Lerror > > For some SEAMCALL/TDCALL the output registers may contain additional error > information. We need to jump to a location where whether returning those > additional regs to 'struct tdx_module_args' depends on \ret. I suppose we can move this into the below conditional :-( The [DS]I register stuff requires a scratch reg to recover, AX being zero provides that. > > .if \saved && \ret > > /* > > * Since RAX==0, it can be used as a scratch register to restore state. > > * > > * [ assumes \saved implies \ret ] > > */ > > popq %rax > > movq %rdi, TDX_MODULE_rdi(%rax) > > movq %rsi, TDX_MODULE_rsi(%rax) > > movq %rax, %rsi > > xor %eax, %eax; > > .endif > > .endif // \host So the reason I want this, is that I feel very strongly that if you cannot write a single coherent wrapper for all this, its calling convention is fundamentally *too* complex / broken.