From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 651FBEB64DD for ; Wed, 5 Jul 2023 20:56:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE4288D0002; Wed, 5 Jul 2023 16:56:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6CF68D0001; Wed, 5 Jul 2023 16:56:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBFAD8D0002; Wed, 5 Jul 2023 16:56:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A5CB68D0001 for ; Wed, 5 Jul 2023 16:56:57 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6E82980784 for ; Wed, 5 Jul 2023 20:56:57 +0000 (UTC) X-FDA: 80978767674.16.6322E24 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 8860B1C0008 for ; Wed, 5 Jul 2023 20:56:54 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="MqB/Cx7O"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of isaku.yamahata@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=isaku.yamahata@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688590614; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ovzEbDAGJ3+ZjzcYiibK38OFMjmG360mQnBH1xSJlPk=; b=jKkEbmbclAWJdYrfzM+tbdDkvC7R/rkkcFs8dlA1nF8Uryuwf9v0hR49s8fEdzMb7TRdke Lq6GBdwjfuYAnJbWIOfkNilW8IRZS5sx+XUWvqeKP9jQDPsJm077naFIXEJs7iE1W0As8T MVLAj9ud73JX4tT+tAI7a1T9e+d1Kq0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="MqB/Cx7O"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of isaku.yamahata@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=isaku.yamahata@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688590614; a=rsa-sha256; cv=none; b=q6/G3RjNR+Gc+kfWyzae0wrR54hvZQDgBtDExp0ETTT3ESF+pYAYd50bpNeOA6ZGId8biv WLRxdjcPN9+ZWZvQ6vhF54aKR6isf4PDBruz2+Bm/Z8zZ19Q4AkICjM38PCMaePsI70QsO 29Z10FSfLwbCrgY2Q9oyywY/ikXw8JI= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1b8b318c5a7so7112615ad.3 for ; Wed, 05 Jul 2023 13:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1688590613; x=1691182613; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ovzEbDAGJ3+ZjzcYiibK38OFMjmG360mQnBH1xSJlPk=; b=MqB/Cx7O1eM1d9pgvNMoEsucDpppcyx1oBo1eJOHnUH/YPkxddkx5hl0nZHaNFEI5c G6ps1xaj4ET6AI9UAOkC08vDCnuoU/mCYSlutjyo55YiTgIId4xDh6OOhzRZF357P7Mw 9NoA7XRB6pVwOyxXzAScmJ0ktaQWLpntLhjDUD0rFzU9CGxVsuznMwjuIrq7/3X6x+2G o455gZcnGVOoykXm6kNg2Efj5rRIc52hj1oRr9OmtjbV6PgJoooFgHb/pzvaw9JKho2B Gf/hNeKmeJz6Zhz7ilOCVjkgwFPgeWPC8olAqtXzFYskQORQKSMO0l3RWQGHl6O9WPRW e9qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688590613; x=1691182613; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ovzEbDAGJ3+ZjzcYiibK38OFMjmG360mQnBH1xSJlPk=; b=aHG40kw3yD48vbEAhRR03j3lRM2DGdUvwCn9vU0G538ZvXpTu6lW6RP9Nr5wCqYuVR AJOBF5mBRK3GvQmY2gMkrgHrGj5rM4wwR1dfpIqdtNA7t2sICUmD6r6ZBdbkWXW7LqAY qvqR2TIrHkEepO8yy7VmpN2hBz3CxGnKQbq3VRZtBGs4LRHU/uWEoZWsHGakgWiC2/NF QEouPUKA6LSDXTmBdSQudUsH3MxU5sE8dd7DovHARcGa8BJd2Bx5yPdUwscEOV3Vpbr8 B7WW/XL72+XssiQ7vHjej04PRjx0y2ydLhRKRLJr3aK2NMyJpz+neGntG+oPUXmgeHxV YGsg== X-Gm-Message-State: ABy/qLbk5cvvaUchH4ba8JEKT95UvixPN+pNQEOmoC09sXLsu0edTZzW P8RLwwNdx1etw1fx0LC0WMYKy10lpLsF6A== X-Google-Smtp-Source: APBJJlEeMt/Iu5iAHsjfGQ7mflNzscvdOd5j6NwpGC/RfdnC/J4hNkVbWBefWE+XhpRumWZ5+EOT3Q== X-Received: by 2002:a17:902:d486:b0:1b2:fa8:d9c9 with SMTP id c6-20020a170902d48600b001b20fa8d9c9mr59542plg.49.1688590613062; Wed, 05 Jul 2023 13:56:53 -0700 (PDT) Received: from localhost ([192.55.54.50]) by smtp.gmail.com with ESMTPSA id x18-20020a170902ec9200b001b022f2aa12sm14405060plg.239.2023.07.05.13.56.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Jul 2023 13:56:52 -0700 (PDT) Date: Wed, 5 Jul 2023 13:56:50 -0700 From: Isaku Yamahata To: "Huang, Kai" Cc: "peterz@infradead.org" , "kvm@vger.kernel.org" , "Raj, Ashok" , "Luck, Tony" , "david@redhat.com" , "bagasdotme@gmail.com" , "Hansen, Dave" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "kirill.shutemov@linux.intel.com" , "Chatre, Reinette" , "Christopherson,, Sean" , "pbonzini@redhat.com" , "mingo@redhat.com" , "tglx@linutronix.de" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "Yamahata, Isaku" , "nik.borisov@suse.com" , "hpa@zytor.com" , "Shahar, Sagi" , "imammedo@redhat.com" , "bp@alien8.de" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" , "x86@kernel.org" , isaku.yamahata@gmail.com Subject: Re: [PATCH v12 20/22] x86/virt/tdx: Allow SEAMCALL to handle #UD and #GP Message-ID: <20230705205650.GJ3436214@ls.amr.corp.intel.com> References: <20230628203823.GR38236@hirez.programming.kicks-ass.net> <42e13ccf7f27a68c0dd64640eed378c38ef40967.camel@intel.com> <20230630100659.GF2533791@hirez.programming.kicks-ass.net> <20230630102141.GA2534364@hirez.programming.kicks-ass.net> <20230630120650.GB2534364@hirez.programming.kicks-ass.net> <20230705102137.GX4253@hirez.programming.kicks-ass.net> <20230705121921.GZ4253@hirez.programming.kicks-ass.net> <3bef89e1bc5935625483cc5bf339c13f643c2c29.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <3bef89e1bc5935625483cc5bf339c13f643c2c29.camel@intel.com> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 8860B1C0008 X-Stat-Signature: p45x179ea3kegeai5z4xz63ctfmbp3qa X-HE-Tag: 1688590614-166872 X-HE-Meta: U2FsdGVkX1/SBmsCHL+g45pQ/2CAVARSNBYLewmb5znOuN4l1RDDh+XnAVZMmuneKa94czYincNkMOJmpRYb/lIYYlF+zyHV1LRlA6kb3bq6mAwJNIdnioffNTLx+WJJ/r4wvRxKrAAoyArEZXhi1GEdBUSAv2VYI1l1sDp5DZtLQH4dgNmYXRngBPCEJyMT6YRCRWy27MbZ0o5x2uRs4GdFbI05RsC6qQ7Y5/KgauABR+RUoP42RTHGz0pkvLh3vq9XQTpVY+aC1D9w/uApWujfBI0Bv5/bmSKwurd5q0LZieHQr/RDKhMrvmpMAC6JBIZpMeG6j2MR7IiUveUQPEwX6QevHbDAclhq0Wx92u3wWiJjTj+ljUxY0R/YdNZLgTVQ97iS9ne/HhSbWkUNK5R6+dx4sh3+eCO4UGBb53ADrPZB71EmKl+HMDpzcmIuT4NMNboN5TYKVb0vDafngHMqAcklQbFnLE8Nuw3/MPE5g3p/gSjZXruD1bLO/lYpJoImwXx9pwl4gT3QQLBTRoieLZxYzxUFK3x9I0R7dPwMgJX1pJI8YY+d8ke65mHNnDveN9WDEth52Hml3rWtfuSoyuDwmVXWjMqBtpTQU4/M/1xB/ozgGJd6F0EvMMosLvTONEXm+aifW8VwOKNuM68218qlnVsKRjZLqBEm5X3VbcI7Eyf2Y7oQ1A2o4m+iCR7JjJnIk2R+aEY/YQL1SHDL1YdDGbh9OvqUEfo41sXPL2Fw1IYraRF/HJ24fmdjT8nSXU4UncLRWgpgs77kgBOSX7fpyyVGlkn8nS3Fp4N4DB6cVHnDY0Std4Kzdrt1PUC+x0gswnvl1ERDp40f68GKDjwVaGRHDKn43jpsyuBT+TztNWWGjrV4VFNIPvTLN9pRqFuxIPhLVIn6Vo37lyPXwfJBpeem2WSSKH7iDWGLl/hdt2+kAAVcVgexFZVgufe0BkaNBSln5z8f6Nq 5zBcJXYJ TyH1LJjNcT/uaeNIVdJqeiSm8tNquFRf8oPgJmECA8RE3AKScQqMTStrto06TPLYhmPFpvDQ7ZKnfZ/8ONBz03BTGTUuEbCt5iDKXVE8b/GaSajYF2q2ncu/zqWNTTftkhLly8OdlepjNT3LUubAtnLJ3HG61nixikvsZ8yILGnRTkcyAkv7sbPb3pCY16UkDu/6lcW/2HTr2KwRevNTSYsSqP8YE7BM3+9RRqhML1u8uhFcNQM+JZ71uOVE82tlD0PUFV0SmPvsu6fzHIMHY6p17HONBuB+/qTt8OM8Pwj0zO3tmUV5QaWCe+2cHaihRxfiNC8SAkFvTJF6traM3s3htVBAPLNg0zn6S3cY0yxlB/S7kDHRk3t12yA6gaJ1Ifu/cCtV+vb7isdF2Y5yKp+GgHJnkVJ9nVG8RKEc9rE1Q96bO9YnoY1wKSEhReHtOF3oRzzd4bxaVoy4WzYIM5lBsrrz+camKU8RBNaNpP4pXo1fQoJmmne2rs5qbyFMV056gv8YVfgzSjYC+lurNXP9ElXoNE4rPz/mp9VrBMqjsXBHGj4oGh/jOfmmLhgZdeBpmCNQ2DRp+9Wk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000021, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 05, 2023 at 12:53:58PM +0000, "Huang, Kai" wrote: > On Wed, 2023-07-05 at 14:19 +0200, Peter Zijlstra wrote: > > On Wed, Jul 05, 2023 at 11:34:53AM +0000, Huang, Kai wrote: > > > > > Yeah I think from long-term's view, since SEAMCALLs to support live migration > > > pretty much uses all RCX/RDX/R8-R15 as input/output, it seems reasonable to > > > unify all of them, although I guess there might be some special handling to > > > VP.VMCALL and/or VP.ENTER, e.g., below: > > > > > > /* TDVMCALL leaf return code is in R10 */ > > > movq %r10, %rax > > > > > > So long-termly, I don't have objection to that. But my thinking is for the > > > first version of TDX host support, we don't have to support all SEAMCALLs but > > > only those involved in basic TDX support. > > > > Since those calls are out now, we should look at them now, there is no > > point in delaying the pain. That then gives us two options: > > > > - we accept them and their wonky calling convention and our code should > > be ready for it. > > > > - we reject them and send the TDX team a message to please try again > > but with a saner calling convention. > > > > Sticking our head in the sand and pretending like they don't exist isn't > > really a viable option at this point. > > OK. I'll work on this. > > But I think even we want to unify __tdx_module_call() and __tdx_hypercall(), the > first step should be making __tdx_module_call() look like __tdx_hypercall()? I > mean from organizing patchset's point of view, we cannot just do in one big > patch but need to split into small patches with each doing one thing. > > By thinking is perhaps we can organize this way: > > 1) Patch(es) to make TDX_MODULE_CALL macro / __tdx_module_call() look like > __tdx_hypercall(). > 2) Add SEAMCALL support based on TDX_MODULE_CALL, e.g., implement __seamcall(). > 3) Unify __tdx_module_call()/__seamcall() with __tdx_hypercall(). > > Does this look good? > > Btw, I've already part 1) based on your code, and sent the patches to Kirill for > review. Should I sent them out first? > > > > > > Also, the new SEAMCALLs to handle live migration all seem to have below > > > statement: > > > > > > AVX, AVX2 May be reset to the architectural INIT state > > > and > > > AVX512 > > > state > > > > > > Which means those SEAMCALLs need to preserve AVX* states too? > > > > Yes, we need to ensure the userspace 'FPU' state is saved before > > we call them. But I _think_ that KVM already does much of that. > > Let me look into this. KVM VCPU_RUN ioctl saves/restores FPU state by kvm_load_guest_fpu() and kvm_put_guest_fpu() which calls fpu_swap_kvm_fpstate(). Other KVM ioctls doesn't modify FPU. Because some SEAMCALLs related for live migration don't preserve FPU state, we need explicit save/restore of FPU state. -- Isaku Yamahata