From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20E5DC433FE for ; Wed, 30 Nov 2022 14:04:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 774586B0072; Wed, 30 Nov 2022 09:04:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 724CF6B0073; Wed, 30 Nov 2022 09:04:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5ECB86B0074; Wed, 30 Nov 2022 09:04:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5244C6B0072 for ; Wed, 30 Nov 2022 09:04:53 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C2A9CAB8B6 for ; Wed, 30 Nov 2022 14:04:52 +0000 (UTC) X-FDA: 80190279624.27.35CFA30 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by imf30.hostedemail.com (Postfix) with ESMTP id 910EA80014 for ; Wed, 30 Nov 2022 14:04:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669817090; x=1701353090; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=L7Trwb2C5sPvmhlDk+PrNgcbJF8XVAhiKvoDPrIpHD8=; b=ApDMiVH4YbyE7inyOuE7YtE9RsuL/KS6+4kxq5P+ROTONV1ewqa95HBW kaI1iJYLbU5K/4p+Ev4p+fs+weRHYwM/3MLY3C/UJz+ffWsjxT51ctHlF r4GqiMFbrTL1Aj9ZfFI+ZyQmqfD3p2QsIizPV/0tJCrSen3XRYVWBsmV6 /CpQmrwjn6PgF8umQ0IuSSIbgBcOfHp0yj+bxnKGvUiv6nCFbKLsdxXSX pBy/gjMHGne3oIfRK+qWPWxVUuV3MmXDE2XfsBtaBFYNtMOZKtSCnAaQr mij807QqhSuSRf3+EOhxkrKFHPPxeBwFz7hI7Tt4nABZ3zycatv73KJDz w==; X-IronPort-AV: E=McAfee;i="6500,9779,10547"; a="303002805" X-IronPort-AV: E=Sophos;i="5.96,206,1665471600"; d="scan'208";a="303002805" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Nov 2022 06:04:36 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10547"; a="646339272" X-IronPort-AV: E=Sophos;i="5.96,206,1665471600"; d="scan'208";a="646339272" Received: from bkalluri-mobl.ger.corp.intel.com (HELO box.shutemov.name) ([10.252.35.191]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Nov 2022 06:04:31 -0800 Received: by box.shutemov.name (Postfix, from userid 1000) id 78574109A09; Wed, 30 Nov 2022 17:04:28 +0300 (+03) Date: Wed, 30 Nov 2022 17:04:28 +0300 From: "kirill.shutemov@linux.intel.com" To: "Huang, Kai" Cc: "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "binbin.wu@linux.intel.com" , "Hansen, Dave" , "Luck, Tony" , "bagasdotme@gmail.com" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "Christopherson,, Sean" , "Chatre, Reinette" , "pbonzini@redhat.com" , "linux-mm@kvack.org" , "Yamahata, Isaku" , "peterz@infradead.org" , "Shahar, Sagi" , "imammedo@redhat.com" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" Subject: Re: [PATCH v7 17/20] x86/virt/tdx: Configure global KeyID on all packages Message-ID: <20221130140428.pkejerx6u6alctyl@box.shutemov.name> References: <8d8285cc5efa6302cf42a3fe2c9153d1a9dbcdac.1668988357.git.kai.huang@intel.com> <3d19683cdc13b7a3884f1e9e75743e922d4630f3.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3d19683cdc13b7a3884f1e9e75743e922d4630f3.camel@intel.com> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669817091; a=rsa-sha256; cv=none; b=GugPZsAjZN/yYxF9qUWp7qTJzbhJLnDvy1aZGzGA/g9QjZdGKR+2Ipl0K6wd7yN+f2wr/W 1eTt7SJLVh7Nsxj1iHPp6FWnIq+jSTcyD1RX36GRUDgHfgGH7kOrk6B118uYV1W4D8sZmg 4srq9D13uOxYVEBjGOCYmpRj9+qgIKw= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ApDMiVH4; spf=none (imf30.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.20) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669817091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LE7Ud0F7atuN64eeopDCkX+IO1SIUqPAEoo4m+HOaL4=; b=0cP8Ihm0JycyGo7TeCSgp696L7FPdih0CXd3AumkdaJw+HfjwfiChm/nCqLeTf2jvlmGiN IpXo7JqJ5H2K50FOiNQR9nfBRvAZKYiThnEq1BrgFGFN6QmI19u5XzzsDFI1QQuGvkknB1 moLkzaIJZ7XNoIA0aUqZPQi6UIx4CX8= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 910EA80014 X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ApDMiVH4; spf=none (imf30.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.20) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) X-Stat-Signature: twn77uc88hgd7hiykixjtwhyrr6muscp X-HE-Tag: 1669817090-763953 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 30, 2022 at 08:34:46AM +0000, Huang, Kai wrote: > On Wed, 2022-11-30 at 11:35 +0800, Binbin Wu wrote: > > On 11/21/2022 8:26 AM, Kai Huang wrote: > > > After the array of TDMRs and the global KeyID are configured to the TDX > > > module, use TDH.SYS.KEY.CONFIG to configure the key of the global KeyID > > > on all packages. > > > > > > TDH.SYS.KEY.CONFIG must be done on one (any) cpu for each package. And > > > it cannot run concurrently on different CPUs. Implement a helper to > > > run SEAMCALL on one cpu for each package one by one, and use it to > > > configure the global KeyID on all packages. > > > > > > Intel hardware doesn't guarantee cache coherency across different > > > KeyIDs. The kernel needs to flush PAMT's dirty cachelines (associated > > > with KeyID 0) before the TDX module uses the global KeyID to access the > > > PAMT. Following the TDX module specification, flush cache before > > > configuring the global KeyID on all packages. > > > > > > Given the PAMT size can be large (~1/256th of system RAM), just use > > > WBINVD on all CPUs to flush. > > > > > > Note if any TDH.SYS.KEY.CONFIG fails, the TDX module may already have > > > used the global KeyID to write any PAMT. Therefore, need to use WBINVD > > > to flush cache before freeing the PAMTs back to the kernel. Note using > > > MOVDIR64B (which changes the page's associated KeyID from the old TDX > > > private KeyID back to KeyID 0, which is used by the kernel) > > > > It seems not accurate to say MOVDIR64B changes the page's associated KeyID. > > It just uses the current KeyID for memory operations. > > The "write" to the memory changes the page's associated KeyID to the KeyID that > does the "write". A more accurate expression perhaps should be MOVDIR64B + > MFENSE, but I think it doesn't matter in changelog. MOVDIR64B KeyID for the cache line, not the page. Integrity tracked on per-cacheline basis. -- Kiryl Shutsemau / Kirill A. Shutemov