From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A82EC7EE25 for ; Mon, 12 Jun 2023 07:58:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 593188E0002; Mon, 12 Jun 2023 03:58:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 542D96B0074; Mon, 12 Jun 2023 03:58:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 432C68E0002; Mon, 12 Jun 2023 03:58:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 31E4D6B0072 for ; Mon, 12 Jun 2023 03:58:43 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DB0121C73C3 for ; Mon, 12 Jun 2023 07:58:42 +0000 (UTC) X-FDA: 80893344084.16.E1E4AA7 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by imf13.hostedemail.com (Postfix) with ESMTP id 443A820015 for ; Mon, 12 Jun 2023 07:58:40 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bxh+8YG7; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf13.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.31) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686556720; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=12+j+aOJSWb76BOWjkyWOr8Fe0QyYrVNBGeDj000NTw=; b=iUWPG37H9YiKi7tbpiR8On5ZiJt9CWA6WAwwEQitRvuo8uDEETzbimaD2/UXmC7gT5YixN 7PlB51Y3fVVVg/OvBTJm+PsCwkCUlzfrKVK9R+WX1Vlg1JEgLAFw6yk6WhwBAWlk27FqDn MGg2SAuoHZLxnOqm0LmhrBAiO5/5J7I= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bxh+8YG7; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf13.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.31) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686556720; a=rsa-sha256; cv=none; b=qc8v/pEPFmKMCAwnFaW6fliVbON0ZCMzpct02+VJv9RL1Eq8nDvm5AVGHJ4xMhDMR6rJqK uQy7XmPZqNzfar9ebSLzU7z0nNI/l8kosozJmi4y7mCP/qJdhYBwOhBso+mJ5Cqd5Ls4tw /VXIRX8kDbpicwG6aovEeBkLKIriLcw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686556720; x=1718092720; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=EEmcYq3XMCh9eZC3lpJIYouGZl8cqpZH7pVSmmfzWgE=; b=bxh+8YG7YxPOTOibDPqEoao1qlaD3MWGMiskVRKFza3BeTAGeGLXOhHD S7e3wyaryJEdKQu5cou/5iVLseUc9gsN2YYbWvG1gQk0zuf+XFoViMDGg /388bIblS/WwTEGUQWJ6dSUpI9fgfUbnKYuSDzHa9Cjfw4L5mPLRra6T8 PXXwBZZyxI0Oxy2QPzkK/jfj5Px3NinSOB9IfRkazX0PwI0hmjo/9nydI kQdLivbc/+BQhIqGb9w1MBgNQ/gTPTnO9EKSuPd+93y0uPmhiJoO5Ynet J7VfWLX3qk/LEKRtSfSxi8Cc4HSbKohIe91Sa/tdn7mdmlpmXOKgBbnGR Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10738"; a="421566821" X-IronPort-AV: E=Sophos;i="6.00,236,1681196400"; d="scan'208";a="421566821" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 00:58:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10738"; a="823885252" X-IronPort-AV: E=Sophos;i="6.00,236,1681196400"; d="scan'208";a="823885252" Received: from smizr3x-mobl3.ger.corp.intel.com (HELO box.shutemov.name) ([10.249.43.127]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 00:58:33 -0700 Received: by box.shutemov.name (Postfix, from userid 1000) id 653A210CC1C; Mon, 12 Jun 2023 10:58:30 +0300 (+03) Date: Mon, 12 Jun 2023 10:58:30 +0300 From: "kirill.shutemov@linux.intel.com" To: "Huang, Kai" Cc: "kvm@vger.kernel.org" , "Hansen, Dave" , "david@redhat.com" , "bagasdotme@gmail.com" , "ak@linux.intel.com" , "Wysocki, Rafael J" , "linux-kernel@vger.kernel.org" , "Chatre, Reinette" , "Christopherson,, Sean" , "pbonzini@redhat.com" , "tglx@linutronix.de" , "linux-mm@kvack.org" , "Yamahata, Isaku" , "Luck, Tony" , "peterz@infradead.org" , "Shahar, Sagi" , "imammedo@redhat.com" , "Gao, Chao" , "Brown, Len" , "sathyanarayanan.kuppuswamy@linux.intel.com" , "Huang, Ying" , "Williams, Dan J" Subject: Re: [PATCH v11 18/20] x86: Handle TDX erratum to reset TDX private memory during kexec() and reboot Message-ID: <20230612075830.jbrdd6ysz4qq7wdf@box.shutemov.name> References: <5aa7506d4fedbf625e3fe8ceeb88af3be1ce97ea.1685887183.git.kai.huang@intel.com> <20230609132301.uvvp27yr5kpenl6f@box.shutemov.name> <58f34b4b81b6d6b37d3386dec0f073e6eb7a97ff.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <58f34b4b81b6d6b37d3386dec0f073e6eb7a97ff.camel@intel.com> X-Rspam-User: X-Stat-Signature: iye17qzy6i3pzq8hn39j3415pmjrd1h8 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 443A820015 X-HE-Tag: 1686556720-474241 X-HE-Meta: U2FsdGVkX18oEDvjjcYMfTBmhoMqZ0nKQ9HzCfEOF3zgo4QiklGRXc5al5ojnKA9QjMwO/PunbLYF0R5tN16fU/79t9SgPUKznoNjFbnveiL8in5AhGPxbIzAFe022oZZ5JcFopi1PAY/Q1QIBfIi1NggqEITieZCc9nVNNr1WxM+bTNbGYj5HENROFjPWB9TmXD9v2zXMM2SjETtabYm/JThtyJW8TQvo2ogV35JlChLqgUbfF7n1OaC5ab7nIEpkerY8RQ9jAbMDtDUpSZ6Np6FtKO3hC4BUNxeX3/yeIPItJgXHEHrvTZNgWEakmz1XXKc8kSsbmIR2f9nqr/4DDVXeu1sViO4XgvwufU5hkS91+Te6Pi2ftw1HRYYW8qXSrJDyyjRz9xVGS06zhBiAFXcok1Bw0VJtBo7hz6yj9+8fYFzJxX9Zi7ygfQ28ZjMPzqfws49sXnupmf4YaAugEfVbB+Ld6bjPcl01k5jYq0Th2YRqJflBmKPQd5U713dayNC6ex28aVwbjlm4Gn5IWRtL5iu2dOH6LS5mxF69fMAol1XTdMB+p7NzyNnnJ5c3lZcXMGllL0WO33cxyEAqRcJEf82oxbc84kOLqRLSUujjqPCeMGK7Xitt0GfloDyo1Y+x5cY03EJRFAWXaD6t1fS4Ui/XKKq9YvNX8phte1E9y9F00Jh3y0o5X9SwVJnAwVvi6TySIsFTTewBEF2AJzB9L+Jyy6poKhSQrrZbtI4JSoTY8WW9j7lZY3mrue3kaI2mCb9vCej51kikuE0Md3SO46T+t5364MKpZ6D02YlyfVepCMdV1HR+AF09xHfrol8/4RH32Z9bOZ1dyZcd+uKa4vWgKzK2MszFfoOYAARwt/w7TFSjT/WvyjA8TvZ3dW19m70MO+OGdWcpb2AczbtVzXZbN2UDX886cmhv76wTcbhlnMGEy1l8Izg7kbCcTt0EwY0BazAneHQME DuZxUu4O IB+ZNSUJXcDr8TroQxjtMWvVeG7HkNisrR4jaNT0gBIQx2WMf+XbrZsyGrvK4rFGOB0zD5E5Xt7Z+gKx1OVgGpOFmb6qk4miQoBAKpQ8s97nu0bODulvw+xgkJt7QIxPJ+IMCFpRswpuSC+iB2zcHOnkPOmTDNhJptJqDEGwfMgrX6tpftDb/WCWSyA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jun 12, 2023 at 03:06:48AM +0000, Huang, Kai wrote: > On Fri, 2023-06-09 at 16:23 +0300, kirill.shutemov@linux.intel.com wrote: > > On Mon, Jun 05, 2023 at 02:27:31AM +1200, Kai Huang wrote: > > > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c > > > index 8ff07256a515..0aa413b712e8 100644 > > > --- a/arch/x86/virt/vmx/tdx/tdx.c > > > +++ b/arch/x86/virt/vmx/tdx/tdx.c > > > @@ -587,6 +587,14 @@ static int tdmr_set_up_pamt(struct tdmr_info *tdmr, > > > tdmr_pamt_base += pamt_size[pgsz]; > > > } > > > > > > + /* > > > + * tdx_memory_shutdown() also reads TDMR's PAMT during > > > + * kexec() or reboot, which could happen at anytime, even > > > + * during this particular code. Make sure pamt_4k_base > > > + * is firstly set otherwise tdx_memory_shutdown() may > > > + * get an invalid PAMT base when it sees a valid number > > > + * of PAMT pages. > > > + */ > > > > Hmm? What prevents compiler from messing this up. It can reorder as it > > wishes, no? > > Hmm.. Right. Sorry I missed. > > > > > Maybe add a proper locking? Anything that prevent preemption would do, > > right? > > > > > tdmr->pamt_4k_base = pamt_base[TDX_PS_4K]; > > > tdmr->pamt_4k_size = pamt_size[TDX_PS_4K]; > > > tdmr->pamt_2m_base = pamt_base[TDX_PS_2M]; > > > > I think a simple memory barrier will do. How does below look? > > --- a/arch/x86/virt/vmx/tdx/tdx.c > +++ b/arch/x86/virt/vmx/tdx/tdx.c > @@ -591,11 +591,12 @@ static int tdmr_set_up_pamt(struct tdmr_info *tdmr, > * tdx_memory_shutdown() also reads TDMR's PAMT during > * kexec() or reboot, which could happen at anytime, even > * during this particular code. Make sure pamt_4k_base > - * is firstly set otherwise tdx_memory_shutdown() may > - * get an invalid PAMT base when it sees a valid number > - * of PAMT pages. > + * is firstly set and place a __mb() after it otherwise > + * tdx_memory_shutdown() may get an invalid PAMT base > + * when it sees a valid number of PAMT pages. > */ > tdmr->pamt_4k_base = pamt_base[TDX_PS_4K]; > + __mb(); If you want to play with barriers, assign pamt_4k_base the last with smp_store_release() and read it first in tdmr_get_pamt() with smp_load_acquire(). If it is non-zero, all pamt_* fields are valid. Or just drop this non-sense and use a spin lock for serialization. -- Kiryl Shutsemau / Kirill A. Shutemov