From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 012B3EB64DC for ; Mon, 3 Jul 2023 15:26:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C48B280013; Mon, 3 Jul 2023 11:26:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7737C280001; Mon, 3 Jul 2023 11:26:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63C8E280013; Mon, 3 Jul 2023 11:26:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 53ECC280001 for ; Mon, 3 Jul 2023 11:26:07 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 199BB1208E9 for ; Mon, 3 Jul 2023 15:26:07 +0000 (UTC) X-FDA: 80970676374.21.F835975 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf30.hostedemail.com (Postfix) with ESMTP id 9718D8000D for ; Mon, 3 Jul 2023 15:26:04 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FkKdX50S; spf=pass (imf30.hostedemail.com: domain of dave.hansen@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688397964; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IplzqWKl9S16eMXUW2iIdkT0fEvefIq+ZB0WA3l/mY8=; b=RBabttIrpImX51vT0CkjkPnfRRU9Zcbf45EFb2bBQ/OJzDkQeOCdQrAYZ/XBXuIskA3HBb wjoHZok8BnLMRg3N+J91VDcqXXrJM1LxmlHnWdVgXkmMQ0CRhEejgNAsZIQFioKvKT1Ux+ FaZ9vfhP7gFoX7actXjmBm4vbq9/vqI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688397964; a=rsa-sha256; cv=none; b=S/EaIwYAByhMk9gFrv3dOO3rgPy/Mv24s8Xg7MszBt9haEFQ10Cc5qWW17P7EDIA1krnU3 WZ6FKHaW/XAPW4FX3uaagIW+B8P0EgwBg5wRg8h5gzGICvVXdUOEStdbjAPnysluS1RvaK lC+p6Vnhfcw9xy6WME7doEOLTOSkp8c= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FkKdX50S; spf=pass (imf30.hostedemail.com: domain of dave.hansen@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688397964; x=1719933964; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=kj6e5XFo1qU/JWRexT6QL5AcQhLeMKD04nwS981R7Mk=; b=FkKdX50StGiEku2b62GqWXpvjWRChndbx21K02HbvAu8iairPPqOmmzW UexKs9TdZ/lnc3fnIzz0XCvFGeeHq5jgE1SbbBXXSov+dBGoOq8b+VINI dOq2JchKKsU81HeAOl+cpWhaOolTJHQufVQimP7bea4omcnx01GBOMfJl 5sKYJRX6yc1oQKG1qcneOhLxSYRCzbvEFU5ljkUSYzRT9BXFMIRcbGBeF Vw2O2INmeQZ30Q35BUAsbUERZ60UlGKX5z2y/wfTT7zcMrSJPVoxabR/q ed/sfzCECDb3GAOtulRUjT9cbliVG1jjHNkoYcjTK2YxpKEzf7jib7Ar0 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10760"; a="428950438" X-IronPort-AV: E=Sophos;i="6.01,178,1684825200"; d="scan'208";a="428950438" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jul 2023 08:26:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10760"; a="668786413" X-IronPort-AV: E=Sophos;i="6.01,178,1684825200"; d="scan'208";a="668786413" Received: from lbates-mobl.amr.corp.intel.com (HELO [10.212.242.115]) ([10.212.242.115]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jul 2023 08:26:01 -0700 Message-ID: <8c080959-e1a5-6768-934d-33eca8e04086@intel.com> Date: Mon, 3 Jul 2023 08:26:00 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH v12 07/22] x86/virt/tdx: Add skeleton to enable TDX on demand Content-Language: en-US To: Peter Zijlstra Cc: Sean Christopherson , Isaku Yamahata , Kai Huang , "kvm@vger.kernel.org" , Ashok Raj , Tony Luck , "david@redhat.com" , "bagasdotme@gmail.com" , "ak@linux.intel.com" , Rafael J Wysocki , "kirill.shutemov@linux.intel.com" , Reinette Chatre , "pbonzini@redhat.com" , "mingo@redhat.com" , "tglx@linutronix.de" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Isaku Yamahata , "nik.borisov@suse.com" , "hpa@zytor.com" , Sagi Shahar , "imammedo@redhat.com" , "bp@alien8.de" , Chao Gao , Len Brown , "sathyanarayanan.kuppuswamy@linux.intel.com" , Ying Huang , Dan J Williams , "x86@kernel.org" References: <104d324cd68b12e14722ee5d85a660cccccd8892.1687784645.git.kai.huang@intel.com> <20230628131717.GE2438817@hirez.programming.kicks-ass.net> <0c9639db604a0670eeae5343d456e43d06b35d39.camel@intel.com> <20230630092615.GD2533791@hirez.programming.kicks-ass.net> <2659d6eef84f008635ba300f4712501ac88cef2c.camel@intel.com> <20230630183020.GA4253@hirez.programming.kicks-ass.net> <20230630190514.GH3436214@ls.amr.corp.intel.com> <20230703104942.GG4253@hirez.programming.kicks-ass.net> <20230703150330.GA83892@hirez.programming.kicks-ass.net> From: Dave Hansen In-Reply-To: <20230703150330.GA83892@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 9718D8000D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jweizb8pbjr6g3fmrb6uuk75t8rcyg9b X-HE-Tag: 1688397964-686853 X-HE-Meta: U2FsdGVkX1/oJzCt0u0Yhxw2jHL4js0VBySif+g72M0EvcyxKfYb3MonRuPg7WEqS2pxmw66ExFF6rmIvGPeH9+bS6sp7b/8hvHOqjSgvB14WOg9ooTOxjmoGWnbaQxyyOFu5h4vDXfjozAqqJOfe/Vn9AIi5kdSoGqbW+BA0SaJW1M2mxr6NCJQQm0wZu9W1vbv6irT9CDMZoaOW64ZHOpa65m+ywQX7Q+DWidebDdxUGeoTkF59Bdqxcd0TxPKSZO2CaeqpJbzo/+cykMPWcD3Wru7EbK/W9i17G53XecYZYTHOqIQIXrxWpLJTwYz2P0ZbGBaIARqqSHrWnZyCLaTbcg48x1whR1TjJJ5ZJb82ma8jfxZ8CEuyRSKazLhb48g+2R8CVGaENWzhizDYHPMwdEjyLTzFhJkxo8g7yxpEkIgrG48Y9s5FCxPkMjjQdDbtLlA3gixsu3zaaeVI3X7zzdLxFEqyfYwXBFp3zKs40VPeQZDaIymG9d3TqJmGzfYiH4RKbq3J/LjUyHda9ABxVugbc5Z9WRILF3b9pixYrGVLsm7o1Kw7U8voLVR6UPLzwVebgydmeZoz7SX+P/jXWYmuwJkeUecu3UFZG8JKjYLr3Y2nwmtrAfhUB9PucRq8AZ6An1HZASlJBp1mzWzo/jARWRwOvPKxXPHQHAyfL1nIA99JGy6Pbabf5Wxnzrm+NQUmGMTd6xzfZhaD1qMSwipuUfGtrCznOkIRRvIsuMPPNLLV3x0Ks44sAYC/F3paIK9sXOB1aEqieZz1GiY7nMe7DUcqH6nPM7St//OwiCeZUJBer3c2UCApmnVnNQjoazNnCyBCodqQRNU53fe6a6JHEHP25XCYuj1cPuS5YkjifGPgpqyM91yBAQvktKzvfUHWLdXlDIc01rxhOtKHWCAquOTDeLzHgcHgcKHMdJhJPp0a9T9Oo5w9KqDA2vu672ykf3LCjt/suQ Yqgmg9n6 av+btvuwfwukF9jVwzrS4jskM18tdq+vghM6C/u88F99xysbChP3l0prTPWex4t8LoVUpvpDTXGR7pPFkYPFK+ObXtxdSfJZEvE7Dya/lc6JDV6VEqG0Zlg4jKUtZaTPihjze10JRUp7Ev7jlS+FMkyTsB5/M1Fbo3OAgWPul806VIzgUXI0UFN7PRhRm8Pa/A8rDMM8fwV+Ls9uZHmi6VSUtddrEmGz7nR/V8mMpMTfK0Tj6ZMUgvBnn3rGRdVY+Q6Kioqji0UELje8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 7/3/23 08:03, Peter Zijlstra wrote: > On Mon, Jul 03, 2023 at 07:40:55AM -0700, Dave Hansen wrote: >> On 7/3/23 03:49, Peter Zijlstra wrote: >>>> There are also latency and noisy neighbor concerns, e.g. we *really* don't want >>>> to end up in a situation where creating a TDX guest for a customer can observe >>>> arbitrary latency *and* potentially be disruptive to VMs already running on the >>>> host. >>> Well, that's a quality of implementation issue with the whole TDX >>> crapola. Sounds like we want to impose latency constraints on the >>> various TDX calls. Allowing it to consume arbitrary amounts of CPU time >>> is unacceptable in any case. >> >> For what it's worth, everybody knew that calling into the TDX module was >> going to be a black hole and that consuming large amounts of CPU at >> random times would drive people bat guano crazy. >> >> The TDX Module ABI spec does have "Leaf Function Latency" warnings for >> some of the module calls. But, it's basically a binary thing. A call >> is either normal or "longer than most". >> >> The majority of the "longer than most" cases are for initialization. >> The _most_ obscene runtime ones are chunked up and can return partial >> progress to limit latency spikes. But I don't think folks tried as hard >> on the initialization calls since they're only called once which >> actually seems pretty reasonable to me. >> >> Maybe we need three classes of "Leaf Function Latency": >> 1. Sane >> 2. "Longer than most" >> 3. Better turn the NMI watchdog off before calling this. :) >> >> Would that help? > > I'm thikning we want something along the lines of the Xen preemptible > hypercalls, except less crazy. Where the caller does: > > for (;;) { > ret = tdcall(fn, args); > if (ret == -EAGAIN) { > cond_resched(); > continue; > } > break; > } > > And then the TDX black box provides a guarantee that any one tdcall (or > seamcall or whatever) never takes more than X ns (possibly even > configurable) and we get to raise a bug report if we can prove it > actually takes longer. It's _supposed_ to be doing something kinda like that. For instance, in the places that need locking, the TDX module essentially does: if (!trylock(&lock)) return -EBUSY; which is a heck of a lot better than spinning in the TDX module. Those module locks are also almost always for things that *also* have some kind of concurrency control in Linux too. *But*, there are also the really nasty calls that *do* take forever. It would be great to have a list of them or, heck, even *enumeration* of which ones can take forever so we don't need to maintain a table.