From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B6C0EB64DD for ; Wed, 5 Jul 2023 07:14:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 138AD6B0071; Wed, 5 Jul 2023 03:14:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E8816B0072; Wed, 5 Jul 2023 03:14:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F19788D0001; Wed, 5 Jul 2023 03:14:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E0D326B0071 for ; Wed, 5 Jul 2023 03:14:56 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A9FE240257 for ; Wed, 5 Jul 2023 07:14:56 +0000 (UTC) X-FDA: 80976696192.14.095EA98 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id 4ABBA40013 for ; Wed, 5 Jul 2023 07:14:53 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kRH0KTcu; spf=none (imf27.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688541294; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EyW9XZN4w1C6P0MPRsEcqqMNHBYbEDNEAhyPTiwy2lQ=; b=dA7LJRlAsweoy1gDmdD0+I4vH2ytw32h4M1aTH5puY3hpDw3qQ5ZacR6zYYMk4wMCv2vqJ 8b66P1j01YqFR6DillxWd8Quv64TPa7WGG7myM7rBgyFMqxe+rbgBB3o/dWoVyGi86m5ui f+Z/IFUqgUmBu4Iy+rFYVdoyKOQSQpc= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kRH0KTcu; spf=none (imf27.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688541294; a=rsa-sha256; cv=none; b=Hwu9XHh1h9WKS6SWrgLueJSgsGtKys560krc+sRbw3/HHoB3yxp2CMI0Y6pq3BxQQ2jmQ+ yqUwgBTGKCHweT3aYmh3nPb4Cv+m1C7ARRQh8+zkfPY8jnKJqEvfESD23PNgpqR5/dwA/T pcM3nzE8+2kuNDUX4/LuuLacauDyMu8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=EyW9XZN4w1C6P0MPRsEcqqMNHBYbEDNEAhyPTiwy2lQ=; b=kRH0KTcuJPnGqXJJIpV8CAvcvR 7sGU5dzETN0Kpb4qVrjCA8NEuHmhNfAkW3SI+JGJUn0H99e8EWj2U6mK8510cx6IUX+LmorMiZhAE x1iw/5XhgpaGnDZmWQCS3fkR646mAeHrW4GBLVjWLebCqaL1/Tylp5XyIuZunabQ28Iueb39afHvN EFAJmbsa4112IrSxNgAyybSaVkGk2+Jt+sa3l9/J17J2PIBXsdb73vZdOhfbeHzEol8AO/eu6TXKW oG9h1h6bguhZNSP0d9wOD/eRAg4fDGJlw2tx9ovAR9BGUTSCADeVBDBK1o9x0ay2icc6GvIEV/ECH L2UZ+mdw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qGwim-009qlL-SN; Wed, 05 Jul 2023 07:14:41 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 236DA300023; Wed, 5 Jul 2023 09:14:39 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 066E82028F056; Wed, 5 Jul 2023 09:14:39 +0200 (CEST) Date: Wed, 5 Jul 2023 09:14:38 +0200 From: Peter Zijlstra To: "kirill.shutemov@linux.intel.com" Cc: Dave Hansen , Sean Christopherson , Isaku Yamahata , Kai Huang , "kvm@vger.kernel.org" , Ashok Raj , Tony Luck , "david@redhat.com" , "bagasdotme@gmail.com" , "ak@linux.intel.com" , Rafael J Wysocki , Reinette Chatre , "pbonzini@redhat.com" , "mingo@redhat.com" , "tglx@linutronix.de" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Isaku Yamahata , "nik.borisov@suse.com" , "hpa@zytor.com" , Sagi Shahar , "imammedo@redhat.com" , "bp@alien8.de" , Chao Gao , Len Brown , "sathyanarayanan.kuppuswamy@linux.intel.com" , Ying Huang , Dan J Williams , "x86@kernel.org" Subject: Re: [PATCH v12 07/22] x86/virt/tdx: Add skeleton to enable TDX on demand Message-ID: <20230705071438.GC462772@hirez.programming.kicks-ass.net> References: <0c9639db604a0670eeae5343d456e43d06b35d39.camel@intel.com> <20230630092615.GD2533791@hirez.programming.kicks-ass.net> <2659d6eef84f008635ba300f4712501ac88cef2c.camel@intel.com> <20230630183020.GA4253@hirez.programming.kicks-ass.net> <20230630190514.GH3436214@ls.amr.corp.intel.com> <20230703104942.GG4253@hirez.programming.kicks-ass.net> <20230703150330.GA83892@hirez.programming.kicks-ass.net> <20230703175556.nn5xozz7dzxjocqm@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230703175556.nn5xozz7dzxjocqm@box.shutemov.name> X-Rspamd-Queue-Id: 4ABBA40013 X-Rspam-User: X-Stat-Signature: hbswanp5ptpkact16xj1u66gahq4ed48 X-Rspamd-Server: rspam01 X-HE-Tag: 1688541293-683597 X-HE-Meta: U2FsdGVkX18bTGzLLEDtgZ+gUNIuMgQf7l6KeMl9hglujY3aijPsTb+yvIMF8qmm1OkiZM+sfG2JqSnXpyF3HvVTgzHxP723QvU68w5K0tOBhWRvxsEbmj9IB5uPSq5SDp1SW0mmn7O+FD8roD5ILg7T9rMIRJfKvGAIZVpew4gxUVNL47eUY9U7G2sYN+d8iBC4mHSK0kua130ZTjDyx87hSIsWcDsbfLPHEjjHb9/lQ2PkoxRl2o0jskHzNEw0aCZ5yG524+i1J3yKT6h3X7ITZDGb3cRC6YUrrydytqY6bbyA6jjC5mFadDLAiMdNeEC8KVGxetXKfeej8cslxLonRh1sNr9GHM7rSe3LS+g4XcXL+El09UDf2kITzIQzrvEUtsVj9vF6z1rtFLa+op0CZhkyn0lBauSupLVNzFVQSL3PuAXW//0PszmVzsz5701glNtAkQu3Z7sCTIUD3hDaLtqxAo+yO/p1LeOnaAgseEDuYFynPlTyFkDPlbKcmJF2Atld2xfgBfImlAPL+kMYmXgd7eM8NMEaJzyHgO5DvKtupcjvQaOxTO/tt/FUEuf+2dgsRYiIqyqd7yow5U4VWea4axYZFFudpLvPd4DLsJBxaGISFxXVC5bHbB2vhNfOJZMGGQAc56jsNGBIMG7NLU0De2xo7XGFkN+zoY6EVj5fFGoN06RC7HvmcbQcQiA3QGKd8Gx/INpWTgd50nn1rbr6au7nkn17i0m0drb3Afdu7Dn7XB3gkP/YOAaZ+vALeFIUf3T0z0ega6JSeTUg/hV2N0R+8x0d8oyfYN4UiqrHbBogB3iQMO8zuKzcZY5DHdgQkERIqqnYEzcl18h/F8Dg4VLW2AGGNIVMY11zfdtouC7coYW4Bd/gM+1xjMXEiSSFZqxLlAL3r4lBXUJKgnB76yjjnhLy/uSSV2W7UdMj76+SSTGQxsm1AQFuEdSHjKMw1jbpnIbtJCC 9z2pYbNi mn3bzGNrdO3xk9eyC960+ISETfJVF0aI7HCA57QOLQVh9rq5gLzaAvHu0gqJkALeXyoUYACcthBeFuh4cb2d51RNLOXI++gomuSttiD3SRVJyHY2PJAdz3NY4l/h5XG/t67FX1MOeAY08a1bjTr1GrXc1TrSLrKaFe/94xwlDHJudhMRI2MQvlTmYnuccbKB0WIRWEIMXx+qaB/468wf0VoKrvvmGrjwdkd7tGhUW9hdkULaRW868ic86+1Y5SYiEIrRpUqpi47Bb9iziGa2p5PQhew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 03, 2023 at 08:55:56PM +0300, kirill.shutemov@linux.intel.com wrote: > On Mon, Jul 03, 2023 at 05:03:30PM +0200, Peter Zijlstra wrote: > > On Mon, Jul 03, 2023 at 07:40:55AM -0700, Dave Hansen wrote: > > > On 7/3/23 03:49, Peter Zijlstra wrote: > > > >> There are also latency and noisy neighbor concerns, e.g. we *really* don't want > > > >> to end up in a situation where creating a TDX guest for a customer can observe > > > >> arbitrary latency *and* potentially be disruptive to VMs already running on the > > > >> host. > > > > Well, that's a quality of implementation issue with the whole TDX > > > > crapola. Sounds like we want to impose latency constraints on the > > > > various TDX calls. Allowing it to consume arbitrary amounts of CPU time > > > > is unacceptable in any case. > > > > > > For what it's worth, everybody knew that calling into the TDX module was > > > going to be a black hole and that consuming large amounts of CPU at > > > random times would drive people bat guano crazy. > > > > > > The TDX Module ABI spec does have "Leaf Function Latency" warnings for > > > some of the module calls. But, it's basically a binary thing. A call > > > is either normal or "longer than most". > > > > > > The majority of the "longer than most" cases are for initialization. > > > The _most_ obscene runtime ones are chunked up and can return partial > > > progress to limit latency spikes. But I don't think folks tried as hard > > > on the initialization calls since they're only called once which > > > actually seems pretty reasonable to me. > > > > > > Maybe we need three classes of "Leaf Function Latency": > > > 1. Sane > > > 2. "Longer than most" > > > 3. Better turn the NMI watchdog off before calling this. :) > > > > > > Would that help? > > > > I'm thikning we want something along the lines of the Xen preemptible > > hypercalls, except less crazy. Where the caller does: > > > > for (;;) { > > ret = tdcall(fn, args); > > if (ret == -EAGAIN) { > > cond_resched(); > > continue; > > } > > break; > > } > > > > And then the TDX black box provides a guarantee that any one tdcall (or > > seamcall or whatever) never takes more than X ns (possibly even > > configurable) and we get to raise a bug report if we can prove it > > actually takes longer. > > TDG.VP.VMCALL TDCALL can take arbitrary amount of time as it handles over > control to the host/VMM. > > But I'm not quite follow how it is different from the host stopping > scheduling vCPU on a random instruction. It can happen at any point and > TDCALL is not special from this PoV. A guest will exit on timer/interrupt and then the host can reschedule; AFAIU this doesn't actually happen with these TDX calls, if control is in that SEAM thing, it stays there until it's done.