From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D486DEB64DA for ; Fri, 30 Jun 2023 21:25:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 246AC8E0052; Fri, 30 Jun 2023 17:25:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F7648E000F; Fri, 30 Jun 2023 17:25:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BF5D8E0052; Fri, 30 Jun 2023 17:25:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EF8CF8E000F for ; Fri, 30 Jun 2023 17:25:00 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C4D7F160158 for ; Fri, 30 Jun 2023 21:25:00 +0000 (UTC) X-FDA: 80960694360.15.041174F Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf02.hostedemail.com (Postfix) with ESMTP id F03358000E for ; Fri, 30 Jun 2023 21:24:58 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=C98VfhGL; spf=pass (imf02.hostedemail.com: domain of 3KUifZAYKCGISEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3KUifZAYKCGISEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688160299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9OuqO+GprsYwzlZrAMxO302oe0WdyZWBaVRgt/0KOVc=; b=PCk/wxFAJAolSjAh0MDK3w5+r2PQoH/+NuGE5SgN3yT40oWb5nXfY1+xKikXapq1/jLmxY GXn1xEl90OUu7rIHlnizQozuewGVSJzu8juisuBtCFfp82bZLN5GpX/50G+92v9TduRE1m sxKQohOV5yNPIMmE+9VIs1rHO2TexMQ= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=C98VfhGL; spf=pass (imf02.hostedemail.com: domain of 3KUifZAYKCGISEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3KUifZAYKCGISEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688160299; a=rsa-sha256; cv=none; b=ALZx6JZ1+s16emrhCy/L35ML3zE97O4Rotm4d4nClapr4cg3MybjfZDSJXWnPrPqsbBuZy X7D9+YXjQR8O3wfVXcL8xX3/6uktWS5RiviPYgKFymvH4DxKakJM6npvRw05QQvafClf0r /+p4aLqbiDQl68pnz37j9HPk4T7jc+A= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-bfebb1beeccso2197815276.2 for ; Fri, 30 Jun 2023 14:24:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688160298; x=1690752298; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9OuqO+GprsYwzlZrAMxO302oe0WdyZWBaVRgt/0KOVc=; b=C98VfhGLNA5xfbBYxeu79VpxIy9gRA6nupPJlAfe0FuPEfw/4GPR1LutGEmbPkU+D6 LCsyyk7D1OMMbPdZuqJ9pRQyj5bW6ckdCA5JJQ6l1LqK6+B3WHfFhFGW1zF4WhSDt95B 76niGAHa2KUaUlGFoU46LfHpWIdnS1uFr74lXzrCCEehASUT/Zkr0bpM7jQ7BSmVFXYz a2YxzWzm0vujdecLtLgB25IIGiMTe9YiFVK4SJ/eBhMlwZW3PKId1fdZAWpVYfILXO43 OkuMAJ9vMYtxDqrigZ3s9PiQR/RD0SJ2XoRA2Xh0FFvQ/TxS2T5r8RXuCsiv02v77dUv kuPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688160298; x=1690752298; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9OuqO+GprsYwzlZrAMxO302oe0WdyZWBaVRgt/0KOVc=; b=ZsjrIjVI3tgDtM3slf6qq7c8cheOpp/KovhZD4RHQ18QQjiet7DG2Qz0wUL6730VKj qLL5bLxKt+US1nPh0HSwecVYDpeSX8IQo8sCi1AAHCte3Nppd0OANT730ms9zFcrZoIZ MCCaYNeWxtqkzCvLJpKsKXeMCgh9MBOaEGnx/8GGyFyqCV2YpE/Ow9M0cnUJ908ZChcq C+86OV1HFbU6/yTL0n3KA2PlAwBrTidlgls7DTR0Ja5UdARFivuWYDlFgDj6yn/5dD7R 9Gs9ODRFKxaJaBDyUBUBhuR8X/bq9iYV2yHj05sIvJ3Wwmuyrs+ITVUdStodoElWULGo GJEw== X-Gm-Message-State: ABy/qLZwSskfnSS19IGoQIgvHRjNCkZ4Ajtsm286CSXVouShzgeR1Qij 4kCRouM/cdduEMqX+SdqlgDlQ9QxCj4= X-Google-Smtp-Source: APBJJlGImEzp5+7kOBgHaxvCjkxiHX5w7NN8ScEyxNJQ7wgzjAyS5wsh+UoGUOwuj07AgYjSnXiJcdrLhIY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:4183:0:b0:c1e:f91c:2691 with SMTP id o125-20020a254183000000b00c1ef91c2691mr37547yba.10.1688160297945; Fri, 30 Jun 2023 14:24:57 -0700 (PDT) Date: Fri, 30 Jun 2023 14:24:56 -0700 In-Reply-To: <20230630190514.GH3436214@ls.amr.corp.intel.com> Mime-Version: 1.0 References: <104d324cd68b12e14722ee5d85a660cccccd8892.1687784645.git.kai.huang@intel.com> <20230628131717.GE2438817@hirez.programming.kicks-ass.net> <0c9639db604a0670eeae5343d456e43d06b35d39.camel@intel.com> <20230630092615.GD2533791@hirez.programming.kicks-ass.net> <2659d6eef84f008635ba300f4712501ac88cef2c.camel@intel.com> <20230630183020.GA4253@hirez.programming.kicks-ass.net> <20230630190514.GH3436214@ls.amr.corp.intel.com> Message-ID: Subject: Re: [PATCH v12 07/22] x86/virt/tdx: Add skeleton to enable TDX on demand From: Sean Christopherson To: Isaku Yamahata Cc: Peter Zijlstra , Kai Huang , "kvm@vger.kernel.org" , Ashok Raj , Tony Luck , "david@redhat.com" , "bagasdotme@gmail.com" , Dave Hansen , "ak@linux.intel.com" , Rafael J Wysocki , "kirill.shutemov@linux.intel.com" , Reinette Chatre , "pbonzini@redhat.com" , "mingo@redhat.com" , "tglx@linutronix.de" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Isaku Yamahata , "nik.borisov@suse.com" , "hpa@zytor.com" , Sagi Shahar , "imammedo@redhat.com" , "bp@alien8.de" , Chao Gao , Len Brown , "sathyanarayanan.kuppuswamy@linux.intel.com" , Ying Huang , Dan J Williams , "x86@kernel.org" Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: F03358000E X-Rspam-User: X-Stat-Signature: zaxkygmrwk9syuzjhtntoutw147m15mx X-Rspamd-Server: rspam01 X-HE-Tag: 1688160298-329861 X-HE-Meta: U2FsdGVkX1+jtZP0XwnIkkTSfPQUzlPWipYXMeY5fRIZ4Akl5Sm+sJKv5TKXXblnhhHl6A9b8AcJd8jWA/NU74q4f1u0b/fN6Jkuqe3V9ac1Aw+REOG6f0zhKVbsWKfZOjyl7T8hgjWCxFdyVA3xjpLoMJKyQBWtPEennPI0G7JS9VFoVvgQB5NeT3oein+BG3gFhNU/f40u0/wops5nF92wsRyLCXyhpuX7orB0c22TEIEvFwIqLEUmJlSEn7x/KNWs5fvAm1Aej1QjiS/Pgxxog93S752cyGXt2x9r+YXHCxXN5uqRgDelrFaCTk2hD4njWHK8oE0UuuRCnPIQasocMiO0lyt5ZbivqIvpTmgSpYnRjpM/PiniwPNuLUg0iYHlKzlk0h4CPQiusK11R03UwsnqcBZk6dj9umWMA1SOXRAdceLeMpiU9ZPKZWRSK77j5sdnm+GzTRmSroZ+Wx+0cBaNwFmEkuWkqiOAbYc2ZgVtqDSN6I/SEj2v+P6k1fYXSe3sETi7i+NOtokdkOPL08kv+ReBjOzJAcAJFjzqv4AvRs7GEKzRFdf7Vqo9yrZ+bU1YTiXhsIVIUz42TA6uhU48fk0f0/Ryz+os7MhJM/GfSmihAM2AlJSZVg8SIawCWFepxV96ztvpXehmbW4sEJ6Fp4dJwKoc2mtAz5d8qQlDYZ0PA7NHwjv0GBo5jzTDsMuCI+6w0gK5ipCLeLTQk/2FzvBT+05v/yu9/GmJP9Tq/k5B6k5acWIBKq8sdfoZud/uWNY83IaDL2r5suVzA9Tf4BMjwe3+SQ6m4yAG5wQgXDHRGrzBGvg21HaF6iAa2IYPE3+xiQ1PZoKyD/8RzTcVuwDxqLYrIZais/qbUvL41adND0fYWKNQNtEAn0+EK9EtdlyGEju29QXKIEOE67wwpmUwB3G6ivHXaS5/jL/V8su0bEbA6RBlURdetw56mwWtuYyHV5gZWow nT79F6oX iwKcuy4JJO8iMDOpLfSNbLktIFnJr+1YdVZeCwOMJw48aEsCwYP+kABXWuaPXB3AmgbwDNtaS1UuwELQIMF+YSYsCWVM5YLV5gTWR4rWOogncfCrGJMH2P1U/RxBd5ldjWBtMHLysIn7QF0OPzHmiatpG/nnMCeA/rct8FNvSd4sxBBH9Cvs5FwcqSSi9tntfEw8MdYiYnmcoiseSrAxkeQEBVwKHJUBWcv1nVFgt09d3c75C1cgE6TVVigZqw04jTP4NhcyNr9nHu6+Yt/WYbY06ivZ5czswbb4F0vE+nK+Qj+65Dx/Zf4TS3KlagMSZY3Saxem/woQUdIHdRLct7/oLOUkw+QYgB+JZ/dfysp5R1GZxAdeRvlWtfhwUqBLlCdMitcHwqzm4EcxCVUJJNeM8ANGTAilqFNBi5VBhlcqfyMO1r2CCcx5/qhMtLg+JnB/+7uIYdcOBZVCO464J6ccaKIZmw9nUiXx5+5PzDV+FK5HSXP+rifEj4xqmqtISCuiNDwZ6JZoMwZemIouE2adD+aJoUE+cNFV1hFwJ0O+5MQZTrtZpjT+WiW7LRRs9LeasXnUWVDVJD5O4BZjElsUkOhL8NlH+dbGe1UgT00AjA3DuGhTEq9YdRFuSlh65aWcjMzs1vMKrUUya2ce7gSVUIV8a1FwbqFqkxRaxB11IpfW/MT4uv117zzudJsXJsohijtO6UfIrHM8OKudLIS3/RPslPdzB63WCLi0JTUrYdiRXLi6cumnAQek8KmwH3+MT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 30, 2023, Isaku Yamahata wrote: > On Fri, Jun 30, 2023 at 08:30:20PM +0200, > Peter Zijlstra wrote: > > > On Fri, Jun 30, 2023 at 09:55:32AM +0000, Huang, Kai wrote: > > > On Fri, 2023-06-30 at 11:26 +0200, Peter Zijlstra wrote: > > > > On Thu, Jun 29, 2023 at 12:10:00AM +0000, Huang, Kai wrote: > > > > > On Wed, 2023-06-28 at 15:17 +0200, Peter Zijlstra wrote: > > > > > > On Tue, Jun 27, 2023 at 02:12:37AM +1200, Kai Huang wrote: > > > > > > > +EXPORT_SYMBOL_GPL(tdx_cpu_enable); > > > > > > > > > > > > I can't find a single caller of this.. why is this exported? > > > > > > > > > > It's for KVM TDX patch to use, which isn't in this series. > > > > > > > > > > I'll remove the export. KVM TDX series can export it. > > > > > > > > Fair enough; where will the KVM TDX series call this? Earlier there was > > > > talk about doing it at kvm module load time -- but I objected (and still > > > > do object) to that. > > > > > > > > What's the current plan? > > > > > > > > > > The direction is still doing it during module load (not my series anyway). But > > > this can be a separate discussion with KVM maintainers involved. > > > > They all on Cc afaict. > > > > > I understand you have concern that you don't want to have the memory & cpu time > > > wasted on enabling TDX by default. For that we can have a kernel command line > > > to disable TDX once for all (we can even make it default). > > > > That's insane, I don't want to totally disable it. I want it done at > > guard creation. Do the whole TDX setup the moment you actually create a > > TDX guast. > > > > Totally killing TDX is stupid, I dunno about that, *totally* killing TDX would make my life a lot simpler ;-) > > just about as stupid as doing it on module load (which equates to always > > doing it). > > > > > Also, KVM will have a module parameter 'enable_tdx'. I am hoping this could > > > reduce your concern too. > > > > I don't get this obsession with doing at module load time :/ Waiting until userspace attempts to create the first TDX guest adds complexity and limits what KVM can do to harden itself. Currently, all feature support in KVM is effectively frozen at module load. E.g. most of the setup code is contained in __init functions, many module-scoped variables are effectively RO after init (though they can't be marked as such until we smush kvm-intel.ko and kvm-amd.ko into kvm.ko, which is tentatively the long-term plan). All of those patterns would get tossed aside if KVM waits until userspace attempts to create the first guest. The userspace experience would also be poor, as KVM can't know whether or TDX is actually supported until the TDX module is fully loaded and configured. KVM waits until VM creation to enable VMX, but that's pure enabling and more or less guaranteed to succeed, e.g. will succeed barring hardware failures, software bugs, or *severe* memory pressure. There are also latency and noisy neighbor concerns, e.g. we *really* don't want to end up in a situation where creating a TDX guest for a customer can observe arbitrary latency *and* potentially be disruptive to VMs already running on the host. Userspace can workaround the second and third issues by spawning a dummy TDX guest as early as possible, but that adds complexity to userspace, especially if there's any desire for it to be race free, e.g. with respect to reporting system capabilities to the control plan. On the flip side, limited hardware availability (unless Intel has changed its tune) and the amount of enabling that's required in BIOS and whatnot makes it highly unlikely that random Linux users are going to unknowingly boot with TDX enabled. That said, if this is a sticking point, let's just make enable_tdx off by default, i.e. force userspace to opt-in. Deployments that *know* they may want to schedule TDX VMs on the host can simply force the module param. And for everyone else, since KVM is typically configured as a module by distros, KVM can be unloaded and reload if the user realizes they want TDX well after the system is up and running. > The KVM maintainers prefer the initialization on kvm_intel.ko loading time. [1] You can say "Sean", I'm not the bogeyman :-) > I can change enable_tdx parameter for kvm_intel.ko instead of boolean. > Something like > > enable_tdx > ondemand: on-demand initialization when creating the first TDX guest > onload: initialize TDX module when loading kvm_intel.ko No, that's the most complex path and makes no one happy. > disable: disable TDX support