From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA196C4332F for ; Wed, 1 Nov 2023 07:30:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B1C88D0033; Wed, 1 Nov 2023 03:30:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43B308D0001; Wed, 1 Nov 2023 03:30:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B3C18D0033; Wed, 1 Nov 2023 03:30:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 15D598D0001 for ; Wed, 1 Nov 2023 03:30:47 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DD9DAA04AB for ; Wed, 1 Nov 2023 07:30:46 +0000 (UTC) X-FDA: 81408563292.18.E59AAC4 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by imf28.hostedemail.com (Postfix) with ESMTP id 31E73C001A for ; Wed, 1 Nov 2023 07:30:43 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="er2QJ/d7"; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf28.hostedemail.com: domain of binbin.wu@linux.intel.com has no SPF policy when checking 134.134.136.100) smtp.mailfrom=binbin.wu@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698823844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gEWzjOHhHZl9KRaYAjNvmTN62/2XLTjDyG++EK1zqCo=; b=ZA/2g156c2i0KtgZkkku1FBLR6PxDSrYb6NYHBug/AbJGWKKQSJuzzKuTW1JvJpuNxTE8g iGaGm196xTU+MsY9zazTaC3lqbrG3ZwOjO5uJei9wnbyHxRliQX+7oEbm4fbfoavQimaKa npjXTUWv5N/I3+C5Qy9YrcPUQL3Gr+4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="er2QJ/d7"; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf28.hostedemail.com: domain of binbin.wu@linux.intel.com has no SPF policy when checking 134.134.136.100) smtp.mailfrom=binbin.wu@linux.intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698823844; a=rsa-sha256; cv=none; b=CRuysiZg9W7qBMYdbwuW3we1b7tM50RyeAd0G9zW8hghEsy9Yw7gpZk6s8NdfBQ7Fh0EDo 5oVtQ3u45ICQZA3z7R3GT2utExx72Raz0NZHUi0Ku7uhheT1brx39nQQMAsv51wvzU4C+7 mweBYV+HoCJEHvViBGU8m3/+OUGWKk0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698823844; x=1730359844; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=38bUYl9zGRcXqgO3kvm3wpVN8MNciUwqMVCV5OnvF8E=; b=er2QJ/d7AotNpisKBRo5fdDhuaQLxKd1yGNkSg+gP1yQR+mrsy2NuNz7 gV9P7+qbsPspIWDOD8+waGk83ZjLSFGYmCBMQZZ1Tdkt4M4wYZPUdriwB Wi6h4wFvCrpcI755Dstzngyyf/yLHahbUNf1jyso9UlUg//MqKg8pduVO q+5KWV7VpBeHg/m+lfQ/ka35MA/OwtHzD1U9C7taZrnkHiKL8ahda+RQt bDDYhiqcaReYgd0QcMOq5Pswg4Ta4Z0GieukUpunrfN20RRZrqpr+Hsaj /xel4RHB3IlxguCFUUGT/vVL+5kjiJuC/T4ED5xqOsQISvBLsgV0EgcGa A==; X-IronPort-AV: E=McAfee;i="6600,9927,10880"; a="454923854" X-IronPort-AV: E=Sophos;i="6.03,267,1694761200"; d="scan'208";a="454923854" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2023 00:30:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10880"; a="884461159" X-IronPort-AV: E=Sophos;i="6.03,267,1694761200"; d="scan'208";a="884461159" Received: from binbinwu-mobl.ccr.corp.intel.com (HELO [10.238.10.126]) ([10.238.10.126]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2023 00:30:30 -0700 Message-ID: Date: Wed, 1 Nov 2023 15:30:12 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v13 09/35] KVM: Add KVM_EXIT_MEMORY_FAULT exit to report faults to userspace To: Sean Christopherson Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Viro , Christian Brauner , "Matthew Wilcox (Oracle)" , Andrew Morton , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Xu Yilun , Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , David Matlack , Yu Zhang , Isaku Yamahata , =?UTF-8?B?TWlja2HDq2wgU2FsYcO8?= =?UTF-8?Q?n?= , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" References: <20231027182217.3615211-1-seanjc@google.com> <20231027182217.3615211-10-seanjc@google.com> From: Binbin Wu In-Reply-To: <20231027182217.3615211-10-seanjc@google.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: fydmunqikiioribrjknykmpotodexn8c X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 31E73C001A X-HE-Tag: 1698823843-434374 X-HE-Meta: U2FsdGVkX1+uUNmy2uPagP7BWrPtYMN3ITWiZahjxSoC8BkxZqxbq7kKOvPdj217t3ThwfLmwAYRA++EZD3APzHjao9YCVjC0OqTtGWJVkOr+WtjLr/EksHlKcQy4f4KOGUDkQ98CSPp8zsMrh/25+4TPTc9S00g8ZaxEEYRgUxvhCSqS9p+3I0E2BWh/Ep9fAo4SmwbWsRuQ8o6lQ7Qktdg+sX67lnW31kNxMAHP2J2EJUmkmCYRQSO2xmp6IlpwbxfPCP7iYrBnFTcBxURebZ5kNbvO7tQobZrW1C78ngWJSRZAMUsFFobEyG3/lWg6PnyDrRzo23+VCYzoLJN8Z3CRSVidA78sGFePXZF2hIPHgIFYwmB8gQyfMvG1Q1rQqlS7A80uZ5mkH5Uw0ZUsfOB7oYFSJm9jE4vR98VcNRuKxgzQE4Jcme48NtTukIadbPpCO7wWCTL/BwifxJJuYbq3gRdlsGJ5DGzZN+dTSNr+l7jadt14y5kna1+NVgxOkV3wXKMHwnHYID//AwwbSH2bIjLgY6zxqsNd7OqsfEPaHyMsGEqTrULLD+i6yqJPMVCBUWFXqk+HD2U/QsAWxsKJYH3Yy94CALbEimgSd0j4bxnzVkdkNhWGR30c0e5ZtvRmOtOZ3VvnmJWmqfCVsfqhg2On256JHbjf8p87h4qHqabJoK/iLkjQlW2K/EKT46pSyG5ir+Jx0R6znXeyOuyBcDL76h709XsbpkuO1A6QIC8EcYQEc3k88bGOW7dtvAdtK3dgmRUj6yDrcP50sn8KFcyrHDOKk1yLc6ZsrEDFEg8A9Q/I1fGeOuaLS+vYx3jHR65YFst7/VPf1H2ykZwcQu82CAvOJGXeDPWz6n8YTW6PwN6L54AlHnR1Oc85QmXXz0bQ80BUmAqmhevr1Xf7zDfpQr4zMURYZC/zIdlUcxT1+9YR4Mh+FaRLDz0/VCOn5JsuSAyLmZ3VQm 5N47O9Zs XP4g6p3kF3WDX50COeRUNPv44zo5SqlMtfR87qMCxd92h1pl+myYofuscaMJMS+86Vd3K/QaGodpAway5CJahV3r8pZHFmkiYatz6ks3f87upuywH/hDUxKnoJVvPKXKXaxnj+ORUsvVpw3mAL8CDZGbCl0aiIJdlbT7kBt/txFOjJzhL+uamryjRkGuAw3OKg41oGwNYXoHlsbjPFzIeXrthJh7Ld3zk1ZPlrsMqg/3AH152+/PxiNuJ5zqGd03fsVGVX+G3GkyHk14n7gI/b2J6oT5stZM+3F+GAEMmSTPf8nZyjDtjc4qAiMEJs6en+BkTHS//7GaLCEsCq3MG1EilXujXNBI3NfEq0DWpApLOUryIM2AMCfne8y3ZPte8gKNjFlU4JekKit+REU7MPddnX4KF3wXS6P+O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 10/28/2023 2:21 AM, Sean Christopherson wrote: > From: Chao Peng > > Add a new KVM exit type to allow userspace to handle memory faults that > KVM cannot resolve, but that userspace *may* be able to handle (without > terminating the guest). > > KVM will initially use KVM_EXIT_MEMORY_FAULT to report implicit > conversions between private and shared memory. With guest private memory, > there will be two kind of memory conversions: > > - explicit conversion: happens when the guest explicitly calls into KVM > to map a range (as private or shared) > > - implicit conversion: happens when the guest attempts to access a gfn > that is configured in the "wrong" state (private vs. shared) > > On x86 (first architecture to support guest private memory), explicit > conversions will be reported via KVM_EXIT_HYPERCALL+KVM_HC_MAP_GPA_RANGE, > but reporting KVM_EXIT_HYPERCALL for implicit conversions is undesriable > as there is (obviously) no hypercall, and there is no guarantee that the > guest actually intends to convert between private and shared, i.e. what > KVM thinks is an implicit conversion "request" could actually be the > result of a guest code bug. > > KVM_EXIT_MEMORY_FAULT will be used to report memory faults that appear to > be implicit conversions. > > Note! To allow for future possibilities where KVM reports > KVM_EXIT_MEMORY_FAULT and fills run->memory_fault on _any_ unresolved > fault, KVM returns "-EFAULT" (-1 with errno == EFAULT from userspace's > perspective), not '0'! Is "-EHWPOISON" case not considered unresolved, so it is not mentioned here? > Due to historical baggage within KVM, exiting to > userspace with '0' from deep callstacks, e.g. in emulation paths, is > infeasible as doing so would require a near-complete overhaul of KVM, > whereas KVM already propagates -errno return codes to userspace even when > the -errno originated in a low level helper. > > Report the gpa+size instead of a single gfn even though the initial usage > is expected to always report single pages. It's entirely possible, likely > even, that KVM will someday support sub-page granularity faults, e.g. > Intel's sub-page protection feature allows for additional protections at > 128-byte granularity. > > Link: https://lore.kernel.org/all/20230908222905.1321305-5-amoorthy@google.com > Link: https://lore.kernel.org/all/ZQ3AmLO2SYv3DszH@google.com > Cc: Anish Moorthy > Cc: David Matlack > Suggested-by: Sean Christopherson > Co-developed-by: Yu Zhang > Signed-off-by: Yu Zhang > Signed-off-by: Chao Peng > Co-developed-by: Sean Christopherson > Signed-off-by: Sean Christopherson > --- > Documentation/virt/kvm/api.rst | 41 ++++++++++++++++++++++++++++++++++ > arch/x86/kvm/x86.c | 1 + > include/linux/kvm_host.h | 11 +++++++++ > include/uapi/linux/kvm.h | 8 +++++++ > 4 files changed, 61 insertions(+) > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index ace984acc125..860216536810 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -6723,6 +6723,26 @@ array field represents return values. The userspace should update the return > values of SBI call before resuming the VCPU. For more details on RISC-V SBI > spec refer, https://github.com/riscv/riscv-sbi-doc. > > +:: > + > + /* KVM_EXIT_MEMORY_FAULT */ > + struct { > + __u64 flags; > + __u64 gpa; > + __u64 size; > + } memory; > + > +KVM_EXIT_MEMORY_FAULT indicates the vCPU has encountered a memory fault that > +could not be resolved by KVM. The 'gpa' and 'size' (in bytes) describe the > +guest physical address range [gpa, gpa + size) of the fault. The 'flags' field > +describes properties of the faulting access that are likely pertinent. > +Currently, no flags are defined. > + > +Note! KVM_EXIT_MEMORY_FAULT is unique among all KVM exit reasons in that it > +accompanies a return code of '-1', not '0'! errno will always be set to EFAULT > +or EHWPOISON when KVM exits with KVM_EXIT_MEMORY_FAULT, userspace should assume > +kvm_run.exit_reason is stale/undefined for all other error numbers. > + > :: > > /* KVM_EXIT_NOTIFY */ > @@ -7757,6 +7777,27 @@ This capability is aimed to mitigate the threat that malicious VMs can > cause CPU stuck (due to event windows don't open up) and make the CPU > unavailable to host or other VMs. > > +7.34 KVM_CAP_MEMORY_FAULT_INFO > +------------------------------ > + > +:Architectures: x86 > +:Returns: Informational only, -EINVAL on direct KVM_ENABLE_CAP. > + > +The presence of this capability indicates that KVM_RUN will fill > +kvm_run.memory_fault if KVM cannot resolve a guest page fault VM-Exit, e.g. if > +there is a valid memslot but no backing VMA for the corresponding host virtual > +address. > + > +The information in kvm_run.memory_fault is valid if and only if KVM_RUN returns > +an error with errno=EFAULT or errno=EHWPOISON *and* kvm_run.exit_reason is set > +to KVM_EXIT_MEMORY_FAULT. > + > +Note: Userspaces which attempt to resolve memory faults so that they can retry > +KVM_RUN are encouraged to guard against repeatedly receiving the same > +error/annotated fault. > + > +See KVM_EXIT_MEMORY_FAULT for more information. > + > 8. Other capabilities. > ====================== > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 6409914428ca..ee3cd8c3c0ef 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -4518,6 +4518,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) > case KVM_CAP_ENABLE_CAP: > case KVM_CAP_VM_DISABLE_NX_HUGE_PAGES: > case KVM_CAP_IRQFD_RESAMPLE: > + case KVM_CAP_MEMORY_FAULT_INFO: > r = 1; > break; > case KVM_CAP_EXIT_HYPERCALL: > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > index 4e741ff27af3..96aa930536b1 100644 > --- a/include/linux/kvm_host.h > +++ b/include/linux/kvm_host.h > @@ -2327,4 +2327,15 @@ static inline void kvm_account_pgtable_pages(void *virt, int nr) > /* Max number of entries allowed for each kvm dirty ring */ > #define KVM_DIRTY_RING_MAX_ENTRIES 65536 > > +static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, > + gpa_t gpa, gpa_t size) > +{ > + vcpu->run->exit_reason = KVM_EXIT_MEMORY_FAULT; > + vcpu->run->memory_fault.gpa = gpa; > + vcpu->run->memory_fault.size = size; > + > + /* Flags are not (yet) defined or communicated to userspace. */ > + vcpu->run->memory_fault.flags = 0; > +} > + > #endif > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h > index bd1abe067f28..7ae9987b48dd 100644 > --- a/include/uapi/linux/kvm.h > +++ b/include/uapi/linux/kvm.h > @@ -274,6 +274,7 @@ struct kvm_xen_exit { > #define KVM_EXIT_RISCV_SBI 35 > #define KVM_EXIT_RISCV_CSR 36 > #define KVM_EXIT_NOTIFY 37 > +#define KVM_EXIT_MEMORY_FAULT 38 > > /* For KVM_EXIT_INTERNAL_ERROR */ > /* Emulate instruction failed. */ > @@ -520,6 +521,12 @@ struct kvm_run { > #define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) > __u32 flags; > } notify; > + /* KVM_EXIT_MEMORY_FAULT */ > + struct { > + __u64 flags; > + __u64 gpa; > + __u64 size; > + } memory_fault; > /* Fix the size of the union. */ > char padding[256]; > }; > @@ -1203,6 +1210,7 @@ struct kvm_ppc_resize_hpt { > #define KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE 228 > #define KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES 229 > #define KVM_CAP_USER_MEMORY2 230 > +#define KVM_CAP_MEMORY_FAULT_INFO 231 > > #ifdef KVM_CAP_IRQ_ROUTING >