From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EFEAC282EC for ; Mon, 10 Mar 2025 14:36:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DBCCA280002; Mon, 10 Mar 2025 10:36:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D470F280001; Mon, 10 Mar 2025 10:36:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC054280002; Mon, 10 Mar 2025 10:36:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 67864280001 for ; Mon, 10 Mar 2025 10:36:03 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 96EF68110D for ; Mon, 10 Mar 2025 14:36:03 +0000 (UTC) X-FDA: 83205891006.28.36B5BC4 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf25.hostedemail.com (Postfix) with ESMTP id E3CECA0009 for ; Mon, 10 Mar 2025 14:35:58 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=R67MNWqp; spf=pass (imf25.hostedemail.com: domain of tabba@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=tabba@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741617358; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7AKenLNL6X3uS3482+fJgsjvOVmCWlfTmQs/079icwE=; b=2z59JShXOBrbF+CTVzb+p5aO/HLz2Ha5Y47SAg9PYDu7IwoO/7xym8CG/sXoSn2dILLrTC 706GXBGv2FtfuKoGqDG2oTeB67Q7FYHRoDc+hq4hgv+akEnZENiUOSNuaXm9kf/xGk0t49 XNqa3S4s5XuCWeSJOz41r1fCZMaoUQM= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=R67MNWqp; spf=pass (imf25.hostedemail.com: domain of tabba@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=tabba@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741617359; a=rsa-sha256; cv=none; b=Bb7zjzOokvsY0RBzQLIF60spAijEmNv63HtLQYSXFquVQWbzDcdck9XzgygNmQELKwjOrn rBQbEtLnMBS+shr+M3H+Cjl/+hPN174+JZ9aT2yLQIbuRMZObgxJraJJ2moo4sSbSWYYCh FCynfRG99cpGng1wbCoytZs0h03t7oc= Received: by mail-qt1-f170.google.com with SMTP id d75a77b69052e-47666573242so537721cf.0 for ; Mon, 10 Mar 2025 07:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1741617357; x=1742222157; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=7AKenLNL6X3uS3482+fJgsjvOVmCWlfTmQs/079icwE=; b=R67MNWqp1KzlrdlS5yxkw59WjtN1L00CeCk0GG+XLdCA68+37TE1q5Kq6w55C+dLR2 +KUhN0aeCc4RkLU8R7tCfcmJgHI1Ze0lVCgIPGHQ9kP8mfF0SWUeoYXCVGXMBxwGfmiJ s50EljpO7DXx98XsVeCtAjClR4D804TixDNjLP6Zzy6O7HPjTF6+nxVck3Djm6/erHlR ejJp3yO/vsoE7ng6b9rnYjuJhC8E18ahMf76dGjyPvzE8B+Av9z318IX0OL4mvQKWYKj iCxZIB/7TRMFTlr/Gg0XVPzFEC5mMYA4pIFRfTSQjLAT8azo1SVG9ZS1bMxp1c//rOJj 4fgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741617357; x=1742222157; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7AKenLNL6X3uS3482+fJgsjvOVmCWlfTmQs/079icwE=; b=Kd6nTXR1vDwgBei8789zNGDvebAfWMDUlvzfNJIrRerA9bzqAgrHQ3yuvzZ3LtahuO X9nT2X+KdHf+Ehda9DNPB6XaEvN/fnVOc062uBMFBFmN/zvIhBuLVbbVnk3mbFUvmfgS ynyIV1W0DW1iss+GIAur6K1adFS49J85V2uQm+G8Iqo4lWWRGhBN1HoJY7o2kuDpWAAi es6tPxJ++WH3b+VA5cU9NoXu8HodMFZijJhYkfFfo0IW4EkqMzQrwxpbW+VU/7KPBsWL gYd0EGwsAPo4N9qUYt+BGCam34+ESe+1zOZ6qVubIPefOg4PcDTe9GenWmKVzx9vvp9h wDgg== X-Forwarded-Encrypted: i=1; AJvYcCXabSMA7LMDsDgphgDmV2NlFjl/RegArnBEKauAw5q/FVyvv9l9GZDaC675wotXiFXjWVHR/awykA==@kvack.org X-Gm-Message-State: AOJu0YyUoGmiDF9pPK1fPzLaZdtMtSH9hgPGSSULDqBPItAar5uwVuLg sJbt1knKQtRoCH8A3t3KA3BgiOTUUbgfy9/mQQv6vgpI9dyfuI/2JzGekqYyxesbfG+9FcpWnzk GrRs/XwRFJKflt7z53hIQARPJQYZMMj63QKAi X-Gm-Gg: ASbGncstl/nYgHTyj8kMbjlrAwYgu8HDvCJ7X1VmFgYOeKZU17RBbd4QsY1cJGB/M0G Y1/CCK3DDKJzA3VrqgFM8973UX9k7CT6rEAYxFVcILv4bvBrI7anLUJGDgiPq0UX8wNgibmByEz a/gxN63KGIK46QGrjeFLBJAqVR X-Google-Smtp-Source: AGHT+IELZMNpcNZxVVAes7Yvp9RPwEJWQwjVaiVupyMZoRqV6OzxPFLq2P3Lxi7fF72RmxV48sWkd2v4B0yhqGQOW8g= X-Received: by 2002:a05:622a:2a0b:b0:471:eab0:ef21 with SMTP id d75a77b69052e-47668a7e96fmr6596751cf.13.1741617357220; Mon, 10 Mar 2025 07:35:57 -0700 (PDT) MIME-Version: 1.0 References: <20250303171013.3548775-3-tabba@google.com> In-Reply-To: From: Fuad Tabba Date: Mon, 10 Mar 2025 14:35:20 +0000 X-Gm-Features: AQ5f1JpC9TjXQKryWVG5dXX_lw6XajYPTX52e9zXWHfytK3c-PmScHapjuQ9jXI Message-ID: Subject: Re: [PATCH v5 2/9] KVM: guest_memfd: Handle final folio_put() of guest_memfd pages To: Ackerley Tng Cc: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org, pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Queue-Id: E3CECA0009 X-Stat-Signature: irwx13fiqghw4jgqmfobo5kt318h1zd6 X-Rspamd-Server: rspam09 X-HE-Tag: 1741617358-849509 X-HE-Meta: U2FsdGVkX19KT1+Hp8htZSQHjCn0FPN3/Lp760AjIWag4NVe6pxcnT9LKIejE2SpZtf/TvzN+ePveP74whQC6sSpaz2BU2McWSLHuaQRIj8LSn+laGfsLK1LxpOFAadNd0ubMNoQJ+tCLQhRZ3pF5TCGfB9rTDAbzijB1lg+K2OCSpKO1UxQkAqCA6xP/hPfQrKAuAU+PpBxq/OT+UIriI3uW82JBeYnEvj5bFXlAOR9FkNV4VNq+xdGPeDi9DQqZc2cBs4MK/MlDmpBcaq+JUfFG7IFRHTlkyy22ln118ijrFKnc9uYQQzd1bZ1RpuXMn/QoKcbLmqmry5vSY+6+K/QmVWkiuvYEWwAW6WDNm3YwM8YTerVzs8Qh099oWV1BTmmjqyPxRKp7taRku/fk2Nj/VAbWpZ/YZ6VMe26v+d587zl6CB/hKmPIK8RS1qFmkuSfYwoydalIN2Cqm+9w/8xjhaCV0yvWA3MOhwo5MG76jw6bWq+xv7W6RjbqVpwgyDL4rppNdP6gG/G482Jz7sLY1QC39gantSvWCfr0PHyDUODXQ+3DVOXFMGaiixJ9YUWFNWpfe7kIhqhd8B/jOUt0or1n/MZJ0r91WpWvJKWp9scp8qr+oqg3IDcmLLyesYfELLQuT6AU7tqVO/AfqT5GsxCcSk+X+t6p3kxEdtSHlCMuJ2MOJPs+MC2n5JIShOU/Ntc9UtYohoPmjG/Ti3V9nQU4EOclG/N0ITa6gKgWrBK3rc8Ncl4RMkMcSTfYMpR7Tyegufq/DYrw13LdAFWh5123jlII4eDsTPKIIlJLWc8QnSzHhxqM5sBab2K/sblfd4DvdIU/y9ZJYmy8Wj59YTwYrYKpFZkLGGSVQPcTvAN51UMGggkkuZyGP5rIwcuxunI7t62yUGYZ5iCD0A4cDzfPjx3X2Mq9HFQbHKFV0xaUQkq6qUK5HEhJmiWB6WKFO6HwN8zXrPWnyG lMnZNC8+ 7JzGAWDTBWcS99eg2NgAi7FoXNMbSxbiUWix+O9oFYwmFMSX3ElpsemgnxThiWhcp3PsvEcrsq0RgzgxY9K9IkF4IJFwhM+6YrNo27uv2Ppp0XNSPRf59+zi1doAA07bsf3ET/EELgDrIyjGhOln2GXCDgkzHO3g5/JYkWYrcyfv5AEO/4TPEKrjWnOInsVLzr6ZbKauL3LoF6zFDt0RXEqHJ8ponw8VQneQ5Hg7UqIkp4zMAtKJbn2BWg3YGtTwlcjEZZLZXKzze2FuoKhSQIXzLL4LtDHpSdvRAl5sRE4eMSL+rwWXHl6/9FafLbOemNCl9t54AeMkrSgXpVwDpRwGK7F0sJeXJihSV1Yw3qfIjfjggwZPUdHInQIxR6FhitrVI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Ackerley, On Mon, 10 Mar 2025 at 14:23, Ackerley Tng wrote: > > Fuad Tabba writes: > > > Hi Ackerley, > > > > On Fri, 7 Mar 2025 at 17:04, Ackerley Tng wrote: > >> > >> Fuad Tabba writes: > >> > >> > Before transitioning a guest_memfd folio to unshared, thereby > >> > disallowing access by the host and allowing the hypervisor to > >> > transition its view of the guest page as private, we need to be > >> > sure that the host doesn't have any references to the folio. > >> > > >> > This patch introduces a new type for guest_memfd folios, which > >> > isn't activated in this series but is here as a placeholder and > >> > to facilitate the code in the subsequent patch series. This will > >> > be used in the future to register a callback that informs the > >> > guest_memfd subsystem when the last reference is dropped, > >> > therefore knowing that the host doesn't have any remaining > >> > references. > >> > > >> > This patch also introduces the configuration option, > >> > KVM_GMEM_SHARED_MEM, which toggles support for mapping > >> > guest_memfd shared memory at the host. > >> > > >> > Signed-off-by: Fuad Tabba > >> > Acked-by: Vlastimil Babka > >> > Acked-by: David Hildenbrand > >> > --- > >> > include/linux/kvm_host.h | 7 +++++++ > >> > include/linux/page-flags.h | 16 ++++++++++++++++ > >> > mm/debug.c | 1 + > >> > mm/swap.c | 9 +++++++++ > >> > virt/kvm/Kconfig | 5 +++++ > >> > 5 files changed, 38 insertions(+) > >> > > >> > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > >> > index f34f4cfaa513..7788e3625f6d 100644 > >> > --- a/include/linux/kvm_host.h > >> > +++ b/include/linux/kvm_host.h > >> > @@ -2571,4 +2571,11 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, > >> > struct kvm_pre_fault_memory *range); > >> > #endif > >> > > >> > +#ifdef CONFIG_KVM_GMEM_SHARED_MEM > >> > +static inline void kvm_gmem_handle_folio_put(struct folio *folio) > >> > +{ > >> > + WARN_ONCE(1, "A placeholder that shouldn't trigger. Work in progress."); > >> > +} > >> > +#endif > >> > + > >> > #endif > >> > >> Following up with the discussion at the guest_memfd biweekly call on the > >> guestmem library, I think this folio_put() handler for guest_memfd could > >> be the first function that's refactored out into (placeholder name) > >> mm/guestmem.c. > >> > >> This folio_put() handler has to stay in memory even after KVM (as a > >> module) is unloaded from memory, and so it is a good candidate for the > >> first function in the guestmem library. > >> > >> Along those lines, CONFIG_KVM_GMEM_SHARED_MEM in this patch can be > >> renamed CONFIG_GUESTMEM, and CONFIG_GUESTMEM will guard the existence of > >> PGTY_guestmem. > >> > >> CONFIG_KVM_GMEM_SHARED_MEM can be introduced in the next patch of this > >> series, which could, in Kconfig, select CONFIG_GUESTMEM. > >> > >> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > >> > index 6dc2494bd002..daeee9a38e4c 100644 > >> > --- a/include/linux/page-flags.h > >> > +++ b/include/linux/page-flags.h > >> > @@ -933,6 +933,7 @@ enum pagetype { > >> > PGTY_slab = 0xf5, > >> > PGTY_zsmalloc = 0xf6, > >> > PGTY_unaccepted = 0xf7, > >> > + PGTY_guestmem = 0xf8, > >> > > >> > PGTY_mapcount_underflow = 0xff > >> > }; > >> > @@ -1082,6 +1083,21 @@ FOLIO_TYPE_OPS(hugetlb, hugetlb) > >> > FOLIO_TEST_FLAG_FALSE(hugetlb) > >> > #endif > >> > > >> > +/* > >> > + * guestmem folios are used to back VM memory as managed by guest_memfd. Once > >> > + * the last reference is put, instead of freeing these folios back to the page > >> > + * allocator, they are returned to guest_memfd. > >> > + * > >> > + * For now, guestmem will only be set on these folios as long as they cannot be > >> > + * mapped to user space ("private state"), with the plan of always setting that > >> > + * type once typed folios can be mapped to user space cleanly. > >> > + */ > >> > +#ifdef CONFIG_KVM_GMEM_SHARED_MEM > >> > +FOLIO_TYPE_OPS(guestmem, guestmem) > >> > +#else > >> > +FOLIO_TEST_FLAG_FALSE(guestmem) > >> > +#endif > >> > + > >> > PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc) > >> > > >> > /* > >> > diff --git a/mm/debug.c b/mm/debug.c > >> > index 8d2acf432385..08bc42c6cba8 100644 > >> > --- a/mm/debug.c > >> > +++ b/mm/debug.c > >> > @@ -56,6 +56,7 @@ static const char *page_type_names[] = { > >> > DEF_PAGETYPE_NAME(table), > >> > DEF_PAGETYPE_NAME(buddy), > >> > DEF_PAGETYPE_NAME(unaccepted), > >> > + DEF_PAGETYPE_NAME(guestmem), > >> > }; > >> > > >> > static const char *page_type_name(unsigned int page_type) > >> > diff --git a/mm/swap.c b/mm/swap.c > >> > index 47bc1bb919cc..241880a46358 100644 > >> > --- a/mm/swap.c > >> > +++ b/mm/swap.c > >> > @@ -38,6 +38,10 @@ > >> > #include > >> > #include > >> > > >> > +#ifdef CONFIG_KVM_GMEM_SHARED_MEM > >> > +#include > >> > +#endif > >> > + > >> > #include "internal.h" > >> > > >> > #define CREATE_TRACE_POINTS > >> > @@ -101,6 +105,11 @@ static void free_typed_folio(struct folio *folio) > >> > case PGTY_hugetlb: > >> > free_huge_folio(folio); > >> > return; > >> > +#endif > >> > +#ifdef CONFIG_KVM_GMEM_SHARED_MEM > >> > + case PGTY_guestmem: > >> > + kvm_gmem_handle_folio_put(folio); > >> > + return; > >> > #endif > >> > default: > >> > WARN_ON_ONCE(1); > >> > diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig > >> > index 54e959e7d68f..37f7734cb10f 100644 > >> > --- a/virt/kvm/Kconfig > >> > +++ b/virt/kvm/Kconfig > >> > @@ -124,3 +124,8 @@ config HAVE_KVM_ARCH_GMEM_PREPARE > >> > config HAVE_KVM_ARCH_GMEM_INVALIDATE > >> > bool > >> > depends on KVM_PRIVATE_MEM > >> > + > >> > +config KVM_GMEM_SHARED_MEM > >> > + select KVM_PRIVATE_MEM > >> > + depends on !KVM_GENERIC_MEMORY_ATTRIBUTES > >> > >> Enforcing that KVM_GENERIC_MEMORY_ATTRIBUTES is not selected should not > >> be a strict requirement. Fuad explained in an offline chat that this is > >> just temporary. > >> > >> If we have CONFIG_GUESTMEM, then this question is moot, I think > >> CONFIG_GUESTMEM would just be independent of everything else; other > >> configs would depend on CONFIG_GUESTMEM. > > > > There are two things here. First of all, the unfortunate naming > > situation where PRIVATE could mean GUESTMEM, or private could mean not > > shared. I plan to tackle this aspect (i.e., the naming) in a separate > > patch series, since that will surely generate a lot of debate :) > > > > Oops. By "depend on CONFIG_GUESTMEM" I meant "depend on the introduction > of the guestmem shim". I think this is a good time to introduce the shim > because the folio_put() callback needs to be in mm and not just in KVM, > which is a loadable module and hence can be removed from memory. > > If we do introduce the shim, the config flag CONFIG_KVM_GMEM_SHARED_MEM > will be replaced by CONFIG_GUESTMEM (or other name), and then the > question on depending on !KVM_GENERIC_MEMORY_ATTRIBUTES will be moot > since I think an mm config flag wouldn't place a constraint on a module > config flag? I see. > When I wrote this, I thought that config flags are easily renamed since > they're an interface and are user-facing, but I realized config flag > renaming seems to be easily renamed based on this search [1]. > > If we're going with renaming in a separate patch series, some mechanism > should be introduced here to handle the case where I'm not talking about renaming the contents of _this_ patch series. I was referring to existing tems such as, CONFIG_KVM_PRIVATE_MEM, which really enabled guestmemfd. > 1. Kernel (and KVM module) is compiled with KVM_GMEM_SHARED_MEM set > 2. KVM is unloaded > 3. folio_put() tries to call kvm_gmem_handle_folio_put() > > > The other part is that, with shared memory in-place, the memory > > attributes are an orthogonal matter. The attributes are the userpace's > > view of what it expects the state of the memory to be, and are used to > > multiplex whether the memory being accessed is guest_memfd or the > > regular (i.e., most likely anonymous) memory used normally by KVM. > > > > This behavior however would be architecture, or even vm-type specific. > > > > I agree it is orthogonal but I'm calling this out because "depends on > !KVM_GENERIC_MEMORY_ATTRIBUTES" means if I set > CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES, I can't use PGTY_guestmem since > CONFIG_KVM_GMEM_SHARED_MEM would get unset. I'll remove this dependence in the respin. Thanks, /fiad > I was trying to test this with a KVM_X86_SW_PROTECTED_VM, setting up for > using the ioctl to convert memory and hit this issue. > > > Cheers, > > /fuad > > > >> > + bool > > [1] https://lore.kernel.org/all/?q=s%3Arename+dfn%3AKconfig+merged