From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 413EEC6FA82 for ; Fri, 23 Sep 2022 15:20:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFE6E80008; Fri, 23 Sep 2022 11:20:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AAD7D80007; Fri, 23 Sep 2022 11:20:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94DD680008; Fri, 23 Sep 2022 11:20:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 82DC880007 for ; Fri, 23 Sep 2022 11:20:25 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 54A7D120991 for ; Fri, 23 Sep 2022 15:20:25 +0000 (UTC) X-FDA: 79943711610.12.39F6710 Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by imf01.hostedemail.com (Postfix) with ESMTP id EA15740012 for ; Fri, 23 Sep 2022 15:20:24 +0000 (UTC) Received: by mail-lf1-f54.google.com with SMTP id a8so751143lff.13 for ; Fri, 23 Sep 2022 08:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date; bh=Zow8yFZS7DL+sQi+dFI8ktR+CVObD/Cj7M/2zMgZrcE=; b=OwesCIlz00EUn7DRsMx0DjzCHBk8S/+Xi2g5fPxGQhSRVaCPRjRDdaU3EoxDGBycWF WjeJOwauK+2vsuvUb0vjkRCZ4RMivueV3nfN+bQNkdWetxD6j+QKSjPsuee2Tp2n38Jz /iJsK25fqBHvk1flzzUqT6xcaD2Kcry+CP91ebbpRaH+7JaO7k+DDGbA80azTpePJHUo C9mi6Q2+k5+aqh7plurR6pEpUr9EvDPbLxCLDRinQNGibIiinnoVBg84EnbuaBKnPpUT cKT++hkuZ7OScqK5B4U9F14Zu14fLK/+p4hCg8Uwqh1UdA/pssHQoaFSPvqyG9cck4Rn GTYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=Zow8yFZS7DL+sQi+dFI8ktR+CVObD/Cj7M/2zMgZrcE=; b=N5jqrlYSM2alFLK583i57KvMm3gSGsKwL87fGAAxrV2ob96/0fcY/kQghikLXHfTcS SGPgB4/u9GFVbVzLvkehIYCWGWAl4XLamYiL3YE3fdlV13H55ppTXRfYnHXEGpCniMl+ ILQ5aJYIbbEK9uyfD6ufVWelivbHqfdf/j0HPRT4QUckGftyeaM+szYn+ZTO4Wbx7o/S Ps39UaZetpPUgtKdo/Ja0p7XEtDUmegDQkcAozAhcokxA+Z5V6D+eN1wXK7PXfnt7PTZ 5KF7VzW2pNIziO3ao6VY+nOMne9DjwSKqCxT6cz4V6O5ShQx99R5o2Sqh+sA/YrgZQrm RdCg== X-Gm-Message-State: ACrzQf3KtYLJN/iInAo0+J1EXpAv3LpMQEm6Wkrcg7X/8HXhB7FHq4CW gIF372esyEI/DcSccQLrWvr7KKttzKB2TKGo6EonlQ== X-Google-Smtp-Source: AMsMyM78W2SpygRyWA6BwLCi+G+Q9AwHIlqubh16dKOB9mvu+zIZ03puXTWepUQ7h9eMZBQnICvUmLJxtqa9A8Z7/3M= X-Received: by 2002:a05:6512:3612:b0:499:aea7:8bed with SMTP id f18-20020a056512361200b00499aea78bedmr3269322lfs.26.1663946423216; Fri, 23 Sep 2022 08:20:23 -0700 (PDT) MIME-Version: 1.0 References: <20220915142913.2213336-1-chao.p.peng@linux.intel.com> <20220915142913.2213336-2-chao.p.peng@linux.intel.com> In-Reply-To: From: Fuad Tabba Date: Fri, 23 Sep 2022 16:19:46 +0100 Message-ID: Subject: Re: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible memfd To: Sean Christopherson Cc: David Hildenbrand , Chao Peng , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , Michael Roth , mhocko@suse.com, Muchun Song , wei.w.wang@intel.com, Will Deacon , Marc Zyngier Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663946424; a=rsa-sha256; cv=none; b=vCbY+u3rRpKhS4PdK2PqLaht5HPcfiscjbIeWfb0S62QJOfyiOUsFnFMV1IsAYPQo31hFt DbLpB4tBDF5l1XZ4jfGfDGDAIXJbq+96m2ufxpfjjKXq1ZK5HgByRupayHZ+pEnwbbpQnF gKyI39MaxV+1Dqaw1mGRO60GVVtBJ1E= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OwesCIlz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of tabba@google.com designates 209.85.167.54 as permitted sender) smtp.mailfrom=tabba@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663946424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zow8yFZS7DL+sQi+dFI8ktR+CVObD/Cj7M/2zMgZrcE=; b=ajjZSWiavb9HPSf/tFHn8X5pL623eAesRCWSDWMEoTqf7E+d5ZKb8Gc5sCLx7kH9fJWGwa eDxRkVNIRfHE4ZM7PxLmBxW97ZqhlZO+6Nl8iieH3gKOvl/KyhYbbLnG4Q1DOXamM/Z73q q7EDSkugY+mnUiLIvqsLsqKgf5szfi8= X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: EA15740012 X-Rspam-User: Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OwesCIlz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of tabba@google.com designates 209.85.167.54 as permitted sender) smtp.mailfrom=tabba@google.com X-Stat-Signature: pwrd37cxcurqx4it54ghdjzuu7cncj5s X-HE-Tag: 1663946424-959877 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On Mon, Sep 19, 2022 at 8:10 PM Sean Christopherson wro= te: > > +Will, Marc and Fuad (apologies if I missed other pKVM folks) > > On Mon, Sep 19, 2022, David Hildenbrand wrote: > > On 15.09.22 16:29, Chao Peng wrote: > > > From: "Kirill A. Shutemov" > > > > > > KVM can use memfd-provided memory for guest memory. For normal usersp= ace > > > accessible memory, KVM userspace (e.g. QEMU) mmaps the memfd into its > > > virtual address space and then tells KVM to use the virtual address t= o > > > setup the mapping in the secondary page table (e.g. EPT). > > > > > > With confidential computing technologies like Intel TDX, the > > > memfd-provided memory may be encrypted with special key for special > > > software domain (e.g. KVM guest) and is not expected to be directly > > > accessed by userspace. Precisely, userspace access to such encrypted > > > memory may lead to host crash so it should be prevented. > > > > Initially my thaught was that this whole inaccessible thing is TDX spec= ific > > and there is no need to force that on other mechanisms. That's why I > > suggested to not expose this to user space but handle the notifier > > requirements internally. > > > > IIUC now, protected KVM has similar demands. Either access (read/write)= of > > guest RAM would result in a fault and possibly crash the hypervisor (at > > least not the whole machine IIUC). > > Yep. The missing piece for pKVM is the ability to convert from shared to= private > while preserving the contents, e.g. to hand off a large buffer (hundreds = of MiB) > for processing in the protected VM. Thoughts on this at the bottom. Just wanted to mention that for pKVM (arm64), this wouldn't crash the hypervisor. A userspace access would crash the userspace process since the hypervisor would inject a fault back. Because of that making it inaccessible from userspace is good to have, but not really vital for pKVM. What is important for pKVM is that the guest private memory is not GUP'able by the host. This is because if it were, it might be possible for a malicious userspace process (e.g., a malicious vmm) to trick the host kernel into accessing guest private memory in a context where it isn=E2=80=99t prepared to handle the fault injected by the hypervisor. This of course might crash the host. > > > This patch introduces userspace inaccessible memfd (created with > > > MFD_INACCESSIBLE). Its memory is inaccessible from userspace through > > > ordinary MMU access (e.g. read/write/mmap) but can be accessed via > > > in-kernel interface so KVM can directly interact with core-mm without > > > the need to map the memory into KVM userspace. > > > > With secretmem we decided to not add such "concept switch" flags and in= stead > > use a dedicated syscall. > > > > I have no personal preference whatsoever between a flag and a dedicated s= yscall, > but a dedicated syscall does seem like it would give the kernel a bit mor= e > flexibility. > > > What about memfd_inaccessible()? Especially, sealing and hugetlb are no= t > > even supported and it might take a while to support either. > > Don't know about sealing, but hugetlb support for "inaccessible" memory n= eeds to > come sooner than later. "inaccessible" in quotes because we might want t= o choose > a less binary name, e.g. "restricted"?. > > Regarding pKVM's use case, with the shim approach I believe this can be d= one by > allowing userspace mmap() the "hidden" memfd, but with a ton of restricti= ons > piled on top. > > My first thought was to make the uAPI a set of KVM ioctls so that KVM cou= ld tightly > tightly control usage without taking on too much complexity in the kernel= , but > working through things, routing the behavior through the shim itself migh= t not be > all that horrific. > > IIRC, we discarded the idea of allowing userspace to map the "private" fd= because > things got too complex, but with the shim it doesn't seem _that_ bad. > > E.g. on the memfd side: > > 1. The entire memfd must be mapped, and at most one mapping is allowed,= i.e. > mapping is all or nothing. > > 2. Acquiring a reference via get_pfn() is disallowed if there's a mappi= ng for > the restricted memfd. > > 3. Add notifier hooks to allow downstream users to further restrict thi= ngs. > > 4. Disallow splitting VMAs, e.g. to force userspace to munmap() everyth= ing in > one shot. > > 5. Require that there are no outstanding references at munmap(). Or if= this > can't be guaranteed by userspace, maybe add some way for userspace t= o wait > until it's ok to convert to private? E.g. so that get_pfn() doesn't= need > to do an expensive check every time. > > static int memfd_restricted_mmap(struct file *file, struct vm_area_stru= ct *vma) > { > if (vma->vm_pgoff) > return -EINVAL; > > if ((vma->vm_end - vma->vm_start) !=3D ) > return -EINVAL; > > mutex_lock(&data->lock); > > if (data->has_mapping) { > r =3D -EINVAL; > goto err; > } > list_for_each_entry(notifier, &data->notifiers, list) { > r =3D notifier->ops->mmap_start(notifier, ...); > if (r) > goto abort; > } > > notifier->ops->mmap_end(notifier, ...); > mutex_unlock(&data->lock); > return 0; > > abort: > list_for_each_entry_continue_reverse(notifier &data->notifiers, l= ist) > notifier->ops->mmap_abort(notifier, ...); > err: > mutex_unlock(&data->lock); > return r; > } > > static void memfd_restricted_close(struct vm_area_struct *vma) > { > mutex_lock(...); > > /* > * Destroy the memfd and disable all future accesses if there are > * outstanding refcounts (or other unsatisfied restrictions?). > */ > if ( || ???) > memfd_restricted_destroy(...); > else > data->has_mapping =3D false; > > mutex_unlock(...); > } > > static int memfd_restricted_may_split(struct vm_area_struct *area, unsi= gned long addr) > { > return -EINVAL; > } > > static int memfd_restricted_mapping_mremap(struct vm_area_struct *new_v= ma) > { > return -EINVAL; > } > > Then on the KVM side, its mmap_start() + mmap_end() sequence would: > > 1. Not be supported for TDX or SEV-SNP because they don't allow adding = non-zero > memory into the guest (after pre-boot phase). > > 2. Be mutually exclusive with shared<=3D>private conversions, and is al= lowed if > and only if the entire gfn range of the associated memslot is shared= . In general I think that this would work with pKVM. However, limiting private<->shared conversions to the granularity of a whole memslot might be difficult to handle in pKVM, since the guest doesn't have the concept of memslots. For example, in pKVM right now, when a guest shares back its restricted DMA pool with the host it does so at the page-level. pKVM would also need a way to make an fd accessible again when shared back, which I think isn't possible with this patch. You were initially considering a KVM ioctl for mapping, which might be better suited for this since KVM knows which pages are shared and which ones are private. So routing things through KVM might simplify things and allow it to enforce all the necessary restrictions (e.g., private memory cannot be mapped). What do you think? Thanks, /fuad