From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDE97C433FE for ; Tue, 18 Oct 2022 13:42:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2261A6B0072; Tue, 18 Oct 2022 09:42:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AE866B0075; Tue, 18 Oct 2022 09:42:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 028A28E0001; Tue, 18 Oct 2022 09:42:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E66776B0072 for ; Tue, 18 Oct 2022 09:42:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id ACAA9404E6 for ; Tue, 18 Oct 2022 13:42:25 +0000 (UTC) X-FDA: 80034184650.08.A5940F8 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf23.hostedemail.com (Postfix) with ESMTP id 40C43140034 for ; Tue, 18 Oct 2022 13:42:24 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id f140so14114665pfa.1 for ; Tue, 18 Oct 2022 06:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TQSORsKSMngPJb5f/eC6Hu6kpkY5kYXJ98QHFM8UvxY=; b=ZHxxlG3UTEot3sPjKtvw1X3aBdRbEmIooxBSsM63jzfeOELeKLeWEK8Ka0UBZMDOP+ akvRFs76C2wKq91BFhqgTb5HB703LMV7esd/A+wWslFPM0+sKw9GkKbhOLqB+y2xkhr8 xAw55/tP2EjBapVx7JcnOuoy3rb21J87PUbxaFqakG14s9yKfbb31M3qi5ybSq2mRACM jSitk4irsc/WL0t07xPDMeWHTF1Fx+OZSUwfySvO4jPCavdKRxOz8qiV/i/KYM1y4DB8 jWL9urfkjvvcFCgxaCtIJ6pE7MoPjXcItlLTszbdaWP+IhZBfYH0+bbwgZYx0bdSzYu+ tT2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TQSORsKSMngPJb5f/eC6Hu6kpkY5kYXJ98QHFM8UvxY=; b=29vom+lelEFGq7tmiyZmugsaitgFFa1PIzcp48hmaxowDihCa3sp8We+0/1iah4qdj C0B0CWKE0JrnKrQvj8XSmJUedUt6RFdwEL3h8g3SiyB9+tzbEmK/zha1lyHDp0COrjni uovX9gd9Y/eweOXudkuRPQqNYL81PVR/5/2fvJw8YKcvGb3d/xfmHya8dJ6Lbe0JVD2i n2LrUqZVJT4rq7+k2iHwdmCrdutYZLZ5jxXezcw0TERctglCCRizwPvPZjgdx73pns/x RQpbuCbzKbJt34PWMAq6yXXxc0nMTypfNUhKJvb9uRXizKTLWT1Yk9bAITyKiTQzAFLt aH+A== X-Gm-Message-State: ACrzQf3jINSltZ7c2gPEYE3VEmTS61f0tGb7rClfZv/c8X9HFwV/DKo9 CLtiBzd1vjbe4o+VaoXSEf48A8Gz5NBovAOhjm87oQ== X-Google-Smtp-Source: AMsMyM457hPoLFIJ8lUjGLgHK/+h3GgdmiyyuIPyEtfxcbOdN4bX0W7oNp+dFIXy83RpDOunf04FJy/8XBLnfxpoQmE= X-Received: by 2002:a63:88c7:0:b0:462:79de:dc75 with SMTP id l190-20020a6388c7000000b0046279dedc75mr2721715pgd.458.1666100542751; Tue, 18 Oct 2022 06:42:22 -0700 (PDT) MIME-Version: 1.0 References: <20220915142913.2213336-1-chao.p.peng@linux.intel.com> <20220915142913.2213336-2-chao.p.peng@linux.intel.com> <20221017161955.t4gditaztbwijgcn@box.shutemov.name> <20221017215640.hobzcz47es7dq2bi@box.shutemov.name> In-Reply-To: <20221017215640.hobzcz47es7dq2bi@box.shutemov.name> From: Vishal Annapurve Date: Tue, 18 Oct 2022 19:12:10 +0530 Message-ID: Subject: Re: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible memfd To: "Kirill A . Shutemov" Cc: "Gupta, Pankaj" , Vlastimil Babka , Chao Peng , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Yu Zhang , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , Michael Roth , mhocko@suse.com, Muchun Song , wei.w.wang@intel.com Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666100544; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TQSORsKSMngPJb5f/eC6Hu6kpkY5kYXJ98QHFM8UvxY=; b=CY9JLT6ihIaFB5dMFNnfeKpOhS9vAsZ81ZxaIpVxEGDLgKKNojY1wCNfFyNdNCh99Cx6V7 7wI1MM2yBmRbv1ip4TnAr5RnYiQ02ilguLXdpxKaCJi7yvNzZbKPDMDaEqky42ICFjicY1 zxR/b9NF954tEJZitBWWTCw0Q4VSQl4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ZHxxlG3U; spf=pass (imf23.hostedemail.com: domain of vannapurve@google.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=vannapurve@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666100544; a=rsa-sha256; cv=none; b=jhAc5a+ZIEpsoEIYbwJh+Jnz9TXO8gBtrTL0e0Qpx2t0xHNpjbooiNGiOkAu10fDeJl/Sg PVaT0rph3WAdoiESqvvIYZmz0k3LtfFqb1gU8+uAxC5PWmx8x+kCc/fsd3CbA4oWRmdWza rwol5sR9ljcrB4BAAFJ2PBuebddFwDk= Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ZHxxlG3U; spf=pass (imf23.hostedemail.com: domain of vannapurve@google.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=vannapurve@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: 783kmaeejkc3z7ifqe49rmyusxzq8awb X-Rspamd-Queue-Id: 40C43140034 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1666100544-885695 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Oct 18, 2022 at 3:27 AM Kirill A . Shutemov wrote: > > On Mon, Oct 17, 2022 at 06:39:06PM +0200, Gupta, Pankaj wrote: > > On 10/17/2022 6:19 PM, Kirill A . Shutemov wrote: > > > On Mon, Oct 17, 2022 at 03:00:21PM +0200, Vlastimil Babka wrote: > > > > On 9/15/22 16:29, Chao Peng wrote: > > > > > From: "Kirill A. Shutemov" > > > > > > > > > > KVM can use memfd-provided memory for guest memory. For normal userspace > > > > > accessible memory, KVM userspace (e.g. QEMU) mmaps the memfd into its > > > > > virtual address space and then tells KVM to use the virtual address to > > > > > setup the mapping in the secondary page table (e.g. EPT). > > > > > > > > > > With confidential computing technologies like Intel TDX, the > > > > > memfd-provided memory may be encrypted with special key for special > > > > > software domain (e.g. KVM guest) and is not expected to be directly > > > > > accessed by userspace. Precisely, userspace access to such encrypted > > > > > memory may lead to host crash so it should be prevented. > > > > > > > > > > This patch introduces userspace inaccessible memfd (created with > > > > > MFD_INACCESSIBLE). Its memory is inaccessible from userspace through > > > > > ordinary MMU access (e.g. read/write/mmap) but can be accessed via > > > > > in-kernel interface so KVM can directly interact with core-mm without > > > > > the need to map the memory into KVM userspace. > > > > > > > > > > It provides semantics required for KVM guest private(encrypted) memory > > > > > support that a file descriptor with this flag set is going to be used as > > > > > the source of guest memory in confidential computing environments such > > > > > as Intel TDX/AMD SEV. > > > > > > > > > > KVM userspace is still in charge of the lifecycle of the memfd. It > > > > > should pass the opened fd to KVM. KVM uses the kernel APIs newly added > > > > > in this patch to obtain the physical memory address and then populate > > > > > the secondary page table entries. > > > > > > > > > > The userspace inaccessible memfd can be fallocate-ed and hole-punched > > > > > from userspace. When hole-punching happens, KVM can get notified through > > > > > inaccessible_notifier it then gets chance to remove any mapped entries > > > > > of the range in the secondary page tables. > > > > > > > > > > The userspace inaccessible memfd itself is implemented as a shim layer > > > > > on top of real memory file systems like tmpfs/hugetlbfs but this patch > > > > > only implemented tmpfs. The allocated memory is currently marked as > > > > > unmovable and unevictable, this is required for current confidential > > > > > usage. But in future this might be changed. > > > > > > > > > > Signed-off-by: Kirill A. Shutemov > > > > > Signed-off-by: Chao Peng > > > > > --- > > > > > > > > ... > > > > > > > > > +static long inaccessible_fallocate(struct file *file, int mode, > > > > > + loff_t offset, loff_t len) > > > > > +{ > > > > > + struct inaccessible_data *data = file->f_mapping->private_data; > > > > > + struct file *memfd = data->memfd; > > > > > + int ret; > > > > > + > > > > > + if (mode & FALLOC_FL_PUNCH_HOLE) { > > > > > + if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) > > > > > + return -EINVAL; > > > > > + } > > > > > + > > > > > + ret = memfd->f_op->fallocate(memfd, mode, offset, len); > > > > > + inaccessible_notifier_invalidate(data, offset, offset + len); > > > > > > > > Wonder if invalidate should precede the actual hole punch, otherwise we open > > > > a window where the page tables point to memory no longer valid? > > > > > > Yes, you are right. Thanks for catching this. > > > > I also noticed this. But then thought the memory would be anyways zeroed > > (hole punched) before this call? > > Hole punching can free pages, given that offset/len covers full page. > > -- > Kiryl Shutsemau / Kirill A. Shutemov I think moving this notifier_invalidate before fallocate may not solve the problem completely. Is it possible that between invalidate and fallocate, KVM tries to handle the page fault for the guest VM from another vcpu and uses the pages to be freed to back gpa ranges? Should hole punching here also update mem_attr first to say that KVM should consider the corresponding gpa ranges to be no more backed by inaccessible memfd?