From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE230C433FE for ; Tue, 15 Nov 2022 09:53:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 621876B0071; Tue, 15 Nov 2022 04:53:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F87B6B0072; Tue, 15 Nov 2022 04:53:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 498F96B0073; Tue, 15 Nov 2022 04:53:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 36BFA6B0071 for ; Tue, 15 Nov 2022 04:53:26 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0DC5380BD9 for ; Tue, 15 Nov 2022 09:53:26 +0000 (UTC) X-FDA: 80135214012.17.D38D8A5 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf10.hostedemail.com (Postfix) with ESMTP id 5C951C0005 for ; Tue, 15 Nov 2022 09:53:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668506004; x=1700042004; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=4aNJ2YOK3rk2Xxfz14Mm6Z9Et6IN8H9Ru4xzr7aFyfQ=; b=XNtcDrGEQfaH/zDGNLFeiImGCJdGSC3G3Pd6jN4IxwS9WubcmR6iwjOj 164y4hvpLcaXMMay8uSBo6bXo6Xara6/BFLZ0/i/sXmRdGUStiLmfmZXQ DRZiT5/FJ75vGRQS1yDCAn0qEkMihMTuo+kS6QjZT3exDPHQEightIq5G qiLn/duIzhlblmF+o9zU35eRepi/IfoOyYuNGxG79DSQSGPsyfKr/cBme WRUU8ZhcpzYwnf+AvrMgmuhmo7O+qZiMkaQMZW+08nnc8/YtWcRTf214q w+gpw7bMq2l3zpa/Z1DWjxe/4bQzZZJMaQp/hTka3PoNlyiSiZXXzJp5e A==; X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="292609052" X-IronPort-AV: E=Sophos;i="5.96,165,1665471600"; d="scan'208";a="292609052" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Nov 2022 01:53:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="702376949" X-IronPort-AV: E=Sophos;i="5.96,165,1665471600"; d="scan'208";a="702376949" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.193.75]) by fmsmga008.fm.intel.com with ESMTP; 15 Nov 2022 01:53:11 -0800 Date: Tue, 15 Nov 2022 17:48:46 +0800 From: Chao Peng To: Michael Roth Cc: "Kirill A. Shutemov" , Vlastimil Babka , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , tabba@google.com, mhocko@suse.com, Muchun Song , wei.w.wang@intel.com Subject: Re: [PATCH v9 1/8] mm: Introduce memfd_restricted system call to create restricted user memory Message-ID: <20221115094846.GB338422@chaop.bj.intel.com> Reply-To: Chao Peng References: <20221025151344.3784230-1-chao.p.peng@linux.intel.com> <20221025151344.3784230-2-chao.p.peng@linux.intel.com> <20221031174738.fklhlia5fmaiinpe@amd.com> <20221101113729.GA4015495@chaop.bj.intel.com> <20221101151944.rhpav47pdulsew7l@amd.com> <20a11042-2cfb-8f42-9d80-6672e155ca2c@suse.cz> <20221114152843.ylxe4dis254vrj5u@box.shutemov.name> <20221114221632.5xaz24adkghfjr2q@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221114221632.5xaz24adkghfjr2q@amd.com> ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=XNtcDrGE; spf=none (imf10.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668506005; a=rsa-sha256; cv=none; b=UDmlXohHiQFPPY85dJ8m1tTKz//iWBtesVO4xI3fe2pnhS5bbLm9zXFAjpWmnwjgJt+pZv Ki9uarjiLdG1HDw+hi3adsnd/YsEF5RLUPgjO3eLWdD9hdt2tdN0U5XXY/jkb6g1rCUVfW eRN90YVjB5tYiZcI/9Qc/VmcKr8YO5w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668506005; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8q+cds1R1mqiq4T3+9viKGyiy3dJZaT8Mcjt0HHJLzc=; b=iKjUqih3yuWhh7YiJ96ee7HySjYI67Hc5trs+VvfeCevv8F/8TBe6oqVD8IVy2yhmxcT/f dVyoiySEG1GQuBvyFUogkqY6/zKv5YydOI+hGWSVbRenUC/65sdyy4GLxjZU72jCu2t6a+ NiUNCCux85f5IVseiUrp5Ax5UkTSooo= X-Stat-Signature: 4jwuxug4t9xuztocpu7mgzjumiam48fe X-Rspamd-Queue-Id: 5C951C0005 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=XNtcDrGE; spf=none (imf10.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) X-Rspamd-Server: rspam11 X-HE-Tag: 1668506004-176714 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 14, 2022 at 04:16:32PM -0600, Michael Roth wrote: > On Mon, Nov 14, 2022 at 06:28:43PM +0300, Kirill A. Shutemov wrote: > > On Mon, Nov 14, 2022 at 03:02:37PM +0100, Vlastimil Babka wrote: > > > On 11/1/22 16:19, Michael Roth wrote: > > > > On Tue, Nov 01, 2022 at 07:37:29PM +0800, Chao Peng wrote: > > > >> > > > > >> > 1) restoring kernel directmap: > > > >> > > > > >> > Currently SNP (and I believe TDX) need to either split or remove kernel > > > >> > direct mappings for restricted PFNs, since there is no guarantee that > > > >> > other PFNs within a 2MB range won't be used for non-restricted > > > >> > (which will cause an RMP #PF in the case of SNP since the 2MB > > > >> > mapping overlaps with guest-owned pages) > > > >> > > > >> Has the splitting and restoring been a well-discussed direction? I'm > > > >> just curious whether there is other options to solve this issue. > > > > > > > > For SNP it's been discussed for quite some time, and either splitting or > > > > removing private entries from directmap are the well-discussed way I'm > > > > aware of to avoid RMP violations due to some other kernel process using > > > > a 2MB mapping to access shared memory if there are private pages that > > > > happen to be within that range. > > > > > > > > In both cases the issue of how to restore directmap as 2M becomes a > > > > problem. > > > > > > > > I was also under the impression TDX had similar requirements. If so, > > > > do you know what the plan is for handling this for TDX? > > > > > > > > There are also 2 potential alternatives I'm aware of, but these haven't > > > > been discussed in much detail AFAIK: > > > > > > > > a) Ensure confidential guests are backed by 2MB pages. shmem has a way to > > > > request 2MB THP pages, but I'm not sure how reliably we can guarantee > > > > that enough THPs are available, so if we went that route we'd probably > > > > be better off requiring the use of hugetlbfs as the backing store. But > > > > obviously that's a bit limiting and it would be nice to have the option > > > > of using normal pages as well. One nice thing with invalidation > > > > scheme proposed here is that this would "Just Work" if implement > > > > hugetlbfs support, so an admin that doesn't want any directmap > > > > splitting has this option available, otherwise it's done as a > > > > best-effort. > > > > > > > > b) Implement general support for restoring directmap as 2M even when > > > > subpages might be in use by other kernel threads. This would be the > > > > most flexible approach since it requires no special handling during > > > > invalidations, but I think it's only possible if all the CPA > > > > attributes for the 2M range are the same at the time the mapping is > > > > restored/unsplit, so some potential locking issues there and still > > > > chance for splitting directmap over time. > > > > > > I've been hoping that > > > > > > c) using a mechanism such as [1] [2] where the goal is to group together > > > these small allocations that need to increase directmap granularity so > > > maximum number of large mappings are preserved. > > > > As I mentioned in the other thread the restricted memfd can be backed by > > secretmem instead of plain memfd. It already handles directmap with care. > > It looks like it would handle direct unmapping/cleanup nicely, but it > seems to lack fallocate(PUNCH_HOLE) support which we'd probably want to > avoid additional memory requirements. I think once we added that we'd > still end up needing some sort of handling for the invalidations. > > Also, I know Chao has been considering hugetlbfs support, I assume by > leveraging the support that already exists in shmem. Ideally SNP would > be able to make use of that support as well, but relying on a separate > backend seems likely to result in more complications getting there > later. > > > > > But I don't think it has to be part of initial restricted memfd > > implementation. It is SEV-specific requirement and AMD folks can extend > > implementation as needed later. > > Admittedly the suggested changes to the invalidation mechanism made a > lot more sense to me when I was under the impression that TDX would have > similar requirements and we might end up with a common hook. Since that > doesn't actually seem to be the case, it makes sense to try to do it as > a platform-specific hook for SNP. > > I think, given a memslot, a GFN range, and kvm_restricted_mem_get_pfn(), > we should be able to get the same information needed to figure out whether > the range is backed by huge pages or not. I'll see how that works out > instead. Sounds a viable solution, just that kvm_restricted_mem_get_pfn() will only give you the ability to check a page, not a range. But you can still call it many times I think. The invalidation callback will be still needed, it gives you the chance to do the restoring. Chao > > Thanks, > > Mike > > > > > -- > > Kiryl Shutsemau / Kirill A. Shutemov