From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BA0BC761A6 for ; Tue, 28 Mar 2023 10:49:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9213E6B0072; Tue, 28 Mar 2023 06:49:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D1496B0074; Tue, 28 Mar 2023 06:49:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7723C6B0075; Tue, 28 Mar 2023 06:49:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6804A6B0072 for ; Tue, 28 Mar 2023 06:49:14 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1DF7880854 for ; Tue, 28 Mar 2023 10:49:14 +0000 (UTC) X-FDA: 80617985028.05.9EE0327 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by imf18.hostedemail.com (Postfix) with ESMTP id A215C1C0005 for ; Tue, 28 Mar 2023 10:49:11 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=UYvQ2VXj; spf=none (imf18.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.31) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680000551; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LGTyogjsCd2AkFUAXz61rZhAR2rcPBDoTZdLie7kVok=; b=PD49s7AnLsg5KFPDj5UrDJXNM7HA5KHF/NeuYrT5VBwDO/m24aq6ER2ALbsawmjdB3lIrN RePk+rykKj7XDkXpGvqA2Ed1yHWaXYG+nBKpFv1y2pZ5xNGjFoQis6poSrxZDkjqDCsmtj Lxm3YDhvluG2yhu021Jkcf/AV3tfEFw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=UYvQ2VXj; spf=none (imf18.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.31) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680000551; a=rsa-sha256; cv=none; b=LxHQsq4j+3argfghD/QeJtyW4c8JeRgqrsyWGS/u/hRDsU6wemrTuXmwgNFkc15Hqqizwb xEOPK58vU+BUJqS7QbiUwiQSiSjGc8NLrgIMh1159CX2SMuOYfy2QRhMm0jVe1vScH6XDy pkz3ocNh90FFFaAg8dUApD9CJ+aCDCs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680000551; x=1711536551; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=AseN6rlr5gwMYM5zhMJq+cjPxZAZKyFMZLKoTjj2S7w=; b=UYvQ2VXjA/JhNqGTnIKTploBocaDba7HCXNlAVomuPc3v15IlEXWnAJb 5z3VU2VLBAIx7C899WFNdRHn3/L5A2z5AnXOsq1AgceRUYbdm7kj2hEFL 03YOSB86q+Pw0QGll2vdPs09KWdOnKS8+h3sso3hVseCeoyFqa1BNDfDp gg3h5+wsnNZs4SYi71QbF9zU2bQV+vpSHv8Zve0jGm1A6Jk6nFr3GlJug SVOB9SScuelfcdQmD1pRHtyncLknBP8ETCq5tfgq+HjUDSYW3IwLzl7Wt a10kDf0mbGdwaLIwF/fU5HltiMxLZKBM+d26iT5jLH/BpVPJCXBczp7D0 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10662"; a="403144899" X-IronPort-AV: E=Sophos;i="5.98,297,1673942400"; d="scan'208";a="403144899" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2023 03:48:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10662"; a="794757674" X-IronPort-AV: E=Sophos;i="5.98,297,1673942400"; d="scan'208";a="794757674" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.192.105]) by fmsmga002.fm.intel.com with ESMTP; 28 Mar 2023 03:48:42 -0700 Date: Tue, 28 Mar 2023 18:41:08 +0800 From: Chao Peng To: Xiaoyao Li Cc: Isaku Yamahata , Ackerley Tng , seanjc@google.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, pbonzini@redhat.com, corbet@lwn.net, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, arnd@arndb.de, naoya.horiguchi@nec.com, linmiaohe@huawei.com, x86@kernel.org, hpa@zytor.com, hughd@google.com, jlayton@kernel.org, bfields@fieldses.org, akpm@linux-foundation.org, shuah@kernel.org, rppt@kernel.org, steven.price@arm.com, mail@maciej.szmigiero.name, vbabka@suse.cz, vannapurve@google.com, yu.c.zhang@linux.intel.com, kirill.shutemov@linux.intel.com, luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, qperret@google.com, tabba@google.com, michael.roth@amd.com, mhocko@suse.com, wei.w.wang@intel.com Subject: Re: [PATCH v10 9/9] KVM: Enable and expose KVM_MEM_PRIVATE Message-ID: <20230328104108.GB2909606@chaop.bj.intel.com> Reply-To: Chao Peng References: <20230128140030.GB700688@chaop.bj.intel.com> <20230308074026.GA2183207@chaop.bj.intel.com> <20230323004131.GA214881@ls.amr.corp.intel.com> <20230324021029.GA2774613@chaop.bj.intel.com> <6cf365a3-dddc-8b74-4d74-04666fbeb53d@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6cf365a3-dddc-8b74-4d74-04666fbeb53d@intel.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A215C1C0005 X-Rspam-User: X-Stat-Signature: eandiyurgfw7cpnhghi9ud1unuansfde X-HE-Tag: 1680000551-905757 X-HE-Meta: U2FsdGVkX1+dnzbP5gKrjT0G4jLMjWSyajsO4SDsgEvJZ1hGJrjyHdrwc6ITw0OVxSA3xlZ8OtOLRawRCzy243H76rcW+q2O0sTm0vZjRZCK3V086KLyRwKOTEaIi95+WNXfVw8Wz9behbGz39QAIy3HTgKilYYx31IgAm69kKrSpYQhQVosrPw280bH3xUSQWgJs3QM9nmxDg7NPDpiCxCcfItzOZ/+b1/RsllVkHGK9e2plBWLORp6JaXp+97rVE2DuVmfQJBSLSTbcbW5g1J0QXx8+qAMC8zD/jTAFZ8YgeNS2xEGUmRieKPByTG1noaI4bMzP5UhKEaunFPuhtfZkk9tx+Q+A8JwpWCvqolJS7X7Wo8OJyWgfiQQN3x+qJSvO/a5NQJgAKPOF/2KQdYJgqOJYKleBqDV2DnYQNoN1Qdi4+Hpkjxxf2HhMuhGwaKzF1kpIXdqKAM/Snd0wI6mnXgLqnM5QtE8w7KhKC+xQyP5X6WZJPuihTCS0zrtv4xUqZYxCOuJO6unsXCAYFf+D6GVpsdvMS5cWFmGX2q72Ri4qoqV6ltPyWy02U8pBt9ZcKHbxmSuAtOgr4fnS3VA3Cjhxmhvv3YzEB8goqVu6eQaqYAC2rPL+L9yllUyV9W39StA6csbHYsGvSMEBUCkxNvsdQ2vehfky0LrendUCoo5i0oEzRts11uYjL5LnGyF9rCJNAmnS5AbQDTlyDw/TPo03+M5CG93jH/I9PUhXr9q+7aJ4hWaR9h2EBHfRpVuTOgHX9qFxvYH/8Wk6V7xfd5lOv++z5UOVy+UGbneOEt7sgNs/I+hbgAO6Qqn5Koc6qyD3vXQfSGn7/dJwkPzTaegp8GbYP4g31CQCvN3CurMFovfmKbCUbXSKmG0XFyvnbCAA7ZLVb08124ZIRX1yyvhxfh6wc4xkKFMLcnu796l4XbJItyiMRUoKONLv+ABRrwmo3tZsaLHNcC d1L8pw+o tRAs205mcBolv+hxjyY3NlJGQqNLGEaDE2cmWxRgzjjK5sQU/2YjlWLGPye1p4abjVkp+TW9owKlncvYvzJfOcxaNIIKkrBTvetffMg7eHFjLlBG52bywAMgY9WzgbZO8pWBdWCMBgZtUFfu7pM4JHnLLPMMDTJCkV0pmLnIUf95Q7h2gn6OZdL80Ir4weMj4qPWB2WHBQg64HyzkkTd12gJkJAXIRAkPxhTPDDTj+YbNRlgr6+FfsjsFQ/wFKKZH0N0NA/3yEA0AK2iCm3iAJyeGWu9RMWvD+MaywlSBgVA1B53ySmDlSFEy2KC/qYc+wb3/4PPzcqbEeKGJfyOzeT3C04vAifgqLSVTSDqCKDcyj8J85/Z1hfA/JcqwijUy9LvCD1m7fRMcq+2s7h0mE7ddnbgftyxL7NtoJG1V3PdpZrxvHEMymobcOC4nX92jxtwU/u1WRb4wod6LYjeJe8RsoxcFAPhTXMYQSxr+mH0ZqSGqHH+tgr5DAgRicRGiL81HVedLea/N5o5tZ9+ie97kcu4ki+Gtlv9kv+GDlnnvZhLWcy9kB3rBebeNFW10YLPFo239YqhE2dGVLZIzPToksA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 24, 2023 at 10:29:25AM +0800, Xiaoyao Li wrote: > On 3/24/2023 10:10 AM, Chao Peng wrote: > > On Wed, Mar 22, 2023 at 05:41:31PM -0700, Isaku Yamahata wrote: > > > On Wed, Mar 08, 2023 at 03:40:26PM +0800, > > > Chao Peng wrote: > > > > > > > On Wed, Mar 08, 2023 at 12:13:24AM +0000, Ackerley Tng wrote: > > > > > Chao Peng writes: > > > > > > > > > > > On Sat, Jan 14, 2023 at 12:01:01AM +0000, Sean Christopherson wrote: > > > > > > > On Fri, Dec 02, 2022, Chao Peng wrote: > > > > > > ... > > > > > > > Strongly prefer to use similar logic to existing code that detects wraps: > > > > > > > > > > > > mem->restricted_offset + mem->memory_size < mem->restricted_offset > > > > > > > > > > > > This is also where I'd like to add the "gfn is aligned to offset" > > > > > > > check, though > > > > > > > my brain is too fried to figure that out right now. > > > > > > > > > > > Used count_trailing_zeros() for this TODO, unsure we have other better > > > > > > approach. > > > > > > > > > > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > > > > > index afc8c26fa652..fd34c5f7cd2f 100644 > > > > > > --- a/virt/kvm/kvm_main.c > > > > > > +++ b/virt/kvm/kvm_main.c > > > > > > @@ -56,6 +56,7 @@ > > > > > > #include > > > > > > #include > > > > > > #include > > > > > > +#include > > > > > > > > > > > #include "coalesced_mmio.h" > > > > > > #include "async_pf.h" > > > > > > @@ -2087,6 +2088,19 @@ static bool kvm_check_memslot_overlap(struct > > > > > > kvm_memslots *slots, int id, > > > > > > return false; > > > > > > } > > > > > > > > > > > +/* > > > > > > + * Return true when ALIGNMENT(offset) >= ALIGNMENT(gpa). > > > > > > + */ > > > > > > +static bool kvm_check_rmem_offset_alignment(u64 offset, u64 gpa) > > > > > > +{ > > > > > > + if (!offset) > > > > > > + return true; > > > > > > + if (!gpa) > > > > > > + return false; > > > > > > + > > > > > > + return !!(count_trailing_zeros(offset) >= count_trailing_zeros(gpa)); > > > > > > This check doesn't work expected. For example, offset = 2GB, gpa=4GB > > > this check fails. > > > > This case is expected to fail as Sean initially suggested[*]: > > I would rather reject memslot if the gfn has lesser alignment than > > the offset. I'm totally ok with this approach _if_ there's a use case. > > Until such a use case presents itself, I would rather be conservative > > from a uAPI perspective. > > > > I understand that we put tighter restriction on this but if you see such > > restriction is really a big issue for real usage, instead of a > > theoretical problem, then we can loosen the check here. But at that time > > below code is kind of x86 specific and may need improve. > > > > BTW, in latest code, I replaced count_trailing_zeros() with fls64(): > > return !!(fls64(offset) >= fls64(gpa)); > > wouldn't it be !!(ffs64(offset) <= ffs64(gpa)) ? As the function document explains, here we want to return true when ALIGNMENT(offset) >= ALIGNMENT(gpa), so '>=' is what we need. It's worthy clarifying that in Sean's original suggestion he actually mentioned the opposite. He said 'reject memslot if the gfn has lesser alignment than the offset', but I wonder this is his purpose, since if ALIGNMENT(offset) < ALIGNMENT(gpa), we wouldn't be possible to map the page as largepage. Consider we have below config: gpa=2M, offset=1M In this case KVM tries to map gpa at 2M as 2M hugepage but the physical page at the offset(1M) in private_fd cannot provide the 2M page due to misalignment. But as we discussed in the off-list thread, here we do find a real use case indicating this check is too strict. i.e. QEMU immediately fails when launch a guest > 2G memory. For this case QEMU splits guest memory space into two slots: Slot#1(ram_below_4G): gpa=0x0, offset=0x0, size=2G Slot#2(ram_above_4G): gpa=4G, offset=2G, size=totalsize-2G This strict alignment check fails for slot#2 because offset(2G) has less alignment than gpa(4G). To allow this, one solution can revert to my previous change in kvm_alloc_memslot_metadata() to disallow hugepage only when the offset/gpa are not aligned to related page size. Sean, How do you think? Chao > > > [*] https://lore.kernel.org/all/Y8HldeHBrw+OOZVm@google.com/ > > > > Chao > > > I come up with the following. > > > > > > >From ec87e25082f0497431b732702fae82c6a05071bf Mon Sep 17 00:00:00 2001 > > > Message-Id: > > > From: Isaku Yamahata > > > Date: Wed, 22 Mar 2023 15:32:56 -0700 > > > Subject: [PATCH] KVM: Relax alignment check for restricted mem > > > > > > kvm_check_rmem_offset_alignment() only checks based on offset alignment > > > and GPA alignment. However, the actual alignment for offset depends > > > on architecture. For x86 case, it can be 1G, 2M or 4K. So even if > > > GPA is aligned for 1G+, only 1G-alignment is required for offset. > > > > > > Without this patch, gpa=4G, offset=2G results in failure of memory slot > > > creation. > > > > > > Fixes: edc8814b2c77 ("KVM: Require gfn be aligned with restricted offset") > > > Signed-off-by: Isaku Yamahata > > > --- > > > arch/x86/include/asm/kvm_host.h | 15 +++++++++++++++ > > > virt/kvm/kvm_main.c | 9 ++++++++- > > > 2 files changed, 23 insertions(+), 1 deletion(-) > > > > > > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h > > > index 88e11dd3afde..03af44650f24 100644 > > > --- a/arch/x86/include/asm/kvm_host.h > > > +++ b/arch/x86/include/asm/kvm_host.h > > > @@ -16,6 +16,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include > > > #include > > > @@ -143,6 +144,20 @@ > > > #define KVM_HPAGE_MASK(x) (~(KVM_HPAGE_SIZE(x) - 1)) > > > #define KVM_PAGES_PER_HPAGE(x) (KVM_HPAGE_SIZE(x) / PAGE_SIZE) > > > +#define kvm_arch_required_alignment kvm_arch_required_alignment > > > +static inline int kvm_arch_required_alignment(u64 gpa) > > > +{ > > > + int zeros = count_trailing_zeros(gpa); > > > + > > > + WARN_ON_ONCE(!PAGE_ALIGNED(gpa)); > > > + if (zeros >= KVM_HPAGE_SHIFT(PG_LEVEL_1G)) > > > + return KVM_HPAGE_SHIFT(PG_LEVEL_1G); > > > + else if (zeros >= KVM_HPAGE_SHIFT(PG_LEVEL_2M)) > > > + return KVM_HPAGE_SHIFT(PG_LEVEL_2M); > > > + > > > + return PAGE_SHIFT; > > > +} > > > + > > > #define KVM_MEMSLOT_PAGES_TO_MMU_PAGES_RATIO 50 > > > #define KVM_MIN_ALLOC_MMU_PAGES 64UL > > > #define KVM_MMU_HASH_SHIFT 12 > > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > > index c9c4eef457b0..f4ff96171d24 100644 > > > --- a/virt/kvm/kvm_main.c > > > +++ b/virt/kvm/kvm_main.c > > > @@ -2113,6 +2113,13 @@ static bool kvm_check_memslot_overlap(struct kvm_memslots *slots, int id, > > > return false; > > > } > > > +#ifndef kvm_arch_required_alignment > > > +__weak int kvm_arch_required_alignment(u64 gpa) > > > +{ > > > + return PAGE_SHIFT > > > +} > > > +#endif > > > + > > > /* > > > * Return true when ALIGNMENT(offset) >= ALIGNMENT(gpa). > > > */ > > > @@ -2123,7 +2130,7 @@ static bool kvm_check_rmem_offset_alignment(u64 offset, u64 gpa) > > > if (!gpa) > > > return false; > > > - return !!(count_trailing_zeros(offset) >= count_trailing_zeros(gpa)); > > > + return !!(count_trailing_zeros(offset) >= kvm_arch_required_alignment(gpa)); > > > } > > > /* > > > -- > > > 2.25.1 > > > > > > > > > > > > -- > > > Isaku Yamahata