From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1434C77B79 for ; Fri, 14 Apr 2023 21:08:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE4A36B0075; Fri, 14 Apr 2023 17:08:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C95176B0078; Fri, 14 Apr 2023 17:08:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B83FF900003; Fri, 14 Apr 2023 17:08:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A93E66B0075 for ; Fri, 14 Apr 2023 17:08:13 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7F3001C621F for ; Fri, 14 Apr 2023 21:08:13 +0000 (UTC) X-FDA: 80681234466.14.E09C26D Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf08.hostedemail.com (Postfix) with ESMTP id C80F216001B for ; Fri, 14 Apr 2023 21:08:11 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Z5HvJTut; spf=pass (imf08.hostedemail.com: domain of 3usA5ZAYKCEs5rn0wpt11tyr.p1zyv07A-zzx8npx.14t@flex--seanjc.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3usA5ZAYKCEs5rn0wpt11tyr.p1zyv07A-zzx8npx.14t@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681506491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b7Jr+FjVXuQEnXDMlE3gznMa+dmQespZNtp7nLwMnLo=; b=y/v27W9cc8vHoZyZjrjjS30YX+gLEvKxaULYun4Ka/w8sA0mLt1ZhiW4AT7j99GAhhhkff Oml+419zbT2AVpb9Sf+dSjAEsutMXuJFDsEsPZMaGlY1/OUKJCFH8Uspi6ZYcOSWMx/UxI zbo1NAuqzcRe4GJQljaj6164wk4TJN0= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Z5HvJTut; spf=pass (imf08.hostedemail.com: domain of 3usA5ZAYKCEs5rn0wpt11tyr.p1zyv07A-zzx8npx.14t@flex--seanjc.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3usA5ZAYKCEs5rn0wpt11tyr.p1zyv07A-zzx8npx.14t@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681506491; a=rsa-sha256; cv=none; b=J6FqEpdbpcIaTF0sJqBT5xIWhf7Xk8FWBX4XKQh0x1XhVSNaQl5vj1j4Oc7Ow9ncM8GZwt zbJwKMxhZsyIo93EIDZzblFE25qvfON4rV21YqfVvlE5Ucts5w8ZrSl0ppyuB675TnnspW VOup3n/SsJZJxjIzAou3ACnky3rKcP0= Received: by mail-pl1-f201.google.com with SMTP id p14-20020a170902a40e00b001a666857bd6so5072861plq.16 for ; Fri, 14 Apr 2023 14:08:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681506490; x=1684098490; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=b7Jr+FjVXuQEnXDMlE3gznMa+dmQespZNtp7nLwMnLo=; b=Z5HvJTutAVSami8OKTwQ0L/Tb2qB7TzKRMxFI28D2AJ1QCpuyyvVHOIi9Fb3XHR9ej LUNfDfNV6JeVjW3E90abkAcvZRFvh1n+aONtio3M8f0noPTpRbjMmxowI+9sS4Xq8hOM Sf8ikJlSVrVt4ndDSkdOQC01mNz5LJQqb5MN5v49d2Xg9pi+/EcTwfMmakOAtcymdSPB 3uhaOLkyYQ+M+WsF5l8y2hIZEq9pNWsbaA7iwUmYpJ328Zz4BxjSvHOlfXGK+mUVM1vi r2OSTO5kfzooBMyGZCEiZPK8V4lIN0PFu5BvTJM4WCQKYevPaAgKuJZuB8K2huoAWqrx vn+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681506490; x=1684098490; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=b7Jr+FjVXuQEnXDMlE3gznMa+dmQespZNtp7nLwMnLo=; b=kVHNnvjJsghWGkQ4/aklPQ/pn972vsOaBkYBxUF9jXzKCuWUDm0iT8iQrdkwekAj3Z E5PxcTcLNo1jH8SgwtxrpymkM1e4p4dGj+Ws6HjD9c8JzprrvQ88n5vwTmNDKuFsyoSo 1hR6aPCD/bylh96KLbTZpNDyGngwWYiXwB9tdireg3aQ/vRmVEb7HIdB7iCHbBM/Q5ks eV1H8NaPKBeQUdwQ1Ai9xbpew5WAYPs/AsKFECodUKO4yYnh9MRgoj+4GcXf1h9QXnGe OdeuS+VclCM1D+9hnBCi7XeL2EhW6eTcf2jB2eZazFxfvRFTbgUh3rQjFTYvGcSC+egN kQiQ== X-Gm-Message-State: AAQBX9cz8TTkNG7Ij1Gbs8TUw1zE/qrLjeOt2MrkHIA7KQtZva5ncH6t 2SzlBJiMGJ8Na6ufdk5b8W3rlg6bo+w= X-Google-Smtp-Source: AKy350bBbN9SHkQpgaNZinbD0G0RBAygFGxlP9+ytFbgc7OhIviX9khLc+w+xI6iW/tqbkBNGnqofgsbIU0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a63:58a:0:b0:513:57ce:d61c with SMTP id 132-20020a63058a000000b0051357ced61cmr1120918pgf.7.1681506490508; Fri, 14 Apr 2023 14:08:10 -0700 (PDT) Date: Fri, 14 Apr 2023 14:08:08 -0700 In-Reply-To: <20230328104108.GB2909606@chaop.bj.intel.com> Mime-Version: 1.0 References: <20230128140030.GB700688@chaop.bj.intel.com> <20230308074026.GA2183207@chaop.bj.intel.com> <20230323004131.GA214881@ls.amr.corp.intel.com> <20230324021029.GA2774613@chaop.bj.intel.com> <6cf365a3-dddc-8b74-4d74-04666fbeb53d@intel.com> <20230328104108.GB2909606@chaop.bj.intel.com> Message-ID: Subject: Re: [PATCH v10 9/9] KVM: Enable and expose KVM_MEM_PRIVATE From: Sean Christopherson To: Chao Peng Cc: Xiaoyao Li , Isaku Yamahata , Ackerley Tng , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, pbonzini@redhat.com, corbet@lwn.net, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, arnd@arndb.de, naoya.horiguchi@nec.com, linmiaohe@huawei.com, x86@kernel.org, hpa@zytor.com, hughd@google.com, jlayton@kernel.org, bfields@fieldses.org, akpm@linux-foundation.org, shuah@kernel.org, rppt@kernel.org, steven.price@arm.com, mail@maciej.szmigiero.name, vbabka@suse.cz, vannapurve@google.com, yu.c.zhang@linux.intel.com, kirill.shutemov@linux.intel.com, luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, qperret@google.com, tabba@google.com, michael.roth@amd.com, mhocko@suse.com, wei.w.wang@intel.com Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: C80F216001B X-Stat-Signature: urk5jd7ctpei5eyust33o7szyt5nabay X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1681506491-541680 X-HE-Meta: U2FsdGVkX19EwWWtnD4WPbHSpQUa06gJZAqFYnJ5N/Lmn/mxS17zYvH9lv62GmBzpv6RDcQFcyhRbofTA3I0C8Ew4RBfpHXNpmQf5rM3TqTNE0sm9H9DE4/Y6lIL78KquUHToOpugm2Vn3MLBlXrtEWJCqOJllq/kQ9ZWPGCQIBlEHBkrJI3FEKpNAt8b1lFu7AJc8CSKoZUNDfoP73nrsjIfBJjBHasQLWWjIAR8eQRhTLb5f8Yt9fJlZ8HVRmzwj+rtHtGGB6IhSLoULoxPLvqH9bkVpq4iodoYTn2eU57clSEg7D+lH70kuLFwP3FOTNLZXnqlfKHdnLSXiYEzPGeUbe7JuoiQkCnV4l+06VZbXGeTu/nH/sJgYM9ivTI75eSWLafvX1yTu9BDyCUoeywrwt0xVw3GphNuPnXVJsuCHgeQK+eIhI03Auy8g9arfRrPV0qmw/Vqa8zVob3ld7I73Hewr7mI1RJuYWIacvev7NrJ/08fjROmwVeZXI0Y3V8IGLHCrlRfKsLhd2z7Jsl/CJ33GDiHygFEWqXDXQ79wx6l/19ucRnb6fn2dFqhD9K172CGW0yLsen9t1w77AJmC57QI2tkcydJDI51TI2or/VZxwtwRd/+CuC5OjI9GwF1qJjaxfaRzU7QiVu4yJ2nKctFJPpNiay+ZD3JwdZOsvXwLSsx2ZHstbQzg+6dGsm/RYM2LvAw2iTIp0cEHx9IUXKoo/ZooHy8aj859+zu3ZAS9r8j/2qrPjUJaE2StycuI/Sp+d40B8MI0q+pwqKQuOVP9bJjFwcXdYHUw4hrg9tN5Mpb/hplbBZbMo5EeLoVYcTqf4aD8LF7G7KcLe8L4aLCDEEvVlxswA+O7snybejt5gtkYYnv/rwHGS1/2q1Ct12yJROsWyUmeQtDNBbQBPEl1+oOUW+Vn3Z5GIxxDCE5mWvffHRZWXuD76VmXrycTINiCv0gkSFdgv 4bnLegXm lb/L+9RHPI3VgJ4NoK3gfdVHrrowsPdI0W9rLzhxx4VXov7Yvnq8/RqPqJ9aV7tc8PTKPVTxV0blP/3aHmvq41+aBhfJeCC3XOJEDt48z38DCFjvuqD50kklJ1sToTrQrOoYvLQgrFop7GoWg7NRF2K2G8PO+0+jvGjhU46hBsEN+1byT2VJ6AqY2f1yVtfP3ETVWKCCrQfL408ooRKOyoe3a+HDKAtYPVgCg0E+kMfJ05hoC50pPxZQ9mrnjr3WZVfZVP0no2OiHvBwRVdvk9BwYobCA7Wr5eBRz5WKtOlsEnl6nOmWBm8097LZx1xfhAy4Fmu28t69FssigOgeFWutAN1LHq4yZGW0ApHIQcHrxwd71rOIOhfLHXN6+DARyTvbCwIHMVh2agCsFDncFsd6lBhZMOd/yVXhYX/e47T2Sw1oXPlP6iNq3uaBKeiRaOW9o8gfWisZqdFQWaSQqpHPcOHiI2QGlp1O1bqdafhlepqW37OxNmaU7Ui9JTMs/KqzNOiQrSOvawLwhyMH2HEhhXm0YD82XL2ElNnIiTesQSyGw321DDDuIZtDbBeN1EdJX+zZJ4ZtS9CrwevCy/JlsJ1VNsdFqLc3y7gxC1S2KJwG47kZlwFnQbdDvPbhudRf4mYcRsK6vZZkd4MiNQYUisfQ3yzF/hVSo0kpJxDJ+KryK5Ngr6aD9RSCr9PSRO5tl1DkNzcZ9VtK0+e+wngS8Nm1LvInUZ039ZJV+grDoFvD8J8oKMEcHna4XVc543yG6ydiNjfSO1yuzNHSuqWPoGg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000216, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 28, 2023, Chao Peng wrote: > On Fri, Mar 24, 2023 at 10:29:25AM +0800, Xiaoyao Li wrote: > > On 3/24/2023 10:10 AM, Chao Peng wrote: > > > On Wed, Mar 22, 2023 at 05:41:31PM -0700, Isaku Yamahata wrote: > > > > On Wed, Mar 08, 2023 at 03:40:26PM +0800, > > > > Chao Peng wrote: > > > > > > > > > On Wed, Mar 08, 2023 at 12:13:24AM +0000, Ackerley Tng wrote: > > > > > > Chao Peng writes: > > > > > > > > > > > > > On Sat, Jan 14, 2023 at 12:01:01AM +0000, Sean Christopherson wrote: > > > > > > > > On Fri, Dec 02, 2022, Chao Peng wrote: > > > > > > > +static bool kvm_check_rmem_offset_alignment(u64 offset, u64 gpa) > > > > > > > +{ > > > > > > > + if (!offset) > > > > > > > + return true; > > > > > > > + if (!gpa) > > > > > > > + return false; > > > > > > > + > > > > > > > + return !!(count_trailing_zeros(offset) >= count_trailing_zeros(gpa)); > > > > > > > > This check doesn't work expected. For example, offset = 2GB, gpa=4GB > > > > this check fails. > > > > > > This case is expected to fail as Sean initially suggested[*]: > > > I would rather reject memslot if the gfn has lesser alignment than > > > the offset. I'm totally ok with this approach _if_ there's a use case. > > > Until such a use case presents itself, I would rather be conservative > > > from a uAPI perspective. > > > > > > I understand that we put tighter restriction on this but if you see such > > > restriction is really a big issue for real usage, instead of a > > > theoretical problem, then we can loosen the check here. But at that time > > > below code is kind of x86 specific and may need improve. > > > > > > BTW, in latest code, I replaced count_trailing_zeros() with fls64(): > > > return !!(fls64(offset) >= fls64(gpa)); > > > > wouldn't it be !!(ffs64(offset) <= ffs64(gpa)) ? > > As the function document explains, here we want to return true when > ALIGNMENT(offset) >= ALIGNMENT(gpa), so '>=' is what we need. > > It's worthy clarifying that in Sean's original suggestion he actually > mentioned the opposite. He said 'reject memslot if the gfn has lesser > alignment than the offset', but I wonder this is his purpose, since > if ALIGNMENT(offset) < ALIGNMENT(gpa), we wouldn't be possible to map > the page as largepage. Consider we have below config: > > gpa=2M, offset=1M > > In this case KVM tries to map gpa at 2M as 2M hugepage but the physical > page at the offset(1M) in private_fd cannot provide the 2M page due to > misalignment. > > But as we discussed in the off-list thread, here we do find a real use > case indicating this check is too strict. i.e. QEMU immediately fails > when launch a guest > 2G memory. For this case QEMU splits guest memory > space into two slots: > > Slot#1(ram_below_4G): gpa=0x0, offset=0x0, size=2G > Slot#2(ram_above_4G): gpa=4G, offset=2G, size=totalsize-2G > > This strict alignment check fails for slot#2 because offset(2G) has less > alignment than gpa(4G). To allow this, one solution can revert to my > previous change in kvm_alloc_memslot_metadata() to disallow hugepage > only when the offset/gpa are not aligned to related page size. > > Sean, How do you think? I agree, a pure alignment check is too restrictive, and not really what I intended despite past me literally saying that's what I wanted :-) I think I may have also inverted the "less alignment" statement, but luckily I believe that ends up being a moot point. The goal is to avoid having to juggle scenarios where KVM wants to create a hugepage, but restrictedmem can't provide one because of a misaligned file offset. I think the rule we want is that the offset must be aligned to the largest page size allowed by the memslot _size_. E.g. on x86, if the memslot size is >=1GiB then the offset must be 1GiB or beter, ditto for >=2MiB and >=4KiB (ignoring that 4KiB is already a requirement). We could loosen that to say the largest size allowed by the memslot, but I don't think that's worth the effort unless it's trivially easy to implement in code, e.g. KVM could technically allow a 4KiB aligned offset if the memslot is 2MiB sized but only 4KiB aligned on the GPA. I doubt there's a real use case for such a memslot, so I want to disallow that unless it's super easy to implement.