From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43CE5C61DA3 for ; Tue, 21 Feb 2023 12:19:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 927256B0075; Tue, 21 Feb 2023 07:19:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B0106B0078; Tue, 21 Feb 2023 07:19:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 777CA6B007B; Tue, 21 Feb 2023 07:19:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 626E06B0075 for ; Tue, 21 Feb 2023 07:19:32 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 38D3B1606D9 for ; Tue, 21 Feb 2023 12:19:32 +0000 (UTC) X-FDA: 80491204584.27.AC70DB1 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf23.hostedemail.com (Postfix) with ESMTP id D131E140004 for ; Tue, 21 Feb 2023 12:19:28 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CIUwa839; spf=none (imf23.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.126) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676981969; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gn1XPx+/KHQbO/aSUiBifjJsyQexH9/bzkjkosqlZKA=; b=mQbi+1u3wvMR9+DawHr+zV4njBZpeoGHECd7AnBHuuRSaXCbrCFxyc6nFXUsmOd4S+UvZd pM/Lhu17w5/lR9z6mjku/qjCessgeVxVs42/Gvlxv1GIUejKrUSFJT4n2CzeB+omw7FEIv 96WMRFqO6tydDPFBtCtxL5XmEWUz7Uk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CIUwa839; spf=none (imf23.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.126) smtp.mailfrom=chao.p.peng@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676981969; a=rsa-sha256; cv=none; b=D1jFxksXpwxXeY7HLruACP1LMUNnFLnl5timJ0BmtMmREZqtSzJ52kv/dzvHSpnB2iLAZx YLURBLoaDF8B4WDqg9IQOaW1damQJI5CZQqew2eM6aMJ7bETme0bCLmq62RfqpRGHTP4L8 uMU2b1Un0GvwuRAOtK5N/oM/V0haV7Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676981968; x=1708517968; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=rNp1uu81FPNgOnZRpB88mbF0GVkK0krzknn+hkCEz0w=; b=CIUwa839qcXxI069niReTswrfK4ytWHiOmF5vZwTD/xDJykACYOgrqbX 5hOsHzMXnoM9opr6DHUp4kDM1pm+dZtA0mqJROcDfsKQFgOm8SjDqAdc/ lTol3N/ca/VVb/tK+3nHSGn5GUe7YtU1/q1UyahD8Xfa4oKNBnNBgNZxc BUcHGdqav1TU4ud9ln0kiCN4xLeo/9Qq52PtQI3uJYUjYceO+2M5/gOpk X0c9ZxQmnUj0g6xfBa7GSPmkmKV1jEI6zZWrQD8xf9fSGGezu0iOTDK0p +YVYwfIlvNt0Go0E7Cub2OSOx7yM/OxdYMMGpWvqQ7CU5dKAk74Ptx6kP g==; X-IronPort-AV: E=McAfee;i="6500,9779,10627"; a="316337827" X-IronPort-AV: E=Sophos;i="5.97,315,1669104000"; d="scan'208";a="316337827" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2023 04:19:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10627"; a="917165304" X-IronPort-AV: E=Sophos;i="5.97,315,1669104000"; d="scan'208";a="917165304" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.192.105]) by fmsmga006.fm.intel.com with ESMTP; 21 Feb 2023 04:19:16 -0800 Date: Tue, 21 Feb 2023 20:11:35 +0800 From: Chao Peng To: Michael Roth Cc: Sean Christopherson , Isaku Yamahata , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Arnd Bergmann , Naoya Horiguchi , Miaohe Lin , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , tabba@google.com, mhocko@suse.com, wei.w.wang@intel.com Subject: Re: [PATCH v10 0/9] KVM: mm: fd-based approach for supporting KVM Message-ID: <20230221121135.GA1595130@chaop.bj.intel.com> Reply-To: Chao Peng References: <20221202061347.1070246-1-chao.p.peng@linux.intel.com> <20230119111308.GC2976263@ls.amr.corp.intel.com> <20230119223704.GD2976263@ls.amr.corp.intel.com> <20230213130102.two7q3kkcf254uof@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230213130102.two7q3kkcf254uof@amd.com> X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: peyh4r8wsyrwzfyrgx9qaoqu1kaom66z X-Rspamd-Queue-Id: D131E140004 X-HE-Tag: 1676981968-112029 X-HE-Meta: U2FsdGVkX1+FN9Aaujy4RDpvWGZi+JgaPctwBtYiAhFUohN8gZNWE3GbYVuQxxWp33uMD9KJ3OCjc/0rc2utTirMx1H3gqGFoK4fsZ9nT10Rlz5fW6dHApsYI7g6GKJO5xRTBfBreGp0Oq7uN9yzY7IPYpcwv35/KQPmlqEeYRYMmU3eBk5H8yYNQ4k8YhHdGz/UgMS1f77S7HmIIzqipcmZCn5B51f5dXlqmFqDANkyQnATQv4GkMNySYncqk7gsh7rFVrvZOvzAUg0DBjpJoL5Fuks6mhkQ7TRbeDfUf2DnEpw7bPREq7zRB/77FWgU3OT+uRszRRdO5QJgnB5iWyksVssMmAc7Y/+nSm2iFUVEorZHkN3hQw8GxsXP1lygdA8zG9ydGNqvxZlLQ/PwvED9vpMlg7kxjzsDtRvr+V85S0YRpbeo1uiESEWKj++GL2lV0/eMMiWE7wsEfPoAdbjL/nA5e8OCiLyGyJWH4+MuK0n0b22RZ3Tf3d6TlaalO++MaKR+shtBw+AYSxSQ3jewcmE2wvXwFY1OFLQ5SWHJF6r0xSefNiV0qjriHsQxFKF80aEDLd/3UzTiT1NkCiViW+6Nk3XZzEtoj2IpzZru8Z1qsMsomKkLLqoihMUVR6XnG39zJRZp5Dkax0pj+vDV/EPJsiX8hFu4TMMgQnzajFj/OFZZ+ZxWuzcpJlJnD5ZH4NLvtP/HfSupArArw9u2a1DuWnxPokVe3mt1MISdhHP+cUrx+iWLKKqWuPHGauygED5xknMqOfkya/OKbvzDU5QnSIlWrbUo9bOIytONkich+3/FqAf1HwcFWSrY8EvbsgEXA1/PRZ7SZYBGx5UegKYOWzbFGksVhCJBBVVDPJucZNYJOLTEQJbtZ5dsY0mo64JpP0xbvO9sJQmIzZId3zZ+4fsDPbsYTUUQgLct3jYpRmmfI/FzFKK4hn2CLmn0SsT2z6CdDfiSnz YWwapbzu CyKcz2l6WuHgJR19BdrRnx3h34YQFS3Fkzrfa5SFkxrY+Hk8YGggvZFtDQ+zsUSSWcCITK3I96ppAENtRCow99OdjADShsvDi08k816GOXTKPq0VzzzvJMXsfZntUiviS2diLaW9JR38iaDF5YP1UEV5Y+FGq9Q9sUNuUKstYtq8PFMPaDUQLD2ezTM0Q9qXJC8sLiMsZcK1SQ4SBKrknBLGT9GSUSReNgzUNEb0DZq+NDwnGpkxaA5KM0VCEY9pbdGXnqd8d6z9Zom6EwfKmlslqtcn2le0HgVSW7GtaJ96gl2L2bZT8adU/zdNSAHqlFpzdx9B0pOzOKzBZl9gOIcV7SSJJeRzy4n9d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > Hi Sean, > > We've rebased the SEV+SNP support onto your updated UPM base support > tree and things seem to be working okay, but we needed some fixups on > top of the base support get things working, along with 1 workaround > for an issue that hasn't been root-caused yet: > > https://github.com/mdroth/linux/commits/upmv10b-host-snp-v8-wip > > *stash (upm_base_support): mm: restrictedmem: Kirill's pinning implementation > *workaround (use_base_support): mm: restrictedmem: loosen exclusivity check What I'm seeing is Slot#3 gets added first and then deleted. When it's gets added, Slot#0 already has the same range bound to restrictedmem so trigger the exclusive check. This check is exactly the current code for. > *fixup (upm_base_support): KVM: use inclusive ranges for restrictedmem binding/unbinding > *fixup (upm_base_support): mm: restrictedmem: use inclusive ranges for issuing invalidations As many kernel APIs treat 'end' as exclusive, I would rather keep using exclusive 'end' for these APIs(restrictedmem_bind/restrictedmem_unbind and notifier callbacks) but fix it internally in the restrictedmem. E.g. all the places where xarray API needs a 'last'/'max' we use 'end - 1'. See below for the change. > *fixup (upm_base_support): KVM: fix restrictedmem GFN range calculations Subtracting slot->restrictedmem.index for start/end in restrictedmem_get_gfn_range() is the correct fix. > *fixup (upm_base_support): KVM: selftests: CoCo compilation fixes > > We plan to post an updated RFC for v8 soon, but also wanted to share > the staging tree in case you end up looking at the UPM integration aspects > before then. > > -Mike This is the restrictedmem fix to solve 'end' being stored and checked in xarray: --- a/mm/restrictedmem.c +++ b/mm/restrictedmem.c @@ -46,12 +46,12 @@ static long restrictedmem_punch_hole(struct restrictedmem *rm, int mode, */ down_read(&rm->lock); - xa_for_each_range(&rm->bindings, index, notifier, start, end) + xa_for_each_range(&rm->bindings, index, notifier, start, end - 1) notifier->ops->invalidate_start(notifier, start, end); ret = memfd->f_op->fallocate(memfd, mode, offset, len); - xa_for_each_range(&rm->bindings, index, notifier, start, end) + xa_for_each_range(&rm->bindings, index, notifier, start, end - 1) notifier->ops->invalidate_end(notifier, start, end); up_read(&rm->lock); @@ -224,7 +224,7 @@ static int restricted_error_remove_page(struct address_space *mapping, } spin_unlock(&inode->i_lock); - xa_for_each_range(&rm->bindings, index, notifier, start, end) + xa_for_each_range(&rm->bindings, index, notifier, start, end - 1) notifier->ops->error(notifier, start, end); break; } @@ -301,11 +301,12 @@ int restrictedmem_bind(struct file *file, pgoff_t start, pgoff_t end, if (exclusive != rm->exclusive) goto out_unlock; - if (exclusive && xa_find(&rm->bindings, &start, end, XA_PRESENT)) + if (exclusive && + xa_find(&rm->bindings, &start, end - 1, XA_PRESENT)) goto out_unlock; } - xa_store_range(&rm->bindings, start, end, notifier, GFP_KERNEL); + xa_store_range(&rm->bindings, start, end - 1, notifier, GFP_KERNEL); rm->exclusive = exclusive; ret = 0; out_unlock: @@ -320,7 +321,7 @@ void restrictedmem_unbind(struct file *file, pgoff_t start, pgoff_t end, struct restrictedmem *rm = file->f_mapping->private_data; down_write(&rm->lock); - xa_store_range(&rm->bindings, start, end, NULL, GFP_KERNEL); + xa_store_range(&rm->bindings, start, end - 1, NULL, GFP_KERNEL); synchronize_rcu(); up_write(&rm->lock); }