From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC8FCC369C2 for ; Tue, 22 Apr 2025 16:50:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C187E6B000D; Tue, 22 Apr 2025 12:50:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC8876B0010; Tue, 22 Apr 2025 12:50:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A43796B0011; Tue, 22 Apr 2025 12:50:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 85AE16B000D for ; Tue, 22 Apr 2025 12:50:44 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EDA421CC957 for ; Tue, 22 Apr 2025 16:50:44 +0000 (UTC) X-FDA: 83362268808.08.7F1A5FB Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf16.hostedemail.com (Postfix) with ESMTP id 47878180015 for ; Tue, 22 Apr 2025 16:50:43 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of cmarinas@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745340643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SEhpfa7C7yS/4QgW58I/mknBa24icRiaeuepPgepQOs=; b=hKfZNkvY1jjEonIi2vioQxxl1Hxv/Y1zOYL3APiX+jP3+PTQhlp2QTxkTp9gUkbbNu3mnd gjChk/c3qjtOHl7ITmnGVOgExQaLBShvOyRVCzqz8h8VBhRqyoJjzyPu86wACVcy7dL7q3 4ENrJaZ3kSVeqxkHJeuvFFnyE13TQlM= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of cmarinas@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745340643; a=rsa-sha256; cv=none; b=zvhBaZJPHL+oKU6LBPsBjjfcNpw7XNZTC4nP33ls4HCzBeRDUQ1zBVbUQBsWeqhDEsSmJx 2mRdZVuAbs796dLHFsAgQQ23COu/ccojbiuKB4Ii7uOCEDVE86K+1mAK9X/bdxXI+n9G0j yz5CG2T6wKhHdnBnjRF7K7ziFvCy0d4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id 3304B61126; Tue, 22 Apr 2025 16:50:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 325B1C4CEE9; Tue, 22 Apr 2025 16:50:35 +0000 (UTC) Date: Tue, 22 Apr 2025 17:50:32 +0100 From: Catalin Marinas To: Jason Gunthorpe Cc: Oliver Upton , Ankit Agrawal , Sean Christopherson , Marc Zyngier , "joey.gouly@arm.com" , "suzuki.poulose@arm.com" , "yuzenghui@huawei.com" , "will@kernel.org" , "ryan.roberts@arm.com" , "shahuang@redhat.com" , "lpieralisi@kernel.org" , "david@redhat.com" , Aniket Agashe , Neo Jia , Kirti Wankhede , "Tarun Gupta (SW-GPU)" , Vikram Sethi , Andy Currid , Alistair Popple , John Hubbard , Dan Williams , Zhi Wang , Matt Ochs , Uday Dhoke , Dheeraj Nigam , Krishnakant Jaju , "alex.williamson@redhat.com" , "sebastianene@google.com" , "coltonlewis@google.com" , "kevin.tian@intel.com" , "yi.l.liu@intel.com" , "ardb@kernel.org" , "akpm@linux-foundation.org" , "gshan@redhat.com" , "linux-mm@kvack.org" , "ddutile@redhat.com" , "tabba@google.com" , "qperret@google.com" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH v3 1/1] KVM: arm64: Allow cacheable stage 2 mapping using VMA flags Message-ID: References: <86wmcbllg2.wl-maz@kernel.org> <20250331145643.GF10839@nvidia.com> <20250407161540.GG1557073@nvidia.com> <20250422135452.GL823903@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250422135452.GL823903@nvidia.com> X-Stat-Signature: 8tmagrura3zjsgjuam5bu76ke4413xye X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 47878180015 X-Rspam-User: X-HE-Tag: 1745340643-899027 X-HE-Meta: U2FsdGVkX18ku25foLoPlRx++52ZDf6Ac2IgFYksrCJ+xQtaRLc/Nro+6J6Vw3n3x+wlcKeQMthFoecRLv7APA7BhtEdzPdUsrgD8ZfUCbPItSon9JnrgMd2BWlP9PArqziZFjU7MGUV/CWkR65C8NlcefAYm6nTToNeMqC+asCTM5VGXongPhIEGIlrFGt4VT1gJnANteGILdWFMjP0g+MRxYmFAjDJLARqbA8S5QUdius0MVaSIitD1cQCzhrDbcBCPPFkDYdhPfFRjEiL0aikxgxlYBBV3eT6qrh6uZowcgVsShXMeERnwUNz9UXUOsZXBlcGpcKsu7HGVrSknIzYkMlE2uktGPJuEodeoaPjLyDuX6FuMovIeszh4zU4fJS8AYvbe1CMJHjdxPIOHyS/LaILaOX8XUWml65kBSR1Nk8SLL60Tw+fhA1dBu6pdRO3214RCJ1sc0mvSBXJ3YgDhpeeQIUyiLggvhcPhVaFKfm9VBRL6QnhmD1B3DEvtRYKoWkXTJk6ScScaNJruN+tUw1Q0CinK9h+n93ZnpOfOsPo7jr7TAsFso8OE61USxbk/++xyNMqRvOfzwv/AocFKUDzFS5mCYZOPQ+AaLbBnim8FzOql2KExLpNojIVQ0HEyhonk4xAW6XYwQACf3yuBJ7VpAPhP3862t+E6jU0f6Kza+NlAsLdYfbYfvWsUN63i7yc8lwIaH/x4+1bbSye5Ijb5mm5T6SgnbEz7kYQg8SleQK8vQN0AYezmQkStZHeSMd2pIwrQTgzFQKOWsQbNAvkWnuf3vhzXBj16bnNFA4pbJ7htazZK/28KzpHi7He0EzsB3iQ67SQKP1BGDdQtbYOsghJ+3KFRBQAt2oL6kZ3nnZvF71tEBrDvC7ensfglM8S3zeQmLhym3LTXJcyvHqnE4iyuhX9eUo1wYUkUtXU+XNmhWsONZI+uGkIpwZ3RPoLB+nkPDMnThI /oHQErLc ZTr1fhRpmaVbP3kiWEGzkszoyoY7nP5nEBgPSfvwpKJc9ckHRCPPZjUn7gH2Huxk4qNIzZ7BuWdP9dmHApDub6kOV870jkbsDv1MJDLgOU8YTT7YwPkn36VmO31IN+j7ffAidx07d0JWaQLHbJIHPuGMcuaXQHGjv0BTRFhHZ+hU0tnkHi9EHMUhc8tKYcSj8VQpFtRKwGkVIiWGMYixAEHeeEfKA3TlULYfAVo8+xwcuUp8sJnZJjqbBqg451UafL8gtaR9vJ2uEFYQTszMYSPEJDa0McMan27iIuBAEpms/RM3JqDbUJgnKzUgVqcgvqIuck/4nJpoUObh0hBLAM0fg7X5oNn77X4S4wMy9kRa7o8Xnhndm03+jJ7PcTBUwbh9S X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Apr 22, 2025 at 10:54:52AM -0300, Jason Gunthorpe wrote: > On Tue, Apr 22, 2025 at 12:49:28AM -0700, Oliver Upton wrote: > > - The memslot flag says userspace expects a particular GFN range to guarantee > > Write-Back semantics. This can be applied to 'normal', kernel-managed memory > > and PFNMAP thingies that have cacheable attributes at host stage-1. > > Userspace doesn't actaully know if it has a cachable mapping from VFIO > though :( Yes, that's why I couldn't figure out how useful a memory slot flag would be. A CAP is definitely useful and some way of preventing migrating between hosts with different capabilities for VM_PFNMAP is needed. However, I think we only want to prevent migration _if_ there's a cacheable VM_PFNMAP, otherwise KVM handles coherency via CMOs already. So, for the above, the VMM needs to know that it somehow got into such situation. If it knows the device (VFIO) capabilities and that the user mapping is Cacheable, coupled with the new KVM CAP, it can infer that Stage 2 will be S2FWB, no need for a memory slot flag. If it doesn't have such information, maybe a new memory slot flag can be used to probe what Stage 2 mapping is going to be: ask for KVM_MEM_PFNMAP_WB. If it fails, Stage 2 is Device/NC and can attempt again with the WB flag. It's a bit of a stretch for the KVM API but IIUC there's no option to query the properties of a memory slot. FWIW, I don't think we need a memory slot flag for host kernel managed memory, its S2FWB already. -- Catalin