From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BEA6C3DA6E for ; Wed, 3 Jan 2024 13:33:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFE1D6B0321; Wed, 3 Jan 2024 08:33:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BACDF6B038B; Wed, 3 Jan 2024 08:33:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9A396B038C; Wed, 3 Jan 2024 08:33:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9692E6B0321 for ; Wed, 3 Jan 2024 08:33:26 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 616A8A1C65 for ; Wed, 3 Jan 2024 13:33:26 +0000 (UTC) X-FDA: 81638091612.04.537C7A9 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id C28E3C0006 for ; Wed, 3 Jan 2024 13:33:24 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704288804; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i2ebhdPpdNJKWVMPY3szJ5Hb8QerDm9zMllrbNVSUnk=; b=X1c1+gf8pMgHIP/ne1UN3ziE42TGlo2My+7s84ZLQ8UJsK2iol2J+yXLHwB0/9gWCJV5st iTgf71lpP4bhsdPO6H2anbKUDcDeFYMtuWGvikQFLVR0gcsoRWTJWmtW7Ixx5Jl2cV70Fd 2uSUQMy+BSqjAg7Ra8LK5b8dtfsiWuQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704288804; a=rsa-sha256; cv=none; b=NPdWLb7yiDKZ5zhI+AjZfm2ugHK3ZXScEpzYvaBd2rU7qbTwREgLEHs1RIETGbSgAvVZ1A sa7SKAOP2mUiIKiOKoFe7C69XRLMSu50asXHu/dodqLHRUYCZFVAgo5qK79saXumA44Xa5 HFW8/hJIWu19/SJbUxnOEUTgv4V5TfI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id C39C461499; Wed, 3 Jan 2024 13:33:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E489CC433C8; Wed, 3 Jan 2024 13:33:17 +0000 (UTC) Date: Wed, 3 Jan 2024 13:33:15 +0000 From: Catalin Marinas To: Jason Gunthorpe Cc: Oliver Upton , Lorenzo Pieralisi , ankita@nvidia.com, maz@kernel.org, suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org, alex.williamson@redhat.com, kevin.tian@intel.com, yi.l.liu@intel.com, ardb@kernel.org, akpm@linux-foundation.org, gshan@redhat.com, linux-mm@kvack.org, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, mochs@nvidia.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com Subject: Re: [PATCH v3 2/2] kvm: arm64: set io memory s2 pte as normalnc for vfio pci devices Message-ID: References: <20231208164709.23101-1-ankita@nvidia.com> <20231208164709.23101-3-ankita@nvidia.com> <20231212181156.GO3014157@nvidia.com> <20240102170908.GG50406@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240102170908.GG50406@nvidia.com> X-Rspam-User: X-Stat-Signature: tobxkgauqpdpbt9httxx5fc7u4nzwpny X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C28E3C0006 X-HE-Tag: 1704288804-979562 X-HE-Meta: U2FsdGVkX1/cZHNmESP0elvEijqzLvHHi1gN5HqisnIg1TxiCTb+QBSxsjSz3tN+jyDf4NFvzgwQpn0qimUwz/+sq1LWUkwTf9/eWaiK2FrledBJLcqjN4hJbz0Xpde7tdxcOBd4lz+uNueitQnZoZ20ta8lie/JetYP/YaHiSXr5CDSQDzNVfrO/ArZc7H6tuEjLHnaSbZEyZKxU2HF27I+WsnsJkA36R8dM14zqpTJLtvZ3k+xNiTzkcCtI5AwcPERDHNJ6HNibA54IZyRjiOxWPD6y8Xew32PjG146B1yuBALPFsJh44LSWm7HHbdBDoXfD4TJTVOG6+G2ciPI150hEhMxhRGo17luElIk8Y1ZRFjkzp6RATLPyTDdUKrvyC3h0b09rZNRTzsBcbvrOkys1t8NQOiQI9lbg0J7n5LuotaA9OeK9XXtNCY6g9eJOoKSYxB/x1xPk/fMVlntx4OmNyoOf8Y5pFRmlhI631MKQePz3NNo5OuGbAUI0VGfzYhuJbheePKefd8Uj6YMVguzODfdX0P480sjGBB4tNA/JXIsru3EOloDsr7i+GZrNp7xT6+aHIJ0GG0ORD5aFBMShtjtnaezMSnbLP7caCy0jnq+5mQ53bVl3cpHenYrO0VOQL4dADod7RPdhM0XtfInU7QB5atyVXSa1NSWTI3AvTZjrwWn492Xua1NLdi2b6wiXTWd/aqTswBLAWy2zZEA6B2Nz6aHoJhPNc4YuNQdZn3UxKVcMWg5eK/lmd23MhZQ7MKETdxAikrPl23WTO3WkHYF5sDKVGbhgeSsDceFBRamWZcaFTWdbMb1sqKIRHXNmYQooAwvmQGU2E5g+oVpLZhp48UuMyo9pQL+xXv1G78rKr+GHEJO1KnjmWPLjLf+UDBLpxbp7hIl627oJykTioRHdPHgkIrIRHAZUx9dUs74gx+3tebqtnlk0lbCe+5Y5Qz8XfGa0nmAXu vVSWfgkh M5YgW0JMzhaXulKpyd+9XbaNjMN7963Oci7Rof6iip/sPCgSNdVm5N+vWZPH2WrlGrHiAvtmmansHkuJlDfgrW9LXGlN+ru0abkeyjwxjQobrBBiwuWzKy44T0WyW7D9QomjSNzUwzuH9mOCtrun8w+KxFTnRZOnNyqFOerTQTqlrwWrnRR6ixa0XtZ5f8uoMmf0jxeV4wyjiBd2twvcZp4WYm0y8/Dfvnub5MDRhutlkKtXUwyNOGrnf3g5k8L6/jhOvU7bIC8tTTXBOEm5oRxEIXQoUcE5kPwHLKkEXQ/XmJTbBsr5Jrq2HcKcS8DgRoOmN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 02, 2024 at 01:09:08PM -0400, Jason Gunthorpe wrote: > On Thu, Dec 21, 2023 at 01:19:18PM +0000, Catalin Marinas wrote: > > If we really want to avoid any aliases (though I think we are spending > > too many cycles on something that's not a real issue), the only way is > > to have fd-based mappings in KVM so that there's no VMM alias. After > > that we need to choose between (2) and (3) since the VMM may no longer > > be able to probe the device and figure out which ranges need what > > attributes. > > If we use a FD then KVM will be invoking some API on the FD to get the > physical memory addreses and we can have that API also return > information on the allowed memory types. I think the part with a VFIO WC flag wouldn't be any different. The fd-based mapping only solves the mismatched alias, otherwise the decision for Normal NC vs Device still lies with the guest driver. > > > Kinda stinks to make the VMM aware of the device, but IMO it is a > > > fundamental limitation of the way we back memslots right now. > > > > As I mentioned above, the limitation may be more complex if the > > intra-BAR attributes are not something readily available in the device > > documentation. Maybe Jason or Ankit can shed some light here: are those > > intra-BAR ranges configurable by the (guest) driver or they are already > > pre-configured by firmware and the driver only needs to probe them? > > Configured by the guest on the fly, on a page by page basis. > > There is no way for the VMM to pre-predict what memory type the VM > will need. The VM must be in control of this. That's a key argument why the VMM cannot do this, unless we come up with some para-virtualised interface and split the device configuration logic between the VMM and the VM. I don't think that's feasible, too much complexity. -- Catalin