From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A14E8CAC5AA for ; Thu, 25 Sep 2025 07:03:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E76B98E0005; Thu, 25 Sep 2025 03:03:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E26D08E0001; Thu, 25 Sep 2025 03:03:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3CDB8E0005; Thu, 25 Sep 2025 03:03:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C11DB8E0001 for ; Thu, 25 Sep 2025 03:03:23 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4391414037D for ; Thu, 25 Sep 2025 07:03:23 +0000 (UTC) X-FDA: 83926881486.25.C590B53 Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by imf08.hostedemail.com (Postfix) with ESMTP id 83B4D16000C for ; Thu, 25 Sep 2025 07:03:21 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qHutfg7p; spf=pass (imf08.hostedemail.com: domain of leon@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1758783801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qEGnlGSl31sTqOX8g325Qy3BxJyeB03/i7rkS5ioaNU=; b=3fjVdv5EhUvVOAQ+2UW3TBor4Q0j3PSlWDR97l5z4YuguLLzFdAs5/9hm5EHA6oTkjh/Fh vGjJ7SGBDp0zcaIzYKBSqY7EtiO1jP6vEjg10tApyEvd8iLOGo8TyoTgpP3wd7WhMCVCrf C6fNK0gYHOTVI7eLda7K61S/IC1Ddao= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qHutfg7p; spf=pass (imf08.hostedemail.com: domain of leon@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1758783801; a=rsa-sha256; cv=none; b=T3qo0CPmilIsrPDfNxomgxd9nKyGK7nqbAFCUBzqoPRkMbzC2CynPZ+I8AKNcdTKHmN/vS nO8hvxfExb4oH/neNO/SYSnpq5OVFo29qcm/H+D3YTtIrPxBtG+0n4GFlOyrfw1h5LdIm9 DjpDFkzZugznsIkZdIPAcwEebU76+Y4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id 161F540817; Thu, 25 Sep 2025 07:03:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 476E0C4CEF4; Thu, 25 Sep 2025 07:03:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758783799; bh=IfFjt1sIn89xjUeeoV2zJsvE3bTjSj86lJ2QWvNYz8w=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qHutfg7p0MJiJzKdbXS/d6R3kdEqDhdXcGmgxvkwI4AM6XuKyeIpgR1GSCsEqIGVB 9ZrAilO0Zj59EMAXR1lQlcpWEMA28nquUz1IKkduHGuS4dyD2OJmp5f3UxncKmQ8+I Dp30uxy2wGUYE3Hja5f4XS00BFzuj+WaXq60qYIzodUdV2tvE04uI5a85RPL6mAqR7 MWKPSQplfkMcH9/nRS03UG60TJBwCq9ZQRhoS60sk8FdhiE0kifJqWTjrMUqN9REZV kUzpbXtt2XKNZVaDT/F1inIn9vj+we8Gfm9meFs0BUoq+pahNKDPioVkPIFajbcBnM waHnKaEouWGfQ== Date: Thu, 25 Sep 2025 10:03:14 +0300 From: Leon Romanovsky To: Alex Williamson Cc: Jason Gunthorpe , Andrew Morton , Bjorn Helgaas , Christian =?iso-8859-1?Q?K=F6nig?= , dri-devel@lists.freedesktop.org, iommu@lists.linux.dev, Jens Axboe , Joerg Roedel , kvm@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Logan Gunthorpe , Marek Szyprowski , Robin Murphy , Sumit Semwal , Vivek Kasireddy , Will Deacon Subject: Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation Message-ID: <20250925070314.GA12165@unreal> References: <1e2cb89ea76a92949d06a804e3ab97478e7cacbb.1757589589.git.leon@kernel.org> <20250922150032.3e3da410.alex.williamson@redhat.com> <20250923150414.GA2608121@nvidia.com> <20250923113041.38bee711.alex.williamson@redhat.com> <20250923174333.GE2608121@nvidia.com> <20250923120932.47df57b2.alex.williamson@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250923120932.47df57b2.alex.williamson@redhat.com> X-Stat-Signature: zb4p9anepj4ig4fz1w1a6oxtbbzr8dmu X-Rspam-User: X-Rspamd-Queue-Id: 83B4D16000C X-Rspamd-Server: rspam04 X-HE-Tag: 1758783801-608269 X-HE-Meta: U2FsdGVkX18nmf7cCuL0e+IOMbnoZOZv+/wWnxfh1Q7ttqjvx/Cv9GzOd+8HmrPSu5DNgupYHljvMYqGTY2yDXIa7xLVT1RRLCIIIPdLoXVfbJNVU0UbqHhdSNTCkLV2Uewun9KVL0b7iYpvWXsMSuKjWqmvdmIv+DrGFoVyOvzzeSvCEdA7a8JvErXXAQv1Uu33E8/BFnVfu7cYw1NyzrWoJbHsJn30XaP3hbcd63ZoNSwm/CEtnoeGomZhF4fPcOCV1bMcpOaqituuVVRdmDi0QGOd4aVBE0P+qLY4dBSQKhzwp8qGPaT6HmZuh/8NqkLYLKVRVpq/HRK9pFXGdvBsm3s9UYMeRpiKIdrceaGqGRWfmf21kqWkpV2ZnEUPbGfTiJr2Iz8cwWH+2CGiyat8ERqzelQbPe6Bj5YrX4XFsPemVOa+3+W31apGwuytgLpqbS8GE8P/pGxFLqZzQQ5hugttqAPA8aUYvCbS31f/PW5tvlKKrrJqEuYoKd1KatE0sJ/YAGpsrht8xgxiLk7FCwA79ndvLSQf8x3TDAwajQwLPGzjueuV6AbZENsLWdDaa6L/ngq/wZzGA7PGrajHjFGCM21RhTCMVXVIronhlOPo90VkdoicoJYqD2270JikYLAWZ1oBb6RxysaKdjwWTFKexlT4yWo+QFkHa1TBvH2q6J89qkTWJb0VabVPCNCcKBXFJqCFfGwQ8GmGww787gmrz1Ewcecw58ALoR7IVU7Az/TYyIaao+f6bxxFIyTXlhW0X8EEH8Gu6CI+vyAahkSP8oNE+s1uMD/KYVVsdIxWH008VRUFK9Nb4gZ0OzDsabAVP+cQNsGrGlP1R0s2tNjo0LQt9f7GjlI2gpru/Xl/4/PwNaiKkCSYrKv8oPvDepjS8Sel9u/HysVzWwjOiCVWeK2DbcRIapvSTAIDr/LBYkQcRxxke6hf+0zFuBBENHg4BQvF62/TVt4 IhE5rRmw RC7AhB+Ro0T5yO5PdCs4walgvhcbShWuU3z3pB07ZceQjNG7qntMNyjlmtsJDLRqSM63YzhKJqKJryvAXpM50o37F0b3BkAB7B1n5fOrS/gOch/+BCYYY4BdEdcvJ+gPZdHNPnoZnWOx3/GWsgsCO9Q1HR8iGTMgRfU9qjNO/8f37ji0cahDtNYZxprvqxpfAFqcMAo/ILf8uU1p6lbfKdhF4HZ78W/NxkrUh6qapr0v/rUGpqk3lqApndZES9CxzTnXaQfj6zphrj7OuyCZSAOra5A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Sep 23, 2025 at 12:09:32PM -0600, Alex Williamson wrote: > On Tue, 23 Sep 2025 14:43:33 -0300 > Jason Gunthorpe wrote: > > > On Tue, Sep 23, 2025 at 11:30:41AM -0600, Alex Williamson wrote: > > > On Tue, 23 Sep 2025 12:04:14 -0300 > > > Jason Gunthorpe wrote: > > > > > > > On Mon, Sep 22, 2025 at 03:00:32PM -0600, Alex Williamson wrote: > > > > > But then later in patch 8/ and again in 10/ why exactly do we cache > > > > > the provider on the vfio_pci_core_device rather than ask for it on > > > > > demand from the p2pdma? > > > > > > > > It makes the most sense if the P2P is activated once during probe(), > > > > it is just a cheap memory allocation, so no reason not to. > > > > > > > > If you try to do it on-demand then it will require more locking. > > > > > > I'm only wondering about splitting to an "initialize/setup" function > > > where providers for each BAR are setup, and a "get provider" interface, > > > which doesn't really seem to be a hot path anyway. Batching could > > > still be done to setup all BAR providers at once. > > > > I agree it is a weird interface, but it is close to the existing weird > > interface :\ > > Seems like it would help if we just positioned it as a "get provider > for BAR" function that happens to initialize all the providers on the > first call, rather than an "enable" function with some strange BAR > argument and provider return. pcim_p2pdma_provider(pdev, bar)? > > It would at least make sense to me then to store the provider on the > vfio_pci_dma_buf object at the time of the get feature call rather than > vfio_pci_core_init_dev() though. That would eliminate patch 08/ and > the inline #ifdefs. I'll change it now. If "enable" function goes to be "get" function, we won't need to store anything in vfio_pci_dma_buf too. At the end, we have exactly two lines "provider = priv->vdev->provider[priv->bar];", which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)" > > > > However, the setup isn't really once per probe(), even in the case of a > > > new driver probing we re-use the previously setup providers. > > > > It uses devm to call pci_p2pdma_release() which NULL's pdev->p2pdma. > > Ah, right. So the /* PCI device was "rebound" to the driver */ comment > is further misleading, a new probe would do a new setup. Thanks, I will fix the comment. Thanks > > Alex >