From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E73FD2F7CE for ; Thu, 17 Oct 2024 11:58:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F26756B007B; Thu, 17 Oct 2024 07:58:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED5EC6B0082; Thu, 17 Oct 2024 07:58:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC57B6B0083; Thu, 17 Oct 2024 07:58:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BDD086B007B for ; Thu, 17 Oct 2024 07:58:20 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0B2C681133 for ; Thu, 17 Oct 2024 11:58:11 +0000 (UTC) X-FDA: 82682945940.08.C63FF99 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf01.hostedemail.com (Postfix) with ESMTP id 39C0940012 for ; Thu, 17 Oct 2024 11:58:09 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=w6k0JD7i; dmarc=none; spf=none (imf01.hostedemail.com: domain of BATV+8fef01ffec99140bf63f+7725+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+8fef01ffec99140bf63f+7725+infradead.org+hch@bombadil.srs.infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729166191; a=rsa-sha256; cv=none; b=NN78eFEpOfRWSiPmhHRop+8OhSL8u+BgunweebuvDqE+csV7byWSjjc0MB4WI5D3T7SccV SEqAxKwRD2BmgZlKHktxikll8D//lDI9DN1tvVMl1c1nFrHYfIxvTVCOjthxgw/krom2/L UeLrWC5qBR4ylhYAfsxREpRgdpFlag0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=w6k0JD7i; dmarc=none; spf=none (imf01.hostedemail.com: domain of BATV+8fef01ffec99140bf63f+7725+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+8fef01ffec99140bf63f+7725+infradead.org+hch@bombadil.srs.infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729166191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EEDlW0Z+27Cz1a36GnGWzG69pvQQcfAeec1992co/rU=; b=Maw7haRLkjndDwof/NdErfROPOr3EEb9wy43jtim+Kr2cxOBS883JzbKS/pESH/hWqqhxH uFcYB9zvgjTevoJ/9kUOsGcvdilQWVsdrkWPAq/cgGzlAtpZrzfyj6Xj42Imppzbl25ccj ePJVWt8fyn73k6zARBUmGMcKjb0MgcE= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=EEDlW0Z+27Cz1a36GnGWzG69pvQQcfAeec1992co/rU=; b=w6k0JD7itqBz1TeMuMUHKr2szV HdrbD5T6KpkzUq67echeUaMGiLU86Jhj67No8E02ULxoUSkbQFEG7QeJlA9i6jCbzVyHdhuhVrCm9 QEbjccVpvAry4xZVtmywe8TpIdwRgeQCAf04vKCPZdaH0ibPWuCoga7WOklEOu7G9flfeP3TOMDbj PeQV3HJ2agNF9brsTbkzHmLHdvpuHq/vmfopnhKxf+H7kerzEqeNPCkgpGhg4A7F5abFkIj1VoBQt 8WiBq5FPcPVE4wxmfiUHJ0j0xYFX5ygiAo17X3gLbyq5MXBm5g5AmNIOBNOYG/WNS1rlPDNX1CXF/ NYo8NzBQ==; Received: from hch by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1t1P8u-0000000Ehiu-2ZiO; Thu, 17 Oct 2024 11:58:12 +0000 Date: Thu, 17 Oct 2024 04:58:12 -0700 From: Christoph Hellwig To: Jason Gunthorpe Cc: Christoph Hellwig , Yonatan Maman , nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, herbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, dri-devel@lists.freedesktop.org, apopple@nvidia.com, bskeggs@nvidia.com, Gal Shalom Subject: Re: [PATCH v1 1/4] mm/hmm: HMM API for P2P DMA to device zone pages Message-ID: References: <20241015152348.3055360-1-ymaman@nvidia.com> <20241015152348.3055360-2-ymaman@nvidia.com> <20241016154428.GD4020792@ziepe.ca> <20241016174445.GF4020792@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241016174445.GF4020792@ziepe.ca> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 39C0940012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: krqdiprs7x7pf9q3pz67pssyq5fqoif6 X-HE-Tag: 1729166289-441274 X-HE-Meta: U2FsdGVkX19hwbFgMnwSPMQrP/Eu86S0xJLqF+WCUO+b4f+THOr3eNifjVItBq7gHc4O1ikzQ8NAO1kW75Ab/5tWCwSrLx9ReDnLPNrZMws3Qn7SAeXsny4lqlyyrNRv7sFW9xZUVnjJLUSTdvyQHtiN+BvEA9TCr4JZCiUK5HBFgrk11SaPEA0IsRXDg+uELjagShjfM+2pFWKm80qR0WTyFCRHpNVVe/I1ilLUZOwD9f5zw2QjFV/J8pK1dLQy5v8UCogUyqnLoLXNdjQ9pTse48FCHxQmyWT0Yz0SbxCjZ/zMLTkMmhXnMHT1zy30ddpB/g31lGV0iikTP4xSHFVh+twe7JRGcU1ic08N1ZhWjuoum2GCG6Qz36MWTiv93f6dcxCzle74AQXc3560a9Qn0XqBKFh3kTJg2JrS2GQL7z9Er4POCNxIzQGyGk1CPYawovhe4TdijwhZF8GgQUKuGNqwjmqy5IQjHwpPlhY2vht4EHBESfMdc/DncUwDG643Jq6VlFQOcCxqUp8ePipw91mPDBkq64HRXXqQw3I5+rrzTS/o+WdN6bHKwbOOeOy+TnS4d5g1mXdpGX/ixnNIvp/3unOBPCNrulw6Lhxi1oezrAn+/zd/qo78fHGbG5m+RJgPGH0C9lnoQ4pbtQ2lgBcUzkMAM8AkQJd8f9tVl9chv5kPmaOAaKPrhkRSBwjyRlt5agwN0I6B0G9OvRz+8G06dMJzA729ZrtcfzDlONdqa0lM8iPR4snguSnVK/kyOfAjlSS6F+0eAH9ubVbNZyiItFf8apcZY2nosjq4Q/1IhgFtPZANwobty+W/aY0GEef/stzuhxAA+qWqXrnTB13YhLCrxsYFzwaCGbFC8QLYVWXKFyZ8WTMgwpzPdNQ3TB/oLk7KRFQnajTBbYCJQV169xAVT3MRFj21UjlnTSccLOhBnFtR7PCvKV0GWoewG/yQjyEx0NAGCcl tkJZ3Z8v ZW4ug6OIjZm7aXfQn26eSt+x+/tpZ8YorxZdubBrqmxV0E+asjDjUHQojXvEJDCSiJSlR4gN6z2Th8xqGd5maSStwUooDKWsFqoyZRB7JEDPpkGhpqFnuXeYSn95wesvvU7ZqhOhInFx4OWcqwkSC2zWJ5/DqM2CZi3MrIMOXyHry9GPTw33iVcFMY2RtSlGqW9zwfHc0SI3OB8epez1aWl66ma03+ap2kNjPES8bLEzAyFrpH2NQw3Ga7HAQ0MmsKROcLgvfUn+pIGnbDB0TRApgZWXxij749cMBFGsVATx59rTMKknoSa8oxR3cB+SzmjkFKTgrwXm6uCL+1lFbZPAGnfpNC9GTDIAk3buFQFHOa53OHPcKpLNRC0m528QS1qeBbwYn4mUlgqbVrFUaqNEYS9TdL7uQHEgOsqwJcI5Ay601Kgc4U3dcRIzcBUBA9U/CPccztKVEkwvcwyoB+E1d2TiJmf2SoWjV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Oct 16, 2024 at 02:44:45PM -0300, Jason Gunthorpe wrote: > > > FWIW, I've been expecting this series to be rebased on top of Leon's > > > new DMA API series so it doesn't have this issue.. > > > > That's not going to make a difference at this level. > > I'm not sure what you are asking then. > > Patch 2 does pci_p2pdma_add_resource() and so a valid struct page with > a P2P ZONE_DEVICE type exists, and that gets returned back to the > hmm/odp code. > > Today odp calls dma_map_page() which only works by chance in limited > cases. With Leon's revision it will call hmm_dma_map_pfn() -> > dma_iova_link() which does call pci_p2pdma_map_type() and should do > the right thing. Again none of this affects the code posted here. It reshuffles the callers but has no direct affect on the patches posted here. (and the current DMA series lacks P2P support, I'm trying to figure out how to properly handle it at the moment). > > IOMMU or not doens't matter much for P2P. The important difference is > > through the host bridge or through a switch. dma_map_page will work > > for P2P through the host brige (assuming the host bridge even support > > it as it also lacks the error handling for when not), but it lacks the > > handling for P2P through a switch. > > On most x86 systems the BAR/bus address of the P2P memory is the same > as the CPU address, so without an IOMMU translation dma_map_page() > will return the CPU/host physical address which is the same as the > BAR/bus address and that will take the P2P switch path for testing. Maybe. Either way the use of dma_map_page is incorrect. > > Jason ---end quoted text---