From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C369ED13588 for ; Mon, 28 Oct 2024 06:37:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A4B16B0092; Mon, 28 Oct 2024 02:37:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 454966B0093; Mon, 28 Oct 2024 02:37:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31CD76B0095; Mon, 28 Oct 2024 02:37:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 11D9E6B0092 for ; Mon, 28 Oct 2024 02:37:48 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D01C41C1DAC for ; Mon, 28 Oct 2024 06:37:21 +0000 (UTC) X-FDA: 82722054828.15.28DB787 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id A23F640003 for ; Mon, 28 Oct 2024 06:37:22 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GwfQjiZK; spf=pass (imf27.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730097387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=h9Vt1G1QhAWZwYF7bg5GNJOdngfwRon2If+BJNxbKvY=; b=RdFDiDGHz053AvDrNrZpAjRrwaKZx7yOwil96I4J4Wy3n7tzAw9pxzmfdH3T8nYUPcmgjZ FL/x9s8PhSqPZBnWRl035VQ04p/ch4dvRD92fj7tkbzNZMkB1CLbFtiE16DwoPYieJ9bCw TwbdxwtDP8lwFMA2G+w06EiaNzGlhmI= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GwfQjiZK; spf=pass (imf27.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730097387; a=rsa-sha256; cv=none; b=S+EdQb7Xv/94gNuTMXk1GmyzZANAV5wG79ZjkmFxiFDC4HCTr1eynhP7SvUeHB7r4X9EiP PyjrqBfGmUJYQIMjoBjDhSRAOpbPUnUPq2IE6ndrzPN5ceijQGPC3959PJDE/AJEOJSDsY 27NWPhjJcWwX1EhGqbdK1IoDdzSnP1g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E1AE65C59CC; Mon, 28 Oct 2024 06:36:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D88BBC4CECD; Mon, 28 Oct 2024 06:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730097464; bh=JSMPxlj7Xh+b8j9DlJV8uewuubV9Tsu6U0l5WWqgJPU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=GwfQjiZKs0yrCxRr/DuLwVj0Hx2O01TJqsx4cOgi4xDB5heJMzcMFklB+BmqVt/Cv 4gTLkD3T4pLeplq55h8kscjuwFaQBaQQl3jJ+S6HDuMUb8Uo4nfwXKbBH4ZpNsFwWc DtTbV7qiK3wP+pOJjE92kpgpVG2/Rp3yxPce9N57yNgHPfhuWe+jA+txqxeG7J3/WR lg7fC+BqgPGzEhchM2jYymbqRn4haIlJdsIa0VzlUlQRhnfBkwl7wj8kV+Z4AwHkx2 BqCjnbHjv5nHEhthh0L9SPFTB5k9zIXGi2bQT8jtQ/tbnPfE55BhKbIxD3tfcRScfY kTj4HSZy9GalA== Date: Mon, 28 Oct 2024 08:37:40 +0200 From: Leon Romanovsky To: Baolu Lu Cc: Jens Axboe , Jason Gunthorpe , Robin Murphy , Joerg Roedel , Will Deacon , Christoph Hellwig , Sagi Grimberg , Keith Busch , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , Marek Szyprowski , =?iso-8859-1?B?Suly9G1l?= Glisse , Andrew Morton , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 05/18] dma: Provide an interface to allow allocate IOVA Message-ID: <20241028063740.GD1615717@unreal> References: <844f3dcf9c341b8178bfbc90909ef13d11dd2193.1730037276.git.leon@kernel.org> <25c32551-32e2-4a44-b0ae-30ad08e06799@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <25c32551-32e2-4a44-b0ae-30ad08e06799@linux.intel.com> X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A23F640003 X-Stat-Signature: ithfrr1yijbxbh76jfqcjrg7xee7nqfj X-HE-Tag: 1730097442-33327 X-HE-Meta: U2FsdGVkX1/jkEECmWU1nKm5ebz4bt7dTmoNsuMK825UJ4kbgl6Lbsf2Gey7D0Q8GY1QxjLpvIAsajMq4kZlHuIIUfIHxatBwj6VXtJwZx4G3lVuv3GH/fmBrG8hQebfIo2HpLD+Lzvl5laeYXlhsJ7RdaVGO/bZ85oOMSEXYkejA7uTIeD3nkcPerVGZFG2wMRvyIvzwAY1WiRjVwtBVktF2qd/NL1HUvm+blDdO0nsQ2R9tp/3tJN0ePTBCmo3vxWc9mh8G/vIdz/vH7X0grfiXTcmrNUANo+87p0NACU6UK/HW1OkircIZAUxL6D36MsbilEH83m2Sd7qyVzN/i3i5KRASh159o/4B83hMa9KJtMXhyPdwxpbPOykFfj94/ASSa3Q4mVZjluOCmgyEAu3WBkwv4CWraYSvl3+PFHBvuVBWe5aq8YtlukIXATIACluOnrP3GvCtSa6HGjDzBmJcCIX4ZDYQ6ZpmRmsyJeV8oibDevW0I0AESnvMaoBgMc4rkg5LpzK6HFx0D+cXz6qzNSxgcTEf76bJapC/9uqmC8RY1RWwFUy0tOLUi4V/5XiJgZVrzL5QspCEQElcfZoWB1EBv3amxlE+q0jpaX0zWlZnBAbtU9B7pjMkplka/711Fxg7UJeTwm3qjNtCx5AGFIv0+3ozkaiKOtwbesksMkGu82dsAgZADsJx1+fmlaJUWqu0Ym9u1FCXTMJSTLcffLLj+52quKw2GzH1zjhmEmJVq2+2uX89XGga5HartLLtmTtQf1fVOvxjFSDWw2SjFYydTDdpKMSQ5i4/1kFwVgy+kubHqzV5s8cyEniFZTYxD/XSNX4wCIuV3FJDA9LIsVhdze5pghQ4WD1NuZZ9O3SIRix6gMdsgvDavGlTU7l+umHCSb3KD+FwLUpEewz9tIljUQlf7K2LTbb4pwAGlm08fL+42KrkncMRE7A3v6AsJyJjESwLDqJa8l Vp2tNW2G RKi4425dglHXDK7sOF/29MValScFSCzgFv6NoTQeM187qcmg7gvIVTkDvR/WByxiIH/yqbx02k//WChh0in3MCF6eCDhty7IEaSRsxrjAq6GGMHCJtl9kWxTSk4Wlwq2ZrjX+rfAuHhoLZMN4ejkKU4hFicAmvIgoKomhjlOP2ecahKOXIvHHU+R4BrBrJWRzqzd8CEjeMOyG5y2PCncUuDMSPMYtPF/gNiFw9YSk5FIRLfCQYoP4qlZLcjFL/QrxPpcO1YGuQjiuRixBFgAFZGfRPQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Oct 28, 2024 at 09:24:08AM +0800, Baolu Lu wrote: > On 2024/10/27 22:21, Leon Romanovsky wrote: > > From: Leon Romanovsky > > > > The existing .map_page() callback provides both allocating of IOVA > > and linking DMA pages. That combination works great for most of the > > callers who use it in control paths, but is less effective in fast > > paths where there may be multiple calls to map_page(). > > > > These advanced callers already manage their data in some sort of > > database and can perform IOVA allocation in advance, leaving range > > linkage operation to be in fast path. > > > > Provide an interface to allocate/deallocate IOVA and next patch > > link/unlink DMA ranges to that specific IOVA. > > > > The API is exported from dma-iommu as it is the only implementation > > supported, the namespace is clearly different from iommu_* functions > > which are not allowed to be used. This code layout allows us to save > > function call per API call used in datapath as well as a lot of boilerplate > > code. > > > > Signed-off-by: Leon Romanovsky > > --- > > drivers/iommu/dma-iommu.c | 79 +++++++++++++++++++++++++++++++++++++ > > include/linux/dma-mapping.h | 15 +++++++ > > 2 files changed, 94 insertions(+) > > > > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c > > index c422e36c0d66..0644152c5aad 100644 > > --- a/drivers/iommu/dma-iommu.c > > +++ b/drivers/iommu/dma-iommu.c > > @@ -1745,6 +1745,85 @@ size_t iommu_dma_max_mapping_size(struct device *dev) > > return SIZE_MAX; > > } > > +static bool iommu_dma_iova_alloc(struct device *dev, > > + struct dma_iova_state *state, phys_addr_t phys, size_t size) > > +{ > > + struct iommu_domain *domain = iommu_get_dma_domain(dev); > > + struct iommu_dma_cookie *cookie = domain->iova_cookie; > > + struct iova_domain *iovad = &cookie->iovad; > > + size_t iova_off = iova_offset(iovad, phys); > > + dma_addr_t addr; > > + > > + if (WARN_ON_ONCE(!size)) > > + return false; > > + if (WARN_ON_ONCE(size & DMA_IOVA_USE_SWIOTLB)) > > + return false; > > + > > + addr = iommu_dma_alloc_iova(domain, > > + iova_align(iovad, size + iova_off), > > + dma_get_mask(dev), dev); > > + if (!addr) > > + return false; > > + > > + state->addr = addr + iova_off; > > + state->__size = size; > > + return true; > > +} > > + > > +/** > > + * dma_iova_try_alloc - Try to allocate an IOVA space > > + * @dev: Device to allocate the IOVA space for > > + * @state: IOVA state > > + * @phys: physical address > I'm curious to know why a physical address is necessary for IOVA space > allocation. Could you please elaborate? The proposed API is not only splitted to allow batching of DMA operations without need of scatter-gather, but also allowed to users without "struct *page" to use it. In IOMMU and DMA layers all operations are performed on physical addresses and the API "request" to provide "struct *page" in dma_map_sg/dma_map_page is not truly needed. In this specific case, the physical address is used to calculate IOVA offset, see "size_t iova_off = iova_offset(iovad, phys);" line, which is needed for NVMe PCI/block layer, as they can have first address to be unaligned and IOVA allocation will need an offset to properly calculate size. HMM and VFIO operate on page granularity and in simple case they don't need alignment. In more advance scenarios, they will benefit from this offset anyway as it will cause to reduce of IOVA space. Thanks