From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C72CEC54E4A for ; Tue, 5 Mar 2024 10:15:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 469216B00AA; Tue, 5 Mar 2024 05:15:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 418506B00AB; Tue, 5 Mar 2024 05:15:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21E416B00AC; Tue, 5 Mar 2024 05:15:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0BDA16B00AA for ; Tue, 5 Mar 2024 05:15:46 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C2DA9C0E00 for ; Tue, 5 Mar 2024 10:15:45 +0000 (UTC) X-FDA: 81862579050.06.9C622D3 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id 417AFA000B for ; Tue, 5 Mar 2024 10:15:44 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XCEvQ40z; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709633744; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZV22DEkctZnBDhVO2mpwxlurhUyOLjlkYP+TkKjvMOA=; b=sFE2TyrwJX3+xEwp1NhGTXq+kKiNmiYI+0LGcHV8wRVBoW6IZUzouSqQHp9dbTYRJkCRtT 1LTVhmErVMGPPZ2SkoX6dHHybTbyN70CIuCD/c6n9bBEdkKtcvrVL9KV91yKat0P5LzCSN ogUY/+L0jymRfiOgonuivaJ5t2HT8A8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XCEvQ40z; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709633744; a=rsa-sha256; cv=none; b=6OxhDWYXHWv1QL2wboHTxUL90f/4RbQurm4tg3vjhXQ1viQGF30oUQ9Rxqz2p0H6dJvrm1 Exrfveif5cnWwvBprd67RhSLVMSG+7/LD9Iiv4P5mp/qxhvtKvjDPsP//2j7pk5LW+is4+ DK/FL8aXQHwD0xah0Y1QZo7flpGRb3s= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6063761489; Tue, 5 Mar 2024 10:15:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F837C433F1; Tue, 5 Mar 2024 10:15:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709633743; bh=XGhek5Egl9EZN5GDA4K/P4lgi/cIEFOzTc1LzkqAZ+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XCEvQ40zftA5CzahxVADDhIsF5HF2PFIaFCKJl08PAjcb8phQRdD1LDvHI07G+6PZ CvBdJkiMC9pln2pklakFZdq+NgxiSqfQtDr0p0vm+4uPAZzhgnykkeCD4vUFgHznRW ZOBma0N0c7UfO8tz8sA3pTWlv0aFd55k7B11BsajPMqaNfTE2HFLWk/aE5awQigsm3 8YGZDR4iQoO0Eh3bVsL7tCcIfm91jjBwAWwMtpH2xHx88UXEnRem5M/qO+QvY/fybd zRSV/V3Bo/1pG1mbTkyZbbDGE1jDM3WdTIBsJpv95ElPKrvNelIborsyuhPgbCuKH7 DHXrXaxWh7khg== From: Leon Romanovsky To: Christoph Hellwig , Robin Murphy , Marek Szyprowski , Joerg Roedel , Will Deacon , Jason Gunthorpe , Chaitanya Kulkarni Cc: Leon Romanovsky , Jonathan Corbet , Jens Axboe , Keith Busch , Sagi Grimberg , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, kvm@vger.kernel.org, linux-mm@kvack.org, Bart Van Assche , Damien Le Moal , Amir Goldstein , "josef@toxicpanda.com" , "Martin K. Petersen" , "daniel@iogearbox.net" , Dan Williams , "jack@suse.com" , Zhu Yanjun Subject: [RFC 02/16] dma-mapping: provide an interface to allocate IOVA Date: Tue, 5 Mar 2024 12:15:12 +0200 Message-ID: <54a3554639bfb963c9919c5d7c1f449021bebdb3.1709631413.git.leon@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 417AFA000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: hunqmut3wyrrtnqje1887yg9sguwrryt X-HE-Tag: 1709633744-938687 X-HE-Meta: U2FsdGVkX19ywEDccTSGEHk/pxdmrq4LtpEPXjjmFI/kDhzo8CQE2pfvFd1fbKmjDEhHpV5goycT5gc/lO3Ptn90JxK8VpNXmznC8sM5+Ihw88jurZf7Ak1XBXynM8YL7acpIVZ/kWjg7DKrdqMghkmJS/eGbezyiy2nNRSzR0eA3d0gnQVE1df6K6fZjtOO6ZD/fasAs0cW1PDQAd274Eh6CVsIp/kls405PGmLYSg8O3eRJwAomRdv70aes832wKhhErioLqD7p2BlPoCLaeO6Fac7iJH76F4HJPf1L0c5LSVgk4vnOONp1A+slUMddD6m9UpUlzDnUn2hSS0j+kERRkUMekHuMD8KskdrCnj80FI2mmes6oDC+Dgu8CFqc+0ddbL3c7jrMT35hfvdANbEBxffqVw/x/roAg1EwNhCIG2mzkEew7F6XHmJJSzCuhc3L5eu9/YEMVmRDjfmvsqWZSTpZVbLH87CmvJd3pADY14Ys7RudX1BcKWR5qnluzUe+3p1kPYate0tnVaJeEQ6HOROePVQj0Hv1kegcmdUg9FJgRxnRUwzMcC7h2g2VwHRcBQQ4JS6tP9LLSOoWg+jN1pAyr29fM5uc2qGyW567h9yT1Qm3SmG9CHPakJ1mgoMp/4EWaIqRitIR9bIK0+/A8lVGkqPM2E8LQWIYlOxnNHW9HZNAtEWJSWmcljh6sw6dlpTaUeBtSFN/p8aK5prbPmKt2QqQFUkwvSGGEJZvwWlVOsZmqNDuo+2c4zYGoqUV37P1DLM/ENgNkE7mDvPCPYp+x0E5ZFBJNEXxlOm6kXt8q57VTAe/5eLVdeS+Cg81VX0EUn6JiFKfymtvTgBZT2Hf4I1BDRUToRz0JilmP+IUMpPEEj0+v1BLvwLekM9eZ1Hc0w9ykzzWX1C1Vut4ickGADxAZ6VvKzTlDKjaXlu97iX+Cin1lufhYgCFbBIQIMXlQQEmgz4AuO qQqesdMw j4pvV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Leon Romanovsky Existing .map_page() callback provides two things at the same time: allocates IOVA and links DMA pages. That combination works great for most of the callers who use it in control paths, but less effective in fast paths. These advanced callers already manage their data in some sort of database and can perform IOVA allocation in advance, leaving range linkage operation to be in fast path. Provide an interface to allocate/deallocate IOVA and next patch link/unlink DMA ranges to that specific IOVA. Signed-off-by: Leon Romanovsky --- include/linux/dma-map-ops.h | 3 +++ include/linux/dma-mapping.h | 20 ++++++++++++++++++++ kernel/dma/mapping.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 4abc60f04209..bd605b44bb57 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -83,6 +83,9 @@ struct dma_map_ops { size_t (*max_mapping_size)(struct device *dev); size_t (*opt_mapping_size)(void); unsigned long (*get_merge_boundary)(struct device *dev); + + dma_addr_t (*alloc_iova)(struct device *dev, size_t size); + void (*free_iova)(struct device *dev, dma_addr_t dma_addr, size_t size); }; #ifdef CONFIG_DMA_OPS diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 4a658de44ee9..176fb8a86d63 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -91,6 +91,16 @@ static inline void debug_dma_map_single(struct device *dev, const void *addr, } #endif /* CONFIG_DMA_API_DEBUG */ +struct dma_iova_attrs { + /* OUT field */ + dma_addr_t addr; + /* IN fields */ + struct device *dev; + size_t size; + enum dma_data_direction dir; + unsigned long attrs; +}; + #ifdef CONFIG_HAS_DMA static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr) { @@ -101,6 +111,9 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr) return 0; } +int dma_alloc_iova(struct dma_iova_attrs *iova); +void dma_free_iova(struct dma_iova_attrs *iova); + dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs); @@ -159,6 +172,13 @@ void dma_vunmap_noncontiguous(struct device *dev, void *vaddr); int dma_mmap_noncontiguous(struct device *dev, struct vm_area_struct *vma, size_t size, struct sg_table *sgt); #else /* CONFIG_HAS_DMA */ +static inline int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + return -EOPNOTSUPP; +} +static inline void dma_free_iova(struct dma_iova_attrs *iova) +{ +} static inline dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 58db8fd70471..b6b27bab90f3 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -183,6 +183,36 @@ void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr, size_t size, } EXPORT_SYMBOL(dma_unmap_page_attrs); +int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->alloc_iova) { + iova->addr = 0; + return 0; + } + + iova->addr = ops->alloc_iova(dev, iova->size); + if (dma_mapping_error(dev, iova->addr)) + return -ENOMEM; + + return 0; +} +EXPORT_SYMBOL(dma_alloc_iova); + +void dma_free_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->free_iova) + return; + + ops->free_iova(dev, iova->addr, iova->size); +} +EXPORT_SYMBOL(dma_free_iova); + static int __dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs) { -- 2.44.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6767EC54798 for ; Tue, 5 Mar 2024 10:22:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E0C9B6B0074; Tue, 5 Mar 2024 05:22:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D6E326B0081; Tue, 5 Mar 2024 05:22:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4C5C6B009F; Tue, 5 Mar 2024 05:22:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9C2AC6B0074 for ; Tue, 5 Mar 2024 05:22:42 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 65F97809F9 for ; Tue, 5 Mar 2024 10:22:42 +0000 (UTC) X-FDA: 81862596564.30.97C6DD9 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id 291B5140007 for ; Tue, 5 Mar 2024 10:22:39 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eS51gVAa; spf=pass (imf26.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709634160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZV22DEkctZnBDhVO2mpwxlurhUyOLjlkYP+TkKjvMOA=; b=AY6DAbIeIxbOkm/gdsxPp1ssLbJy33jzp/P/ciBh4cQ3ZOmeRE8fa6Wl1Tyw/4siT2XhIH BG6OzepM1iLzc94pDIpHVJvVTnQfAIged+l/xQm90nKluY9AKDg71cPZvuzFYlmSRqudMs 1CyWI+MT79GuyAhRk3fnISawQ2c7Eqo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709634160; a=rsa-sha256; cv=none; b=rNX72NwYQLnvA8n9uhVUINgW6BpZj97T8J8Fn73f5luP4l1reF3EadIGiCg/nyA8J1s4nA fO4z/t4dxrRH+qbVbyua54+U4dMyb3FAgrrllMZa2Vo/IGIGzFTxuATGZs/DvIaqAaCMCx P+XUBX2VfKKkJpBZwJDENptWeuU4vdw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eS51gVAa; spf=pass (imf26.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 94904CE19DC; Tue, 5 Mar 2024 10:22:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 09399C433F1; Tue, 5 Mar 2024 10:22:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709634155; bh=XGhek5Egl9EZN5GDA4K/P4lgi/cIEFOzTc1LzkqAZ+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eS51gVAasYJ+YmP3qPog2KbD15dWSXhXc4+SaN+qhGISnjGipSsNrL3biMp40Hc7N 1BVi9y1wJDuy9tk9SIGILkIwvJp5ARv8HrREX8TWbPJuuO/JzUSLYpfyDgcRNlIQTR 2k5m3myNAU7p4VUf5S9pc6CaOOqsKxascob8TpGUaVDFhddASE3rZmxeKSCZQMvdnu tLTU8Bv3H9G/8U2/bVo2D5QJJrh3cxadwsrdepN5EtpTvHMl+l1nwRObFaxrzDtkZK UCd3k20Yfjl9Bo9ygi/4OSseMRBGEPCMfqKMlu+A6JHDOjRMYNF6CHGW8cImfa1GlC AL9el8MxW3vRQ== From: Leon Romanovsky To: Christoph Hellwig , Robin Murphy , Marek Szyprowski , Joerg Roedel , Will Deacon , Jason Gunthorpe , Chaitanya Kulkarni Cc: Leon Romanovsky , Jonathan Corbet , Jens Axboe , Keith Busch , Sagi Grimberg , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, kvm@vger.kernel.org, linux-mm@kvack.org, Bart Van Assche , Damien Le Moal , Amir Goldstein , "josef@toxicpanda.com" , "Martin K. Petersen" , "daniel@iogearbox.net" , Dan Williams , "jack@suse.com" , Zhu Yanjun Subject: [RFC 02/16] dma-mapping: provide an interface to allocate IOVA Date: Tue, 5 Mar 2024 12:22:03 +0200 Message-ID: <54a3554639bfb963c9919c5d7c1f449021bebdb3.1709631413.git.leon@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 291B5140007 X-Rspam-User: X-Stat-Signature: hunqmut3wyrrtnqje1887yg9sguwrryt X-Rspamd-Server: rspam03 X-HE-Tag: 1709634159-13676 X-HE-Meta: U2FsdGVkX182aXi4h68PFv0f8MbUdgwMkafzYzm3cvBwb+F3/zZeVGT6pYX6kEyAXqYdJxJSs/GLPLLdARh5cLADx8jHv1A2iSTyzAkb2yOQpUeJB3JxBHLMRqIbvdN7Bwr3lQsh0HQzUWOCpotrej28kuZYPPHZ+o1kgfOfYR+ouLg4arZe+d9eKs3YTDl/QA/PLc9qC8mm9+VjBB+m52bsFGD+rs8MCi/AoE6bnoux3JDQeQv/FmUddkAtCGpbV+KT+01wPWVb7gbwfS+PsyyM9WIue2MJi+6h1MohsAtl+K8cxV5nq7p9+4OTgROwKRdnkT8C6Lz2o0w/icD76YHU5Lv+I+gW1dyBejI++bQoguQOnMo75FJjQVBPazq5+6rdiwukPIYpOSb9KA9ZnJDDJ9F2gjA0mpVx4W4oSldwlMsVRJKnxczHqQ14Rm4qGnY5JYiFEmM+/tGCYhRlHyOLbRPxtxNny1QAmz4EFcsc/onrzSPZFzTzKX+OZOy75Wup4JKfnKqRYbeiZ3nuRhxiHrFlfHXD9UrJQAL5ybVcakFGWxmYrvyeZWvKzIrsXoT4G/8j1VCEwwWV14xL55B9XKO9XFkNtvfYkWPLqP1kNZ/JyvZYlHCqNiKrC6hw4K4hL0LDHxrQogbV2MRj722OmM3Y1ySFR9x0tlWshrGYbXjuOa/xNFDS/lj3qRcwzElC0XfJFQTd4c0IuUfCeU5ZHhkpwFo4C2xCfUS8qRiC/mas9ROksFq+31UIXo2NxOocMbzpzXq8aORTUBfOS/JFMBkPLGY6fP3yI3b4741ikK/6OxiGr4td7x8lKKrUJy76nb174V6tUfSdB0sJ00+uqaoNlkKXO0E3D5lg/Jf26+HvYOpIhgCMY0vnL5rXW81VXkgG8vnzWWaOhDG6A8emirRSfho76H/NtEiNnFh6iiMFagPm2Sxi5DIJAtT1SPexwonpHNoBjwpjYdk G7fw6cJF YZ0W3W+s5Vb+jdESQ1qoyCXVXqs4vl7m7Y9AFqFRJdQ+sNNWSmhUyDCyqAlrYHZnRX4Iq+ETxL2OADMHkt7EDMN57r8KEFhHTs+yYLMpH3tlv6ixeZBwJj6qy3caPVZyxS60wJVwd93PRJlsHUuUIZIaBv7lnXQ64LNBqIMLm9b/NATKJWrzWWAQws+rDt3W8CJ3E5Uvcp+lOt7lXyHBttXabTX/4QZOm4zYqu6PTcyeE/5TKw2tDxgMDIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Message-ID: <20240305102203.KYNHj-BxhyLs9IMGSugnkF_H_wESJYr2RJm4dUxUXs4@z> From: Leon Romanovsky Existing .map_page() callback provides two things at the same time: allocates IOVA and links DMA pages. That combination works great for most of the callers who use it in control paths, but less effective in fast paths. These advanced callers already manage their data in some sort of database and can perform IOVA allocation in advance, leaving range linkage operation to be in fast path. Provide an interface to allocate/deallocate IOVA and next patch link/unlink DMA ranges to that specific IOVA. Signed-off-by: Leon Romanovsky --- include/linux/dma-map-ops.h | 3 +++ include/linux/dma-mapping.h | 20 ++++++++++++++++++++ kernel/dma/mapping.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 4abc60f04209..bd605b44bb57 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -83,6 +83,9 @@ struct dma_map_ops { size_t (*max_mapping_size)(struct device *dev); size_t (*opt_mapping_size)(void); unsigned long (*get_merge_boundary)(struct device *dev); + + dma_addr_t (*alloc_iova)(struct device *dev, size_t size); + void (*free_iova)(struct device *dev, dma_addr_t dma_addr, size_t size); }; #ifdef CONFIG_DMA_OPS diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 4a658de44ee9..176fb8a86d63 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -91,6 +91,16 @@ static inline void debug_dma_map_single(struct device *dev, const void *addr, } #endif /* CONFIG_DMA_API_DEBUG */ +struct dma_iova_attrs { + /* OUT field */ + dma_addr_t addr; + /* IN fields */ + struct device *dev; + size_t size; + enum dma_data_direction dir; + unsigned long attrs; +}; + #ifdef CONFIG_HAS_DMA static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr) { @@ -101,6 +111,9 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr) return 0; } +int dma_alloc_iova(struct dma_iova_attrs *iova); +void dma_free_iova(struct dma_iova_attrs *iova); + dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs); @@ -159,6 +172,13 @@ void dma_vunmap_noncontiguous(struct device *dev, void *vaddr); int dma_mmap_noncontiguous(struct device *dev, struct vm_area_struct *vma, size_t size, struct sg_table *sgt); #else /* CONFIG_HAS_DMA */ +static inline int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + return -EOPNOTSUPP; +} +static inline void dma_free_iova(struct dma_iova_attrs *iova) +{ +} static inline dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 58db8fd70471..b6b27bab90f3 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -183,6 +183,36 @@ void dma_unmap_page_attrs(struct device *dev, dma_addr_t addr, size_t size, } EXPORT_SYMBOL(dma_unmap_page_attrs); +int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->alloc_iova) { + iova->addr = 0; + return 0; + } + + iova->addr = ops->alloc_iova(dev, iova->size); + if (dma_mapping_error(dev, iova->addr)) + return -ENOMEM; + + return 0; +} +EXPORT_SYMBOL(dma_alloc_iova); + +void dma_free_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->free_iova) + return; + + ops->free_iova(dev, iova->addr, iova->size); +} +EXPORT_SYMBOL(dma_free_iova); + static int __dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs) { -- 2.44.0