From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6864BC3600B for ; Mon, 31 Mar 2025 19:10:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7B52280002; Mon, 31 Mar 2025 15:10:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2C68280001; Mon, 31 Mar 2025 15:10:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CBC3280002; Mon, 31 Mar 2025 15:10:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6B671280001 for ; Mon, 31 Mar 2025 15:10:54 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 14C3453089 for ; Mon, 31 Mar 2025 19:10:54 +0000 (UTC) X-FDA: 83282788428.21.E4D7151 Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf01.hostedemail.com (Postfix) with ESMTP id 0D2F640003 for ; Mon, 31 Mar 2025 19:10:51 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=KK9KNglT; spf=pass (imf01.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.50 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743448252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Dt32gY7PII308zLmCJyd2JbPBk7pRfyqNOcGW9IblNM=; b=rvE+vdd9rXqihUH9Bo5L4u2aLOo9YZtVNF+gT6ZHfDF7zWAxBhcUHHfZtwU5dhh2OZauEY ++ihqFwwDy67irPmeXTOqL3iLsq7v9cZgUHKu5YOO/idYgDGbaPabTzJb0JuTCSAvoCq7z /NAZ7Icop1KpbAxLa0QUgHLs3PS+Waw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743448252; a=rsa-sha256; cv=none; b=GA//PX592znvjQ+MGj/CrybvNDObsmg7SHx83BYuy8N/9+jtNB5ZDG+nSktP/f27wIr76C isJXS7SqJNVyBcxgP6v7xRKutRbQbXVHZgJlCRIpBrhvBn9mv1j1XpVbwxBa1l0rz/9PQl C2dSiMCekJfYgXrLE1OADU/xG7jTBqg= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=KK9KNglT; spf=pass (imf01.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.50 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-6e8fd49b85eso72454756d6.0 for ; Mon, 31 Mar 2025 12:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1743448251; x=1744053051; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Dt32gY7PII308zLmCJyd2JbPBk7pRfyqNOcGW9IblNM=; b=KK9KNglTuFtfY8aDYlpwAKlKvSrjpn4Dv/qqVnhwVLirL+yMrTIwd5pYXP7yqSacHZ 0pRVrM5eXMs0uo6xbirDsOdYfVwQvJDt+qfh36+Ek+hSHAhycaei77mboXZ42pDWX7M1 2hK0SmzFLFv0hwjhGnDiOgA1O73KCg7L0WHo/gdIIfFXYSuIF12bznvlcyMSTUw8rDn2 dhjDaxWmhYoEwO8lww/1pDvoy6azp2L0XuxN8d2EnDtBzQer+0bwA9GQ09J1EWw9rfyT w8UPPKLJ0GAAz1OGJQFwFj0DXs/CRbAVNeKMLAGIy/1iRFayTUEo+OOVG+ViJfUTmHe0 EnTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743448251; x=1744053051; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Dt32gY7PII308zLmCJyd2JbPBk7pRfyqNOcGW9IblNM=; b=j+scd/7vT4RnebUqkE8D/CfQyXJs+/nmqe+ff5GbcaiyluvUcW4n0HmYRUg/g6vsdy ZmUNWrBQ0KglqsN2J6lLLkp+nRZbTD1t39RBoatix1MMpl2GtWGfRwW/ku8C8v+2AFWl 4gJMUPU9DwbAUZVPQGHDpzjLnLy1QxxnH5pq2DaoH7x2nESSdEperUnUHosmvLXez0re qUTiYlEYBuHif4YqmlbFoI05AOUX7uYGt9vt6SPQADexz5iJzcwJKbskLo2CxmnV+mDH rPTf+mFvNabK/lrR+F7auyxbPkVj7XUDXE8pIjTdPi7l3eVrjMYpvEsnp1DZLcNsPsGV N1Ew== X-Forwarded-Encrypted: i=1; AJvYcCVokWXStz/ND1nxNL9JATRvR2AIcipnxgWAE3o2JVV4hjDUgowxfK4U1tIZ0UzgViiZYq5fWNUj1A==@kvack.org X-Gm-Message-State: AOJu0Yxg7Cm3bcDBsDYrCILbd7DiVNoLH6k7D8kUrTg1ri/QzgHAO9Ac y4Tv0E3L8DQ708y7n/FSnl/G1L90SdWxDphkaALUnVLn0yC89473EezCc6F39P8= X-Gm-Gg: ASbGnctNiDjfG51cnKkJCaNQDMoqC17Do0FATCNvHwAhj8yhbruUvqKkbcTkBlzSmxJ RvqSnj1CpmOZNZWkulNhCfM3nJEHRE5+occoAn8JHY/AxQ8xYqw4HvMHNbcBywA3bPA6695baPI 7W5o8N5do0iRWZZNUdE4AhOPXHHre0wMrvhmRjgbGb07uC1IYf8pLs3EAURPouP3sXYClOFr/wh whiAGvWROAnkHOgxh0Bfule1tiYxpI65PjBstCx1iop5s6k30TaWbq9EHmXSWBkmr8jIzijL+zo JltfYpm1DkfDRz6vHnSFACzH8YcpKpmofw7N4Gs9rKS17FnrLtrpQhsOU9h5lcf84DKr3f/pB+Q SqxAsAeQnq6yZ0TtWX5/5eh4= X-Google-Smtp-Source: AGHT+IEZHkY9cceG9Xv7rEu7PB72dn7pTp6BwVJLHoJ1MHilW0TdQewa+AGCIOjjtFFGEUJX/PJSGA== X-Received: by 2002:a05:6214:19cd:b0:6df:99f7:a616 with SMTP id 6a1803df08f44-6eed5f4e587mr150303596d6.2.1743448250916; Mon, 31 Mar 2025 12:10:50 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-167-219-86.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.167.219.86]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6eec9797136sm49986056d6.103.2025.03.31.12.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Mar 2025 12:10:50 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1tzKX3-00000001EjT-33eb; Mon, 31 Mar 2025 16:10:49 -0300 Date: Mon, 31 Mar 2025 16:10:49 -0300 From: Jason Gunthorpe To: Marek Szyprowski Cc: Leon Romanovsky , Robin Murphy , Christoph Hellwig , Jens Axboe , Joerg Roedel , Will Deacon , Sagi Grimberg , Keith Busch , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Andrew Morton , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, Randy Dunlap Subject: Re: [PATCH v7 00/17] Provide a new two step DMA mapping API Message-ID: <20250331191049.GB186258@ziepe.ca> References: <20250220124827.GR53094@unreal> <1166a5f5-23cc-4cce-ba40-5e10ad2606de@arm.com> <20250312193249.GI1322339@unreal> <20250319175840.GG10600@ziepe.ca> <1034b694-2b25-4649-a004-19e601061b90@samsung.com> <20250322004130.GS126678@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: 6e3qsfkwcamcghfy7rteq7w8fwmfxqj5 X-Rspam-User: X-Rspamd-Queue-Id: 0D2F640003 X-Rspamd-Server: rspam08 X-HE-Tag: 1743448251-479443 X-HE-Meta: U2FsdGVkX19Qu8LcN3tTDQHlquUWNMoSN+TPDW/Q9iJqFeTQNrx+glsw7ZEi/Jh/PjFh4mMeiPfCx9C2fb9KxnoLwgQ02OX2MMXfCVMKoa9HyXKvWNLHcN6+OFbaB8OUutMJIl/Fc4eopGiqGdBhJ5zdeRIEeG/yjAi3VXW6zEcCStI5zYzKAZzQajsLk0UUqN8cXNiGHa8CrdUkPF3wGurelBvU+5A8hB894oilxMKNCi/YQx5WsrPCtxHlBJfIDp04rWVRqlUsVZf7JKoA5s59R1rUre7wSw1FM+VBibDEClAtDa7jA54EcWveeTV5mWN2z0emcL3DHeS8Il3hNSOrjI/ZoXKSwxer25EiTDhbfzMSYTHuWHkuywGJd1BPffFl39ZxnHWihJ152YQ1wi8w3FFGwIW/uocIqZzxeJzoKooQRtZaQzyhaRq5f53LQl+8tZOsiEapZRvZGvgxo5gu9xOVX2RNrk95wRWQd1VfKlVBHDLoPlZpIpqpaYNNMSfj/Fl5AolHFEje9Mq4EVFsZQyRjEsQZFGAsQyWVm1l8+G4kooSgDYvBOOATTHDaMrBxnY0CiwDznL9Hd2G4qf42FIudMYEjabDbaUZw6Kh01oEOq0vFlGfJ8mWnT8X3ESjZmKFzuDT8EE/rxAVYSf0FAO2OvcAZSUr5ly3Aby9J90fCrWjYMR+w9cRTgPurdNuybMCEhOuBbWpKPymjw56zPCzH5e2wwaK1l1cllFipFVlRKhgvEnpkcrvKzngjfG5cS7y48CS61yEvDbIp94z0UWTj5CcxV+9sGpOjrhcBDc6zNBSspK2+rtlZhvbdEr8CjbhVumNixxqNHVZ1IYthrwYvNYFVnSDf24SbTlp6+XuNxXCIG9GER8F5n5ziUpleDIO8q9yw/hrtxufx6YnGxixvuBRk4++0GGNrBRgBbJ+E3qN9m7coB2RyQTVQrp1W2klPToDSzh8fYl wHm7NpP+ WmYznVljkX/JY6+w3kQxiEEyI7+91UWx7T45x7mDMt19PPDhMAGhf32z9YjLiLf4GLNxN1UgTX5akRXhGCY/r7uhNJs/D9lPhdHGOeyhX74wKDzopDw4hF6rsTpwyjyc5ljGNec3qzcGsA1TXpcWFNI0ey3xYoavrMvwa2zNh81H7IbY9B2vfnEMrUcnRVI+i2TCdafVAcwWPitRxZcAqXu9lIvAmjds5oSIZ7hDFe6DIOajzpzsZD8GrD4e6UvDRssT/RPczbZaNXLAhaTenyP0PErPV5+miiaJz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Mar 28, 2025 at 03:18:39PM +0100, Marek Szyprowski wrote: > What kind of a fix is needed to dma_map_resource()/dma_unmap_resource() > API to make it usable with P2P DMA? It looks that this API is closest to > the mentioned dma_map_phys() and has little clients, so potentially > changing the function signature should be quite easy. I looked at this once, actually it was one of my first ideas to get the VFIO DMABUF side going. I gave up on it pretty fast because of how hard it would be to actually use.. Something like this: dma_addr_t dma_map_p2p(struct device *dma_dev, struct device *mmio_dev, phys_addr_t phys_addr, u8 *unmap_flags, size_t size, enum dma_data_direction dir, unsigned long attrs); void dma_unmap_p2p(struct device *dev, dma_addr_t addr, u8 *unmap_flags, size_t size, enum dma_data_direction dir, unsigned long attrs); Where 'unmap_flags' would have to be carried by the caller between map and unmap. mmio_dev is part of the P2P metadata that dma_map_sg gets out of the struct page via the pgmap. unmap_flags is the big challenge, there is not an easy data structure to put that u8 as a per-call array into in the DMABUF space. It could maybe work not bad for VFIO, but the DRM DMABUF drivers would be a PITA to store and this wouldn't be some trivial conversion I could do to all drivers in a few hours. But, the big killer is that such an API would not be suitable for performance path, so even DMABUF might not want to use it. If we start to fix the performance things then we end up back to this series's API design where you have the notion of DMA transaction. The more of the micro optimizations that this series has the closer the APIs will be.. So, IDK, we could make some small painfull progress for VFIO with the above, but replacing scatterlist would be off the table, we'd have to continue to live with the DMABUF scatterlist abuse, and there is no advancement toward a performance struct-page-less DMA API. Jason