From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D2E2C7EE31 for ; Thu, 26 Jun 2025 13:48:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB9C96B009B; Thu, 26 Jun 2025 09:48:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C6AB66B009C; Thu, 26 Jun 2025 09:48:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B59576B009E; Thu, 26 Jun 2025 09:48:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9C4FB6B009B for ; Thu, 26 Jun 2025 09:48:26 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0463E801C9 for ; Thu, 26 Jun 2025 13:48:25 +0000 (UTC) X-FDA: 83597681412.19.394FA22 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf17.hostedemail.com (Postfix) with ESMTP id EFA844000F for ; Thu, 26 Jun 2025 13:48:23 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=ZnkswdR1; spf=pass (imf17.hostedemail.com: domain of ptesarik@suse.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=ptesarik@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750945704; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/zjavZaKvDBVJ5OFAiCUTocE/OnzRRLYP9bsN9qkVUk=; b=Alf5FhrU1Uw1QFo0+8in0EorrqwV/TRc/YbuYIYhTAFea/Jp7pfK+OtKNo3eniPvAZ+OMa kZSwObF3R1UPHe3g223hKamLP5tzCoMup5jYH4jP/Onnp0o7gABI3mFHX0ncGIzfBfOknJ OE3L5dcVop0/dnsyA4ebmZuU37Rso0k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750945704; a=rsa-sha256; cv=none; b=sNk73uO0HL0NwXzjUAPv4vfDIhKVbYKVlmzTsn0pqDqR6F18UWaf6Esb1HKCsrArIhKYRg ilJmrWu6VfGqLI1E58PjiMGvQLiwMNzB/R+1+5SpVmBFA46jGNrzaw/ml9QBLS2IzR4rxF YiK6RGUFQXitDgK7pSgtWZGULBVhQY4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=ZnkswdR1; spf=pass (imf17.hostedemail.com: domain of ptesarik@suse.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=ptesarik@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-45306976410so549405e9.3 for ; Thu, 26 Jun 2025 06:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1750945702; x=1751550502; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=/zjavZaKvDBVJ5OFAiCUTocE/OnzRRLYP9bsN9qkVUk=; b=ZnkswdR1fp3dfqwiGdN5GjM5FDz3Ewdz8wirlnE0iqbgoez7N++c3cy76p4GYTxHf+ xvwLlEefHH7aTsHODTNjfah5lT661juahRv/xw2muzEl+Kjgrfdm9aYGXf7Kx1Ue9Cek oAUQ/SqKstmr09Jejmuhqnn1bD/GfT0TUT+YqGJufYP24yh2dKSaSm11LX+7OH0i1PMG pw+yhJo7Qzhuv4A1X+Y+xENfOzynT7bzPbcdiKpvUx2jhFE8jZfitAmf+vPOq4jQuPny LrwcXMuQhATyCw/lXI1xWVqQFjm1iVCIy/yAf5hxH8bcN/QfdsKSRn+mSZvOQ4TZ3Ytw 8CrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750945702; x=1751550502; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/zjavZaKvDBVJ5OFAiCUTocE/OnzRRLYP9bsN9qkVUk=; b=iZ03gT49oSmZ4DHWvOrqvquCcK0O2qLxILcZGan0kfLuWiScanCVgphjVIH3CdxtQz UWSgSOXueccF3uzPjLChJTqoPFqJPyNaVO5u00QzvoS5dU2L1VKxy37dome4Gpd3Ww3f P44n0U4NX+eMglX4XOm/Jsze66zwu0PZxNUQMVgy1YkSWIqhOXTTecrlxqVlxVCCSVX5 dPSyj1rb5fqlX/8EHBTVqiNs8ORyHHtlKAjEJmDfU1mOn7TruV0mrTwJfqV9Ec+f7W7L 4jI5BJv/J1cYhw48iTPXJX0mgANdq9W0InYP7WCKFUxdSpELBhXnIOQIcVZXnl53zk4s AYuA== X-Forwarded-Encrypted: i=1; AJvYcCWC/lEneVO37Wexe2dqTHt+ltjLvL+rg9uovElKy4B9Da78SA+UAj+C85/IUJ5oNAbAGklAd6bUHw==@kvack.org X-Gm-Message-State: AOJu0YzAMJSVpHfE3pI3pe4SEJTGcMvXArgUYgcM8Ue5zu+qgF1khHnO LxVoKOo/esY43XUMHwiQahZztTZsdVxcBx/U8o7Fw4gOimN02Db4YN/knmdzo++8mbk= X-Gm-Gg: ASbGncuQUkGNU9BCJLjhMSvg4gC2CR6EnJWU3cGnpMaeM97iOLyak4bG6R4xVXqKpZ9 3eFdTaw69dE4JgSFaGmBCBZhjid5JOKtjIwwA5Y8A138oT7ZBuKjHzooFeYXv/dlGmOTNBUT0IC fSae8z4WSDwReVgbud0E8yqZK7SzQ+3K9+ersrPnLjnp5z9Rn9BIlzJSJqJS8jylEIISj2SHV71 wqFVnf69rd6sWapClm/gYgOe0mmKQZYQnZovofsitv9tKS6pEpy9/e/q3V3B6MaOWfdHt7Q6uR9 xad6nAhZahqlpYVd7XyZAjIiN75YYNe4z4x/qP40G4dH1XGkwhrltuHsMdfNDHChWJvtNw== X-Google-Smtp-Source: AGHT+IGqI8V9vaVv3uCndCWBOBFY2VODqcpVFqg2M7hP6cPXVkW6VuBfDCJqR6MPjmx0smQ1u0Oo6g== X-Received: by 2002:a05:600c:a11a:b0:442:e608:12a6 with SMTP id 5b1f17b1804b1-45381ae86c6mr19913825e9.1.1750945702200; Thu, 26 Jun 2025 06:48:22 -0700 (PDT) Received: from mordecai.tesarici.cz ([193.86.92.181]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4538a390d11sm20938185e9.7.2025.06.26.06.48.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Jun 2025 06:48:21 -0700 (PDT) Date: Thu, 26 Jun 2025 15:48:18 +0200 From: Petr Tesarik To: Robin Murphy Cc: Marek Szyprowski , Bagas Sanjaya , Jonathan Corbet , Andrew Morton , Leon Romanovsky , Keith Busch , Caleb Sander Mateos , Sagi Grimberg , Jens Axboe , John Garry , "open list:DOCUMENTATION" , open list , "open list:MEMORY MANAGEMENT" , iommu@lists.linux.dev Subject: Re: [PATCH 7/8] docs: dma-api: update streaming DMA API physical address constraints Message-ID: <20250626154818.46b0cfb1@mordecai.tesarici.cz> In-Reply-To: References: <20250624133923.1140421-1-ptesarik@suse.com> <20250624133923.1140421-8-ptesarik@suse.com> <20250626070602.3d42b607@mordecai.tesarici.cz> X-Mailer: Claws Mail 4.3.1 (GTK 3.24.50; x86_64-suse-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EFA844000F X-Stat-Signature: xzz4repz96jwunm4qetkqtiaimizsus9 X-Rspam-User: X-HE-Tag: 1750945703-992807 X-HE-Meta: U2FsdGVkX1++rOFBIJa3HCw0ZIQjAUpJcwa/RkhyfnbVhmFJMofIsTbLrt8+4SAN2LMuQabKoI76oB/5hxTgnmvzxMbknj46jHqY1QILSuANYUjt+siDIAQHcgDqMqN8hAqbZnES+rrZZlfijj65Twkk4d7vSyEXf+3rKrr58ydp8t5ND/BbNuOTIkuoFxLXwLz5R2VFbNYkrBUzBhxRWpINKkM/toIDSDR4xO+1uliuP97XbgG8vrjaRDZdhyvbD1roiVLw98mvW8d8cbbdxiE+DMyHkWmA0rQBHqVWA6Gycb9Jo5bblHJKrsX428Jh2LMO23ADaDGaHk+Q8FMav2ik1v+rx5o58dsOtztNUQ0mVLKaVXBbmwVw5aR4nT8rzGnXrd6esZz1ueIRAq+D7O+6/hHCobntJ1jT94S4C3DoiHEEMu8h1vLZ4F+TQP5lruEHr1lH7egOEHtKoDJDeePQJZj6hSvfpGW4oayAqs2AZ+015TPfN5N3P8frGwvSDJPKTOhuOCLUTGOzV1J0+M4MMVxmCmGwGBKGlkLUvunT7LWyVg/9yjOMtaSV2avtL+FV2wSbepX5zNF4qEk2BT7mWyyHVTnZ0Xv/FOxDGXzPkySELASDspFd/RoZe51iFEB6g/Wy2nI0RXhMEPnkFda+92H6144UOxK1GtjZRXybYd0kNLzmKB1PuJNCo+BXolghPnjxfsTom6GKvZ+9cDnceJ4iMerasSc7c4ItJF+wTAVsM0BtpjvB/4n+TUZwCuhzWgFbzxzVJtitV1gATp+b/R5zo/fHPi77J3nZpZ4qyohT3sik+gDsT3vZ4917wbuD+iV0bbn9IWwdYYJha0lDYtx+9xRUMuZjEg9TIQiM9Qub5jECOD9FX886ua0e7Fa92srBH8XCJ9HpokAC1Z+7MLYm4OuKNdJfbsSMz+0HWPs4Fhshzjlz/cU0OyFrWTDxEsmBmrFf7cuvvUJ HUcydjPC kiwGtfLKtyJ37yYcBYiVc/DrkzQGE15uYUo5YtGFsPc11zkOnQThgHUa8nGkREw10svaVMZIx5fnle9TN2Wa8OnGSJoxncrwWEBVxOaHCsmQPMK/whq/gEe760ZMaxcr7NZaS39TX2oUcx58QlkYL5AIKzYyFL2aH6vo6GPkia6R666OgUffyBfXbiAv6Dv/Ai3i06jrR1Zw03yLm0Lvz5k4Kf1OkPRXCX8pgbrUPOjgCIpTtZ5GuWXduZChvGnuKFYlxSxPgCX4mzD7wRaUW+3r2G0vHwyBlcgPS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 26 Jun 2025 10:58:00 +0100 Robin Murphy wrote: > On 2025-06-26 6:06 am, Petr Tesarik wrote: > > On Thu, 26 Jun 2025 08:49:17 +0700 > > Bagas Sanjaya wrote: > > > >> On Tue, Jun 24, 2025 at 03:39:22PM +0200, Petr Tesarik wrote: > >>> diff --git a/Documentation/core-api/dma-api.rst b/Documentation/core-api/dma-api.rst > >>> index cd432996949c..65132ec88104 100644 > >>> --- a/Documentation/core-api/dma-api.rst > >>> +++ b/Documentation/core-api/dma-api.rst > >>> @@ -210,18 +210,12 @@ DMA_BIDIRECTIONAL direction isn't known > >>> this API should be obtained from sources which guarantee it to be > >>> physically contiguous (like kmalloc). > >>> > >>> - Further, the DMA address of the memory must be within the dma_mask of > >>> - the device. To ensure that the memory allocated by kmalloc is within > >>> - the dma_mask, the driver may specify various platform-dependent flags > >>> - to restrict the DMA address range of the allocation (e.g., on x86, > >>> - GFP_DMA guarantees to be within the first 16MB of available DMA > >>> - addresses, as required by ISA devices). > >>> - > >>> - Note also that the above constraints on physical contiguity and > >>> - dma_mask may not apply if the platform has an IOMMU (a device which > >>> - maps an I/O DMA address to a physical memory address). However, to be > >>> - portable, device driver writers may *not* assume that such an IOMMU > >>> - exists. > >>> + Mapping may also fail if the memory is not within the DMA mask of the > >>> + device. However, this constraint does not apply if the platform has > >>> + an IOMMU (a device which maps an I/O DMA address to a physical memory > >>> + address), or the kernel is configured with SWIOTLB (bounce buffers). > >>> + It is reasonable to assume that at least one of these mechanisms > >>> + allows streaming DMA to any physical address. > > > > Now I realize this last sentence may be contentious... > > The whole paragraph is wrong as written, not least because it is > conflating two separate things: "any physical address" is objectively > untrue, since SWIOTLB can only bounce from buffers within by the > kernel's linear/direct map, i.e. not highmem, not random memory > carveouts, and and definitely not PAs which are not RAM at all. I see, saying "any" was indeed too strong. > Secondly, even if the source buffer *is* bounceable/mappable, there is > still no guarantee at all that it can actually be made to appear at a > DMA address within an arbitrary DMA mask. We aim for a general > expectation that 32-bit DMA masks should be well-supported (but still > not 100% guaranteed), but anything smaller can absolutely still have a > high chance of failing, e.g. due to the SWIOTLB buffer being allocated > too high or limited IOVA space. Of course this cannot be guaranteed. The function may always fail and return DMA_MAPPING_ERROR. No doubts about it. > > @Marek, @Robin Do you agree that device drivers should not be concerned > > about the physical address of a buffer passed to the streaming DMA API? > > > > I mean, are there any real-world systems with: > > * some RAM that is not DMA-addressable, > > * no IOMMU, > > * CONFIG_SWIOTLB is not set? > > Yes, almost certainly, because "DMA-addressable" depends on individual > devices. You can't stop a user from sticking, say, a Broadcom 43xx WiFi > card into a PCI slot on an i.MX6 board with 2GB of RAM that *starts* > just above its 31-bit DMA capability. People are still using AMD Seattle > machines, where even though arm64 does have SWIOTLB it's essentially > useless since RAM starts up around 40 bits IIRC (and although they do > also have SMMUs for PCI, older firmware didn't advertise them). Some of these scenarios can never work properly because of hardware limitations. There's nothing software can do about a bus master which cannot address any RAM in the machine. I'm not trying to claim that an operating system kernel can do magic and square the circle. If that's how it sounded, then my wording needs to be improved. IIUC the expected audience of this document are device driver authors. They want a clear guidance on how they should allocate buffers for the streaming DMA API. Now, it is my understanding that device drivers should *not* have to care about the physical location of a buffer passed to the streaming DMA API. Even if a bus master implements less than 32 address bits in hardware, I'm convinced that device drivers should not have to examine the system to check if an IOMMU is available and try to guess whether a buffer must be bounced, and how exactly the bounce buffer should be allocated. If we can agree on this, I can iron out the details for a v2 of this patch series. > > FWIW if _I_ received a bug report that a device driver fails to submit > > I/O on such a system, I would politely explain the reporter that their > > kernel is misconfigured, and they should enable CONFIG_SWIOTLB. > > It's not really that simple. SWIOTLB, ZONE_DMA, etc. require platform > support, which end users can't just turn on if it's not there to begin with. I know this very well. As you may not be aware, my ultimate goal is to get rid of ZONE_DMA and instead enhance the buddy allocator to allow allocations within an arbitrary physical address range, which will not rely on platform support. But that's another story; for now, let's just agree on how the DMA API is supposed to work. Petr T