From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08F25C28B2E for ; Mon, 10 Mar 2025 18:45:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D72A280005; Mon, 10 Mar 2025 14:44:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 660D0280004; Mon, 10 Mar 2025 14:44:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50244280005; Mon, 10 Mar 2025 14:44:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2F4CD280004 for ; Mon, 10 Mar 2025 14:44:59 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 66CE4B87BE for ; Mon, 10 Mar 2025 18:45:00 +0000 (UTC) X-FDA: 83206518360.03.A4A17FA Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id 67BA1C0005 for ; Mon, 10 Mar 2025 18:44:58 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741632298; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kMeY2azJlaEYJvWqYppeQbwJYolhSJED1SI07Gzmp/k=; b=TLJo4qkm0bwpxfyks2nSeTUxWFyijInDTcwu+QMt3xxviaFFfWm8E5RukZ7LDXbXq1J1HK /O+KDoluI00Ku8QDPWyNeeb0c6ncV7yppNJPtbPdUL52wyU+p3eUqTSZfrma9hYowM8Iv1 7wp/kIJcfSoaSYjZ0Be/STuBvGWlr5c= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741632298; a=rsa-sha256; cv=none; b=F7AYtWczboR+G41eOvV3TLS3P0D6A3qR+/rn4cjHDqyD+AhJNptYZ/6CYtYZ24zQ1ZAtkA JJ897qxYi0Eh/mB92Ij2LXvz3phvZ8Va/HkyubcupQdfC9Oem4r5s0hCU6Ci43fFUByZTs +G9/+E1qVbQIqxe9zzq6Mgyxh1/hejU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ADD28152B; Mon, 10 Mar 2025 11:45:08 -0700 (PDT) Received: from [10.57.39.174] (unknown [10.57.39.174]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9AB873F673; Mon, 10 Mar 2025 11:44:53 -0700 (PDT) Message-ID: <0b057c55-fe02-4c83-af69-37770dc83eb8@arm.com> Date: Mon, 10 Mar 2025 18:44:51 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC 06/12] dma: direct: Provide accessor to dmem region To: Maxime Ripard Cc: Andrew Morton , Marek Szyprowski , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , Maarten Lankhorst , Thomas Zimmermann , David Airlie , Simona Vetter , Tomasz Figa , Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , linux-mm@kvack.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org References: <20250310-dmem-cgroups-v1-0-2984c1bc9312@kernel.org> <20250310-dmem-cgroups-v1-6-2984c1bc9312@kernel.org> <2af9ea85-b31d-49c9-b574-38c33cc89cef@arm.com> <20250310-expert-piculet-of-fascination-3813cd@houat> From: Robin Murphy Content-Language: en-GB In-Reply-To: <20250310-expert-piculet-of-fascination-3813cd@houat> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 67BA1C0005 X-Stat-Signature: m6fxphu88g85us4pcnsmd8rs6m5qcyjr X-Rspam-User: X-HE-Tag: 1741632298-933524 X-HE-Meta: U2FsdGVkX18qtOuwmiHtI95MkL431j6iuyRBAJMSwkP/2PAHwJE64yxwGt0EC+vGmHzcWVCiPhqER9Efsljr/9MFubU1/znQQLLdINRnbhbixEkV/kwZE5HPBjk8e+FJ1McBmbgEa9tfW3hWK7Q4GctHCLbGZ0fHz5+hEpN5F5PKKix/MTLuq2yCltaVFA+U0hv1HOqvq/piYqqxTC5JscXv+N2XVDrTMJMJ6o1j0M4RKEedpmbNf+NM0NbI4LcLOagTXtn7tyYxgQiUExvn5/E6ghflOXI+w4el3DDUBZxm6L/edY7lrvZiOqM64ZmRY80yDlUObBJgp6N8sT9BIV/ok1nH9c5KNJPhTXelkKANVFhzqmPI/Dgg0JH2LPi23nEvMi04Sa0iTVNtCrjxic59gsqBVOmOKefYCAj3eb1TYU303Gi/gAlTEyd4QHoCc4n+uaiGjjFNsZZNN99NivpQyWk3zSmyv+Xgz9Yg6rr3lUvSsWiWYaZmXuxPqiLNT/MTD5nsTSMRpzjSOESbtGBs6W8ouXfeWKW/6L3W7TH9UOLvCr6TSNkwJuim893wan3/c5kta5c5IUXNSMd7iu8chhWi/8xkNtePxcwADaTOVE1PI+YxTQM4brApMJr/YMeFUeDZMyZlHwEN8iF6Z3Ir/l2bja2tJJ767vXouGWxMfz+ZORysuD1z+PNKq92f0TgUi6oHEzId89rtIkA5tevMSr8SCNiUtb/YvTL3Qv5Vk1ZSmZc5CpJNVio2dIBcqSiTC56U6VIDyYg//PxUVHeYLxDE3e4r+2CaWAfNQFrsak4m/TlXOV1ilY2P8WoUTKs1ZHw1uhsB1pP+2J2nFgeTJP6b35Bq0z3Ieuzpdoo0lMkoFIQRm9QB7JS679C/GkhMNGQIP5i2fUMJUs9rPnIc8OaLhZWh4hg+NTQBPiOgl1xGg+W8niu4C/Y1zxUdPnStUeTe9b4SiAzFUq J2lF3ynq z7E2Vcf+T96PKhvrofdmmXA2atFnP9gM9gLBwOke9/qp2NcYRQ1NVFvZLut4pJ4unhA4SIVsbWpFIvxD01MrMdj+Wvcljn7/F4wko9gvGnlsb1CoAcBv65TtLzZQTvbL+qekkGe2TtVK3veo+Fg6VVKWawjLynO0YEj968LToTMdoGw/fondYM19A1vnav4eKTCK/7VeuAlt344AOWv7jQ5v1jITHvKwYCqZd X-Bogosity: Ham, tests=bogofilter, spamicity=0.003612, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025-03-10 4:28 pm, Maxime Ripard wrote: > On Mon, Mar 10, 2025 at 02:56:37PM +0000, Robin Murphy wrote: >> On 2025-03-10 12:06 pm, Maxime Ripard wrote: >>> Consumers of the direct DMA API will have to know which region their >>> device allocate from in order for them to charge the memory allocation >>> in the right one. >> >> This doesn't seem to make much sense - dma-direct is not an allocator >> itself, it just provides the high-level dma_alloc_attrs/dma_alloc_pages/etc. >> interfaces wherein the underlying allocations _could_ come from CMA, but >> also a per-device coherent/restricted pool, or a global coherent/atomic >> pool, or the regular page allocator, or in one weird corner case the SWIOTLB >> buffer, or... > > I guess it wasn't super clear, but what I meant is that it's an > allocator to the consumer: it gets called, and returns a buffer. How it > does so is transparent to the device, and on the other side of the > abstraction. > > I do agree that the logic is complicated to follow, and that's what I > was getting at in the cover letter. Right, but ultimately my point is that when we later end up with: struct dmem_cgroup_region * dma_get_dmem_cgroup_region(struct device *dev) { if (dma_alloc_direct(dev, get_dma_ops(dev))) return dma_direct_get_dmem_cgroup_region(dev); = dma_contiguous_get_dmem_cgroup_region(dev); it's objectively wrong given what dma_alloc_direct() means in context: void *dma_alloc_attrs(...) { if (dma_alloc_direct(dev, ops)) cpu_addr = dma_direct_alloc(...); where dma_direct_alloc() may then use at least 5 different allocation methods, only one of which is CMA. Accounting things which are not CMA to CMA seems to thoroughly defeat the purpose of having such fine-grained accounting at all. This is why the very notion of "consumers of dma-direct" should fundamentally not be a thing IMO. Drivers consume the DMA API interfaces, and the DMA API ultimately consumes various memory allocators, but what happens in between is nobody else's business; dma-direct happens to represent *some* paths between the two, but there are plenty more paths to the same (and different) allocators through other DMA API implementations as well. Which route a particular call takes to end up at a particular allocator is not meaningful unless you are the DMA ops dispatch code. Or to put it another way, to even go for the "dumbest possible correct solution", the plumbing of dma_get_dmem_cgroup_region() would need to be about as complex and widespread as the plumbing of dma_alloc_attrs() itself ;) I think I see why a simple DMA attribute couldn't be made to work, as dmem_cgroup_uncharge() can't simply look up the pool the same way dmem_cgroup_try_charge() found it, since we still need a cg for that and get_current_dmemcs() can't be assumed to be stable over time, right? At the point I'm probably starting to lean towards a whole new DMA op with a properly encapsulated return type (and maybe a long-term goal of consolidating the 3 or 4 different allocation type we already have), or just have a single dmem region for "DMA API memory" and don't care where it came from (although I do see the issues with that too - you probably wouldn't want to ration a device-private pool the same way as global system memory, for example) Thanks, Robin.