From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B80EC3600C for ; Thu, 3 Apr 2025 15:47:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27383280003; Thu, 3 Apr 2025 11:47:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 22164280001; Thu, 3 Apr 2025 11:47:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E9A9280003; Thu, 3 Apr 2025 11:47:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E3FD6280001 for ; Thu, 3 Apr 2025 11:47:28 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AF694140B84 for ; Thu, 3 Apr 2025 15:47:29 +0000 (UTC) X-FDA: 83293162218.03.2BE3DDF Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by imf25.hostedemail.com (Postfix) with ESMTP id 0AC24A000B for ; Thu, 3 Apr 2025 15:47:27 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BHiu9zpQ; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of mripard@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=mripard@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743695248; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EaxIEi2MzHezSZZXiYbZ5wYqTfobu9WblfTbW9XFbBU=; b=UQAnPhIMnnfN5D9WBvHk8UdKl9sSGpNLmlRbm1oHBIHMnSCjWS5Nz9ZI0KEQExxxbWSuG7 pUA77Gr9o5eqQUgrpKiRQSmX+2tJBH1x9Tp+xqT4EUUMoct7uikw92ldOL4ldR7h2/HwAl CjmfnQGza4MnvJ6rscJg6nwLndChPdc= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BHiu9zpQ; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of mripard@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=mripard@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743695248; a=rsa-sha256; cv=none; b=bSVWneHn0t+AcZ/zQaLTzriW7/mLr5vbSw++yGTDI4gXpYwPe6DpsuPVX2qkW++ueXlMc6 lX75z/Ho+wO4ZuvreW3pLOYH/UqSCcKYmQ/6Mu7F+hi1rwknZIDasPZnx/Y+UmkyPXIPmd VZM2GD447EVMoFKT90ce+UGgDRh4GEc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id 29A6843AE7; Thu, 3 Apr 2025 15:47:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10A82C4CEE5; Thu, 3 Apr 2025 15:47:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1743695246; bh=EaxIEi2MzHezSZZXiYbZ5wYqTfobu9WblfTbW9XFbBU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=BHiu9zpQPAcxHwkl1Fk/pF6J7ENRms/xF8j934S+wm5det9lbVcH1nobBOu15k51V RmDBtTgJZHYPAdwvl4kISbu5SX2/wkYQ0u1jiGP4hQtG7pu7SP7BNPvH7vTC1xrEIo ltfbDK2HY+se5xOmcWrS/NNPQn+JU+xDg4xJp/d+sv7rTQqOUQghv20wtk5dTFtOFB veMF3kZMzeJReUYQtauA8m+blHkxnPv0xw+OKZwrM1t6or5a9UTXUI5ozuHTVEJYt1 gw8ANUXA7riCX5CUAfPQlKcGsacCkcw1I4VNuBhzP9A+Vm0J7vijuKj1sAzGiztkWX iVhsYPqKATeJg== Date: Thu, 3 Apr 2025 17:47:23 +0200 From: Maxime Ripard To: Christian =?utf-8?B?S8O2bmln?= Cc: Dave Airlie , Andrew Morton , Marek Szyprowski , Robin Murphy , Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , Maarten Lankhorst , Thomas Zimmermann , Simona Vetter , Tomasz Figa , Mauro Carvalho Chehab , Ben Woodard , Hans Verkuil , Laurent Pinchart , linux-mm@kvack.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Subject: Re: [PATCH RFC 00/12] dma: Enable dmem cgroup tracking Message-ID: <20250403-quick-salamander-of-charisma-cab289@houat> References: <20250310-dmem-cgroups-v1-0-2984c1bc9312@kernel.org> <20250310-eccentric-wonderful-puffin-ddbb26@houat> <5ed87c80-6fe3-4f8c-bb98-ca07f1db8c34@amd.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ladwbtqhxfoyjegt" Content-Disposition: inline In-Reply-To: <5ed87c80-6fe3-4f8c-bb98-ca07f1db8c34@amd.com> X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 0AC24A000B X-Stat-Signature: wdk9z86473gufn8b7s4qguasejrrmtrs X-Rspam-User: X-HE-Tag: 1743695247-322061 X-HE-Meta: U2FsdGVkX18LWvsj042kksQeVerCxhD+WmIJEJ+IuNgBjhd84TsyHeJrTodokMiICVZGW+j7dG+Sc00/htx4mls+WGag8LA9mGel1/FLVxpP+vRqDVVUow1nImurx2G4ALHZUo7ZlrK9d7SB3888HXaQwVTtCu3Nc7ZOXUAbhTsUYmAQdgXmdOUpRBmOESWoRtGVLm8eQogqnMcE8fEsMpiRhXdZHBaCdm/EOwiKWhjVcQv8SfA8bzsGi6f7F8fgBIJscz8lVREWt7eUrDG96WsSrGl0XQUV/mgxV6MiS2lCMD1E4pGusfDxLmybpK7k6MQPjTRST8TphaJR5mD2RSQ8D/jl2rG4TXKWjk088rB4OHbZRaHA/5DVpVeJoJ1M6em7EyRXP94uogZaRLjmx1jlBJdg0on9LpzSmQOfrh79axEqmXUQ4zCemxE5bCqbGo5ZMMmXvZJoVlYKPMTrLyLqXLgGWwSdFhH7H/2WFmJDugUzeOMIh6f2au+zTBma88W8DnuxqWpT3AdYUWmLzh8+/PGJKxuRvU/7s0+RmDn14Eak+u7aoNXfxrEDLfEOQSq+rgOpSk4Z0NyMmLYvyvo3KGiYaGQnYym4QHb+jOGdqiKa7B9NCdoUXrpuoAuYes97y46kpfAMWVL9J2IDySJ+3/3+SjnIk8J9CBhHz7CkCMJqCBpJOfu9JXUbTJ4LWnCQyEmd4//Z00rmNmLbR2NZXdph4orTgG+0ksz4yblmT+xbM1vJaDemUrnc8sVtwv7cTAx2o6ChZnY0kDCG/h9LNapTJ73LMbiRnz6rc7abKk9c9G7iGSGV4sXqceEOHWBHkWp+/Lv5OmcRwq82Gey/qHI/HyOFWWBheYhRUw2GTPEkKutU+Dr7UuOywllFSt5UZOs/sYoNhX3HT4ovj93Mx9ih0ETv0+RhT3SywzF7oLXkFiiUiWrm+Kd4RT8xQT5GNzRvfGjtZ8OZeyl qnGABSdc ay2epUQskqCtfixIgWZqLCrr8qLsvDVc3SIUB99zq3EaQogeLXK+HFSFQ6sNUOLanuZfeDUQMvooH0lbpLVGbzipfODxcgB9oKvzn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --ladwbtqhxfoyjegt Content-Type: text/plain; protected-headers=v1; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: [PATCH RFC 00/12] dma: Enable dmem cgroup tracking MIME-Version: 1.0 On Thu, Apr 03, 2025 at 09:39:52AM +0200, Christian K=F6nig wrote: > > For the UMA GPU case where there is no device memory or eviction > > problem, perhaps a configurable option to just say account memory in > > memcg for all allocations done by this process, and state yes you can > > work around it with allocation servers or whatever but the behaviour > > for well behaved things is at least somewhat defined. >=20 > We can have that as a workaround, but I think we should approach that > differently. >=20 > With upcoming CXL even coherent device memory is exposed to the core > OS as NUMA memory with just a high latency. >=20 > So both in the CXL and UMA case it actually doesn't make sense to > allocate the memory through the driver interfaces any more. With > AMDGPU for example we are just replicating mbind()/madvise() within > the driver. >=20 > Instead what the DRM subsystem should aim for is to allocate memory > using the normal core OS functionality and then import it into the > driver. >=20 > AMD, NVidia and Intel have HMM working for quite a while now but it > has some limitations, especially on the performance side. >=20 > So for AMDGPU we are currently evaluating udmabuf as alternative. That > seems to be working fine with different NUMA nodes, is perfectly memcg > accounted and gives you a DMA-buf which can be imported everywhere. >=20 > The only show stopper might be the allocation performance, but even if > that's the case I think the ongoing folio work will properly resolve > that. I mean, no, the showstopper to that is that using udmabuf has the assumption that you have an IOMMU for every device doing DMA, which is absolutely not true on !x86 platforms. It might be true for all GPUs, but it certainly isn't for display controllers, and it's not either for codecs, ISPs, and cameras. And then there's the other assumption that all memory is under the memory allocator control, which isn't the case on most recent platforms either. We *need* to take CMA into account there, all the carved-out, device specific memory regions, and the memory regions that aren't even under Linux supervision like protected memory that is typically handled by the firmware and all you get is a dma-buf. Saying that it's how you want to workaround it on AMD is absolutely fine, but DRM as a whole should certainly not aim for that, because it can't. Maxime --ladwbtqhxfoyjegt Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRcEzekXsqa64kGDp7j7w1vZxhRxQUCZ+6tigAKCRDj7w1vZxhR xQFfAQDBTdwSGeM/HRXug8mlHyT5psOLiGa5pibxIgH2xR/VmgEA0w5A63Bu6RYa JKVhD2+5cuBaPVUha8mXQIEsqmEOGgI= =Li+I -----END PGP SIGNATURE----- --ladwbtqhxfoyjegt--