From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1266FC30653 for ; Thu, 27 Jun 2024 17:16:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 901996B00A1; Thu, 27 Jun 2024 13:16:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B1006B00A3; Thu, 27 Jun 2024 13:16:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 778AA6B00A5; Thu, 27 Jun 2024 13:16:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 56C016B00A1 for ; Thu, 27 Jun 2024 13:16:14 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B53ED1A0D1C for ; Thu, 27 Jun 2024 17:16:13 +0000 (UTC) X-FDA: 82277321826.14.C50B46B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id E805B100020 for ; Thu, 27 Jun 2024 17:16:11 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jdT2GY36; spf=pass (imf14.hostedemail.com: domain of mripard@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=mripard@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719508554; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6WK/vchWH/J+FIqlBT/gc+KXMENDsuVvKQbQ+B3nKLU=; b=FiIf8M4zmXFoTGc0EYrOOGYedxS9Dp2od2Iw0O0PoA6phLYtovJBFyY4KGuTAu+XoDaVkV d3+LRGDqD6Mm/pHPFMUoaMi+r7fVGHsuWnO0/B4IlBAwkhN7E0I9uCCQW2uGZU0t1YfkmZ mX8o1Mv0p4GlLYtpikSdcTE/DRkZR9E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719508554; a=rsa-sha256; cv=none; b=SZM4e9/LFBqlzqvC3+/rCrbiUSZbpuyODdlb4pyJcIbRfjWecd3jsGdP6K3E6zOpkvT3Ip QK2ze/46VsQiRZqF/LihNmsVO0ap7yoBdJiifyCBsQuWJlW4q83cVJ1bm4QT0qREkQwvJL REnlDAcEO7DrGIKNMnP7UYwK8JSuAS0= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jdT2GY36; spf=pass (imf14.hostedemail.com: domain of mripard@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=mripard@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id ED21961E14; Thu, 27 Jun 2024 17:16:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CB2DC2BBFC; Thu, 27 Jun 2024 17:16:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719508570; bh=sSeDFPLANRtdCM3W5AaKh+hXR3lM6y6nXqJeEH89EL0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=jdT2GY3651qQ8ySaOjVCeu1i6mUXn4kbzneO/06+6TL90hewAkhxeRwFIfQY2C3+a hRPnYCpnbQrMjqi4GjpSn2CqyXE8MVqbgBRTwq9Mat2vjPspxeFhMi1oGL234JB+HW vIvEFfR49tfh7YBT6j6zBoVLacpuKruzBkVjDeVeUAci2DvClA1WRoL8SlP+mYLuZv vR16hFCG8zK5xThzW2Q3wvVlXPB3bdkcWVEir6e8V60E2gSXokjDUXPEEJKtn2+zQz SDzgGknxNP9tUsP9KQkpxeBtklw7JadnygMfF/VyG8pPRiOkVBEn6Y7zBAGXoTBdj3 eFiEJ7l0D/L/g== Date: Thu, 27 Jun 2024 19:16:07 +0200 From: Maxime Ripard To: Maarten Lankhorst Cc: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Jonathan Corbet , David Airlie , Daniel Vetter , Thomas Zimmermann , Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org Subject: Re: [RFC PATCH 2/6] drm/cgroup: Add memory accounting DRM cgroup Message-ID: <20240627-paper-vicugna-of-fantasy-c549ed@houat> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> <20240627154754.74828-3-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="4x267kq4clbw44gl" Content-Disposition: inline In-Reply-To: <20240627154754.74828-3-maarten.lankhorst@linux.intel.com> X-Stat-Signature: 4ut6hpqxmrddus6bn1wdrffoc3bmziba X-Rspamd-Queue-Id: E805B100020 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719508571-424751 X-HE-Meta: U2FsdGVkX1/m7tBKvjJms/I1BgEJ2UkH/arAS6G823VwRQwh1jX4WcElOfxJ+jvik0z+cOxq6mkmy/Jd8A8vj8ywpQMdYUqnidV1itfssrDeLZqV8JwC1M8Qh+JQATdUScISifalbHPELWtH4QoEuofvpT+kDjR6rRKh3ZrmbRl67vqJYunAyb7DH6WhhON0FT/UzKx7+//B7RYiSVHPFCPiDAMugG9rfatfj7qPYS/7rIinBzBCx6ntCUTdW8CRzLpJY3jc27by1/eb1lpPxGbB7b77I/joL98gZt5i/Rp9CUyEj6hRsmbhgM/p3PqtcAHOFBjgQpOtU50+rWTpba/Xk6QpW7UqjGoTi8GIIXmj0Gq0KIQxNRVYdGb7WHxirEE9AfzDhZiEwppADEdjyHpsFceAFTd9U0CO1uNPgfNK8oCziatb/eNsOtfaYEKoBv7h9MPGZfEk732PmBSQjAIIkzdkX92Pa/8yzDMLc0lQCX0cUTdRZtTW73vSWPJ4i8R5qzysLZJcZsGJwolPU8YOQiGO1BSNP0b+gDPytpv4CjvfVQVQlCudhGuWDuhZkmeiT7uTBeZr5huCorZIfpF/fWvKahKtubzBwWznAkXPzuOq46QMNxVfha+CcB3rAuxnlaznNCqTWlMDFEVn+ih4j7D/0zrCt0XORgkjjuViWeUKrqmU4N7VwjtET6awwBuisU5YX5AGPP8Y6/iKMT4tn/tatbb8eU26lRQTnbBtU9jpOxHd3oBlLDzlBACP5weR708dN3HzfcQ2+vCliLsGV9u9iHKxLDmopJZxXVwLshFq9Fti/TMouth9SQ+4PO9czvJKDjUl32eV1rV2Dxv9zQCGK7hP+04q9dx626KgY5JViX6Ppmfnhf+P6f14k/f1CBeWDzRoZ4e9FrD1eplGBy5aoYWc5ij8bMZUAh+JOTXVm0dCq/oXLyRdhaAIbCS2vDL6vPWY42MR4mi kbrvYcqD lltXHMcsfxI28mrVdwoyO/fcalav+cbnTltUV4NlX2Mxk/oS5YcoRd+Z5l0nlXcYjlxwSOMMYD1UbBAZqprjJQFgrjMSd8wM9ARGjZ/rehXAinGclOjiT9/7LOBY8RZ2TeJ4GWJZ7Bz8E23FtFZjg3x3GZtEdnDCVIG6mpAtR7iplCoj0Z8Os7gQdgHcwVpdrcbweYrVa2BCjgGhavP1Lt+dHYWs/ljI20jAGK+TYqzAC4VpRRP1fB5JrE0c6pXp4GW2z+YNwE9ZJuV9caNGd8LjQ0nI5RGBryP1nzucJ8+T3bIpmPhdaLQZybZt8Kj7MDwbunP2cy0ukQ/lnClFZkbYlLu8qqMEQCkPMqvjLCVbQJQiPFWCBbDQE0vjPcUkxk2ITAbJU094Xf+0xwRyiY+7grA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --4x267kq4clbw44gl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, Thanks for working on this! On Thu, Jun 27, 2024 at 05:47:21PM GMT, Maarten Lankhorst wrote: > The initial version was based roughly on the rdma and misc cgroup > controllers, with a lot of the accounting code borrowed from rdma. >=20 > The current version is a complete rewrite with page counter; it uses > the same min/low/max semantics as the memory cgroup as a result. >=20 > There's a small mismatch as TTM uses u64, and page_counter long pages. > In practice it's not a problem. 32-bits systems don't really come with > >=3D4GB cards and as long as we're consistently wrong with units, it's > fine. The device page size may not be in the same units as kernel page > size, and each region might also have a different page size (VRAM vs GART > for example). >=20 > The interface is simple: > - populate drmcgroup_device->regions[..] name and size for each active > region, set num_regions accordingly. > - Call drm(m)cg_register_device() > - Use drmcg_try_charge to check if you can allocate a chunk of memory, > use drmcg_uncharge when freeing it. This may return an error code, > or -EAGAIN when the cgroup limit is reached. In that case a reference > to the limiting pool is returned. > - The limiting cs can be used as compare function for > drmcs_evict_valuable. > - After having evicted enough, drop reference to limiting cs with > drmcs_pool_put. >=20 > This API allows you to limit device resources with cgroups. > You can see the supported cards in /sys/fs/cgroup/drm.capacity > You need to echo +drm to cgroup.subtree_control, and then you can > partition memory. >=20 > Signed-off-by: Maarten Lankhorst > Co-developed-by: Friedrich Vock I'm sorry, I should have wrote minutes on the discussion we had with TJ and Tvrtko the other day. We're all very interested in making this happen, but doing a "DRM" cgroup doesn't look like the right path to us. Indeed, we have a significant number of drivers that won't have a dedicated memory but will depend on DMA allocations one way or the other, and those pools are shared between multiple frameworks (DRM, V4L2, DMA-Buf Heaps, at least). This was also pointed out by Sima some time ago here: https://lore.kernel.org/amd-gfx/YCVOl8%2F87bqRSQei@phenom.ffwll.local/ So we'll want that cgroup subsystem to be cross-framework. We settled on a "device" cgroup during the discussion, but I'm sure we'll have plenty of bikeshedding. The other thing we agreed on, based on the feedback TJ got on the last iterations of his series was to go for memcg for drivers not using DMA allocations. It's the part where I expect some discussion there too :) So we went back to a previous version of TJ's work, and I've started to work on: - Integration of the cgroup in the GEM DMA and GEM VRAM helpers (this works on tidss right now) - Integration of all heaps into that cgroup but the system one (working on this at the moment) - Integration into v4l2 (next on my list) Maxime --4x267kq4clbw44gl Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRcEzekXsqa64kGDp7j7w1vZxhRxQUCZn2eVwAKCRDj7w1vZxhR xReyAP4+9TGUgVyaERT/3Z2Q6QCqUYta9dlEvaZaPjxpE/PzCgD9GtTKU1rmEBJN XhhmHQ04LONiQxG4Qp4fDvfFOussXgY= =InwG -----END PGP SIGNATURE----- --4x267kq4clbw44gl--