From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 319A1C2BD09 for ; Fri, 28 Jun 2024 14:05:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 885D66B0092; Fri, 28 Jun 2024 10:05:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 834806B0095; Fri, 28 Jun 2024 10:05:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D4886B0096; Fri, 28 Jun 2024 10:05:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 481EE6B0092 for ; Fri, 28 Jun 2024 10:05:05 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 56C30A18AE for ; Fri, 28 Jun 2024 14:05:04 +0000 (UTC) X-FDA: 82280468928.16.D4508FD Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id EEC308003E for ; Fri, 28 Jun 2024 14:05:00 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JrZl5nCo; spf=pass (imf02.hostedemail.com: domain of mripard@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=mripard@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719583477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RDiJrIbR1M21eICsVZ2YwlztW0UReH8eIgtOLJ8Hs7w=; b=uIn76L7jf92G4sEcrF/sQoNNFbgUDBxd/INCcX8WsvKNPfpbIFDUL42R/CiNNKbWrbffP2 dVbmf10E7W+Ce+h7wNQiLN3QZTO+bt6OYcqE2fd10DleJc6a8DQSE6B8ondvrt7d+/azk+ duKjPjd2sN84wyCRvRvI2Q7OMzzeb0s= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JrZl5nCo; spf=pass (imf02.hostedemail.com: domain of mripard@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=mripard@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719583477; a=rsa-sha256; cv=none; b=ShHjmSdKf8O6Doe5SFG3WeDT63ZrpBoSQkfV1UYY5i1t5Zl42/8sBJPlaVXhRtMgFXJRdV Tsn8b1ofdnXetwTu4Me5ymvK3AU3MatcEfhfNsX0p+3Mo6qIU7d8NDZQ8idOU9QBcklhbV U1MVyGYsqBNcajP/IVbuVwU7O33xFHk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3DB1CCE3B5F; Fri, 28 Jun 2024 14:04:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 22FC3C116B1; Fri, 28 Jun 2024 14:04:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719583495; bh=gPsBksBd9/r5KyiQkMqtaVzJ3Jnw9xxaAJaOJd1IL5A=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=JrZl5nCoSOg1oFczaYNXOld1XvShBNSqpPU19+yaDf6ERI30kJ5Dorzs0krZKwCBX oETKiAlQbjACn116sCeEjTwKL5u4AH4m943lebVVjS8pc3ejDEUrBsFdc+0gXFgRm1 vU2W/gzhdm9dudHQSSQpaPEew9f1v4vyUOOa4dPaQCx0PIMX13Sl2SF1rs+yZNCa/o PDHnlDCgSwVaGcAzxdt+Coft0zyfppm/k9KrjJ+aJoYnxsdJO7OFGeKc3mYsvBlavD unyIJDu9Xzf7TZiwSoHgPgGhF4XEdPdWJ/SAWYmEX404qo0FB/O8kkJoNSIX9xSyMU Jm+53XnRA8tKA== Date: Fri, 28 Jun 2024 16:04:53 +0200 From: Maxime Ripard To: Maarten Lankhorst Cc: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Jonathan Corbet , David Airlie , Daniel Vetter , Thomas Zimmermann , Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org Subject: Re: [RFC PATCH 2/6] drm/cgroup: Add memory accounting DRM cgroup Message-ID: <20240628-romantic-emerald-snake-7b26ca@houat> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> <20240627154754.74828-3-maarten.lankhorst@linux.intel.com> <20240627-paper-vicugna-of-fantasy-c549ed@houat> <6cb7c074-55cb-4825-9f80-5cf07bbd6745@linux.intel.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="7fymz6tlpfxqxwqn" Content-Disposition: inline In-Reply-To: <6cb7c074-55cb-4825-9f80-5cf07bbd6745@linux.intel.com> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EEC308003E X-Stat-Signature: bsa1bnx3cutgpmjuoh96y9pnfg5djqwx X-Rspam-User: X-HE-Tag: 1719583500-869511 X-HE-Meta: U2FsdGVkX1/hFEUegwfT4hImxH3KZH1YmPcLZsQ0w4/DaLp++u6OlCQ+lgVPpXtOzFzBBqVclJOB4uSo/70jeG+3r1FK8Be6zv4Hk8hXSjZLH17irprVlIhFF+qlzBwDEJa9FUR1pTPMnOjm9/7v2gYZOXWf0gy6HQKG7Cl9hI8udRbwWpO/jGl53SElW5l0hTSP47vAL7gG57aVSZmDkyIbxMBt/equN+fMvZLvHIe6ay5mHt8BIlmf2HoNOuWzjiw9U3UYmTh99zhRAiNR5pbXYy3WFaWSMq+qWUP8HsEhsYsbaOBrCxnILDhwuniYN1X5kc20jIpaJx3MACWolPtIrp83xdm3CqLXLZIvApJHaQSpEkJZ6KbLcIKJxIuHkpjpYuN0RtcmtUXHzgdRkGElBa+6GaDoLL4Su7ieCPSR4sVdnl0OOW0Ix2mhHD82KPzsbyYGdpjUmA4EcJ20UpX1y1BN0UoPDwyHaTBh8pG4U1M5mdzNyiDVTXVJQrBvVFVh8T6x4geVX6SrdPIDHZSVWEXaOm4xrW0sQ1YlZtI+SLB75YkfZ3e9OjkxQlo2Okc1elTEjkqmEH7Q7rfoe5hl49i7yRHw9jNR1mjRUn0LerZ0EWjbpbFLjeAgPEcAVB8cvfq3j79UlQV8WA1S8kCfnXH2RgnVvvacJmb6U6bJSPmCLu09CHcIZuyYVpGodTTdTVuXkKR1P1xKoU4varGUhS5ygpbiJpPkHugrluL+9qJh94zqgfLJc3AwHR1tVqHi8f/nUKcH0GUs+P50EHaTSLyP7gGUmsd//MCYq9+z0RdslEiCSOFz/PWmkL9R3MDHKjSHcAakaFeXgaUpsdqx3ZQOkZpNu0E7dEvgmc/BydvGVy2PyDm1q7O7WsY2cJ36F/1SDpwdlSahC5jVfE5JkWcwT4iY16JGNws+BQZZ1+8A4J+ZZEzvxFUBHwRMjwfBaKxnCOGT1sXIXjw EfzUzdGb jlhGRQ0MFwjuS8lWpcf/sECwAHyqrn67Eg09KpKns+cr9ufzTwo0eJf94kkZ0b3CxewRBCR8xTT8O1xF5BfsWhT1P2CvQSpSfhpOkiQ/NWxc9H2AuLiGdUdTvA6rdKDPwPwZcBS86MoSyaoN4x/UXcJ/w2WmL+sXtSzKWiP/CD8YSJPmioeqgEqys1sNqH9H4IDod2ibPprtUP+/w9XJ/eQ8upOXSejW7rgeWCBCV/GHiWWWtsqgHbPzxv9E+LUTwK9YO23oYjDQwUAQiXuMsN0thv1KcdXdeFbtESVR71VzS1ZFmZnRaOOvY6UNqWc7nrguhUlDw2yI/PKeV+eA1Qz4q8mNdjrZC8mURnfD5G0jiQEG3Oj4C2t7c2tfkxcAdc90/rY4vVdbjvbQovp56F8F5GQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --7fymz6tlpfxqxwqn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, On Thu, Jun 27, 2024 at 09:22:56PM GMT, Maarten Lankhorst wrote: > Den 2024-06-27 kl. 19:16, skrev Maxime Ripard: > > Hi, > >=20 > > Thanks for working on this! > >=20 > > On Thu, Jun 27, 2024 at 05:47:21PM GMT, Maarten Lankhorst wrote: > > > The initial version was based roughly on the rdma and misc cgroup > > > controllers, with a lot of the accounting code borrowed from rdma. > > >=20 > > > The current version is a complete rewrite with page counter; it uses > > > the same min/low/max semantics as the memory cgroup as a result. > > >=20 > > > There's a small mismatch as TTM uses u64, and page_counter long pages. > > > In practice it's not a problem. 32-bits systems don't really come with > > > > =3D4GB cards and as long as we're consistently wrong with units, it= 's > > > fine. The device page size may not be in the same units as kernel page > > > size, and each region might also have a different page size (VRAM vs = GART > > > for example). > > >=20 > > > The interface is simple: > > > - populate drmcgroup_device->regions[..] name and size for each active > > > region, set num_regions accordingly. > > > - Call drm(m)cg_register_device() > > > - Use drmcg_try_charge to check if you can allocate a chunk of memory, > > > use drmcg_uncharge when freeing it. This may return an error code, > > > or -EAGAIN when the cgroup limit is reached. In that case a refere= nce > > > to the limiting pool is returned. > > > - The limiting cs can be used as compare function for > > > drmcs_evict_valuable. > > > - After having evicted enough, drop reference to limiting cs with > > > drmcs_pool_put. > > >=20 > > > This API allows you to limit device resources with cgroups. > > > You can see the supported cards in /sys/fs/cgroup/drm.capacity > > > You need to echo +drm to cgroup.subtree_control, and then you can > > > partition memory. > > >=20 > > > Signed-off-by: Maarten Lankhorst > > > Co-developed-by: Friedrich Vock > > I'm sorry, I should have wrote minutes on the discussion we had with TJ > > and Tvrtko the other day. > >=20 > > We're all very interested in making this happen, but doing a "DRM" > > cgroup doesn't look like the right path to us. > >=20 > > Indeed, we have a significant number of drivers that won't have a > > dedicated memory but will depend on DMA allocations one way or the > > other, and those pools are shared between multiple frameworks (DRM, > > V4L2, DMA-Buf Heaps, at least). > >=20 > > This was also pointed out by Sima some time ago here: > > https://lore.kernel.org/amd-gfx/YCVOl8%2F87bqRSQei@phenom.ffwll.local/ > >=20 > > So we'll want that cgroup subsystem to be cross-framework. We settled on > > a "device" cgroup during the discussion, but I'm sure we'll have plenty > > of bikeshedding. > >=20 > > The other thing we agreed on, based on the feedback TJ got on the last > > iterations of his series was to go for memcg for drivers not using DMA > > allocations. > >=20 > > It's the part where I expect some discussion there too :) > >=20 > > So we went back to a previous version of TJ's work, and I've started to > > work on: > >=20 > > - Integration of the cgroup in the GEM DMA and GEM VRAM helpers (this > > works on tidss right now) > >=20 > > - Integration of all heaps into that cgroup but the system one > > (working on this at the moment) >=20 > Should be similar to what I have then. I think you could use my work to > continue it. >=20 > I made nothing DRM specific except the name, if you renamed it the device > resource management cgroup and changed the init function signature to tak= e a > name instead of a drm pointer, nothing would change. This is exactly what > I'm hoping to accomplish, including reserving memory. I've started to work on rebasing my current work onto your series today, and I'm not entirely sure how what I described would best fit. Let's assume we have two KMS device, one using shmem, one using DMA allocations, two heaps, one using the page allocator, the other using CMA, and one v4l2 device using dma allocations. So we would have one KMS device and one heap using the page allocator, and one KMS device, one heap, and one v4l2 driver using the DMA allocator. Would these make different cgroup devices, or different cgroup regions? > The nice thing is that it should be similar to the memory cgroup controll= er > in semantics, so you would have the same memory behavior whether you use = the > device cgroup or memory cgroup. >=20 > I'm sad I missed the discussion, but hopefully we can coordinate more now > that we know we're both working on it. :) Yeah, definitely :) Maxime --7fymz6tlpfxqxwqn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRcEzekXsqa64kGDp7j7w1vZxhRxQUCZn7DBAAKCRDj7w1vZxhR xWlJAP0UNWQ0gcwWNN/Im1DgFf7X2Yi5sSP1W1uEZE9I0hrrPAEAuZ5tF02to94P dCchG/vB5gsWEnB2EerIOPqG6gW4tw4= =J4LC -----END PGP SIGNATURE----- --7fymz6tlpfxqxwqn--