From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F313CDDE7F for ; Wed, 23 Oct 2024 15:26:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 92F056B0083; Wed, 23 Oct 2024 11:26:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DEBB6B0085; Wed, 23 Oct 2024 11:26:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77FE86B0088; Wed, 23 Oct 2024 11:26:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5A3406B0083 for ; Wed, 23 Oct 2024 11:26:20 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 79BFB1C64C7 for ; Wed, 23 Oct 2024 15:25:59 +0000 (UTC) X-FDA: 82705242774.24.A77552E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id CB65120020 for ; Wed, 23 Oct 2024 15:25:57 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ALF4GfAx; spf=pass (imf13.hostedemail.com: domain of llong@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=llong@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729697022; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sUh0g17SqyKPeTvfCPnxLjLxXG/Lptz+4Dsb0wobQHw=; b=TKxLZ+u1YnLM2v90f5l77tgZYie9Aek6py0rpbKtgp//Cl/9uj3D1SFzWJ6Y2yGil8d1P4 uORiGsEjQCdSZ5wQBh6jKNZa9kZE3PCBx1OGtGFZscvf9jfeJYLvIBMuLxqt/cbtF51JwX CqKPLCABr9R8hR+klmImUfc2oJagOOM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729697022; a=rsa-sha256; cv=none; b=i+ACrnQyveukCPfRwZEcDDt2sQ/8v6/Aeorwfl+Lfg2d/mMpSQz1vjGrfsUh/eIpSSzq7F /aG20g08Dpybngza39Ebgl64fhfpHH9QQefvzOqqB0MsDBixxFnq6mERzmK+Br6VC//ER5 vKOhNcJzqqgL2xBVRAVxalsRBXr9Ebk= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ALF4GfAx; spf=pass (imf13.hostedemail.com: domain of llong@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=llong@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1729697175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sUh0g17SqyKPeTvfCPnxLjLxXG/Lptz+4Dsb0wobQHw=; b=ALF4GfAxMYSAwmb0MscpgT5ycslvdxMrN8zR3YIBd989d+t20P9u+9ZQvRpiXeGLpqEbPG UtKLkYUum/IXa+gkXDmtjM02sCaBEx17LhIUpOQDCG0BCzxl/+EK07JYvtjCbn/o7YKKK2 m0y/UASML+vozj/yLuBPYK9EaLCcyaY= Received: from mail-ot1-f72.google.com (mail-ot1-f72.google.com [209.85.210.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-102-1QN1TIUKNESGnH1gSMkHmA-1; Wed, 23 Oct 2024 11:26:13 -0400 X-MC-Unique: 1QN1TIUKNESGnH1gSMkHmA-1 Received: by mail-ot1-f72.google.com with SMTP id 46e09a7af769-7181971cfa0so5247616a34.0 for ; Wed, 23 Oct 2024 08:26:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729697173; x=1730301973; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:subject:user-agent:mime-version:date:message-id:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sUh0g17SqyKPeTvfCPnxLjLxXG/Lptz+4Dsb0wobQHw=; b=PudUf5xyH+fR4Q9Tqe4Ys7lk/IRtiIPCDB6KOqwK8rv0lV3i+aG/AO9cdmJfCXhohL FQ4KlUTEb3doIRErb9Yr4KcSgn58ByAPPqxJgz49UeOa+dijDlg2Q+H2QgRsvPDiK9Ra KLVQV6mlo0Y87b2XsHXXsXeN4OIWHfCqbbBG+JoPphZzAfHH8lScVnlqkKVcW2KgUwhY H53PHndZ9noxpTv4KZBZ3KpnR6Xz7nJIPbhRryEvEgBRgT/9KiflRUK8T1jan4G/v+ou qlQ4Qmg+TOQk+lpKgp+uXY2fyBCANu5x1jAqVs2kwarQf6tStDBiMXQzUGY/uEcICOUA 4TEg== X-Forwarded-Encrypted: i=1; AJvYcCXwbh05CKGk3Ay2XodUU1FfoLZpXv9F75o7xd8ObLh/LJ1SSR7u5wTJuNH7DKRPr+wtf7V+M5iSfg==@kvack.org X-Gm-Message-State: AOJu0Yywh/N30z5BoCY7Qheg2uK6eTEIJ9KwBUgB1e3wqYidF4GRhjr7 jsZp/qZ9eAqcqGltWVcmmxgWp7a7cbx4W9ceqsGUNNHR/hywPQnninC2ig0vTPLKcyG/Y3FQeT6 /1cGF63DmQCY+fm6RX4w7hOdq+VFRzGe/CdsiwtWKhxq5L+cf X-Received: by 2002:a05:6830:4428:b0:718:119:ee15 with SMTP id 46e09a7af769-7184b2d43abmr3322085a34.10.1729697173023; Wed, 23 Oct 2024 08:26:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFZ8HZNZZ1NdptD5eLWfYWIKUwYpZpDtDkhL+yfDSXkQRcv/wze/ury8WoK60x0GCFuj5xePQ== X-Received: by 2002:a05:6830:4428:b0:718:119:ee15 with SMTP id 46e09a7af769-7184b2d43abmr3322057a34.10.1729697172730; Wed, 23 Oct 2024 08:26:12 -0700 (PDT) Received: from ?IPV6:2601:188:ca00:a00:f844:fad5:7984:7bd7? ([2601:188:ca00:a00:f844:fad5:7984:7bd7]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6ce008fb5e0sm40518826d6.33.2024.10.23.08.26.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 23 Oct 2024 08:26:12 -0700 (PDT) From: Waiman Long X-Google-Original-From: Waiman Long Message-ID: <813cc1d5-1648-4900-ae56-5405e52926df@redhat.com> Date: Wed, 23 Oct 2024 11:26:10 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/7] kernel/cgroup: Add "dev" memory accounting cgroup To: Maarten Lankhorst , intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, Maxime Ripard References: <20241023075302.27194-1-maarten.lankhorst@linux.intel.com> <20241023075302.27194-2-maarten.lankhorst@linux.intel.com> In-Reply-To: <20241023075302.27194-2-maarten.lankhorst@linux.intel.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: wo6g8ep5scib39tnemh144phyfjdb3ur X-Rspamd-Queue-Id: CB65120020 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1729697157-440195 X-HE-Meta: U2FsdGVkX1+XD2FjymjkS0YrrG9juyJsmTQAAiVpKmZNL0719nndXUfqA0ZWPomV95+Pv739rvJ1xCidy/guA/eP/xyF4Sf8jTRgkGhv/YAzAfXI/2xCSbfehsJg3fzzuVAHBNDfWmfhuXe4UGN2iOCnl+Z1IBoiROvomjmE4VAgYfM6PYTLz02ZDkGnZaKqqGw+0xOJTFC++ykxjs8dY5d9ZBnbX6bk0Li/q9nc//NURiG4ppx5AsTKDh8ISnELwEa46D5ikqcR90LDWZojrfKXQneR3qRQMGh/G+J/dxCBPUBxZrf1YFYSIIBJCKiGkFTu8NF4YTi0NlYU7t9MGNz9o1PPUGxRmBnA4PY/CeBo8sKZ3jlVw/NppBmfkuEv15eVnnYuOTlKa9Dk+DquPgFR/eorjiKjyZygK7COEvZvO//6VCCfNmBFze3YquwzW2Sk/n603pZsfAAdUNtUgjWhY8npIcNoWhQNxf/D3U7VkhkPuPR3vafjiJwUMAMPZGbTO3hHAe1c54YfGjLeH5edqQ2/UXsBvuGVhzeR8/ZjBLAiTh0IAd+nx2kgjz65z8brmImC4cmu69+O3mk3B5EEi/3fZQ21ZR2Je8SvNej0HVFX3undYzHcZILtWiSMKZOOunn9PV3WWX9UFknoTiVSUtTafttDJdyAvdZrQMb8Zk0NTJbIPDiX1JK0VBSnUtPvH4A4fOkoG6A3mtOAr9NtqlkgYFzGTkav63+cwBd5jomZ4p1d2QXOr3baQ3ntf5RxjBcNcYePaq8mKlKqTq4dcBT4odg4W76JTANegZgDYkb1sVZyd4f+lFaHGcwmGLpPzZAfSw02Xj8pbK+8bm7UOpcockHhSvZmTYPxt6+p4HgrQMHgw5RQvvlqSx/dgK2H6cjwJblgxm2WJef/e0YWz4v1d0aCCxpoi9pYqGoidhp6Yswn5SN4tmXjE19OhpbsDy8rnZbdT6oratx e85IuHUT STauHjDbC2V3nKvisiCy3Hw2vaP/kY+z3RHXOKdpQe29bJtgamt/3bv5Y41+Qm8/FJ5e7vkh+AUzDzi7sAuR2c6IKT4snhBRLUPLZSChybTDiJyxIB/ljjSjAGSM3dqlK2K4KaXtENINNIT+eyT6QOc2rABCVK+lqI7O/57UDipc7FtoEnpClKqvxu0d4cqSd8ExIfxh46aGvv2xLnHOz4jRluSa0Gc9FLtiMw5ivFRl+OZcRLPPE29BQ9wVdcukkeKZxwZrA41k2feCPWe14FD8srDYoy0hQAYe3EjFK8x8gj93uOFOy3kHlKyyKxqI0RpCtTf+GZPGQdsi5DmF9rHucBb1krbua5wVDwkj20CE/VUzm3iRb0gOI0jyQDJ66wYpAlgkBzB/KIHvASCtZ5mSMg4kukbnm627DUnzQxgtXYlInZ3nGTferYduzk72LjDcLX+gpKk3feNraB/7ycAxQEw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 10/23/24 3:52 AM, Maarten Lankhorst wrote: > The initial version was based roughly on the rdma and misc cgroup > controllers, with a lot of the accounting code borrowed from rdma. > > The current version is a complete rewrite with page counter; it uses > the same min/low/max semantics as the memory cgroup as a result. > > There's a small mismatch as TTM uses u64, and page_counter long pages. > In practice it's not a problem. 32-bits systems don't really come with >> =4GB cards and as long as we're consistently wrong with units, it's > fine. The device page size may not be in the same units as kernel page > size, and each region might also have a different page size (VRAM vs GART > for example). > > The interface is simple: > - populate dev_cgroup_try_charge->regions[..] name and size for each active > region, set num_regions accordingly. > - Call (dev,drmm)_cgroup_register_device() > - Use dev_cgroup_try_charge to check if you can allocate a chunk of memory, > use dev_cgroup__uncharge when freeing it. This may return an error code, > or -EAGAIN when the cgroup limit is reached. In that case a reference > to the limiting pool is returned. > - The limiting cs can be used as compare function for > dev_cgroup_state_evict_valuable. > - After having evicted enough, drop reference to limiting cs with > dev_cgroup_pool_state_put. > > This API allows you to limit device resources with cgroups. > You can see the supported cards in /sys/fs/cgroup/dev.region.capacity > You need to echo +dev to cgroup.subtree_control, and then you can > partition memory. > > Co-developed-by: Friedrich Vock > Signed-off-by: Friedrich Vock > Co-developed-by: Maxime Ripard > Signed-off-by: Maxime Ripard > Signed-off-by: Maarten Lankhorst > --- > Documentation/admin-guide/cgroup-v2.rst | 51 ++ > Documentation/core-api/cgroup.rst | 9 + > Documentation/core-api/index.rst | 1 + > Documentation/gpu/drm-compute.rst | 54 ++ > include/linux/cgroup_dev.h | 91 +++ > include/linux/cgroup_subsys.h | 4 + > include/linux/page_counter.h | 2 +- > init/Kconfig | 7 + > kernel/cgroup/Makefile | 1 + > kernel/cgroup/dev.c | 893 ++++++++++++++++++++++++ > mm/page_counter.c | 4 +- > 11 files changed, 1114 insertions(+), 3 deletions(-) > create mode 100644 Documentation/core-api/cgroup.rst > create mode 100644 Documentation/gpu/drm-compute.rst > create mode 100644 include/linux/cgroup_dev.h > create mode 100644 kernel/cgroup/dev.c Just a general comment. Cgroup v1 has a legacy device controller in security/device_cgroup.c which is no longer available in cgroup v2. So if you use the name device controller, the documentation must be clear that it is completely different and have no relationship from the device controller in cgroup v1. Cheers, Longman