From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A646C7115A for ; Thu, 19 Jun 2025 00:42:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC0358D0002; Wed, 18 Jun 2025 20:42:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A725F8D0001; Wed, 18 Jun 2025 20:42:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 960AF8D0002; Wed, 18 Jun 2025 20:42:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7F4B58D0001 for ; Wed, 18 Jun 2025 20:42:57 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A38E0C105E for ; Thu, 19 Jun 2025 00:42:56 +0000 (UTC) X-FDA: 83570300352.06.D2A733B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 6463D4000B for ; Thu, 19 Jun 2025 00:42:54 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RSAZ2oPC; spf=pass (imf27.hostedemail.com: domain of airlied@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=airlied@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750293774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Pe2k0icdwqpd4w/wA67sebAWmODKh8BWDjcuUYYdUoQ=; b=JmgNcaUlKlnJ6RB7ik1nwXC7NIG4SFL2b/PiVytbB4xpG0QWhMTtSLiX3+7VPkSs5E/LFE +bHk5cPGv2AqTGIVUZGNUctXj1YQs0bUgiAKoWX1pIrtnOCtKY712+3WASyoYgs2R9mSWA 2FHUUhPC10eh6ogX6QWdUDcw6eX/Sf8= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RSAZ2oPC; spf=pass (imf27.hostedemail.com: domain of airlied@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=airlied@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750293774; a=rsa-sha256; cv=none; b=QiRFggzEQXHbpZFenxfBg+Xn8qw2YedwZS1fFJemEhO7KIsmG04YZY/G3+57DqqF5Tx2Ny lRrrYGDuGRsGNkn6R6C1yAesdxBvmFvJzTJy9uxk1YpSliySj254LT3X27miK24jq4OmUz P7cqpXMMG6B3f6KfrZ0xxJIBqKqMUVU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750293773; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pe2k0icdwqpd4w/wA67sebAWmODKh8BWDjcuUYYdUoQ=; b=RSAZ2oPCCPAwbr20VecaQD1dp3sGljgSFyZuC2KhjdWQOV0/tvP9JDOcXE5BrYdtH7Jw0A l0hZyHC3tdpS4PB/3fhzAZIrmlBa4B3Ba0UUEvXV7Nf9Ny7X9FJSxwfXdqiWWGNuSvl4SO gHYV5Zk5n74kiYcttMIWb378CKDKyYU= Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-12-JmNHo31ENEWIq5WfuylDHA-1; Wed, 18 Jun 2025 20:42:50 -0400 X-MC-Unique: JmNHo31ENEWIq5WfuylDHA-1 X-Mimecast-MFC-AGG-ID: JmNHo31ENEWIq5WfuylDHA_1750293770 Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-3138e65efe2so134516a91.1 for ; Wed, 18 Jun 2025 17:42:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750293769; x=1750898569; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pe2k0icdwqpd4w/wA67sebAWmODKh8BWDjcuUYYdUoQ=; b=jry+n//TvCWJXKcD/mUSZmNPg2zrEvKrZeGvr+aKdfLPMvik5QUhv8j9Gi7MFCw0gn h60o7IoaMJ7sGSRPc7bRvVDeStSjlo7/U3XBPzMC4dYP8TKEztaaTMIw2soRxagIPuwX OyVp4c9NH9UzMRAnEvwEa4ruM4K6ypa5F7whCf7aZVwC9Wl4WcwlAqzJQRk2PbS+PZeL 7F/+kJCcgG5av3FVvz932q6dTa9ivkqBPnd11tiWWrh5yhXU/QBwxmP8CHTSTztmhE4W Sdd3UBp8ALKakCKEEoX1mk5GhEpgT5YRi7bJ/VobdikKxb9PscKOlIUXqDnJXSvXDor1 PI6w== X-Forwarded-Encrypted: i=1; AJvYcCU0BC+zXzWRlsQkUk8RwrPf1UISL3553yu3Z/OLSv2QMTLYWgG7Kz7ihl1qa/mu4P/YGC9lOAdbjQ==@kvack.org X-Gm-Message-State: AOJu0YzOIvvHqVEfZiMAZhN9lSb3SmF8MABzfRMoxdRNUVDywfMt0OS4 jwMbvsPvroX+ZmUCWmU7uHuMwIa0aesJHmNxIX4gYm70pXg+aJ/51dgjZeN0lzhIk+k5vxE/EwT J1Q5L1BOuKD9k/JDamD0ZLU+rp4J5dIr3Yd9ND0tPTAWkbbbkT/xHA9A4Ca2ucW5tgQHJjA5tfK Y1+Kdgr2a0duKm7Tc+UgYVzyVIVW8= X-Gm-Gg: ASbGncsLU9LIjNz0/lAy8KJ893Xdxu0Fzx6hYXIeK/SLpd3QE1tKjUKQiQIxkfALYfk +c+dtT6BOVdZDDf3KwSuseFX5qUk4U6c/i0a/ylpaGH+kzCtC1YsZlp9vJ20CFwb/mY4/Q4TauV YHCA== X-Received: by 2002:a17:90b:3143:b0:311:c596:5c6f with SMTP id 98e67ed59e1d1-313f1d5efd9mr32256521a91.17.1750293769502; Wed, 18 Jun 2025 17:42:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHIIlFuzoH/5qCUw2+angE5OPLXFvYSG3YUBq+YUMApbItuXGjmzNmKE5PwmZXsw8dCzGlzAruYOOBot7UeVb8= X-Received: by 2002:a17:90b:3143:b0:311:c596:5c6f with SMTP id 98e67ed59e1d1-313f1d5efd9mr32256493a91.17.1750293769067; Wed, 18 Jun 2025 17:42:49 -0700 (PDT) MIME-Version: 1.0 References: <20250618040618.564650-1-airlied@gmail.com> <6yxpihotsrg73dmlr2fajga2b7qbdnsroi2tq7alohrqt56dx3@sjyoy4yg2ck7> In-Reply-To: <6yxpihotsrg73dmlr2fajga2b7qbdnsroi2tq7alohrqt56dx3@sjyoy4yg2ck7> From: David Airlie Date: Thu, 19 Jun 2025 10:42:37 +1000 X-Gm-Features: AX0GCFuJhXX4C_1px9RQtna8BYsmm3A0K_QbTxFUqdVZfCNxrpaTdYIrldCVZjI Message-ID: Subject: Re: [PATCH 1/2] mm: add gpu active/reclaim per-node stat counters To: Shakeel Butt Cc: Dave Airlie , dri-devel@lists.freedesktop.org, Christian Koenig , Matthew Brost , Johannes Weiner , linux-mm@kvack.org, Andrew Morton X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: HJHCGoHpgEm52Sr8XpqaG520ca_XGAnMZy_q6EZY48g_1750293770 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 6463D4000B X-Stat-Signature: 5pum8nop95djx19wt51fcajox3jnbusq X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1750293774-766560 X-HE-Meta: U2FsdGVkX1+nl1Esqf2TnLz50k7fpBSUdh34abrb94pmES5OD1x4Oo8XbcHPRRzylrS7tI5RzKjwq2AHgCA+zgNGAh+6N12LIbqz50oTJ68l3yBySTBvBDOzuDEBYOffRjfEJTTTdNsKE17KhJN7hGMH0cLWV5Pe1xsWQ2DKtesKmwQgYm/MEJk53QNqvgb9i7Bpt+4cdusBz7oDQH/cWy3U0T2uFdDYC37ihe2nTh1hIeD2YTpjsXXTBZLXHBO9NMlSEUXeQWu0gEtoqYa6NmYl0hEIUmgTQzziVXRa5deOlZdMkjq4W+0tTzlayts5UFzCeZyXh+TDFRS6YOPEaIAy2I2Lbdofko0AzLV/kPa1bYicf+Z/n+BatrMwDwRmPzy9gxolP9oOI47t7CKqx2oHWMR++eg9UnJ1kChaCCRY1SUR9J3Fo5vAB41cGtqfWzvd0y7GHQV1UBVXtGhiOcyIKbgS2AFb63huTPqB34D+axin+YPXtFyS39PRRyOfhSNgMZKhMxhCMLL8M7k5ogEmqNyFh6/A3OQ+jj4TzChNw0gSI1XRjz71oxpEP5qorgjVvoFwWHEKRVIF20UFJIvd9xXCcgL/XW9ttJkSCriEBOFcJUYjcNzT3wsR2vVrBD62+rpPmg2kUrlQ34FIzyPf0/duUDEqa0kaD78dExinCH9lI0xRjWr4VXLoHVumZribupr1dIfdnjZUT6uWnSZGZWoYhyQKDScDRQKWGL85Te+t4tbCRyXm8+hl2vDKSogBBtxyztBKFC2d7loHw0+UMX217+jlskBD8n0Xa8OB+SF9Yp5fo+J/0Uds/Z6NMFfClhqwGkOogdLg1xTvpQpiIQQbflggkHZHysDU/KDtfdz1wd0LZub9nuAiHsswYvPrP67Y+ypASjbBg1pvEZRmi3DcI9RoLKXoyZfUVF1ynjjYPYn0nKZ0X+LsyXYlzJL4Z8HNAnxoqW965J0 FZuBSqnd XT/x5/NCBiJV8KhLkBwnHnZXSbUX1ozw4wr17K0VhUbpF/qyPvGf3AIuVy4XG3ZyFSzU9owGFWRH4sGDqp/S6RbjKbsRDuVE4xClnMulztHzUygr0PRy61+gPz4oQrAdzqvfP3WxVc0BMluMTsWK1aCsY3oohAGBlOGUWaBaudMIx6GNyqfqIIKhyBlEapn3EvktoI9H2Vd2vRcGVHLUYX03bKhFI0fWCxOcVYRGLGsmNmFKE+9mYDDDqWHQ650bj6PXv5Ww/po3zRleu01ti3f6iq8LoE03KUMBwUuG0DEb/h8W3BfD/NrHSad2WGZ48k+A/nrGijxkTgOr//Fcg38blNHFC20WEracdL5S4kQ40BJAhg/muhEOQRc/wjYPyo8W05KaWv0RD7If9JpIcyCek5yDkX2imnee4qflmrxWQn4CR4HDDyu/stAMLfM3FeY9WFBsboNIs80FQtczXEe3K4Vs+Xxm5dIVThZbp75f2aNz39n+dGLxblks841cYW9+x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 19, 2025 at 10:33=E2=80=AFAM Shakeel Butt wrote: > > On Wed, Jun 18, 2025 at 02:06:17PM +1000, Dave Airlie wrote: > > From: Dave Airlie > > > > While discussing memcg intergration with gpu memory allocations, > > it was pointed out that there was no numa/system counters for > > GPU memory allocations. > > > > With more integrated memory GPU server systems turning up, and > > more requirements for memory tracking it seems we should start > > closing the gap. > > > > Add two counters to track GPU per-node system memory allocations. > > > > The first is currently allocated to GPU objects, and the second > > is for memory that is stored in GPU page pools that can be reclaimed, > > by the shrinker. > > > > Cc: Christian Koenig > > Cc: Matthew Brost > > Cc: Johannes Weiner > > Cc: linux-mm@kvack.org > > Cc: Andrew Morton > > Signed-off-by: Dave Airlie > > > > --- > > > > I'd like to get acks to merge this via the drm tree, if possible, > > > > Dave. > > --- > > Documentation/filesystems/proc.rst | 6 ++++++ > > drivers/base/node.c | 5 +++++ > > fs/proc/meminfo.c | 6 ++++++ > > include/linux/mmzone.h | 2 ++ > > mm/show_mem.c | 9 +++++++-- > > mm/vmstat.c | 2 ++ > > 6 files changed, 28 insertions(+), 2 deletions(-) > > > > diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesys= tems/proc.rst > > index 5236cb52e357..45f61a19a790 100644 > > --- a/Documentation/filesystems/proc.rst > > +++ b/Documentation/filesystems/proc.rst > > @@ -1095,6 +1095,8 @@ Example output. You may not have all of these fie= lds. > > CmaFree: 0 kB > > Unaccepted: 0 kB > > Balloon: 0 kB > > + GPUActive: 0 kB > > + GPUReclaim: 0 kB > > HugePages_Total: 0 > > HugePages_Free: 0 > > HugePages_Rsvd: 0 > > @@ -1273,6 +1275,10 @@ Unaccepted > > Memory that has not been accepted by the guest > > Balloon > > Memory returned to Host by VM Balloon Drivers > > +GPUActive > > + Memory allocated to GPU objects > > +GPUReclaim > > + Memory in GPU allocator pools that is reclaimable > > Can you please explain a bit more about these GPUActive & GPUReclaim? > Please correct me if I am wrong, GPUActive is the total memory used by > GPU objects and GPUReclaim is the subset of GPUActive which is > reclaimable (possibly through shrinkers). Currently, GPUActive is total memory used by active GPU objects. GPUReclaim is the amount of memory (not a subset of Active) that is being stored in GPU reusable pools, that can be retrieved via a simple shrinker. (this memory usually has different page table attributes, uncached or writecombined). Example workflow: User allocates cached system RAM for GPU object: Active increases, Free cached system RAM, Active decreases. User allocates write combined system RAM for GPU object: Active increases Free write combined system RAM Active decreases, Reclaim increases User allocates another WC system RAM object: Reclaim decreases Active increases Shrinker shrinks the pool: Reclaim decreases. There could be in the future a 3rd type of memory which I'm not sure it if necessary to account at this level, but it would be Active memory that the driver considers discardable, and could be shrunk easily, but I'm not seeing much consistency on usage in drivers for this, or even what use case it is needed for, so I'm not going to address it yet. This could end up in Reclaim, but I'd need to see the use cases for it. Dave.