From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E99CBC83038 for ; Tue, 1 Jul 2025 08:14:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DA486B009E; Tue, 1 Jul 2025 04:14:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B2666B00A0; Tue, 1 Jul 2025 04:14:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C8656B00A1; Tue, 1 Jul 2025 04:14:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 454B56B009E for ; Tue, 1 Jul 2025 04:14:27 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 166F0B8663 for ; Tue, 1 Jul 2025 08:07:00 +0000 (UTC) X-FDA: 83614965000.08.5320F7C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id CAA4EC0013 for ; Tue, 1 Jul 2025 08:06:57 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d0RcGJUR; spf=pass (imf28.hostedemail.com: domain of airlied@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=airlied@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751357217; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/xLK3zn6hnNDwcaFv8tczFp10l9/ZMwRevm7NRDXVYA=; b=MviPLQfg1RXNmKo+zcatbmtJLwHSSSkajbQmMVUKxpzu6u9BzYZ7XparVmC8HTGBKQzl2L 47Nqy7l+c1WOTTChsErZnU27QKxArhtyptyKWr399RANtew77RGuhiEjqWyuaHh2QXTmQW k7oEmlOojVUArHQUjSCsC918tc5hGKY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d0RcGJUR; spf=pass (imf28.hostedemail.com: domain of airlied@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=airlied@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751357217; a=rsa-sha256; cv=none; b=IwkS//zDwFsOfUBtPBnnalQMyR8pEnD163wgFt8bbeEdQHyXApJS3OzuaUWnzf2VFrbCwF SJlb+FlTHtyT5k+xYL610mhnRl1KyqOFdQl9bx2cqpjT0bUAQJdLPd/f+wDdoK2VsnJHmh r2rFHAx1GkTjS6PMwwHXh3Zwk8CG91w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751357217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/xLK3zn6hnNDwcaFv8tczFp10l9/ZMwRevm7NRDXVYA=; b=d0RcGJUR5QYYQt0hlNCOkyqYGgYknznC50rDsPY3yBR8hForOZIj3HZdOLsCjvsMG5xStJ /mAr7sYoGOPf7noINFllxMWP0bKcD79Eip2bRJuAFG9yw0Z7oesC34+EKDzgo1UN6qeJoE Y6gBiq/fbN1QaE8vwd8IiXpYihKe+WE= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-351-v-VWKZh5Olaq9P6R7GqkCg-1; Tue, 01 Jul 2025 04:06:56 -0400 X-MC-Unique: v-VWKZh5Olaq9P6R7GqkCg-1 X-Mimecast-MFC-AGG-ID: v-VWKZh5Olaq9P6R7GqkCg_1751357215 Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-315af0857f2so3032417a91.0 for ; Tue, 01 Jul 2025 01:06:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751357214; x=1751962014; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/xLK3zn6hnNDwcaFv8tczFp10l9/ZMwRevm7NRDXVYA=; b=qEGficwX6+YxpZugKj59pbIuXczmjDuEKbXhenG+2RTIetlDXaMsa7ob4xy8jpB57b yBV2HQ9K07LjtPELwo++vqBvdybQX9STdv6HSoAfHHdsX6M783nfQBQOzSMF/5gAgQ4m uDnXWmDQUyksfAoY/JWzdghGbU2nRzEe5NBrYVIGUz1pSRm7S2YVfPdbVNAvw5Key8IE OqD3PJ+y6lpt+GLdOJ/v4N6O4bIs6hc3rwMjJvK5NWqq+qHzgcwgIXQj/VUP/uiTEzP5 owf9p8Lu/fHFkzX5cgf3HIYK+SjQHdwzew+3vyHpYoR3j5rfF59xnCVEE4ZbyLKQyICk nVZg== X-Forwarded-Encrypted: i=1; AJvYcCVgDRax7GHP01CLBFK2s3FgDRPBuZSo6mFg6OvqmbHQVuH8ncbZkUSu4zKdHE0yQRLpq610hmSEDw==@kvack.org X-Gm-Message-State: AOJu0YxKHX+idcwElz6rA329qfuhJ671ATVZKrJfAA+7WO5deVSgGyET qPW7uG9Sd4UpkHdHzGq6InCyYDZSyz0GT/wNMk/QiJ2UIHc3qOXt/uHUpn2k+US7Vu3fDP2sDjS C5Ki+18mbsq/8+Q4fCSYsKsKv2MbH52CHhv+oqdyWI0DfBBVPiesQXjlhe0g50p+PymkLohPbo7 ovCkUdrMbUr8aqQclIfWzbuJdanGRJOjzr4RQ= X-Gm-Gg: ASbGncv9gFWBkDotNEtwoFGK4Z7/suUmPPoXWwdH6TZbyid5a4giuplnY6Fzo5/rabh 57nrYgQbgaA2wcFFPe/w+QhGjjnRER2HtIVqZJ0dqLP8RGPubUZGKYjhoeJSRBxsw5yP27qxzgD xX/Q== X-Received: by 2002:a17:90a:e7d0:b0:311:a314:c2d1 with SMTP id 98e67ed59e1d1-318c910dd2amr21627768a91.6.1751357214172; Tue, 01 Jul 2025 01:06:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGnPC7dxHW1zo1ojXv0A3+oYFSOsb2eKLl2v85+jsxM/Gh14AF4yL16yb/X0R8sTooo30Idk4IynWvdgK97ZxE= X-Received: by 2002:a17:90a:e7d0:b0:311:a314:c2d1 with SMTP id 98e67ed59e1d1-318c910dd2amr21627730a91.6.1751357213713; Tue, 01 Jul 2025 01:06:53 -0700 (PDT) MIME-Version: 1.0 References: <20250630045005.1337339-1-airlied@gmail.com> <20250630045005.1337339-13-airlied@gmail.com> <20a90668-3ddf-4153-9953-a2df9179a1b1@amd.com> In-Reply-To: From: David Airlie Date: Tue, 1 Jul 2025 18:06:42 +1000 X-Gm-Features: Ac12FXxXfpBXAeVSqi2RcEI9hELPChB3FlteEOITqF5PoK6jsBCqVxmbsPeC3lw Message-ID: Subject: Re: [PATCH 12/17] ttm: add objcg pointer to bo and tt To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: Dave Airlie , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, Johannes Weiner , Dave Chinner , Kairui Song X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: N5pZgowWILl_UzvpLp1M4110ifVxWRHPXbF9IrAu3ws_1751357215 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: o9jc65eu1u4wycwyjxjbpwjrapyhmdq4 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: CAA4EC0013 X-Rspam-User: X-HE-Tag: 1751357217-447954 X-HE-Meta: U2FsdGVkX18iMw+FZFwcZssrGLuAWvJMon9PsqfeDf+60DLhdGGLEJnKld16ucH6gZAbfTCaqsZkiEXIwfqNT9x2W5igWT+wxj05Q6YGyXNpsFB1Cysa2HB3sKOOtLgNV6RH51eajjc5Org0fplTNG3GIm0TlG353HlHS29ePvsbsQrVJyau4yBC3CDmgbjwObq/7wsek5u/ojWiCU9DOSaa9qDTqFamplhkxJoywa3setfaOFiYGNUJQn+GgzGEEs6M6KBUUGgpM6CJ5CTlfv74VKCqZQ8jBS/UKNd50xWkSYUB4+1pS4HukAQrjIuzR4wIi9fpqnj1ao4xm+r7akzSwMyQgAmOFwJEe4FoC6Dcd4HOLowmc1EfrBE6c8bu52IkSfYjcuLHRxc1WkEH466mb52JBbV+QBe5dcP8pIZByVqb/Nqc0OYZ6iBmB4y2isjeCgXNdVRofU8AHuGluIvzkRayjZom/I/+O9albgrzQoz3opcsaOG7/WqzfHdWVzf6k6WuxWIyErjDJt/XAenB6dH4fFib/Sq5GvILDBK/t5gt5t4SIbjOb3KQhbLfRe+ke/4R6th99C5iqt7PfOU958FPWrL3eWW9PRI6mVSDEX6u3LKJKVpqDaOCLyy9CjNhy5AYJktKZZJAomsJhHIWymutUoZhtATl6dM1noCxFNwrXTKwZKIQFbir8COMODLt/Laol/MfWB8xzjsIlwjPTvmIE2ZD7B6ggGV4doc2m8WH1/Gr2hA/oau52w3vWm8cxy9ihGgOWLkE1+iBwshrJ/0t0YIntoIqE0Qzl9ulnNjM/ROQ8W4WxwYn8QzUmZA432GBH6ba702miTA/auQvupsFwJ12PPvptwAbRc3/DINZuoFC54RzALpSkRVXeoJ3fVE7xMfLe0DuMIlunuZGrjuXyRY3PgdYjqjcQyy21dirF0UBbDoJurZJNRYnZKJqlikzK+uKoyEHEIY zYGNrbkU kMHSiwlRYW1K74NpVX5HFnDCMdTaOgyXKTmTT/eshc38to1u+l3Opjil/K6NGB2ElqLaUxj2zXI04XWXqerH5CP8UExxo8p4nXsxhXY1SdfBDXbzSCVSjdi3eVyRE7wcDNhnodgZNkmgS+oQILDoSRN1yzvB9YCFzRmLUSdWqaJ9gfXbPZwZHrmMaILZQctA/r92FEFc8ER2fsceN3nEHtF0iBun/D9G8X66fugXpOq0MHVxGanig0zE/CG1FMMie595NnYzgmWoA0/e/nz0wrKFHgP0jab4YsYyjvTd5B5IxHX0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 1, 2025 at 5:22=E2=80=AFPM Christian K=C3=B6nig wrote: > > On 30.06.25 23:33, David Airlie wrote: > > On Mon, Jun 30, 2025 at 8:24=E2=80=AFPM Christian K=C3=B6nig > > wrote: > >> > >> On 30.06.25 06:49, Dave Airlie wrote: > >>> From: Dave Airlie > >>> > >>> This just adds the obj cgroup pointer to the bo and tt structs, > >>> and sets it between them. > >>> > >>> Signed-off-by: Dave Airlie > >>> --- > >>> drivers/gpu/drm/ttm/ttm_tt.c | 1 + > >>> include/drm/ttm/ttm_bo.h | 6 ++++++ > >>> include/drm/ttm/ttm_tt.h | 2 ++ > >>> 3 files changed, 9 insertions(+) > >>> > >>> diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_t= t.c > >>> index 8f38de3b2f1c..0c54d5e2bfdd 100644 > >>> --- a/drivers/gpu/drm/ttm/ttm_tt.c > >>> +++ b/drivers/gpu/drm/ttm/ttm_tt.c > >>> @@ -162,6 +162,7 @@ static void ttm_tt_init_fields(struct ttm_tt *ttm= , > >>> ttm->caching =3D caching; > >>> ttm->restore =3D NULL; > >>> ttm->backup =3D NULL; > >>> + ttm->objcg =3D bo->objcg; > >>> } > >>> > >>> int ttm_tt_init(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > >>> diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h > >>> index 099dc2604baa..f26ec0a0273f 100644 > >>> --- a/include/drm/ttm/ttm_bo.h > >>> +++ b/include/drm/ttm/ttm_bo.h > >>> @@ -135,6 +135,12 @@ struct ttm_buffer_object { > >>> * reservation lock. > >>> */ > >>> struct sg_table *sg; > >>> + > >>> + /** > >>> + * @objcg: object cgroup to charge this to if it ends up using = system memory. > >>> + * NULL means don't charge. > >>> + */ > >>> + struct obj_cgroup *objcg; > >>> }; > >>> > >>> #define TTM_BO_MAP_IOMEM_MASK 0x80 > >>> diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h > >>> index 15d4019685f6..c13fea4c2915 100644 > >>> --- a/include/drm/ttm/ttm_tt.h > >>> +++ b/include/drm/ttm/ttm_tt.h > >>> @@ -126,6 +126,8 @@ struct ttm_tt { > >>> enum ttm_caching caching; > >>> /** @restore: Partial restoration from backup state. TTM privat= e */ > >>> struct ttm_pool_tt_restore *restore; > >>> + /** @objcg: Object cgroup for this TT allocation */ > >>> + struct obj_cgroup *objcg; > >>> }; > >> > >> We should probably keep that out of the pool and account the memory to= the BO instead. > >> > > > > I tried that like 2-3 patch posting iterations ago, you suggested it > > then, it didn't work. It has to be done at the pool level, I think it > > was due to swap handling. > > When you do it at the pool level the swap/shrink handling is broken as we= ll, just not for amdgpu. > > See xe_bo_shrink() and drivers/gpu/drm/xe/xe_shrinker.c on how XE does it= . I've read all of that, but I don't think it needs changing yet, though I do think I probably need to do a bit more work on the ttm backup/restore paths to account things, but again we suffer from the what happens if your cgroup runs out of space on a restore path, similiar to eviction. Blocking the problems we can solve now on the problems we've no idea how to solve means nobody gets experience with solving anything. > So the best we can do is to do it at the resource level because that is c= ommon for everybody. > > This doesn't takes swapping on amdgpu into account, but that should not b= e that relevant since we wanted to remove that and switch to the XE approac= h anyway. I don't understand, we cannot do it at the resource level, I sent patches to try, they don't fundamentally work properly, so it isn't going to fly. We can solve it at the pool level, so we should, if we somehow rearchitect things later to solve it at the resource level, but I feel we'd have to make swap handling operate at the resource level instead of tt level to have any chance. Swapping via the backup/restore paths should be accounted properly, since moving pages out to swap one way cgroups can reduce the memory usage, if we can't account that swapped pages aren't removed from the page count, then it isn't going to work properly. Dave.