From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CC59C0218D for ; Thu, 30 Jan 2025 02:02:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB5B62800AF; Wed, 29 Jan 2025 21:02:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C3EE32800A8; Wed, 29 Jan 2025 21:02:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB81F2800AF; Wed, 29 Jan 2025 21:02:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8973B2800A8 for ; Wed, 29 Jan 2025 21:02:46 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0C988A03E0 for ; Thu, 30 Jan 2025 02:02:46 +0000 (UTC) X-FDA: 83062469532.07.8A5B574 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf20.hostedemail.com (Postfix) with ESMTP id 308DD1C000A for ; Thu, 30 Jan 2025 02:02:43 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Cfl4J7fP; spf=pass (imf20.hostedemail.com: domain of yuanchu@google.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=yuanchu@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738202564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zjUEKGcrUN8SEEUn3045QyzgbV2Muq4d8woD1luLXEo=; b=DpbFbQf+qx4jAqX6oewHO8D7FHcAV0YEe2qaAn8+a9H3tGYMaQEXGtAh0+NhwLw/Wlid9D bnHVW3iMigUZ+Bj6X7Hz/ubSSlU3UqLbmniD3YLJIcy2yeb4ODRr+w4Qz00j2qINnoY0+i Ni23KJ+yyuDAj/TpnA1PM0WEBuO/viE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738202564; a=rsa-sha256; cv=none; b=SY2Vqvjpk5Z/Fu62EIUvXzHO9hlFrJMlsl+TTYvl99a1iu2NhHEQ32ER5kCnqU2W22ofFA 35LWaSWN8AQFiymcC7xyrzYz13Ecd2WOtBo8SvtOnHz18Y1gCsh6ZC7AgFCLXulADLr2XE j6vl/q35MLSzWIQgba0X9zTn1qiy+5k= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Cfl4J7fP; spf=pass (imf20.hostedemail.com: domain of yuanchu@google.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=yuanchu@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-21625b4f978so63075ad.0 for ; Wed, 29 Jan 2025 18:02:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738202563; x=1738807363; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zjUEKGcrUN8SEEUn3045QyzgbV2Muq4d8woD1luLXEo=; b=Cfl4J7fPBe8vURM9GaZPuoXqdTKUKCQrE1e7hLMlsSg0/3cG3Zb8jlu2zPZWMxNyuM 9RJPVkGIu9lLMqGAP05N6sbjwHw/NrCIxAWvo8QVYGPK4jmBED7hOZrafIqV9jpAwHFy 0f3tYHTgCh3bqood7HfRdxBM+Q0EpHhFx0rXiEy30fqvmt/z2Uq1g0T0n8+jEtWTglAR Av7h3eHxmH+yaN1NfAIe5Zxc1NizopydA2xYD/0I/QwBSVII21RaVHIh4MIOLqOh6sZa aw9gjz3B7TvTc03c4U6Om1/hZyNI+8B+TO2EgutjCtZpx1ytQbEk2eVT7q6y7AmYWtHc GKqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738202563; x=1738807363; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zjUEKGcrUN8SEEUn3045QyzgbV2Muq4d8woD1luLXEo=; b=a0BFnKG5oCqa6yuw7qWESFNx6OrzzoL+cYBv5KfJI3LmjIFD6mbb0x6yQbGsU4pGJp nkItHK1IgSGryWAGeaRgfvSBJ9ECM/wiSd15P+sOjJjOBPViChNSrIkEfwmzV7XnZRSs GeX+enujS/JfMRCglMX1+Z9tFVRLpT7ZhNIVE3g0geHFI2+8DMW6KjS5rj1quSnBJKHf 5eahb8YnDAgVLKYKcQOIcE7FFykcZ4BPRW/yGXoQe9MJ2JUoq0K5sRudyDdv47TdUp/+ UOW+FrBdH3UOTJc/P7gp9wDolb5ycdJpMTBoEZJSjbn6d4VIGBysPx2SSupnQ1ppvQHM 5bKQ== X-Forwarded-Encrypted: i=1; AJvYcCUGnSlhVQZV2/PEkUlbPor+mY2Y7Qs9dS4YOwtZa8rHH3NpnmHf4nxVq6n0tt2STP8sKDQ2Vu8FLA==@kvack.org X-Gm-Message-State: AOJu0YxeadfnbEp6HvZ3AUYz46rTDdM3hd7E4kBJQXZgO0JgzMUUmguz 1g0J5A5ZAHS9OBOly9NHsiHqD3HHR3A/gKwjHlflaEMjxdzrx6p6LXDerWs5J3INgjY4F9b6nw6 ebrZsktzjA7YgDsb6yxHBe183AuO1MHTPDOO0 X-Gm-Gg: ASbGnctbYDYdX9fZlhbC+2i1miJLbI+Zbn9Q3ZrJksRaez0WFVI5TxaoPXOF8ZDzUP7 KzXphAvWD955o+P29o2d9QUDJTPkXrwWCgYqEn3P+CuQ+3/9lLlfLo8SVirUdzC3fxZ2KJ/9pCj 7NshCpUf7MSNm4Xmes+zfHjsY6jw== X-Google-Smtp-Source: AGHT+IFE4A3aWM8yz21uOrZOg4SChpOzryWR+ItPTzgeX3qzwaFRLrgsLB6RJr5+4uESj8JF/qvVZaYyrmavlvqmppA= X-Received: by 2002:a17:902:fc4b:b0:215:9ab0:402 with SMTP id d9443c01a7336-21de24504dfmr1287495ad.18.1738202562597; Wed, 29 Jan 2025 18:02:42 -0800 (PST) MIME-Version: 1.0 References: <20241211195329.60224-1-sj@kernel.org> In-Reply-To: <20241211195329.60224-1-sj@kernel.org> From: Yuanchu Xie Date: Wed, 29 Jan 2025 18:02:26 -0800 X-Gm-Features: AWEUYZmCJuhnrHdMr4LNNVLS2bbqy0msDks-3N4T6wXtjurzCtTNwSUNxZIoRCo Message-ID: Subject: Re: [PATCH v4 0/9] mm: workingset reporting To: SeongJae Park Cc: David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Lance Yang , Randy Dunlap , Muhammad Usama Anjum , Tejun Heo , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , =?UTF-8?Q?Eugenio_P=C3=A9rez?= , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Mike Rapoport , Shuah Khan , Christian Brauner , Daniel Watson , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org, linux-kselftest@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: jgcds887yix8cks6tmqexrrcsyaf7nyf X-Rspamd-Queue-Id: 308DD1C000A X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738202563-389795 X-HE-Meta: U2FsdGVkX19SrSyL/cRx3oMMqvYz3Tdy5n+J0F/VwW/1o8tebyNhmj+oJ6mTuGSyoomInSoouNUNMCVdHGjUmqEcsv8vLPlsvfcONgY1wh8jy6dz3FslXhZZ90hfwBMByMF/D7C5rf6ZbmNBcnHxGTSGKi0VCTUJFoeIGj8uDgLKAhmkweWD7Th5tO/7P4WMm/SpszxjIn88GSGg2uWPzZQmuIKOxLCgK0+fb606itG4C0mJa0Q224/G9X1iid88PRb5W/nHy4eFE22lw2PlIHMJ65ZGmMbxQw48tV/E+xpqu99y3zYKPgqdZq2PRNewjmLyyeG3X0JlzPbdhxEPUATBWg9bDdDANohNRyldEa+8NFZjLFCtysKWdkAMcW1wfzRecwSYnzc6iUif64L3OGHMPheZAXcaGlg4ocvddpJNcurC5eFWACn2azVM2wJndkvaWsELthUn8vUl5BNz7OdjCtSVX4uHoOm00ZuIKJKw3UNKGnsRzM++4HkmUlBai00u0X0xAZRLyDR7J994g1wRnLB/mg+Rg+WR6BfvbzNQyc91J5tkiAb9XVMKWOVENX87j7R2MSvgVv+kC2gpFuaNWuzp9qhVg8dT6u4T12xZUmOIKsQitLgCRWeoDsXHH9m05/PrHmOjlsCI9v1gUgrP1aK6OemWKs1LzaCX9rn14FdCjv2KNiTgA/h03yxmfh9RUQbq7KM1HAlY8NTwSPQrtE0HXqs7ZN2gkQaj6BNe4ESDAZvA1NrKtM5Ler5qX4fT1bfjaABigKJnwnPe0ugABdOD2+6Rj5y5zdm1QRfhFEoZZfl6HZWUNsoGnfLjVNnKoFimhmOcIoaUs0y/qUsY3scT3DNae2jgVx3D30xitw1UFCNpzuZDAKA7KDmoXQWXaU9DnlemPHxaC6YuqPfgnp3gGGsWNcxbJiE8wyFa22N7lyAoAKoEny5fi5jDnWzNJxRsZ52j0Zkjn+T yLlfqOt+ Icgy/gmhN74HjyAN0YRWvILQoDw5/5dLms7J8QMLYVleQjUG9xy7hjYGSGITCdANU9/B2+sUpYfnB2hIjxcbc03je0g+63UHJ2/eUBugmH12gjNEE93IGeT9DjoxTu0SFCQv/kzVQOGYqxl8NHd6lV5673woqqFoKTAW/PImWxTDSHlHb6vpU/QtytFEcLRZ7RLLsH19DWgq3Vm/m6wGz3/PlYQUghWSMw5qS3bnaGLXy0YatWZhgRjv9oo97RR5D2syCvQrVzfN9BKvDu3HjRnmLnSS+CJ6a113/8XIxH/C1X1s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Dec 11, 2024 at 11:53=E2=80=AFAM SeongJae Park wrot= e: > > On Fri, 6 Dec 2024 11:57:55 -0800 Yuanchu Xie wrote: > > > Thanks for the response Johannes. Some replies inline. > > > > On Tue, Nov 26, 2024 at 11:26\u202fPM Johannes Weiner wrote: > > > > > > On Tue, Nov 26, 2024 at 06:57:19PM -0800, Yuanchu Xie wrote: > > > > This patch series provides workingset reporting of user pages in > > > > lruvecs, of which coldness can be tracked by accessed bits and fd > > > > references. However, the concept of workingset applies generically = to > > > > all types of memory, which could be kernel slab caches, discardable > > > > userspace caches (databases), or CXL.mem. Therefore, data sources m= ight > > > > come from slab shrinkers, device drivers, or the userspace. > > > > Another interesting idea might be hugepage workingset, so that we c= an > > > > measure the proportion of hugepages backing cold memory. However, w= ith > > > > architectures like arm, there may be too many hugepage sizes leadin= g to > > > > a combinatorial explosion when exporting stats to the userspace. > > > > Nonetheless, the kernel should provide a set of workingset interfac= es > > > > that is generic enough to accommodate the various use cases, and ex= tensible > > > > to potential future use cases. > > > > > > Doesn't DAMON already provide this information? > > > > > > CCing SJ. > > Thanks for the CC. DAMON was really good at visualizing the memory > > access frequencies last time I tried it out! > > Thank you for this kind acknowledgement, Yuanchu! > > > For server use cases, > > DAMON would benefit from integrations with cgroups. The key then would= be a > > standard interface for exporting a cgroup's working set to the user. > > I show two ways to make DAMON supports cgroups for now. First way is mak= ing > another DAMON operations set implementation for cgroups. I shared a roug= h idea > for this before, probably on kernel summit. But I haven't had a chance t= o > prioritize this so far. Please let me know if you need more details. Th= e > second way is extending DAMOS filter to provide more detailed statistics = per > DAMON-region, and adding another DAMOS action that does nothing but only > accounting the detailed statistics. Using the new DAMOS action, users wi= ll be > able to know how much of specific DAMON-found regions are filtered out by= the > given filter. Because we have DAMOS filter type for cgroups, we can know= how > much of workingset (or, warm memory) belongs to specific groups. This ca= n be > applied to not only cgroups, but for any DAMOS filter types that exist (e= .g., > anonymous page, young page). > > I believe the second way is simpler to implement while providing informat= ion > that sufficient for most possible use cases. I was anyway planning to do= this. For a container orchestrator like kubernetes, the node agents need to be able to gather the working set stats at a per-job level. Some jobs can create sub-hierarchies as well, so it's important that we have hierarchical stats. Do you think it's a good idea to integrate DAMON to provide some aggregate stats in a memory controller file? With the DAMOS cgroup filter, there can be some kind of interface that a DAMOS action or the damo tool could call into. I feel that would be a straightforward and integrated way to support cgroups. Yuanchu