From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F257D6D232 for ; Wed, 27 Nov 2024 19:40:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B68D36B0083; Wed, 27 Nov 2024 14:40:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B17F26B0092; Wed, 27 Nov 2024 14:40:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DFFA6B0093; Wed, 27 Nov 2024 14:40:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7CC766B0083 for ; Wed, 27 Nov 2024 14:40:45 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ED6FB1C5D87 for ; Wed, 27 Nov 2024 19:40:44 +0000 (UTC) X-FDA: 82832892198.25.80EF55A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id CC4E780017 for ; Wed, 27 Nov 2024 19:40:31 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Y4OdQFkp; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732736440; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P9ARbAgzgy3FD2EtjuO3lF3YkVEWJiey6PREkbybpCE=; b=krj0iiLrJwDvlmaXQs0a+ucHAMetx7h+sTqZT2BtKDIccX7yymlrClun+y3I01PgabgJaZ dLUsQUtKASWVzCapUlMCsagIAj+1G10WWtTsaskkznIZmY2j1VgiidTeSeOc8FwyVvu9dp bc5q6aSjGzT/v6x/SBR/QbPxqVSBzKA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732736440; a=rsa-sha256; cv=none; b=QsiBQnDOfH6h1ABBdqZJcEhemjbq+EmM1Mf6ec04eBlujtkv2ffBZvNemPuBvci69B2tef w+NzgoSg4F+bOAjNhRjy2nRkXR5w8U/MuYUxR7ZVSedpVgOByzQ8Wyo6KLTvaE/bHAxA56 MS4OlClvC6MFABMZMJaDcjVOkmzxmtY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Y4OdQFkp; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5EDD35C58FF; Wed, 27 Nov 2024 19:39:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EF0AC4CECC; Wed, 27 Nov 2024 19:40:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732736441; bh=FNpsPBymSvEX1rHGNSh2J+QjHOLtO8xKZwskRMfxD8A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y4OdQFkp2nt/VvgqWAzMb+U036mIXv7j2LJHKF/dpUn954h4y4SzH9pfQm5Llm4xC Xn3EHI2RFYk+gF5oK9ediPguZG96lc468mGXVKDVabS6zPzl3wbCS/MnXPq9wNWUoD 88QCHf1aG6/UUPTBoYdYVcG4036FWpAv+gaqp53RLXgsDDy3Z7Nu4AVunbRKleiL8c 4+D8sjtuIsIxt+phU4/VoCafcBznr1o2zDnmKGhv34cusEtthtKdEUJGBXWRWILSOo OA57OHPSkfugMlhiW2Ny9LvwIpITFIFKLXiw77pynSGgenHRCGTb3GbcimZ9kP5jMT 7BPLErjejFueQ== From: SeongJae Park To: Johannes Weiner Cc: SeongJae Park , Yuanchu Xie , David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Lance Yang , Randy Dunlap , Muhammad Usama Anjum , Tejun Heo , Michal =?iso-8859-1?Q?Koutn=FD?= , Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , Eugenio =?iso-8859-1?Q?P=E9rez?= , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Mike Rapoport , Shuah Khan , Christian Brauner , Daniel Watson , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, damon@lists.linux.dev Subject: Re: [PATCH v4 0/9] mm: workingset reporting Date: Wed, 27 Nov 2024 11:40:38 -0800 Message-Id: <20241127194038.84382-1-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241127072604.GA2501036@cmpxchg.org> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: 9cjacbmqduy1p9tbphnk44zb8wdzoiwo X-Rspamd-Queue-Id: CC4E780017 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1732736431-599364 X-HE-Meta: U2FsdGVkX1/AwSWr1AckGj6Q1O2RpHsnPwGZ/dydDHSrxHdDjBbMlBZ1uVpwlPAbfOeooX3UVDO9uV5IuxzMTzQm6YkNhoupMLIEW69nFyJDgJ7TCruHGIfatTA1aS2Tm65GCpmPaDVcvIoFeXoSmqE0myrPvPm3YTutGDCFVFa3Rkkgp4Bk6Dvi/pXUcT7j8k4Q7CoOzYf3zEkPnebH3T34/GbetRkmG1ECXnLT4WOQUA8lqJLqjfe7XNxkQ1lh01s/bLTiuJrVzZEzidU4EBa2SnNjzigSKFr52s4t2AlHtGiOaTMmy/zl3XFi+ITz/EZQ7N8nrMbeEvhj1Ifr1uSZpu4Om1W2ffJK2xfLEKSITDLkU6CJkOXXg5y1/1z8pi1lJFFf2n4tCG0ubGjtjndKzHFTvBEPkFPcrAsgUG1NJ6hh1k264OJYL4qkyfohTFaIT6ZjSnVgFwoWpj2LF5ohf6iqpjQ7koig7wbSpVKK9VhAh2sSHq/k6ESTOnZ9luTRCZdgbRZCx6VbawjyoXnu2RKkWoHtNifQhMsKECmEaQObFkzCMsTpDUjhzIYlbZlr9FsgwYb6gZ9OCoGZtmSKTWKrziUdrqmO7iX+ERukR/dLk+2pIwBI+AJDUb51iPC8/DEhRrb9lqOxUgDV7Z3PuJGPmuFnjDs4YSOh1Cfx1Rc/U91+t0ia37PbWXESZHByI33MWIlKHsJ+800YViB9fohGgWvc4n/2ikOUbCJK5XPUW9pprUINFTI3gD9DENR6KdHxTmVlieJEdz5n7fKB3xC2qBTu1KhAsut9G3fN3uq3zo2vscNQELmBkueFkF3mXnbNUnfR7JClOHJjTxKrPHg3EwZdIHyYZp7ehBi6i59wQSNagtVvkH96gxqcAin5CM+XxHrOLcfVa5bH8AvzdFuLvLaJ51iMTmKPihgvat4XsOdCoPdGjeLWOApDxrW4VhjpyQ0Hkm1P8GP TGeF+6MY mtnYw+riihshHzdGyF9jdAz3rGko+iAi6Al51cB0AE19XWZkKtXIKc512gt8hPuR/cl7NCcIhnirvRUYRZvPaVTH057wFx0lWBAoIlzkiCX1tBlNKFTWB4LBfW1w6mulwedmrDYvZ3rPtjj3Y2M4i7k9mCzPqSnHZ438k8NrNO9NYXvw6sF1DI9TJS6ZTjZWxAG3IbIBGSdBLbk03+LlteJrIUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: + damon@lists.linux.dev I haven't thoroughly read any version of this patch series due to my laziness, sorry. So I may saying something completely wrong. My apology in advance, and please correct me in the case. > On Tue, Nov 26, 2024 at 06:57:19PM -0800, Yuanchu Xie wrote: > > This patch series provides workingset reporting of user pages in > > lruvecs, of which coldness can be tracked by accessed bits and fd > > references. DAMON provides data access patterns of user pages. It is not exactly named as workingset but a superset of the information. Users can therefore get the workingset from DAMON-provided raw data. So I feel I have to ask if DAMON can be used for, or help at achieving the purpose of this patch series. Depending on the detailed definition of workingset, of course, the workingset we can get from DAMON might not be technically same to what this patch series aim to provide, and the difference could be somewhat that makes DAMON unable to be used or help here. But I cannot know if this is the case with only this cover letter. > > However, the concept of workingset applies generically to > > all types of memory, which could be kernel slab caches, discardable > > userspace caches (databases), or CXL.mem. Therefore, data sources might > > come from slab shrinkers, device drivers, or the userspace. > > Another interesting idea might be hugepage workingset, so that we can > > measure the proportion of hugepages backing cold memory. However, with > > architectures like arm, there may be too many hugepage sizes leading to > > a combinatorial explosion when exporting stats to the userspace. > > Nonetheless, the kernel should provide a set of workingset interfaces > > that is generic enough to accommodate the various use cases, and extensible > > to potential future use cases. This again sounds similar to what DAMON aims to provide, to me. DAMON is designed to be easy to extend for vairous use cases and internal mechanisms. Specifically, it separates access check mechanisms and core logic into different layers, and provides an interface to use for implementing extending DAMON with new mechanisms. DAMON's two access check mechanisms for virtual address spaces and the physical address space are made using the interface, indeed. Also there were RFC patch series extending DAMON for NUMA-specific and write-only access monitoring using NUMA hinting fault and soft-dirty PTEs as the internal mechanisms. My humble understanding of the major difference between DAMON and workingset reporting is the internal mechanism. Workingset reporting uses MGLRU as the access check mechanism, while current access check mechanisms for DAMON are using page table accessed bits checking as the major mechanism. I think DAMON can be extended to use MGLRU as its another internal access check mechanism, but I understand that there could be many things that I overseeing. Yuanchu, I think it would help me and other reviewers better understand this patch series if you could share that. And I will also be more than happy to help you and others better understanding what DAMON can do or not with the discussion. > > Doesn't DAMON already provide this information? > > CCing SJ. Thank you for adding me, Johannes :) [...] > It does provide more detailed insight into userspace memory behavior, > which could be helpful when trying to make sense of applications that > sit on a rich layer of libraries and complicated runtimes. But here a > comparison to DAMON would be helpful. 100% agree. Thanks, SJ [...]