From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E949C433EF for ; Wed, 23 Feb 2022 15:21:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B13D8D000D; Wed, 23 Feb 2022 10:21:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 660638D0001; Wed, 23 Feb 2022 10:21:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 503008D000D; Wed, 23 Feb 2022 10:21:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id 2F2A78D0001 for ; Wed, 23 Feb 2022 10:21:25 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BCCB792DE9 for ; Wed, 23 Feb 2022 15:21:24 +0000 (UTC) X-FDA: 79174408488.21.97F7E1D Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf01.hostedemail.com (Postfix) with ESMTP id 0C8EC40010 for ; Wed, 23 Feb 2022 15:21:23 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C7C27B82079; Wed, 23 Feb 2022 15:21:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95711C340E7; Wed, 23 Feb 2022 15:21:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1645629680; bh=VSQR+hx02n7ejqcnmXQ4srcfdfDnZENuxQVT/EaK31A=; h=From:To:Cc:Subject:Date:From; b=gA5a3UNquyhltWkY1aMDYPpOIwRgGUfagIY7vYRNdG4Xtod8tfIuKS+xpOXvqJQHI gzQLOWlrhvPvB4lnQzzqjcWA7W0xSclVmKkrM2Yb3MbkRXNZDxtcdi/bxptpdnx7O6 JA3HZj5mqEk1u1CGAt4Yy1Hx/bq3Vs/BDDIi3r6a9Ii8zHHkXbYi0VtNdDqS213h6+ idlRWwZw+vcis6dNNkOWpJn87whlrGKQYGGzw42jXUEnZ+64c6867nd7whU1LIsLR1 FPiIHdD/3jmcKHekDDGLNHcH7Um7TyOyTEA/ZdS3kSWVo8yFyAWN8rSUV/4uJsWJ8R h6brnx4CXQPDg== From: SeongJae Park To: akpm@linux-foundation.org Cc: corbet@lwn.net, skhan@linuxfoundation.org, rientjes@google.com, xhao@linux.alibaba.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, SeongJae Park Subject: [PATCH 00/12] Introduce DAMON sysfs interface Date: Wed, 23 Feb 2022 15:20:39 +0000 Message-Id: <20220223152051.22936-1-sj@kernel.org> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gA5a3UNq; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf01.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0C8EC40010 X-Stat-Signature: o57n71g1ew6egi13sad1ot6j9dzhbitn X-HE-Tag: 1645629683-673157 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Chages from Previous Version (RFC) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D Compared to the RFC version of this patchset (https://lore.kernel.org/linux-mm/20220217161938.8874-1-sj@kernel.org/), = this version contains below changes. - Implement all DAMON debugfs interface providing features - Writeup documents - Add more selftests Introduction =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D DAMON's debugfs-based user interface (DAMON_DBGFS) served very well, so f= ar. However, it unnecessarily depends on debugfs, while DAMON is not aimed to= be used for only debugging. Also, the interface receives multiple values vi= a one file. For example, schemes file receives 18 values. As a result, it is inefficient, hard to be used, and difficult to be extended. Especially, keeping backward compatibility of user space tools is getting only challe= nging. It would be better to implement another reliable and flexible interface a= nd deprecate DAMON_DBGFS in long term. For the reason, this patchset introduces a sysfs-based new user interface= of DAMON. The idea of the new interface is, using directory hierarchies and having one dedicated file for each value. For a short example, users can= do the virtual address monitoring via the interface as below: # cd /sys/kernel/mm/damon/admin/ # echo 1 > kdamonds/nr # echo 1 > kdamonds/0/contexts/nr # echo vaddr > kdamonds/0/contexts/0/operations # echo 1 > kdamonds/0/contexts/0/targets/nr # echo $(pidof ) > kdamonds/0/contexts/0/targets/0/pid # echo on > kdamonds/0/state A brief representation of the files hierarchy of DAMON sysfs interface is= as below. Childs are represented with indentation, directories are having '= /' suffix, and files in each directory are separated by comma. /sys/kernel/mm/damon/admin =E2=94=82 kdamonds/nr =E2=94=82 =E2=94=82 0/state,pid =E2=94=82 =E2=94=82 =E2=94=82 contexts/nr =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 0/operations =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 monitoring_attrs/ =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 intervals= /sample_us,aggr_us,update_us =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 nr_region= s/min,max =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 targets/nr =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 0/pid =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= regions/nr =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= =E2=94=82 0/start,end =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= =E2=94=82 ... =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 ... =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 schemes/nr =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 0/action =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 access_pa= ttern/ =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= sz/min,max =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= nr_accesses/min,max =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= age/min,max =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 quotas/ms= ,sz,reset_interval_ms =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82= weights/sz,nr_accesses,age =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 watermark= s/metric,interval_us,high,mid,low =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 stats/nr_= tried,sz_tried,nr_applied,sz_applied,qt_exceeds =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 ... =E2=94=82 =E2=94=82 ... Detailed usage of the files will be described in the final Documentation = patch of this patchset. Main Difference Between DAMON_DBGFS and DAMON_SYSFS --------------------------------------------------- At the moment, DAMON_DBGFS and DAMON_SYSFS provides same features. One important difference between them is their exclusiveness. DAMON_DBGFS wo= rks in an exclusive manner, so that no DAMON worker thread (kdamond) in the syst= em can run concurrently and interfere somehow. For the reason, DAMON_DBGFS asks= users to construct all monitoring contexts and start them at once. It's not a = big problem but makes the operation a little bit complex and unflexible. For more flexible usage, DAMON_SYSFS moves the responsibility of preventi= ng any possible interference to the admins and work in a non-exclusive manner. = That is, users can configure and start contexts one by one. Note that DAMON respects both exclusive groups and non-exclusive groups of contexts, in a manner similar to that of reader-writer locks. That is, if any exclusive monitoring contexts (e.g., contexts that started via DAMON_DBGFS) are run= ning, DAMON_SYSFS does not start new contexts, and vice versa. Future Plan of DAMON_DBGFS Deprecation =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Once this patchset is merged, DAMON_DBGFS development will be frozen. Th= at is, we will maintain it to work as is now so that no users will be break. Bu= t, it will not be extended to provide any new feature of DAMON. The support wi= ll be continued only until next LTS release. After that, we will drop DAMON_DB= GFS. User-space Tooling Compatibility -------------------------------- As DAMON_SYSFS provides all features of DAMON_DBGFS, all user space tooli= ng can move to DAMON_SYSFS. As we will continue supporting DAMON_DBGFS until ne= xt LTS kernel release, user space tools would have enough time to move to DAMON_= SYSFS. The official user space tool, damo[1], is already supporting both DAMON_S= YSFS and DAMON_DBGFS. Both correctness tests[2] and performance tests[3] of D= AMON using DAMON_SYSFS also passed. [1] https://github.com/awslabs/damo [2] https://github.com/awslabs/damon-tests/tree/master/corr [3] https://github.com/awslabs/damon-tests/tree/master/perf Complete Git Tree =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D You can get the complete git tree from https://git.kernel.org/sj/h/damon/sysfs/patches/v1. Sequence of Patches =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D First two patches (patches 1-2) make core changes for DAMON_SYSFS. The f= irst one (patch 1) allows non-exclusive DAMON contexts so that DAMON_SYSFS can= work in non-exclusive mode, while the second one (patch 2) adds size of DAMON = enum types so that DAMON API users can safely iterate the enums. Third patch (patch 3) implements basic sysfs stub for virtual address spa= ces monitoring. Note that this implements only sysfs files and DAMON is not linked. Fourth patch (patch 4) links the DAMON_SYSFS to DAMON so that us= ers can control DAMON using the sysfs files. Following six patches (patches 5-10) implements other DAMON features that DAMON_DBGFS supports one by one (physical address space monitoring, DAMON= -based operation schemes, schemes quotas, schemes prioritization weights, scheme= s watermarks, and schemes stats). Following patch (patch 11) adds a simple selftest for DAMON_SYSFS, and th= e final one (patch 12) documents DAMON_SYSFS. SeongJae Park (12): mm/damon/core: Allow non-exclusive DAMON start/stop mm/damon/core: Add number of each enum type values mm/damon: Implement a minimal stub for sysfs-based DAMON interface mm/damon/sysfs: Link DAMON for virtual address spaces monitoring mm/damon/sysfs: Support physical address space monitoring mm/damon/sysfs: Support DAMON-based Operation Schemes mm/damon/sysfs: Support DAMOS quotas mm/damon/sysfs: Support schemes prioritization weights mm/damon/sysfs: Support DAMOS watermarks mm/damon/sysfs: Support DAMOS stats selftests/damon: Add a test for DAMON sysfs interface Docs/admin-guide/mm/damon/usage: Document DAMON sysfs interface Documentation/admin-guide/mm/damon/usage.rst | 349 ++- include/linux/damon.h | 6 +- mm/damon/Kconfig | 7 + mm/damon/Makefile | 1 + mm/damon/core.c | 23 +- mm/damon/dbgfs.c | 2 +- mm/damon/reclaim.c | 2 +- mm/damon/sysfs.c | 2684 ++++++++++++++++++ tools/testing/selftests/damon/Makefile | 1 + tools/testing/selftests/damon/sysfs.sh | 306 ++ 10 files changed, 3364 insertions(+), 17 deletions(-) create mode 100644 mm/damon/sysfs.c create mode 100755 tools/testing/selftests/damon/sysfs.sh --=20 2.17.1