[PATCH 00/10] mm/damon: misc fixes and improvements

[PATCH 00/10] mm/damon: misc fixes and improvements

[SeongJae Park]

Add miscelleneous and non-urgent fixes and improvements for DAMON code,
selftests, and documents.

SeongJae Park (10):
  mm/damon/core: initialize ->esz_bp from damos_quota_init_priv()
  selftests/damon/_damon_sysfs: check errors from nr_schemes file reads
  selftests/damon/_damon_sysfs: find sysfs mount point from /proc/mounts
  selftests/damon/_damon_sysfs: use 'is' instead of '==' for 'None'
  selftests/damon: classify tests for functionalities and regressions
  Docs/admin-guide/mm/damon/usage: fix wrong example of DAMOS filter
    matching sysfs file
  Docs/admin-guide/mm/damon/usage: fix wrong schemes effective quota
    update command
  Docs/mm/damon/design: use a list for supported filters
  Docs/mm/damon/maintainer-profile: change the maintainer's timezone
    from PST to PT
  Docs/mm/damon/maintainer-profile: allow posting patches based on
    damon/next tree

 Documentation/admin-guide/mm/damon/usage.rst  |  6 +-
 Documentation/mm/damon/design.rst             | 46 +++++----
 Documentation/mm/damon/maintainer-profile.rst | 13 +--
 mm/damon/core.c                               |  1 +
 tools/testing/selftests/damon/Makefile        | 13 ++-
 tools/testing/selftests/damon/_damon_sysfs.py | 95 +++++++++++--------
 6 files changed, 100 insertions(+), 74 deletions(-)


base-commit: fc7314cb6b750187a1366e0bf9da4c3ca8cfd064
-- 
2.39.2

[PATCH 01/10] mm/damon/core: initialize ->esz_bp from damos_quota_init_priv()

[SeongJae Park]

damos_quota_init_priv() function should initialize all private fields of
struct damos_quota.  However, it is not initializing ->esz_bp field.
This could result in use of uninitialized variable from
damon_feed_loop_next_input() function.  There is no such issue at the
moment because every caller of the function is passing damos_quota
object that already having the field zero value.  But we cannot
guarantee the future, and the function is not doing what it is
promising.  A bug is a bug.  This fix is for preventing possible future
issues.

Fixes: 9294a037c015 ("mm/damon/core: implement goal-oriented feedback-driven quota auto-tuning")
Signed-off-by: SeongJae Park <sj@kernel.org>
---
 mm/damon/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/damon/core.c b/mm/damon/core.c
index 172095e68c5d..6392f1cc97a3 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -346,6 +346,7 @@ static struct damos_quota *damos_quota_init(struct damos_quota *quota)
 	quota->charged_from = 0;
 	quota->charge_target_from = NULL;
 	quota->charge_addr_from = 0;
+	quota->esz_bp = 0;
 	return quota;
 }
 
-- 
2.39.2

[PATCH 02/10] selftests/damon/_damon_sysfs: check errors from nr_schemes file reads

[SeongJae Park]

DAMON context staging method in _damon_sysfs.py is not checking the
returned error from nr_schemes file read.  Check it.

Fixes: f5f0e5a2bef9 ("selftests/damon/_damon_sysfs: implement kdamonds start function")
Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index f80fdcef507c..fffa74a78bd7 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -341,6 +341,8 @@ class DamonCtx:
         nr_schemes_file = os.path.join(
                 self.sysfs_dir(), 'schemes', 'nr_schemes')
         content, err = read_file(nr_schemes_file)
+        if err is not None:
+            return err
         if int(content) != len(self.schemes):
             err = write_file(nr_schemes_file, '%d' % len(self.schemes))
             if err != None:
-- 
2.39.2

[PATCH 03/10] selftests/damon/_damon_sysfs: find sysfs mount point from /proc/mounts

[SeongJae Park]

_damon_sysfs.py assumes sysfs is mounted at /sys.  In some systems, that
might not be true.  Find the mount point from /proc/mounts file content.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index fffa74a78bd7..5367e98817a9 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -2,7 +2,18 @@
 
 import os
 
-sysfs_root = '/sys/kernel/mm/damon/admin'
+ksft_skip=4
+
+sysfs_root = None
+with open('/proc/mounts', 'r') as f:
+    for line in f:
+        dev_name, mount_point, dev_fs = line.split()[:3]
+        if dev_fs == 'sysfs':
+            sysfs_root = '%s/kernel/mm/damon/admin' % mount_point
+            break
+if sysfs_root is None:
+    print('Seems sysfs not mounted?')
+    exit(ksft_skip)
 
 def write_file(path, string):
     "Returns error string if failed, or None otherwise"
-- 
2.39.2

[PATCH 05/10] selftests/damon: classify tests for functionalities and regressions

[SeongJae Park]

DAMON selftests can be classified into two categories: functionalities
and regressions.  Functionality tests are for checking if the function
is working as specified, while the regression tests are basically
reproducers of previously reported and fixed bugs.  The tests of the
categories are mixed in the selftests Makefile.  Separate those for
easier understanding of the types of tests.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/Makefile | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 06c248880172..29a22f50e762 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -7,16 +7,21 @@ TEST_GEN_FILES += debugfs_target_ids_pid_leak
 TEST_GEN_FILES += access_memory
 
 TEST_FILES = _chk_dependency.sh _debugfs_common.sh
+
+# functionality tests
 TEST_PROGS = debugfs_attrs.sh debugfs_schemes.sh debugfs_target_ids.sh
+TEST_PROGS += sysfs.sh
+TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
+TEST_PROGS += damos_quota.py damos_quota_goal.py damos_apply_interval.py
+TEST_PROGS += reclaim.sh lru_sort.sh
+
+# regression tests (reproducers of previously found bugs)
 TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh
 TEST_PROGS += debugfs_duplicate_context_creation.sh
 TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += debugfs_target_ids_read_before_terminate_race.sh
 TEST_PROGS += debugfs_target_ids_pid_leak.sh
-TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
+TEST_PROGS += sysfs_update_removed_scheme_dir.sh
 TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
-TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
-TEST_PROGS += damos_quota.py damos_quota_goal.py damos_apply_interval.py
-TEST_PROGS += reclaim.sh lru_sort.sh
 
 include ../lib.mk
-- 
2.39.2

[PATCH 06/10] Docs/admin-guide/mm/damon/usage: fix wrong example of DAMOS filter matching sysfs file

[SeongJae Park]

The example usage of DAMOS filter sysfs files, specifically the part of
'matching' file writing for memcg type filter, is wrong.  The intention
is to exclude pages of a memcg that already getting enough care from a
given scheme, but the example is setting the filter to apply the scheme
to only the pages of the memcg.  Fix it.

Fixes: 9b7f9322a530 ("Docs/admin-guide/mm/damon/usage: document DAMOS filters of sysfs")
Closes: https://lore.kernel.org/r/20240317191358.97578-1-sj@kernel.org
Cc: <stable@vger.kernel.org> # 6.3.x
Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 69bc8fabf378..3ce3f0aaa1d5 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -434,7 +434,7 @@ pages of all memory cgroups except ``/having_care_already``.::
     # # further filter out all cgroups except one at '/having_care_already'
     echo memcg > 1/type
     echo /having_care_already > 1/memcg_path
-    echo N > 1/matching
+    echo Y > 1/matching
 
 Note that ``anon`` and ``memcg`` filters are currently supported only when
 ``paddr`` :ref:`implementation <sysfs_context>` is being used.
-- 
2.39.2

[PATCH 07/10] Docs/admin-guide/mm/damon/usage: fix wrong schemes effective quota update command

[SeongJae Park]

To update effective size quota of DAMOS schemes on DAMON sysfs file
interface, user should write 'update_schemes_effective_quotas' to the
kdamond 'state' file.  But the document is mistakenly saying the input
string as 'update_schemes_effective_bytes'.  Fix it (s/bytes/quotas/).

Fixes: a6068d6dfa2f ("Docs/admin-guide/mm/damon/usage: document effective_bytes file")
Cc: <stable@vger.kernel.org> # 6.9.x
Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 3ce3f0aaa1d5..e58ceb89ea2a 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -153,7 +153,7 @@ Users can write below commands for the kdamond to the ``state`` file.
 - ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
   action tried regions directory for each DAMON-based operation scheme of the
   kdamond.
-- ``update_schemes_effective_bytes``: Update the contents of
+- ``update_schemes_effective_quotas``: Update the contents of
   ``effective_bytes`` files for each DAMON-based operation scheme of the
   kdamond.  For more details, refer to :ref:`quotas directory <sysfs_quotas>`.
 
@@ -342,7 +342,7 @@ Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the
 effective size quota is further adjusted.  Reading ``effective_bytes`` returns
 the current effective size quota.  The file is not updated in real time, so
 users should ask DAMON sysfs interface to update the content of the file for
-the stats by writing a special keyword, ``update_schemes_effective_bytes`` to
+the stats by writing a special keyword, ``update_schemes_effective_quotas`` to
 the relevant ``kdamonds/<N>/state`` file.
 
 Under ``weights`` directory, three files (``sz_permil``,
-- 
2.39.2

[PATCH 08/10] Docs/mm/damon/design: use a list for supported filters

[SeongJae Park]

Filters section is listing currently supported filter types in a normal
paragraph.  Since the number of types are higher than four, it is not
easy to read for only specific types.  Use a list for easier finding of
specific types.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 46 +++++++++++++++++--------------
 1 file changed, 26 insertions(+), 20 deletions(-)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index f2baf617184d..1873755358af 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -461,26 +461,32 @@ number of filters for each scheme.  Each filter specifies the type of target
 memory, and whether it should exclude the memory of the type (filter-out), or
 all except the memory of the type (filter-in).
 
-Currently, anonymous page, memory cgroup, young page, address range, and DAMON
-monitoring target type filters are supported by the feature.  Some filter
-target types require additional arguments.  The memory cgroup filter type asks
-users to specify the file path of the memory cgroup for the filter.  The
-address range type asks the start and end addresses of the range.  The DAMON
-monitoring target type asks the index of the target from the context's
-monitoring targets list.  Hence, users can apply specific schemes to only
-anonymous pages, non-anonymous pages, pages of specific cgroups, all pages
-excluding those of specific cgroups, pages that not accessed after the last
-access check from the scheme, pages that accessed after the last access check
-from the scheme, pages in specific address range, pages in specific DAMON
-monitoring targets, and any combination of those.
-
-To handle filters efficiently, the address range and DAMON monitoring target
-type filters are handled by the core layer, while others are handled by
-operations set.  If a memory region is filtered by a core layer-handled filter,
-it is not counted as the scheme has tried to the region.  In contrast, if a
-memory regions is filtered by an operations set layer-handled filter, it is
-counted as the scheme has tried.  The difference in accounting leads to changes
-in the statistics.
+For efficient handling of filters, some types of filters are handled by the
+core layer, while others are handled by operations set.  In the latter case,
+hence, support of the filter types depends on the DAMON operations set.  In
+case of the core layer-handled filters, the memory regions that excluded by the
+filter are not counted as the scheme has tried to the region.  In contrast, if
+a memory regions is filtered by an operations set layer-handled filter, it is
+counted as the scheme has tried.  This difference affects the statistics.
+
+Below types of filters are currently supported.
+
+- anonymous page
+  - Applied to pages that containing data that not stored in files.
+  - Handled by operations set layer.  Supported by only ``paddr`` set.
+- memory cgroup
+  - Applied to pages that belonging to a given cgroup.
+  - Handled by operations set layer.  Supported by only ``paddr`` set.
+- young page
+  - Applied to pages that are accessed after the last access check from the
+    scheme.
+  - Handled by operations set layer.  Supported by only ``paddr`` set.
+- address range
+  - Applied to pages that belonging to a given address range.
+  - Handled by the core logic.
+- DAMON monitoring target
+  - Applied to pages that belonging to a given DAMON monitoring target.
+  - Handled by the core logic.
 
 
 Application Programming Interface
-- 
2.39.2

[PATCH 09/10] Docs/mm/damon/maintainer-profile: change the maintainer's timezone from PST to PT

[SeongJae Park]

The document says the maintainer is working on only PST.  The maintainer
respects daylight saving system, though.  Update the time zone to PT.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/maintainer-profile.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/mm/damon/maintainer-profile.rst b/Documentation/mm/damon/maintainer-profile.rst
index 5a306e4de22e..ea42f57cf9dc 100644
--- a/Documentation/mm/damon/maintainer-profile.rst
+++ b/Documentation/mm/damon/maintainer-profile.rst
@@ -48,9 +48,9 @@ Review cadence
 --------------
 
 The DAMON maintainer does the work on the usual work hour (09:00 to 17:00,
-Mon-Fri) in PST.  The response to patches will occasionally be slow.  Do not
-hesitate to send a ping if you have not heard back within a week of sending a
-patch.
+Mon-Fri) in PT (Pacific Time).  The response to patches will occasionally be
+slow.  Do not hesitate to send a ping if you have not heard back within a week
+of sending a patch.
 
 
 .. [1] https://git.kernel.org/akpm/mm/h/mm-unstable
-- 
2.39.2

[PATCH 10/10] Docs/mm/damon/maintainer-profile: allow posting patches based on damon/next tree

[SeongJae Park]

The document mentions any patches for review should based on mm-unstable
instead of damon/next.  It should be the recommended process, but
sometimes patches based on damon/next could be posted for some reasons.
Actually, the DAMON-based tiered memory management patchset[1] was
written on top of 'young page' DAMOS filter patchset, which was in
damon/next tree as of the writing.

Allow such case and just ask such things to be clearly specified.

[1] https://lore.kernel.org/20240405060858.2818-1-honggyu.kim@sk.com

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/maintainer-profile.rst | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/mm/damon/maintainer-profile.rst b/Documentation/mm/damon/maintainer-profile.rst
index ea42f57cf9dc..8213cf61d38a 100644
--- a/Documentation/mm/damon/maintainer-profile.rst
+++ b/Documentation/mm/damon/maintainer-profile.rst
@@ -20,9 +20,10 @@ management subsystem maintainer.  After more sufficient tests, the patches will
 be queued in mm-stable [3]_ , and finally pull-requested to the mainline by the
 memory management subsystem maintainer.
 
-Note again the patches for review should be made against the mm-unstable
-tree [1]_ whenever possible.  damon/next is only for preview of others' works
-in progress.
+Note again the patches for mm-unstable tree [1]_ are queued by the memory
+management subsystem maintainer.  If the patches requires some patches in
+damon/next tree [2]_ which not yet merged in mm-unstable, please make sure the
+requirement is clearly specified.
 
 Submit checklist addendum
 -------------------------
-- 
2.39.2

Re: [PATCH 00/10] mm/damon: misc fixes and improvements

[SeongJae Park]

Andrew, please add DAMON selftests patchset[1] that I posted yesterday before
this patchset.  Otherwise, patches would get conflicts.

[1] https://lore.kernel.org/20240502172718.74166-1-sj@kernel.org


Thanks,
SJ

On Fri,  3 May 2024 11:03:08 -0700 SeongJae Park <sj@kernel.org> wrote:

> Add miscelleneous and non-urgent fixes and improvements for DAMON code,
> selftests, and documents.
> 
> SeongJae Park (10):
>   mm/damon/core: initialize ->esz_bp from damos_quota_init_priv()
>   selftests/damon/_damon_sysfs: check errors from nr_schemes file reads
>   selftests/damon/_damon_sysfs: find sysfs mount point from /proc/mounts
>   selftests/damon/_damon_sysfs: use 'is' instead of '==' for 'None'
>   selftests/damon: classify tests for functionalities and regressions
>   Docs/admin-guide/mm/damon/usage: fix wrong example of DAMOS filter
>     matching sysfs file
>   Docs/admin-guide/mm/damon/usage: fix wrong schemes effective quota
>     update command
>   Docs/mm/damon/design: use a list for supported filters
>   Docs/mm/damon/maintainer-profile: change the maintainer's timezone
>     from PST to PT
>   Docs/mm/damon/maintainer-profile: allow posting patches based on
>     damon/next tree
> 
>  Documentation/admin-guide/mm/damon/usage.rst  |  6 +-
>  Documentation/mm/damon/design.rst             | 46 +++++----
>  Documentation/mm/damon/maintainer-profile.rst | 13 +--
>  mm/damon/core.c                               |  1 +
>  tools/testing/selftests/damon/Makefile        | 13 ++-
>  tools/testing/selftests/damon/_damon_sysfs.py | 95 +++++++++++--------
>  6 files changed, 100 insertions(+), 74 deletions(-)
> 
> 
> base-commit: fc7314cb6b750187a1366e0bf9da4c3ca8cfd064
> -- 
> 2.39.2