linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests
@ 2023-12-02  0:08 SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 1/5] selftests/damon: add a DAMON interface wrapper python module SeongJae Park
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

DAMON provides almost all control to the user via its sysfs interface.
For that, the interface provides plenty of files and hierarchies.  The
interface is simple enough to be controlled by shell commands including
'cat', 'echo', and redirection.  However, due to the number of files and
the hierarchies, doing that repeatedly is quite tedious.  As a result,
DAMON selftests are containing only simple test cases rather than real
functionality tests.  Having a wrapper script that can be reused to
implement more functionality tests could be helpful.  Writing such
wrapper with shell script might be challenging and not easy to further
maintain and extend for future DAMON interface extensions, though.

To this end, implement a Python-written DAMON sysfs interface wrapper
that could be easily managed and extended for future DAMON interface
extensions.  Further implement one simple functionality test and a
corner case regression test for a previously found bug, using the
wrapper module.  In fact, the bug was found by the test this patchset is
introducing.

Note that the Python wrapper is not supporting full features of DAMON
interface, but only some of those that essential for the tests that this
patchset is introducing.  The wrapper would extended to support more
features, but only with essential ones for such future tests.  The
wrapper will hence keep being simple, small, and constrained.  For
convenient and general use cases of DAMON, users should use DAMON
user-space tools for such purpose, like damo[1].

[1] https://github.com/damonitor/damo

Patches Sequence
----------------

This patchset is constructed with five patches.  The first three patches
implement the Python-written DAMON sysfs interface wrapper in small
steps.  The basic data structure (first patch), kdamond startup command
(second patch), and finally DAMOS tried bytes command (third patch).

Then two patches for adding selftests using the wrapper follows.  The
fourth patch implements a basic functionality test of DAMON for working
set estimation accuracy.  Finally, the fifth patch implements a corner
case test for a previously found bug.

SeongJae Park (5):
  selftests/damon: add a DAMON interface wrapper python module
  selftests/damon/_damon: implement sysfs-based kdamonds start function
  selftests/damon/_damon: implement sysfs updat_schemes_tried_bytes
    command
  selftests/damon: add a test for update_schemes_tried_regions sysfs
    command
  selftests/damon: add a test for update_schemes_tried_regions hang bug

 tools/testing/selftests/damon/Makefile        |   3 +
 tools/testing/selftests/damon/_damon.py       | 322 ++++++++++++++++++
 tools/testing/selftests/damon/access_memory.c |  41 +++
 ...sysfs_update_schemes_tried_regions_hang.py |  33 ++
 ...te_schemes_tried_regions_wss_estimation.py |  48 +++
 5 files changed, 447 insertions(+)
 create mode 100644 tools/testing/selftests/damon/_damon.py
 create mode 100644 tools/testing/selftests/damon/access_memory.c
 create mode 100755 tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_hang.py
 create mode 100755 tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_wss_estimation.py


base-commit: 1be383c41197b82cfd51b2edc7ee515c0b786496
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [RFC PATCH 1/5] selftests/damon: add a DAMON interface wrapper python module
  2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
@ 2023-12-02  0:08 ` SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 2/5] selftests/damon/_damon: implement sysfs-based kdamonds start function SeongJae Park
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

DAMON sysfs interface supports almost every DAMON functionality.  For
that, it provides a number of files and hierarchies that not simple to
be manually manipulated, or writing shell script every time.  For the
reason, DAMON is not having good functionality selftests.  Adding an
existing DAMON user space tool and using it could also be a way, but it
would add unnecessarily huge change that not essential for the test
purpose.  Write a minimum python module that can further extended to be
used as an easy-to-use DAMON functionality test purpose DAMON user
interface wrapper.

Note that this commit is implementing only basic data structures for
representing the sysfs files.  More operations will be implemented by
following commits, as needed for specific real test cases.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon.py | 102 ++++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 tools/testing/selftests/damon/_damon.py

diff --git a/tools/testing/selftests/damon/_damon.py b/tools/testing/selftests/damon/_damon.py
new file mode 100644
index 000000000000..78101846ab66
--- /dev/null
+++ b/tools/testing/selftests/damon/_damon.py
@@ -0,0 +1,102 @@
+# SPDX-License-Identifier: GPL-2.0
+
+class DamosAccessPattern:
+    size = None
+    nr_accesses = None
+    age = None
+    scheme = None
+
+    def __init__(self, size=None, nr_accesses=None, age=None):
+        self.size = size
+        self.nr_accesses = nr_accesses
+        self.age = age
+
+        if self.size == None:
+            self.size = [0, 2**64 - 1]
+        if self.nr_accesses == None:
+            self.nr_accesses = [0, 2**64 - 1]
+        if self.age == None:
+            self.age = [0, 2**64 - 1]
+
+class Damos:
+    action = None
+    access_pattern = None
+    # todo: Support quotas, watermarks, stats, tried_regions
+    idx = None
+    context = None
+
+    def __init__(self, action='stat', access_pattern=DamosAccessPattern()):
+        self.action = action
+        self.access_pattern = access_pattern
+        self.access_pattern.scheme = self
+
+class DamonTarget:
+    pid = None
+    # todo: Support target regions if test is made
+    idx = None
+    context = None
+
+    def __init__(self, pid):
+        self.pid = pid
+
+class DamonAttrs:
+    sample_us = None
+    aggr_us = None
+    update_us = None
+    min_nr_regions = None
+    max_nr_regions = None
+    context = None
+
+    def __init__(self, sample_us=5000, aggr_us=100000, update_us=1000000,
+            min_nr_regions=10, max_nr_regions=1000):
+        self.sample_us = sample_us
+        self.aggr_us = aggr_us
+        self.update_us = update_us
+        self.min_nr_regions = min_nr_regions
+        self.max_nr_regions = max_nr_regions
+
+class DamonCtx:
+    ops = None
+    monitoring_attrs = None
+    targets = None
+    schemes = None
+    kdamond = None
+    idx = None
+
+    def __init__(self, ops='paddr', monitoring_attrs=DamonAttrs(), targets=[],
+            schemes=[]):
+        self.ops = ops
+        self.monitoring_attrs = monitoring_attrs
+        self.monitoring_attrs.context = self
+
+        self.targets = targets
+        for idx, target in enumerate(self.targets):
+            target.idx = idx
+            target.context = self
+
+        self.schemes = schemes
+        for idx, scheme in enumerate(self.schemes):
+            scheme.idx = idx
+            scheme.context = self
+
+class Kdamond:
+    state = None
+    pid = None
+    contexts = None
+    idx = None      # index of this kdamond between siblings
+    kdamonds = None # parent
+
+    def __init__(self, contexts=[]):
+        self.contexts = contexts
+        for idx, context in enumerate(self.contexts):
+            context.idx = idx
+            context.kdamond = self
+
+class Kdamonds:
+    kdamonds = []
+
+    def __init__(self, kdamonds=[]):
+        self.kdamonds = kdamonds
+        for idx, kdamond in enumerate(self.kdamonds):
+            kdamond.idx = idx
+            kdamond.kdamonds = self
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [RFC PATCH 2/5] selftests/damon/_damon: implement sysfs-based kdamonds start function
  2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 1/5] selftests/damon: add a DAMON interface wrapper python module SeongJae Park
@ 2023-12-02  0:08 ` SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 3/5] selftests/damon/_damon: implement sysfs updat_schemes_tried_bytes command SeongJae Park
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

Extend the test-writing-purpose DAMON sysfs wrapper Python module to
support turning DAMON on with basic parameter inputs.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon.py | 206 ++++++++++++++++++++++++
 1 file changed, 206 insertions(+)

diff --git a/tools/testing/selftests/damon/_damon.py b/tools/testing/selftests/damon/_damon.py
index 78101846ab66..6b99f87a5f1e 100644
--- a/tools/testing/selftests/damon/_damon.py
+++ b/tools/testing/selftests/damon/_damon.py
@@ -1,5 +1,28 @@
 # SPDX-License-Identifier: GPL-2.0
 
+import os
+
+sysfs_root = '/sys/kernel/mm/damon/admin'
+
+def write_file(path, string):
+    "Returns error string if failed, or None otherwise"
+    string = '%s' % string
+    try:
+        with open(path, 'w') as f:
+            f.write(string)
+    except Exception as e:
+        return '%s' % e
+    return None
+
+def read_file(path):
+    '''Returns the read content and error string.  The read content is None if
+    the reading failed'''
+    try:
+        with open(path, 'r') as f:
+            return f.read(), None
+    except Exception as e:
+        return None, '%s' % e
+
 class DamosAccessPattern:
     size = None
     nr_accesses = None
@@ -18,6 +41,35 @@ class DamosAccessPattern:
         if self.age == None:
             self.age = [0, 2**64 - 1]
 
+    def sysfs_dir(self):
+        return os.path.join(self.scheme.sysfs_dir(), 'access_pattern')
+
+    def stage(self):
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'sz', 'min'), self.size[0])
+        if err != None:
+            return err
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'sz', 'max'), self.size[1])
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'nr_accesses', 'min'),
+                self.nr_accesses[0])
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'nr_accesses', 'max'),
+                self.nr_accesses[1])
+        if err != None:
+            return err
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'age', 'min'), self.age[0])
+        if err != None:
+            return err
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'age', 'max'), self.age[1])
+        if err != None:
+            return err
+
 class Damos:
     action = None
     access_pattern = None
@@ -30,6 +82,39 @@ class Damos:
         self.access_pattern = access_pattern
         self.access_pattern.scheme = self
 
+    def sysfs_dir(self):
+        return os.path.join(
+                self.context.sysfs_dir(), 'schemes', '%d' % self.idx)
+
+    def stage(self):
+        err = write_file(os.path.join(self.sysfs_dir(), 'action'), self.action)
+        if err != None:
+            return err
+        err = self.access_pattern.stage()
+        if err != None:
+            return err
+
+        # disable quotas
+        err = write_file(os.path.join(self.sysfs_dir(), 'quotas', 'ms'), '0')
+        if err != None:
+            return err
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'quotas', 'bytes'), '0')
+        if err != None:
+            return err
+
+        # disable watermarks
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'watermarks', 'metric'), 'none')
+        if err != None:
+            return err
+
+        # disable filters
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'filters', 'nr_filters'), '0')
+        if err != None:
+            return err
+
 class DamonTarget:
     pid = None
     # todo: Support target regions if test is made
@@ -39,6 +124,18 @@ class DamonTarget:
     def __init__(self, pid):
         self.pid = pid
 
+    def sysfs_dir(self):
+        return os.path.join(
+                self.context.sysfs_dir(), 'targets', '%d' % self.idx)
+
+    def stage(self):
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'regions', 'nr_regions'), '0')
+        if err != None:
+            return err
+        return write_file(
+                os.path.join(self.sysfs_dir(), 'pid_target'), self.pid)
+
 class DamonAttrs:
     sample_us = None
     aggr_us = None
@@ -55,6 +152,40 @@ class DamonAttrs:
         self.min_nr_regions = min_nr_regions
         self.max_nr_regions = max_nr_regions
 
+    def interval_sysfs_dir(self):
+        return os.path.join(self.context.sysfs_dir(), 'monitoring_attrs',
+                'intervals')
+
+    def nr_regions_range_sysfs_dir(self):
+        return os.path.join(self.context.sysfs_dir(), 'monitoring_attrs',
+                'nr_regions')
+
+    def stage(self):
+        err = write_file(os.path.join(self.interval_sysfs_dir(), 'sample_us'),
+                self.sample_us)
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.interval_sysfs_dir(), 'aggr_us'),
+                self.aggr_us)
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.interval_sysfs_dir(), 'update_us'),
+                self.update_us)
+        if err != None:
+            return err
+
+        err = write_file(
+                os.path.join(self.nr_regions_range_sysfs_dir(), 'min'),
+                self.min_nr_regions)
+        if err != None:
+            return err
+
+        err = write_file(
+                os.path.join(self.nr_regions_range_sysfs_dir(), 'max'),
+                self.max_nr_regions)
+        if err != None:
+            return err
+
 class DamonCtx:
     ops = None
     monitoring_attrs = None
@@ -79,6 +210,46 @@ class DamonCtx:
             scheme.idx = idx
             scheme.context = self
 
+    def sysfs_dir(self):
+        return os.path.join(self.kdamond.sysfs_dir(), 'contexts',
+                '%d' % self.idx)
+
+    def stage(self):
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'operations'), self.ops)
+        if err != None:
+            return err
+        err = self.monitoring_attrs.stage()
+        if err != None:
+            return err
+
+        nr_targets_file = os.path.join(
+                self.sysfs_dir(), 'targets', 'nr_targets')
+        content, err = read_file(nr_targets_file)
+        if err != None:
+            return err
+        if int(content) != len(self.targets):
+            err = write_file(nr_targets_file, '%d' % len(self.targets))
+            if err != None:
+                return err
+        for target in self.targets:
+            err = target.stage()
+            if err != None:
+                return err
+
+        nr_schemes_file = os.path.join(
+                self.sysfs_dir(), 'schemes', 'nr_schemes')
+        content, err = read_file(nr_schemes_file)
+        if int(content) != len(self.schemes):
+            err = write_file(nr_schemes_file, '%d' % len(self.schemes))
+            if err != None:
+                return err
+        for scheme in self.schemes:
+            err = scheme.stage()
+            if err != None:
+                return err
+        return None
+
 class Kdamond:
     state = None
     pid = None
@@ -92,6 +263,27 @@ class Kdamond:
             context.idx = idx
             context.kdamond = self
 
+    def sysfs_dir(self):
+        return os.path.join(self.kdamonds.sysfs_dir(), '%d' % self.idx)
+
+    def start(self):
+        nr_contexts_file = os.path.join(self.sysfs_dir(),
+                'contexts', 'nr_contexts')
+        content, err = read_file(nr_contexts_file)
+        if err != None:
+            return err
+        if int(content) != len(self.contexts):
+            err = write_file(nr_contexts_file, '%d' % len(self.contexts))
+            if err != None:
+                return err
+
+        for context in self.contexts:
+            err = context.stage()
+            if err != None:
+                return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'state'), 'on')
+        return err
+
 class Kdamonds:
     kdamonds = []
 
@@ -100,3 +292,17 @@ class Kdamonds:
         for idx, kdamond in enumerate(self.kdamonds):
             kdamond.idx = idx
             kdamond.kdamonds = self
+
+    def sysfs_dir(self):
+        return os.path.join(sysfs_root, 'kdamonds')
+
+    def start(self):
+        err = write_file(os.path.join(self.sysfs_dir(),  'nr_kdamonds'),
+                '%s' % len(self.kdamonds))
+        if err != None:
+            return err
+        for kdamond in self.kdamonds:
+            err = kdamond.start()
+            if err != None:
+                return err
+        return None
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [RFC PATCH 3/5] selftests/damon/_damon: implement sysfs updat_schemes_tried_bytes command
  2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 1/5] selftests/damon: add a DAMON interface wrapper python module SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 2/5] selftests/damon/_damon: implement sysfs-based kdamonds start function SeongJae Park
@ 2023-12-02  0:08 ` SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 4/5] selftests/damon: add a test for update_schemes_tried_regions sysfs command SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 5/5] selftests/damon: add a test for update_schemes_tried_regions hang bug SeongJae Park
  4 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

Implement update_schemes_tried_bytes command in the test-writing-purpose
DAMON sysfs interface wrapper Python module.  It is not only making the
update, but also read the updated value from the sysfs interface and
store on the Kdamond python objects so that user of the module can get
it by reading the field of the object.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tools/testing/selftests/damon/_damon.py b/tools/testing/selftests/damon/_damon.py
index 6b99f87a5f1e..e98cf4b6a4b7 100644
--- a/tools/testing/selftests/damon/_damon.py
+++ b/tools/testing/selftests/damon/_damon.py
@@ -76,6 +76,7 @@ class Damos:
     # todo: Support quotas, watermarks, stats, tried_regions
     idx = None
     context = None
+    tried_bytes = None
 
     def __init__(self, action='stat', access_pattern=DamosAccessPattern()):
         self.action = action
@@ -284,6 +285,19 @@ class Kdamond:
         err = write_file(os.path.join(self.sysfs_dir(), 'state'), 'on')
         return err
 
+    def update_schemes_tried_bytes(self):
+        err = write_file(os.path.join(self.sysfs_dir(), 'state'),
+                'update_schemes_tried_bytes')
+        if err != None:
+            return err
+        for context in self.contexts:
+            for scheme in context.schemes:
+                content, err = read_file(os.path.join(scheme.sysfs_dir(),
+                    'tried_regions', 'total_bytes'))
+                if err != None:
+                    return err
+                scheme.tried_bytes = int(content)
+
 class Kdamonds:
     kdamonds = []
 
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [RFC PATCH 4/5] selftests/damon: add a test for update_schemes_tried_regions sysfs command
  2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
                   ` (2 preceding siblings ...)
  2023-12-02  0:08 ` [RFC PATCH 3/5] selftests/damon/_damon: implement sysfs updat_schemes_tried_bytes command SeongJae Park
@ 2023-12-02  0:08 ` SeongJae Park
  2023-12-02  0:08 ` [RFC PATCH 5/5] selftests/damon: add a test for update_schemes_tried_regions hang bug SeongJae Park
  4 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

Add a basic test for verifying accuracy of DAMON's access monitoring
functionality.  The test starts a program of artificial access pattern,
monitor the accesses using DAMON, and check if DAMON gets working set
size with reasonable amount of error rate.  Note that the acceptable
error rate is set with only a naive assumption.  Failure of this test
may not always mean DAMON is broken, and we could adjust it in future.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/Makefile        |  2 +
 tools/testing/selftests/damon/access_memory.c | 41 ++++++++++++++++
 ...te_schemes_tried_regions_wss_estimation.py | 48 +++++++++++++++++++
 3 files changed, 91 insertions(+)
 create mode 100644 tools/testing/selftests/damon/access_memory.c
 create mode 100755 tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_wss_estimation.py

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index d2105d41ea25..1363987709c6 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -4,6 +4,7 @@
 TEST_GEN_FILES += huge_count_read_write
 TEST_GEN_FILES += dbgfs_target_ids_read_before_terminate_race
 TEST_GEN_FILES += dbgfs_target_ids_pid_leak
+TEST_GEN_FILES += access_memory
 
 TEST_FILES = _chk_dependency.sh _debugfs_common.sh
 TEST_PROGS = debugfs_attrs.sh debugfs_schemes.sh debugfs_target_ids.sh
@@ -11,6 +12,7 @@ TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh
 TEST_PROGS += debugfs_duplicate_context_creation.sh
 TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
+TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
 TEST_PROGS += reclaim.sh lru_sort.sh
 TEST_PROGS += dbgfs_target_ids_read_before_terminate_race.sh
 TEST_PROGS += dbgfs_target_ids_pid_leak.sh
diff --git a/tools/testing/selftests/damon/access_memory.c b/tools/testing/selftests/damon/access_memory.c
new file mode 100644
index 000000000000..585a2fa54329
--- /dev/null
+++ b/tools/testing/selftests/damon/access_memory.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Artificial memory access program for testing DAMON.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+
+int main(int argc, char *argv[])
+{
+	char **regions;
+	clock_t start_clock;
+	int nr_regions;
+	int sz_region;
+	int access_time_ms;
+	int i;
+
+	if (argc != 4) {
+		printf("Usage: %s <number> <size (bytes)> <time (ms)>\n",
+				argv[0]);
+		return -1;
+	}
+
+	nr_regions = atoi(argv[1]);
+	sz_region = atoi(argv[2]);
+	access_time_ms = atoi(argv[3]);
+
+	regions = malloc(sizeof(*regions) * nr_regions);
+	for (i = 0; i < nr_regions; i++)
+		regions[i] = malloc(sz_region);
+
+	for (i = 0; i < nr_regions; i++) {
+		start_clock = clock();
+		while ((clock() - start_clock) * 1000 / CLOCKS_PER_SEC <
+				access_time_ms)
+			memset(regions[i], i, 1024 * 1024 * 10);
+	}
+	return 0;
+}
diff --git a/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_wss_estimation.py b/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_wss_estimation.py
new file mode 100755
index 000000000000..2aa66f4ad2dc
--- /dev/null
+++ b/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_wss_estimation.py
@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+import subprocess
+import time
+
+import _damon
+
+def main():
+    # access three 10 MiB memory regions, 1 second per each
+    sz_region = 10 * 1024 * 1024
+    proc = subprocess.Popen(['./access_memory', '3', '%d' % sz_region, '1000'])
+    kdamonds = _damon.Kdamonds([_damon.Kdamond(
+            contexts=[_damon.DamonCtx(
+                ops='vaddr',
+                targets=[_damon.DamonTarget(pid=proc.pid)],
+                schemes=[_damon.Damos(
+                    access_pattern=_damon.DamosAccessPattern(
+                        nr_accesses=[1, 20]))] # schemes
+                )] # contexts
+            )]) # kdamonds
+
+    err = kdamonds.start()
+    if err != None:
+        print('kdmaond start failed: %s' % err)
+        exit(1)
+
+    wss_collected = []
+    while proc.poll() == None:
+        time.sleep(0.1)
+        err = kdamonds.kdamonds[0].update_schemes_tried_bytes()
+        if err != None:
+            print('tried bytes update failed: %s' % err)
+            exit(1)
+
+        wss_collected.append(
+                kdamonds.kdamonds[0].contexts[0].schemes[0].tried_bytes)
+
+    wss_collected.sort()
+    for percentile in [25, 75]:
+        sample = wss_collected[int(len(wss_collected) * percentile / 100)]
+        error_rate = abs(sample - sz_region) / sz_region
+        print('%d-th percentile error %f' % (percentile, error_rate))
+        if error_rate > 0.5:
+            exit(1)
+
+if __name__ == '__main__':
+    main()
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [RFC PATCH 5/5] selftests/damon: add a test for update_schemes_tried_regions hang bug
  2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
                   ` (3 preceding siblings ...)
  2023-12-02  0:08 ` [RFC PATCH 4/5] selftests/damon: add a test for update_schemes_tried_regions sysfs command SeongJae Park
@ 2023-12-02  0:08 ` SeongJae Park
  4 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-02  0:08 UTC (permalink / raw)
  Cc: SeongJae Park, Andrew Morton, Shuah Khan, damon, linux-mm,
	linux-kselftest, linux-kernel

The update_schemees_tried_{regions,bytes} command was able to be
indefinitely hang in some corner cases.  It has fixed by introducing a
timeout for the command[1].  Add a test for the corner case to not
introduce the problem again.

[1] https://lore.kernel.org/damon/20231124213840.39157-1-sj@kernel.org/

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/Makefile        |  1 +
 ...sysfs_update_schemes_tried_regions_hang.py | 33 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100755 tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_hang.py

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 1363987709c6..d90bdba28ff4 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -12,6 +12,7 @@ TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh
 TEST_PROGS += debugfs_duplicate_context_creation.sh
 TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
+TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
 TEST_PROGS += reclaim.sh lru_sort.sh
 TEST_PROGS += dbgfs_target_ids_read_before_terminate_race.sh
diff --git a/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_hang.py b/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_hang.py
new file mode 100755
index 000000000000..14187be3b004
--- /dev/null
+++ b/tools/testing/selftests/damon/sysfs_update_schemes_tried_regions_hang.py
@@ -0,0 +1,33 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+import subprocess
+import time
+
+import _damon
+
+def main():
+    proc = subprocess.Popen(['sleep', '2'])
+    kdamonds = _damon.Kdamonds([_damon.Kdamond(
+            contexts=[_damon.DamonCtx(
+                ops='vaddr',
+                targets=[_damon.DamonTarget(pid=proc.pid)],
+                schemes=[_damon.Damos(
+                    access_pattern=_damon.DamosAccessPattern(
+                        nr_accesses=[200, 200]))] # schemes
+                )] # contexts
+            )]) # kdamonds
+
+    err = kdamonds.start()
+    if err != None:
+        print('kdmaond start failed: %s' % err)
+        exit(1)
+
+    while proc.poll() == None:
+        err = kdamonds.kdamonds[0].update_schemes_tried_bytes()
+        if err != None:
+            print('tried bytes update failed: %s' % err)
+            exit(1)
+
+if __name__ == '__main__':
+    main()
-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2023-12-02  0:08 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-02  0:08 [RFC PATCH 0/5] selftests/damon: add Python modules for DAMON control and functionality tests SeongJae Park
2023-12-02  0:08 ` [RFC PATCH 1/5] selftests/damon: add a DAMON interface wrapper python module SeongJae Park
2023-12-02  0:08 ` [RFC PATCH 2/5] selftests/damon/_damon: implement sysfs-based kdamonds start function SeongJae Park
2023-12-02  0:08 ` [RFC PATCH 3/5] selftests/damon/_damon: implement sysfs updat_schemes_tried_bytes command SeongJae Park
2023-12-02  0:08 ` [RFC PATCH 4/5] selftests/damon: add a test for update_schemes_tried_regions sysfs command SeongJae Park
2023-12-02  0:08 ` [RFC PATCH 5/5] selftests/damon: add a test for update_schemes_tried_regions hang bug SeongJae Park

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox