From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIM_ADSP_ALL,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B8D5C433E0 for ; Tue, 7 Jul 2020 14:50:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 453B420786 for ; Tue, 7 Jul 2020 14:50:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="eNtkr/tr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 453B420786 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DD7B28D001F; Tue, 7 Jul 2020 10:50:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DADAF8D001B; Tue, 7 Jul 2020 10:50:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC31F8D001F; Tue, 7 Jul 2020 10:50:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id B62248D001B for ; Tue, 7 Jul 2020 10:50:48 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6F1BE824805A for ; Tue, 7 Jul 2020 14:50:48 +0000 (UTC) X-FDA: 77011566576.03.hat11_3e1714326eb5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 3D3DF28A4E9 for ; Tue, 7 Jul 2020 14:50:48 +0000 (UTC) X-HE-Tag: hat11_3e1714326eb5 X-Filterd-Recvd-Size: 14421 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 7 Jul 2020 14:50:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1594133448; x=1625669448; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sPPawWKOCJ+KeiYfy+BMQZKBX+GVe2sPytsHFts75C8=; b=eNtkr/trVIobmxNefr5YJl1gsHaZXMv8xaD4bZ1/rXSvFRXi4N7DEJYO YQ+uHkWgKUbt5AEs1kgzpuP47xwbAnIPFjuH0C0XCLvvOloIq8+fsfFsb 4Y3snb+I34IiHk+qhxOuu7j/OTmuyR0nAvVjiXAXLBv1XTq71J9/lBJSt E=; IronPort-SDR: PSfc9SmJSsmmnz3c++8acCuxf9WSMe1W5waLdPvWIMD5zF2pvqDqvqSWi+tSHglAE/VSSm3SsN QuSLTxmIohrA== X-IronPort-AV: E=Sophos;i="5.75,324,1589241600"; d="scan'208";a="49757050" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-e7be2041.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 07 Jul 2020 14:50:47 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2a-e7be2041.us-west-2.amazon.com (Postfix) with ESMTPS id E34E6A1E3F; Tue, 7 Jul 2020 14:50:43 +0000 (UTC) Received: from EX13D31EUA004.ant.amazon.com (10.43.165.161) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 7 Jul 2020 14:50:43 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.214) by EX13D31EUA004.ant.amazon.com (10.43.165.161) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 7 Jul 2020 14:50:25 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v5 11/11] Docs/damon: Document physical memory monitoring support Date: Tue, 7 Jul 2020 16:45:40 +0200 Message-ID: <20200707144540.21216-12-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200707144540.21216-1-sjpark@amazon.com> References: <20200707144540.21216-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.161.214] X-ClientProxiedBy: EX13D35UWC001.ant.amazon.com (10.43.162.197) To EX13D31EUA004.ant.amazon.com (10.43.165.161) X-Rspamd-Queue-Id: 3D3DF28A4E9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds description for the physical memory monitoring usage in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/faq.rst | 7 ++- Documentation/admin-guide/mm/damon/index.rst | 1 - .../admin-guide/mm/damon/mechanisms.rst | 29 +++++----- Documentation/admin-guide/mm/damon/plans.rst | 7 --- Documentation/admin-guide/mm/damon/usage.rst | 53 ++++++++++++++----- 5 files changed, 60 insertions(+), 37 deletions(-) delete mode 100644 Documentation/admin-guide/mm/damon/plans.rst diff --git a/Documentation/admin-guide/mm/damon/faq.rst b/Documentation/a= dmin-guide/mm/damon/faq.rst index f55d1d719999..ff630cf5fce1 100644 --- a/Documentation/admin-guide/mm/damon/faq.rst +++ b/Documentation/admin-guide/mm/damon/faq.rst @@ -44,10 +44,9 @@ constructions and actual access checks can be implemen= ted and configured on the DAMON core by the users. In this way, DAMON users can monitor any addre= ss space with any access check technique. =20 -Nonetheless, DAMON provides a vma tracking and PTE Accessed bit check ba= sed -implementation of the address space dependent functions for the virtual = memory -by default, for a reference and convenient use. In near future, we will= also -provide that for physical memory address space. +Nonetheless, DAMON provides vma/rmap tracking and PTE Accessed bit check= based +implementations of the address space dependent functions for the virtual= memory +and the physical memory by default, for a reference and convenient use. =20 =20 Can I simply monitor page granularity? diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation= /admin-guide/mm/damon/index.rst index c6e657f8e90c..6e36149053fa 100644 --- a/Documentation/admin-guide/mm/damon/index.rst +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -32,4 +32,3 @@ workloads and systems. faq mechanisms eval - plans diff --git a/Documentation/admin-guide/mm/damon/mechanisms.rst b/Document= ation/admin-guide/mm/damon/mechanisms.rst index 16066477bb2c..fb33d8d8a09c 100644 --- a/Documentation/admin-guide/mm/damon/mechanisms.rst +++ b/Documentation/admin-guide/mm/damon/mechanisms.rst @@ -25,9 +25,11 @@ files, and backing devices would be supportable. Also= , if some architectures or kernel module support special access check primitives for specific ad= dress space, those will be easily configurable. =20 -DAMON currently provides an implementation of the primitives for the vir= tual -address space. It uses VMA for the target address range identification = and PTE -Accessed bit for the access check. +DAMON currently provides an implementation of the primitives for the phy= sical +and virtual address spaces. The implementation for the physical address= space +ask users to manually set the monitoring target address ranges while the +implementation for the virtual address space uses VMA for the target add= ress +range identification. Both uses PTE Accessed bit for the access check. =20 Below four sections describe the address independent core mechanisms and= the five knobs for tuning, that is, ``sampling interval``, ``aggregation @@ -113,26 +115,29 @@ memory mapping changes and applies it to the abstra= cted target area only for each of a user-specified time interval (``regions update interval``). =20 =20 -Virtual Address Space Specific Low Primitives -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Address Space Specific Low Primitives +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -This is for the DAMON's reference implementation of the virtual memory a= ddress -specific low level primitive only. +This is for the DAMON's reference implementation of the address space sp= ecific +low level primitive only. =20 =20 PTE Accessed-bit Based Access Check ----------------------------------- =20 -The implementation uses PTE Accessed-bit for basic access checks. That = is, it -clears the bit for next sampling target page and checks whether it set a= gain -after one sampling period. To avoid disturbing other Accessed bit users= such -as the reclamation logic, this implementation adjusts the ``PG_Idle`` an= d -``PG_Young`` appropriately, as same to the 'Idle Page Tracking'. +Both of the implementations for physical and virtual address spaces use = PTE +Accessed-bit for basic access checks. That is, those clears the bit for= next +sampling target page and checks whether it set again after one sampling = period. +To avoid disturbing other Accessed bit users such as the reclamation log= ic, the +implementations adjust the ``PG_Idle`` and ``PG_Young`` appropriately, a= s same +to the 'Idle Page Tracking'. =20 =20 VMA-based Target Address Range Construction ------------------------------------------- =20 +This is for the virtual address space specific primitives implementation= . + Only small parts in the super-huge virtual address space of the processe= s are mapped to the physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with = some diff --git a/Documentation/admin-guide/mm/damon/plans.rst b/Documentation= /admin-guide/mm/damon/plans.rst deleted file mode 100644 index 765344f02eb3..000000000000 --- a/Documentation/admin-guide/mm/damon/plans.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0 - -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Future Plans -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -TBD. diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation= /admin-guide/mm/damon/usage.rst index 573fcb4c57a7..356281078d4d 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -10,15 +10,16 @@ DAMON provides below three interfaces for different u= sers. This is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the = DAMON=E2=80=99s major features in a human-friendly way. It may not be highly tuned fo= r - special cases, though. It supports virtual address space monitoring o= nly. + special cases, though. It supports both virtual and physical address = spaces + monitoring. - *debugfs interface.* This is for privileged user space programmers who want more optimized = use of DAMON. Using this, users can use DAMON=E2=80=99s major features by re= ading from and writing to special debugfs files. Therefore, you can write a= nd use your personalized DAMON debugfs wrapper programs that reads/writes the debugfs files instead of you. The DAMON user space tool is also a ref= erence - implementation of such programs. It supports virtual address space - monitoring only. + implementation of such programs. It supports both virtual and physica= l + address spaces monitoring. - *Kernel Space Programming Interface.* This is for kernel space programmers. Using this, users can utilize e= very feature of DAMON most flexibly and efficiently by writing kernel space @@ -48,9 +49,11 @@ Recording Data Access Pattern ----------------------------- =20 The ``record`` subcommand records the data access pattern of target work= loads -in a file (``./damon.data`` by default). You can specify the target as = either -process id of running target or a command for execution of it. Below ex= ample -shows a command target usage:: +in a file (``./damon.data`` by default). You can specify the target wit= h 1) +the command for execution of the monitoring target process, 2) pid of ru= nning +target process, or 3) the special keyword, 'paddr', if you want to monit= or the +system's physical memory address space. Below example shows a command t= arget +usage:: =20 # cd /tools/damon/ # damo record "sleep 5" @@ -61,6 +64,15 @@ of the process. Below example shows a pid target usag= e:: # sleep 5 & # damo record `pidof sleep` =20 +Finally, below example shows the use of the special keyword, 'paddr':: + + # damo record paddr + +In this case, the monitoring target regions defaults to the largetst 'Sy= stem +RAM' region specified in '/proc/iomem' file. Note that the initial moni= toring +target region is maintained rather than dynamically updated like the vir= tual +memory address spaces monitoring case. + The location of the recorded file can be explicitly set using ``-o`` opt= ion. You can further tune this by setting the monitoring attributes. To know= about the monitoring attributes in detail, please refer to :doc:`mechanisms`. @@ -303,15 +315,25 @@ check it again:: Target PIDs ----------- =20 -Users can get and set the pids of monitoring target processes by reading= from -and writing to the ``pids`` file. For example, below commands set proce= sses -having pids 42 and 4242 as the processes to be monitored and check it ag= ain:: +To monitor the virtual memory address spaces of specific processes, user= s can +get and set the pids of monitoring target processes by reading from and = writing +to the ``pids`` file. For example, below commands set processes having = pids 42 +and 4242 as the processes to be monitored and check it again:: =20 # cd /damon # echo 42 4242 > pids # cat pids 42 4242 =20 +Users can also monitor the physical memory address space of the system b= y +writing a special keyword, "``paddr\n``" to the file. In this case, rea= ding the +file will show ``-1``, as below:: + + # cd /damon + # echo paddr > pids + # cat pids + -1 + Note that setting the pids doesn't start the monitoring. =20 =20 @@ -326,6 +348,10 @@ file-mapped area. Or, some users might know the init= ial access pattern of their workloads and therefore want to set optimal initial regions for the 'ada= ptive regions adjustment'. =20 +In contrast, DAMON do not automatically sets and updates the monitoring = target +regions in case of physical memory monitoring. Therefore, users should = set the +monitoring target regions by themselves. + In such cases, users can explicitly set the initial monitoring target re= gions as they want, by writing proper values to the ``init_regions`` file. Ea= ch line of the input should represent one region in below form.:: @@ -344,10 +370,11 @@ region of process 42, and another couple of address= ranges, ``20-40`` and 4242 20 40 4242 50 100" > init_regions =20 -Note that this sets the initial monitoring target regions only. DAMON w= ill -automatically updates the boundary of the regions after one ``regions up= date -interval``. Therefore, users should set the ``regions update interval``= large -enough. +Note that this sets the initial monitoring target regions only. In case= of +virtual memory monitoring, DAMON will automatically updates the boundary= of the +regions after one ``regions update interval``. Therefore, users should = set the +``regions update interval`` large enough in this case, if they don't wan= t the +update. =20 =20 Record --=20 2.17.1