From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.0 required=3.0 tests=BAYES_00,DKIM_ADSP_ALL, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CF34C433E1 for ; Wed, 5 Aug 2020 07:04:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 14E9D22B45 for ; Wed, 5 Aug 2020 07:04:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="sUA6GMFQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14E9D22B45 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ADFAC6B0006; Wed, 5 Aug 2020 03:04:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB72B6B0007; Wed, 5 Aug 2020 03:04:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A51E6B000A; Wed, 5 Aug 2020 03:04:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 468DA6B0006 for ; Wed, 5 Aug 2020 03:04:41 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B35BB3625 for ; Wed, 5 Aug 2020 07:04:40 +0000 (UTC) X-FDA: 77115627120.17.bit76_301729b26fad Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 48CDF180D0185 for ; Wed, 5 Aug 2020 07:04:40 +0000 (UTC) X-HE-Tag: bit76_301729b26fad X-Filterd-Recvd-Size: 12461 Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Wed, 5 Aug 2020 07:04:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1596611080; x=1628147080; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OZo+vTKyPigtYlE9tXWQERpNEX8zpQE9PZnr8q+Ltb4=; b=sUA6GMFQ0kIF3/U/qhI6dmFfXUy497pbBoSUzfCYWij/oTCXD9jQf6wj HLLEjau5COQHttpAHkLjSjdTaOGZAJiwLnl991P7wdzsnzo+yi9GZO7cT P/j/N8lJigvQ4F3TjknDw7jdxtyVWl56uexxSGLd0+hy4xO2mr33Lfq4e Q=; IronPort-SDR: cCGpUd6OuigIzbeDCIZdv0SZ/X1hxGWx6gKPQ7Vz5EYMYbeZ3Fb9/hrPtQyaivgBvl6A2GyGli L3sQsjnsqFvQ== X-IronPort-AV: E=Sophos;i="5.75,436,1589241600"; d="scan'208";a="47559040" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1e-27fb8269.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6001.iad6.amazon.com with ESMTP; 05 Aug 2020 07:04:39 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1e-27fb8269.us-east-1.amazon.com (Postfix) with ESMTPS id 7D2B3A056E; Wed, 5 Aug 2020 07:04:28 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 5 Aug 2020 07:04:27 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.160.26) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 5 Aug 2020 07:04:08 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v6 10/10] Docs/DAMON: Document physical memory monitoring support Date: Wed, 5 Aug 2020 08:59:51 +0200 Message-ID: <20200805065951.18221-11-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200805065951.18221-1-sjpark@amazon.com> References: <20200805065951.18221-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.160.26] X-ClientProxiedBy: EX13D24UWB002.ant.amazon.com (10.43.161.159) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Rspamd-Queue-Id: 48CDF180D0185 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON documents for the physical memory monitoring support. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 42 ++++++++++++++++---- Documentation/vm/damon/design.rst | 29 +++++++++----- Documentation/vm/damon/faq.rst | 5 +-- 3 files changed, 54 insertions(+), 22 deletions(-) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation= /admin-guide/mm/damon/usage.rst index cf0d44ce0ac9..88b8e9254a7e 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -10,15 +10,16 @@ DAMON provides below three interfaces for different u= sers. This is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the = DAMON=E2=80=99s major features in a human-friendly way. It may not be highly tuned fo= r - special cases, though. It supports only virtual address spaces monito= ring. + special cases, though. It supports both virtual and physical address = spaces + monitoring. - *debugfs interface.* This is for privileged user space programmers who want more optimized = use of DAMON. Using this, users can use DAMON=E2=80=99s major features by re= ading from and writing to special debugfs files. Therefore, you can write a= nd use your personalized DAMON debugfs wrapper programs that reads/writes the debugfs files instead of you. The DAMON user space tool is also a ref= erence - implementation of such programs. It supports only virtual address spa= ces - monitoring. + implementation of such programs. It supports both virtual and physica= l + address spaces monitoring. - *Kernel Space Programming Interface.* This is for kernel space programmers. Using this, users can utilize e= very feature of DAMON most flexibly and efficiently by writing kernel space @@ -49,8 +50,10 @@ Recording Data Access Pattern =20 The ``record`` subcommand records the data access pattern of target work= loads in a file (``./damon.data`` by default). You can specify the target wit= h 1) -the command for execution of the monitoring target process, or 2) pid of -running target process. Below example shows a command target usage:: +the command for execution of the monitoring target process, 2) pid of ru= nning +target process, or 3) the special keyword, 'paddr', if you want to monit= or the +system's physical memory address space. Below example shows a command t= arget +usage:: =20 # cd /tools/damon/ # damo record "sleep 5" @@ -61,6 +64,15 @@ of the process. Below example shows a pid target usag= e:: # sleep 5 & # damo record `pidof sleep` =20 +Finally, below example shows the use of the special keyword, 'paddr':: + + # damo record paddr + +In this case, the monitoring target regions defaults to the largetst 'Sy= stem +RAM' region specified in '/proc/iomem' file. Note that the initial moni= toring +target region is maintained rather than dynamically updated like the vir= tual +memory address spaces monitoring case. + The location of the recorded file can be explicitly set using ``-o`` opt= ion. You can further tune this by setting the monitoring attributes. To know= about the monitoring attributes in detail, please refer to the @@ -319,20 +331,34 @@ check it again:: # cat target_ids 42 4242 =20 +Users can also monitor the physical memory address space of the system b= y +writing a special keyword, "``paddr\n``" to the file. Because physical = address +space monitoring doesn't support multiple targets, reading the file will= show a +fake value, ``-1``, as below:: + + # cd /damon + # echo paddr > target_ids + # cat target_ids + -1 + Note that setting the target ids doesn't start the monitoring. =20 =20 Initial Monitoring Target Regions --------------------------------- =20 -In case of the debugfs based monitoring, DAMON automatically sets and up= dates -the monitoring target regions so that entire memory mappings of target -processes can be covered. However, users might want to limit the monitor= ing +In case of the virtual address space monitoring, DAMON automatically set= s and +updates the monitoring target regions so that entire memory mappings of = target +processes can be covered. However, users might want to limit the monito= ring region to specific address ranges, such as the heap, the stack, or speci= fic file-mapped area. Or, some users might know the initial access pattern = of their workloads and therefore want to set optimal initial regions for th= e 'adaptive regions adjustment'. =20 +In contrast, DAMON do not automatically sets and updates the monitoring = target +regions in case of physical memory monitoring. Therefore, users should = set the +monitoring target regions by themselves. + In such cases, users can explicitly set the initial monitoring target re= gions as they want, by writing proper values to the ``init_regions`` file. Ea= ch line of the input should represent one region in below form.:: diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/d= esign.rst index 727d72093f8f..0666e19018fd 100644 --- a/Documentation/vm/damon/design.rst +++ b/Documentation/vm/damon/design.rst @@ -35,27 +35,34 @@ two parts: 1. Identification of the monitoring target address range for the address= space. 2. Access check of specific address range in the target space. =20 -DAMON currently provides the implementation of the primitives for only t= he -virtual address spaces. Below two subsections describe how it works. +DAMON currently provides the implementations of the primitives for the p= hysical +and virtual address spaces. Below two subsections describe how those wor= k. =20 =20 PTE Accessed-bit Based Access Check ----------------------------------- =20 -The implementation for the virtual address space uses PTE Accessed-bit f= or -basic access checks. It finds the relevant PTE Accessed bit from the ad= dress -by walking the page table for the target task of the address. In this w= ay, the -implementation finds and clears the bit for next sampling target address= and -checks whether the bit set again after one sampling period. This could = disturb -other kernel subsystems using the Accessed bits, namely Idle page tracki= ng and -the reclaim logic. To avoid such disturbances, DAMON makes it mutually -exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` = page -flags to solve the conflict with the reclaim logic, as Idle page trackin= g does. +Both of the implementations for physical and virtual address spaces use = PTE +Accessed-bit for basic access checks. Only one difference is the way of +finding the relevant PTE Accessed bit(s) from the address. While the +implementation for the virtual address walks the page table for the targ= et task +of the address, the implementation for the physical address walks every = page +table having a mapping to the address. In this way, the implementations= find +and clear the bit(s) for next sampling target address and checks whether= the +bit(s) set again after one sampling period. This could disturb other ke= rnel +subsystems using the Accessed bits, namely Idle page tracking and the re= claim +logic. To avoid such disturbances, DAMON makes it mutually exclusive wi= th Idle +page tracking and uses ``PG_idle`` and ``PG_young`` page flags to solve = the +conflict with the reclaim logic, as Idle page tracking does. =20 =20 VMA-based Target Address Range Construction ------------------------------------------- =20 +This is only for the virtual address space primitives implementation. T= hat for +the physical address space simply asks users to manually set the monitor= ing +target address ranges. + Only small parts in the super-huge virtual address space of the processe= s are mapped to the physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with = some diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.= rst index 088128bbf22b..6469d54c480f 100644 --- a/Documentation/vm/damon/faq.rst +++ b/Documentation/vm/damon/faq.rst @@ -43,10 +43,9 @@ constructions and actual access checks can be implemen= ted and configured on the DAMON core by the users. In this way, DAMON users can monitor any addre= ss space with any access check technique. =20 -Nonetheless, DAMON provides vma tracking and PTE Accessed bit check base= d +Nonetheless, DAMON provides vma/rmap tracking and PTE Accessed bit check= based implementations of the address space dependent functions for the virtual= memory -by default, for a reference and convenient use. In near future, we will -provide those for physical memory address space. +and the physical memory by default, for a reference and convenient use. =20 =20 Can I simply monitor page granularity? --=20 2.17.1