From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_ADSP_ALL, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C265C4361B for ; Wed, 16 Dec 2020 09:51:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AD1322313B for ; Wed, 16 Dec 2020 09:51:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD1322313B Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1F0D76B0073; Wed, 16 Dec 2020 04:51:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A1206B0074; Wed, 16 Dec 2020 04:51:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0920B8D0002; Wed, 16 Dec 2020 04:51:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0036.hostedemail.com [216.40.44.36]) by kanga.kvack.org (Postfix) with ESMTP id E72CC6B0073 for ; Wed, 16 Dec 2020 04:51:53 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B2E4B19B3B for ; Wed, 16 Dec 2020 09:51:53 +0000 (UTC) X-FDA: 77598678906.03.root67_5004e6e2742b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 959A528A4E8 for ; Wed, 16 Dec 2020 09:51:53 +0000 (UTC) X-HE-Tag: root67_5004e6e2742b X-Filterd-Recvd-Size: 12226 Received: from smtp-fw-9103.amazon.com (smtp-fw-9103.amazon.com [207.171.188.200]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Dec 2020 09:51:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1608112312; x=1639648312; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qGKT1F1qVkIMFajZCvJm8BTClvxWeUPf3j13t8snc08=; b=S9mfGPH6TlpAbKZhPCDQXIYkTaXujw2YuXVw/YQ8SdSoW8SMsdmjIQjj qG0NmEpIbKmz7Z/OZ8EUxO1u+rKOW7msQeZeFUeUSSiyCDPXr5HOU6ZgQ uLV4dpREju0NJ/VIw/3WgAf67K4hAuUa9b36ybjgr19fU0asSxk1S330V s=; X-IronPort-AV: E=Sophos;i="5.78,424,1599523200"; d="scan'208";a="903547195" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2c-87a10be6.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9103.sea19.amazon.com with ESMTP; 16 Dec 2020 09:51:51 +0000 Received: from EX13D31EUA001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-87a10be6.us-west-2.amazon.com (Postfix) with ESMTPS id EB466A0262; Wed, 16 Dec 2020 09:47:19 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.161.31) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 16 Dec 2020 09:47:02 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v10 11/13] Docs/DAMON: Document physical memory monitoring support Date: Wed, 16 Dec 2020 10:42:19 +0100 Message-ID: <20201216094221.11898-12-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201216094221.11898-1-sjpark@amazon.com> References: <20201216094221.11898-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.161.31] X-ClientProxiedBy: EX13D16UWB001.ant.amazon.com (10.43.161.17) To EX13D31EUA001.ant.amazon.com (10.43.165.15) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON documents for the physical memory monitoring support. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 42 ++++++++++++++++---- Documentation/vm/damon/design.rst | 29 +++++++++----- Documentation/vm/damon/faq.rst | 5 +-- 3 files changed, 54 insertions(+), 22 deletions(-) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation= /admin-guide/mm/damon/usage.rst index cf0d44ce0ac9..3e2f1519c96a 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -10,15 +10,16 @@ DAMON provides below three interfaces for different u= sers. This is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the = DAMON=E2=80=99s major features in a human-friendly way. It may not be highly tuned fo= r - special cases, though. It supports only virtual address spaces monito= ring. + special cases, though. It supports both virtual and physical address = spaces + monitoring. - *debugfs interface.* This is for privileged user space programmers who want more optimized = use of DAMON. Using this, users can use DAMON=E2=80=99s major features by re= ading from and writing to special debugfs files. Therefore, you can write a= nd use your personalized DAMON debugfs wrapper programs that reads/writes the debugfs files instead of you. The DAMON user space tool is also a ref= erence - implementation of such programs. It supports only virtual address spa= ces - monitoring. + implementation of such programs. It supports both virtual and physica= l + address spaces monitoring. - *Kernel Space Programming Interface.* This is for kernel space programmers. Using this, users can utilize e= very feature of DAMON most flexibly and efficiently by writing kernel space @@ -49,8 +50,10 @@ Recording Data Access Pattern =20 The ``record`` subcommand records the data access pattern of target work= loads in a file (``./damon.data`` by default). You can specify the target wit= h 1) -the command for execution of the monitoring target process, or 2) pid of -running target process. Below example shows a command target usage:: +the command for execution of the monitoring target process, 2) pid of ru= nning +target process, or 3) the special keyword, 'paddr', if you want to monit= or the +system's physical memory address space. Below example shows a command t= arget +usage:: =20 # cd /tools/damon/ # damo record "sleep 5" @@ -61,6 +64,15 @@ of the process. Below example shows a pid target usag= e:: # sleep 5 & # damo record `pidof sleep` =20 +Finally, below example shows the use of the special keyword, 'paddr':: + + # damo record paddr + +In this case, the monitoring target regions defaults to the largetst 'Sy= stem +RAM' region specified in '/proc/iomem' file. Note that the initial moni= toring +target region is maintained rather than dynamically updated like the vir= tual +memory address spaces monitoring case. + The location of the recorded file can be explicitly set using ``-o`` opt= ion. You can further tune this by setting the monitoring attributes. To know= about the monitoring attributes in detail, please refer to the @@ -319,20 +331,34 @@ check it again:: # cat target_ids 42 4242 =20 +Users can also monitor the physical memory address space of the system b= y +writing a special keyword, "``paddr\n``" to the file. Because physical = address +space monitoring doesn't support multiple targets, reading the file will= show a +fake value, ``42``, as below:: + + # cd /damon + # echo paddr > target_ids + # cat target_ids + 42 + Note that setting the target ids doesn't start the monitoring. =20 =20 Initial Monitoring Target Regions --------------------------------- =20 -In case of the debugfs based monitoring, DAMON automatically sets and up= dates -the monitoring target regions so that entire memory mappings of target -processes can be covered. However, users might want to limit the monitor= ing +In case of the virtual address space monitoring, DAMON automatically set= s and +updates the monitoring target regions so that entire memory mappings of = target +processes can be covered. However, users might want to limit the monito= ring region to specific address ranges, such as the heap, the stack, or speci= fic file-mapped area. Or, some users might know the initial access pattern = of their workloads and therefore want to set optimal initial regions for th= e 'adaptive regions adjustment'. =20 +In contrast, DAMON do not automatically sets and updates the monitoring = target +regions in case of physical memory monitoring. Therefore, users should = set the +monitoring target regions by themselves. + In such cases, users can explicitly set the initial monitoring target re= gions as they want, by writing proper values to the ``init_regions`` file. Ea= ch line of the input should represent one region in below form.:: diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/d= esign.rst index 727d72093f8f..0666e19018fd 100644 --- a/Documentation/vm/damon/design.rst +++ b/Documentation/vm/damon/design.rst @@ -35,27 +35,34 @@ two parts: 1. Identification of the monitoring target address range for the address= space. 2. Access check of specific address range in the target space. =20 -DAMON currently provides the implementation of the primitives for only t= he -virtual address spaces. Below two subsections describe how it works. +DAMON currently provides the implementations of the primitives for the p= hysical +and virtual address spaces. Below two subsections describe how those wor= k. =20 =20 PTE Accessed-bit Based Access Check ----------------------------------- =20 -The implementation for the virtual address space uses PTE Accessed-bit f= or -basic access checks. It finds the relevant PTE Accessed bit from the ad= dress -by walking the page table for the target task of the address. In this w= ay, the -implementation finds and clears the bit for next sampling target address= and -checks whether the bit set again after one sampling period. This could = disturb -other kernel subsystems using the Accessed bits, namely Idle page tracki= ng and -the reclaim logic. To avoid such disturbances, DAMON makes it mutually -exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` = page -flags to solve the conflict with the reclaim logic, as Idle page trackin= g does. +Both of the implementations for physical and virtual address spaces use = PTE +Accessed-bit for basic access checks. Only one difference is the way of +finding the relevant PTE Accessed bit(s) from the address. While the +implementation for the virtual address walks the page table for the targ= et task +of the address, the implementation for the physical address walks every = page +table having a mapping to the address. In this way, the implementations= find +and clear the bit(s) for next sampling target address and checks whether= the +bit(s) set again after one sampling period. This could disturb other ke= rnel +subsystems using the Accessed bits, namely Idle page tracking and the re= claim +logic. To avoid such disturbances, DAMON makes it mutually exclusive wi= th Idle +page tracking and uses ``PG_idle`` and ``PG_young`` page flags to solve = the +conflict with the reclaim logic, as Idle page tracking does. =20 =20 VMA-based Target Address Range Construction ------------------------------------------- =20 +This is only for the virtual address space primitives implementation. T= hat for +the physical address space simply asks users to manually set the monitor= ing +target address ranges. + Only small parts in the super-huge virtual address space of the processe= s are mapped to the physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with = some diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.= rst index 088128bbf22b..6469d54c480f 100644 --- a/Documentation/vm/damon/faq.rst +++ b/Documentation/vm/damon/faq.rst @@ -43,10 +43,9 @@ constructions and actual access checks can be implemen= ted and configured on the DAMON core by the users. In this way, DAMON users can monitor any addre= ss space with any access check technique. =20 -Nonetheless, DAMON provides vma tracking and PTE Accessed bit check base= d +Nonetheless, DAMON provides vma/rmap tracking and PTE Accessed bit check= based implementations of the address space dependent functions for the virtual= memory -by default, for a reference and convenient use. In near future, we will -provide those for physical memory address space. +and the physical memory by default, for a reference and convenient use. =20 =20 Can I simply monitor page granularity? --=20 2.17.1