From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C37AAC433E1 for ; Thu, 20 Aug 2020 07:11:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 60AB52078B for ; Thu, 20 Aug 2020 07:11:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="OEy3Y8pO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60AB52078B Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 772C88D0005; Thu, 20 Aug 2020 03:11:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 723A68D0003; Thu, 20 Aug 2020 03:11:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 613678D0005; Thu, 20 Aug 2020 03:11:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0240.hostedemail.com [216.40.44.240]) by kanga.kvack.org (Postfix) with ESMTP id 480848D0003 for ; Thu, 20 Aug 2020 03:11:40 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 07BED8248047 for ; Thu, 20 Aug 2020 07:11:40 +0000 (UTC) X-FDA: 77170076760.08.offer37_2d074fd2702e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id CC56B1819E63B for ; Thu, 20 Aug 2020 07:11:39 +0000 (UTC) X-HE-Tag: offer37_2d074fd2702e X-Filterd-Recvd-Size: 9055 Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Thu, 20 Aug 2020 07:11:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1597907500; x=1629443500; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=8eYpAR681chumFtUVHT4O5U5Ye5aFsGlvA09RlvhVzE=; b=OEy3Y8pOQNmgHiVt0c7lxS4lKb6RdHDDaxNDS05KzwldPJclTo0YcyeL xLLppxUW2P9GO+L2nnBEWRXiYeAMRKytg2fTnT8v4yZYiWQJCcun9CrAF 9iKjgJKpavYCdyIX/dfHyI8gQWa8YDAaKtQrrKeoqTrtEZ4ZsGAJhvrmW I=; X-IronPort-AV: E=Sophos;i="5.76,332,1592870400"; d="scan'208";a="50288937" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6001.iad6.amazon.com with ESMTP; 20 Aug 2020 07:11:38 +0000 Received: from EX13MTAUEA001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com (Postfix) with ESMTPS id 562B9A20E8; Thu, 20 Aug 2020 07:11:26 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 Aug 2020 07:11:25 +0000 Received: from u3f2cd687b01c55.ant.amazon.com (10.43.162.38) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 Aug 2020 07:11:09 +0000 From: SeongJae Park To: Shakeel Butt CC: SeongJae Park , SeongJae Park , , Andrea Arcangeli , , , , , , Brendan Higgins , Qian Cai , Colin Ian King , Jonathan Corbet , "David Hildenbrand" , , "Du, Fan" , , Greg Thelen , Ian Rogers , , "Kirill A. Shutemov" , , Mel Gorman , Minchan Kim , Ingo Molnar , , "Peter Zijlstra (Intel)" , Randy Dunlap , Rik van Riel , David Rientjes , Steven Rostedt , , , , , , Vlastimil Babka , Vladimir Davydov , Yang Shi , Huang Ying , , , Linux MM , , LKML Subject: Re: [RFC v7 00/10] DAMON: Support Physical Memory Address Space Monitoring Date: Thu, 20 Aug 2020 09:10:52 +0200 Message-ID: <20200820071052.24271-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.43.162.38] X-ClientProxiedBy: EX13D17UWB003.ant.amazon.com (10.43.161.42) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Rspamd-Queue-Id: CC56B1819E63B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 19 Aug 2020 18:21:44 -0700 Shakeel Butt wrote: > On Tue, Aug 18, 2020 at 12:25 AM SeongJae Park wrote: > > > > From: SeongJae Park > > > > Changes from Previous Version > > ============================= > > > > - Use 42 as the fake target id for paddr instead of -1 > > - Fix a typo > > > > Introduction > > ============ > > > > DAMON[1] programming interface users can extend DAMON for any address space by > > configuring the address-space specific low level primitives with appropriate > > ones including their own implementations. However, because the implementation > > for the virtual address space is only available now, the users should implement > > their own for other address spaces. Worse yet, the user space users who rely > > on the debugfs interface and user space tool, cannot implement their own. > > > > This patchset implements another reference implementation of the low level > > primitives for the physical memory address space. With this change, hence, the > > kernel space users can monitor both the virtual and the physical address spaces > > by simply changing the configuration in the runtime. Further, this patchset > > links the implementation to the debugfs interface and the user space tool for > > the user space users. > > > > Note that the implementation supports only the user memory, as same to the idle > > page access tracking feature. > > > > [1] https://lore.kernel.org/linux-mm/20200706115322.29598-1-sjpark@amazon.com/ > > > > I am still struggling to find the benefit of this feature the way it > is implemented i.e. region based physical address space monitoring. > What exactly am I supposed to do for a given hot (or cold) physical > region? In a containerized world, that region can contain pages from > any cgroup. I can not really do anything about the accesses PHY-DAMON > provides me for a region. Technically speaking, this patchset introduces an implementation of DAMON's low level primitives for physical address space of LRU-listed pages. In other words, it is not designed for cgroups case. Also, please note that this patchset is only RFC, because it aims to only show the future plan of DAMON and get opinions about the concept before being serious. It will be serious only after the DAMON patchset is merged. Maybe I didn' made this point clear in the CV, sorry. I will state this clearly in the next spin. However, owing to the flexible design of DAMON, you can still use DAMON for cgroups case, though you need to make some efforts. There could be a number of ways. First, you could figure out the physical address regions for the target cgroups by yourself, set the target regions by yourself in 'damon_ctx' object and pass it to 'damon_start()'. For dynamic page allocations, you could check if a monitored region belongs to your target cgroup or not from your action making code, which could be implemented in the '->sample_cb()' or '->aggregate_cb()' callbacks. Note that you can even update the regions inside the callbacks. That is, you can remove regions containing pages of other containers, add new pages allocated for your target containers, adjust regions having pages of both other container and your target containers to represent only your target container's. Second, you could expand DAMON for cgroups by implementing your own low level primitives. You could also reuse some of the current implementation. For example, you could implement only '->init_target_regions' and '->update_target_regions' callbacks again so that only the pages of your target cgroup belongs in the target regions. However, if you need to monitor non-LRU-listed pages, you should implement '->prepare_access_checks()' and '->check_accesses()' callbacks. > > Now if you give me per-page information that would be useful as I can > at least get per-cgroup accesses (idle or re-use data) but that would > be as costly as Page Idle Tracking. So, seems you are saying about the 'adaptive regions adjustment' disabled page granularity monitoring case. Indeed. Same information comes with same overhead. Moreover, in the page granularity monitoring case, DAMON will make more space overhead (at least 8 bytes per page), because DAMON will represent each page as a physical address region having start address and end address, while Idle Pages Tracking can use only pfn. I'm planning optimizations for this page granularity case as a future work. However, if you don't strictly need page granularity accuracy, you could reduce the overhead by using larger granularity. That is, you can set the monitoring granularity as you want while the adaptive regions adjustment is disabled. You could even use variable granularity in this case using the callbacks mentioned above. So, DAMON is a framework rather than a tool. Though it comes with basic applications using DAMON as a framework (e.g., the virtual address space low primitives implementation, DAMON debugfs interface, and the DAMON user space tool) that could be useful in simple use cases, you need to code your application on it if your use cases are out of the simple cases. I will also develop more of such applications for more use-cases, but it will be only after the framework is complete enough to be merged in the mainline. Thanks, SeongJae Park