From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96BC8C433E0 for ; Thu, 24 Dec 2020 07:02:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 093E0227BF for ; Thu, 24 Dec 2020 07:02:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 093E0227BF Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 06B158D0072; Thu, 24 Dec 2020 02:02:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F37B08D0063; Thu, 24 Dec 2020 02:02:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E265C8D0072; Thu, 24 Dec 2020 02:02:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id CA2D38D0063 for ; Thu, 24 Dec 2020 02:02:43 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 86679180AD82F for ; Thu, 24 Dec 2020 07:02:43 +0000 (UTC) X-FDA: 77627283006.06.road44_2113aa52746f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 636041004C9DA for ; Thu, 24 Dec 2020 07:02:43 +0000 (UTC) X-HE-Tag: road44_2113aa52746f X-Filterd-Recvd-Size: 6297 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Thu, 24 Dec 2020 07:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1608793363; x=1640329363; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=oUJoo3u6FSyIrW2Sz/3hj3BeSsKUKhSGefxN2L8Gdj8=; b=VruMRaX7/d+yfnM+M5AZvQckdkoQxbbAfR4rjw8sQXdGV9wuQ3yVU95c oIA5VB769ifacPMFfQQrhw48ilU1Ltusj8U+bwQajUc3ChIBpjPdRyOAb +4z+vppd5yi4VT6/CLD7dD+upRdokLVA/tJnfmNiWAslpzAP+Umxevxgi o=; X-IronPort-AV: E=Sophos;i="5.78,444,1599523200"; d="scan'208";a="98775651" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-22cc717f.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 24 Dec 2020 07:02:35 +0000 Received: from EX13D31EUA001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2a-22cc717f.us-west-2.amazon.com (Postfix) with ESMTPS id A6CDCA1E7D; Thu, 24 Dec 2020 07:02:32 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.161.193) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 24 Dec 2020 07:02:16 +0000 From: SeongJae Park To: Shakeel Butt CC: SeongJae Park , , "Andrea Arcangeli" , , , , , , Brendan Higgins , Qian Cai , Colin Ian King , Jonathan Corbet , "David Hildenbrand" , , Marco Elver , "Du, Fan" , , "Greg Thelen" , Ian Rogers , , "Kirill A. Shutemov" , Mark Rutland , Mel Gorman , Minchan Kim , Ingo Molnar , , "Peter Zijlstra (Intel)" , Randy Dunlap , Rik van Riel , David Rientjes , Steven Rostedt , Mike Rapoport , , Shuah Khan , , , Vlastimil Babka , Vladimir Davydov , Yang Shi , Huang Ying , , , Linux MM , , LKML Subject: Re: [PATCH v23 01/15] mm: Introduce Data Access MONitor (DAMON) Date: Thu, 24 Dec 2020 08:02:01 +0100 Message-ID: <20201224070201.9607-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.43.161.193] X-ClientProxiedBy: EX13D43UWC004.ant.amazon.com (10.43.162.42) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 23 Dec 2020 14:49:57 -0800 Shakeel Butt wrote: > On Wed, Dec 23, 2020 at 8:34 AM SeongJae Park wrote: > [snip] > > > Overall the patch looks good to me. Two concerns I have are if we > > > should damon_callback here or with the real user and the regions part > > > of primitive abstraction. For the first one, I don't have any strong > > > opinion but for the second one I do. > > > > I'd like to keep 'damon_callback' part here, to let API users know how the > > monitoring result will be available to them. > > > > For the 'regions' part, I will rename relevant things as below in the next > > version, to reduce any confusion. > > > > init_target_regions() -> init() > > update_target_regions() -> update() > > regions_update_interval -> update_interval > > last_regions_update -> last_update > > > > > > > > More specifically the question is if sampling and adaptive region > > > adjustment are general enough to be part of core monitoring context? > > > Can you give an example of a different primitive/use-case where these > > > would be beneficial. > > > > I think all adress spaces having some spatial locality and monitoring requests > > that need to have upper-bound overhead and best-effort accuracy could get > > benefit from it. The primitives targetting 'virtual address spaces' and the > > 'physical address space' clearly showed the benefit. > > I am still not much convinced on the 'physical address space' use-case > or the way you are presenting it. I understand the concern. I also once thought the mechanism might not work well for the physical address space because we cannot expect much spatial locality in the space. However, it turned out that there is some (temporal) spatial locality that enough to make DAMON work reasonably well. The word, 'reasonably well' might be controversial. With the mechanism, DAMON provides only 'best-effort' accuracy, rather than 100% accuracy. Our goal is to make the information accurate enough only for DRAM-centric optimizations. I'd like to also note that there are knobs that you can use to make minimum quality higher (nr_min_regions) while setting the upperbound of the monitoring overhead (nr_max_regions). What I can say for now is that we ran DAMON for physical address space of our production systems (shared detail in the 'Real-workd User Story' section of coverletter[1]) and the result was reasonable enough to convince the owner of the systems. [1] https://lore.kernel.org/linux-mm/20201215115448.25633-1-sjpark@amazon.com/ > Anyways I think we start with what you have and if in future there is a > use-case where regions adjustment does not make sense, we can change it then. 100% agreed, and thank you for understanding my argument. Thanks, SeongJae Park