From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0704CC4338F for ; Wed, 4 Aug 2021 07:41:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 95BA660F35 for ; Wed, 4 Aug 2021 07:41:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 95BA660F35 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E09048D0038; Wed, 4 Aug 2021 03:41:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D92428D002D; Wed, 4 Aug 2021 03:41:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C32EC8D0038; Wed, 4 Aug 2021 03:41:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A26E58D002D for ; Wed, 4 Aug 2021 03:41:15 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5DEB822028 for ; Wed, 4 Aug 2021 07:41:15 +0000 (UTC) X-FDA: 78436602510.04.B3BF19F Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by imf08.hostedemail.com (Postfix) with ESMTP id EA5F8300ACB9 for ; Wed, 4 Aug 2021 07:41:14 +0000 (UTC) Received: by mail-qk1-f170.google.com with SMTP id c9so1898910qkc.13 for ; Wed, 04 Aug 2021 00:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to; bh=+YKN8zFIulxmNIlW3F0W91JQ1d2GvyA2/2s6T5fcLjA=; b=h+X3Tahl8foSVSHxt2y7HIFQH0IexcgUU3OGrFSOU1IilFE+BsDs4VTPwaNgbpKY5J dkIsSYDqDPp1pFXAr73L22WMYXdPCsPW4WLlK2iqAuphF6yLKem2lcOQEqXyhUA8oek9 xSQ7y8RgNiCiRgGT2JtByjR/BxMg78Y5btA0HDx7tqO8Xl9JWBDo5Y2y1VmVq4h1QuU5 RJTCkxAC9KJ6RHOwIO/V8Mmdc6N5mQPRPukjKuD3hAimpnqORR6zTGrmRYNlbQBI/3xf Fx/Blv67OzXMHDRLS6f556aV8UbnQRpqrQ1QLwIAs5O5eRk9teDmhHlixmU6CaTbJpQX Lrtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to; bh=+YKN8zFIulxmNIlW3F0W91JQ1d2GvyA2/2s6T5fcLjA=; b=cOCuQ0PLDlE+SvrLP21S/QYEgrbCkboqRf/8rlMHL8avuMxWjhQYaOEldOZyNRpVDW QgQ15vEulxgEE5CztAz0oc5PX6vfBsUYxm4IVeNfTsQvMCrAdvIn/8v+Oh1CKeIp62DE rIPeFwpdibuoPh8P+v8fZamg6jC/lLnbQekV61cYzQpx3mk8FjO+/9RK0VTf/tkzk0yL P3kn1ztcB/Iz1V8weIjaN9eWTxjeQt/t/QpY0h82WZWeRVv9Okq+p9d7YbxBQ3fSSZqI UKBRh3RbN1s8ys7AsPY8T/M+kNTVmIJ9wndvKYwAF7I3eJ8V78qLUfyLmXF3Skp7W1qf wLSQ== X-Gm-Message-State: AOAM5328TQaNz349Udi3pm9uTbZ/iG8WlbqJUk8ldKPhFYTxQBRKDc1o SIZ819Pxfq8xw+U4Uj+MDIk= X-Google-Smtp-Source: ABdhPJwLdZO1sAy3jiAaknQrXnaR0L2WJdQu0ZfiPIgYHgAkdE1YZuIY/FL9pb4yUW2nrclDkLHkqg== X-Received: by 2002:a05:620a:a8f:: with SMTP id v15mr24407916qkg.257.1628062874185; Wed, 04 Aug 2021 00:41:14 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id v11sm602529qtc.0.2021.08.04.00.41.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Aug 2021 00:41:13 -0700 (PDT) From: SeongJae Park X-Google-Original-From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , Shakeel Butt , SeongJae Park , Jonathan.Cameron@huawei.com, amit@kernel.org, Jonathan Corbet , David Hildenbrand , dwmw@amazon.com, foersleo@amazon.de, Greg Thelen , jgowans@amazon.com, mheyne@amazon.de, David Rientjes , sieberf@amazon.com, Vlastimil Babka , linux-damon@amazon.com, Linux MM , linux-doc@vger.kernel.org, LKML , Wei Xu , Paul Turner , Yu Zhao , Dave Hansen Subject: Re: [PATCH v34 00/13] Introduce Data Access MONitor (DAMON) Date: Wed, 4 Aug 2021 07:41:07 +0000 Message-Id: <20210804074107.3021-1-sjpark@amazon.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210802082424.19584-1-sjpark@amazon.de> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EA5F8300ACB9 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=h+X3Tahl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.170 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Stat-Signature: 8zxwmgmog3fxj7mgcxyjc95t5rmzgtko X-HE-Tag: 1628062874-838511 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park Hello Andrew, On Mon, 2 Aug 2021 08:24:24 +0000 SeongJae Park wrote: > From: SeongJae Park > > Hello Andrew, > > On Wed, 28 Jul 2021 08:36:43 +0000 SeongJae Park wrote: > > [...] > > Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We > > didn't find serious problems since v26[5], which was posted about four months > > ago. so I'm thinking this patchset has passed the minimum qualification. If > > you think there are more things to be done before this patchset is merged in > > the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to > > consider merging this patchset into '-mm' tree. > > I'm wondering if you had a chance to consider that. If you had the chance but > this patchset didn't convince you, could you please let me know your concerns > so that I can make some progress? Because nearly three weeks passed since this patchset is posted, I considered rebasing it on the latest -mm tree and posting it as v35. But, apparently it makes no much sense because we found nothing to fix or improve. And, this version can still cleanly be applied on top of the latest -mm tree. So, instead of merely increasing the version number, I'd like to describe why I believe this need to be merged into the -mm tree and eventually the mainline. 1. Merging this patchset will not bother other developers Most changes in this patchset are for DAMON-dedicated new source files. There is a change[1] for existing files, which makes PG_Idle independent of Idle Page Tracking, but it is only small. Therefore, merging this patchset will not increase the complexity of the other parts or introduce a regression. 2. Merging this patchset will not bother other users DAMON utilizes a mechanism that designed to minimize and limit the monitoring overhead. That said, DAMON can be opt out in the compile time for users who don't want it. Even though it is compiled, it does nothing at all unless a user explicitly asks it to do some works. Therefore, merging this patchset will not silently introduce any additional overhead to users. 3. This patchset is deployed to real users We are currently using DAMON patchset for profiling production workloads, as described in 'Real-world User Story' section of the cover letter. It is also deployed to real users other than us via Amazon Linux[2,3]. A few companies and several researchers outside Amazon have publicly and/or privately shown their interests in DAMON. 4. The downstream-only maintenance overhead is significant Following development works based on DAMON[4,5,6] are also ongoing. Because all the works are currently in downstream only, the maintenance overhead is not small for us. Once DAMON is upstreamed, the overhead will significantly be reduced. 5. This patchset is reviewed and apparently is stabilized Since the first version of DAMON patchset is posted (2020-01-20), it has evolved a lot. All patches of this patchset got at least one 'Reviewed-by:' or 'Acked-by:' tag by v31[7], which have posted about seven weeks ago (2021-06-21). After that, we found and fixed only minor issues. We also got a few more 'Acked-by:' tags. Since v34, which has posted about three weeks ago, we found no more issues. We are also continuously running extensive DAMON-dedicated tests. The tests include unit tests, self tests, functional tests, performance tests, and static code analysis. Some of those are also publicly available[8]. [1] https://lore.kernel.org/linux-mm/20210716081449.22187-5-sj38.park@gmail.com/ [2] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon [3] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon [4] https://lore.kernel.org/linux-mm/20201216084404.23183-1-sjpark@amazon.com/ [5] https://lore.kernel.org/linux-mm/20201216094221.11898-1-sjpark@amazon.com/ [6] https://lore.kernel.org/linux-mm/20210720131309.22073-1-sj38.park@gmail.com/ [7] https://lore.kernel.org/linux-mm/20210621083108.17589-1-sj38.park@gmail.com/ [8] https://github.com/awslabs/damon-tests If you think above explanation makes sense, please consider merging this into the -mm tree. Else, if this doesn't convince you, please let me know your concerns or what I'm missing, so that I can make some progress. Thanks, SeongJae Park [...]