From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 536C3C433EF for ; Mon, 30 May 2022 05:04:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E14C68D0003; Mon, 30 May 2022 01:04:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D9C718D0001; Mon, 30 May 2022 01:04:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C60838D0003; Mon, 30 May 2022 01:04:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B57B58D0001 for ; Mon, 30 May 2022 01:04:23 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 7AC7A8078E for ; Mon, 30 May 2022 05:04:23 +0000 (UTC) X-FDA: 79521218406.06.53ECA2C Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf01.hostedemail.com (Postfix) with ESMTP id 25DA340057 for ; Mon, 30 May 2022 05:04:17 +0000 (UTC) Received: by mail-ej1-f54.google.com with SMTP id l19so704632ejr.8 for ; Sun, 29 May 2022 22:04:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7XirfjVzzLTx46Nwwu3/b9HkXUr3i8KCBR76hKC0bJ0=; b=DGhV6BHo9SqfMyLJjJ3wu1Uvioz9+CdI8xcclct5k9zkFmC91TC/GcQFrLJtjk+3JK Fxl+U4K7mXsZnykWWVzdXkfszUw0iElQsMpT5qlzr2k73LsOyT04ud0ANyOPX9XWRX2L Z08pqCkTditNbugrDOtozQj72LnxxWQmacSinPdkdZu7eHp94yxT9a8xn3/zCzIwXg6F FHYwXtF5TpiQQ9tGpL9pjo2RYdsQksdw4Co538yxxUfxku3NxhuL3sp5O+RMxoxBzYCR ggMDQNmHPVKvbVt895CzU+hBFdgdCzjL36cZ8gCfSrqCS2Et46lAtoL+Fo1gekZbS/lh r6Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7XirfjVzzLTx46Nwwu3/b9HkXUr3i8KCBR76hKC0bJ0=; b=61w39YY6DduG7+gn/PQIbIuiqLHVpeqGO5BmVkOE7l6ys6uA1I7EkYfngJv5twACro NeIk7FnjtE2tFZQQJvUl1bvb4Zu1v+x2/8SNnG4VndfoedIMi8Y3AcmKW2+S5Ef3fBQC AbqSMPl/ig04W/G0+OMwu3B+GbGTW7+U+hntt91PMG9nmDUTnUW7AS9n3KG+YdR6W3LE mTOtGMoTj3FAi0q9a97wg1BJGw8LVW90sZRVxxTukC0lxPDg8+/N+Z21LmPNSJKnityW HyiCjE+71EEjXeJuEEMMb27vi97dxpUWiLHE9OixR9fzhQQVMZ8RGxMz+nXHFro6zErL j5lA== X-Gm-Message-State: AOAM531kIH3kX2qwH3xL2ybXP0rksz/hWuGyo9CM55FHls/mrL5yhqnp ba91tC+oX+vxAEKf/NIHkKfLa8gTVrvxuuJQSmA= X-Google-Smtp-Source: ABdhPJxzcsWyiFUNFhiLW9ZBQqLkxaf3fKI1k4l28bbKdikEdXxdXZVuWKK+6Sq/Nt4qfH6wn4CQzCbctovuEx5Xndo= X-Received: by 2002:a17:907:d17:b0:6ff:2f91:5b82 with SMTP id gn23-20020a1709070d1700b006ff2f915b82mr14190903ejc.192.1653887061641; Sun, 29 May 2022 22:04:21 -0700 (PDT) MIME-Version: 1.0 References: <20220427175049.50057-1-sj@kernel.org> <20220529195411.102379-1-sj@kernel.org> In-Reply-To: <20220529195411.102379-1-sj@kernel.org> From: Barry Song <21cnbao@gmail.com> Date: Mon, 30 May 2022 17:04:10 +1200 Message-ID: Subject: Re: DAMON VA regions don't split on an large Android APP To: SeongJae Park Cc: Rongwei Wang , Andrew Morton , Linux-MM , LKML , Matthew Wilcox , shuah@kernel.org, Brendan Higgins , foersleo@amazon.de, sieberf@amazon.com, Shakeel Butt , sjpark@amazon.de, tuhailong@gmail.com, Song Jiang , =?UTF-8?B?5byg6K+X5piOKFNpbW9uIFpoYW5nKQ==?= , =?UTF-8?B?5p2O5Z+56ZSLKHdpbmsp?= , xhao@linux.alibaba.com, damon@lists.linux.dev Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: tucjrxezy8p65f9zoifuymhw3fpu1wr3 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DGhV6BHo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 25DA340057 X-HE-Tag: 1653887057-597673 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, May 30, 2022 at 7:54 AM SeongJae Park wrote: > > On Wed, 27 Apr 2022 17:50:49 +0000 sj@kernel.org wrote: > > > Hello Rongwei and Barry, > > > > On Wed, 27 Apr 2022 19:44:23 +1200 Barry Song <21cnbao@gmail.com> wrote: > > > > > On Wed, Apr 27, 2022 at 6:56 PM Rongwei Wang > > > wrote: > > > > > > > > > > > > > > > > On 4/27/22 7:19 AM, Barry Song wrote: > [...] > > > > > > I guess the cause might be: > > > in case a region is very big like 10GiB, we have only 1MiB hot pages > > > in this large region. > > > damon will randomly pick one page to sample, but the page has only > > > 1MiB/10GiB, thus > > > less than 1/10000 chance to hit the hot 1MiB. so probably we need > > > 10000 sample periods > > > to hit the hot 1MiB in order to split this large region? > > > > > > @SeongJae, please correct me if I am wrong. > > > > I think your theory makes sense. There was a similar concern, so we made DAMON > > to split regions into 3 sub-regions when we don't see advance[1]. My current > > rough idea for improving DAMON accuracy is making it more aggressive while > > keeping the monitoring overhead low. > > > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/sj/linux.git/tree/mm/damon/core.c?h=damon/next-2022-04-21-08-31-on-v5.18-rc3-mmots-2022-04-20-17-37#n1053 > > > > > > > > > > > > > I'm not sure whether sets init_regions can deal with the above problem, > > > > or dynamic choose one or limited number VMA to monitor. > > > > > > > > > > I won't set a limited number of VMA as this will make the damon too hard to use > > > as nobody wants to make such complex operations, especially an Android > > > app might have more than 8000 VMAs. > > > > > > I agree init_regions might be the right place to enhance the situation. > > > > 'init_regions' has developed for the purpose, where user space knows some good > > information for starting point of the regions adjustment, and thus want to hint > > DAMON. Nevertheless, it might not work as expected, because DAMON > > automatically updates the target regions to cover all VMAs as much as it can. > > I have posted a patchset for the use case yesterday[1]. > > > > [1] https://lore.kernel.org/linux-mm/20220426231750.48822-1-sj@kernel.org/ > > FWIW, the patchset for the fixed virtual address space ranges monitoring has > merged in the mainline[1]. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=98931dd95fd489fcbfa97da563505a6f071d7c77 > nice to know that. Thanks, though it doesn't fix my problem as I am looking for a solution to collect precise monitoring data automatically and economically. > > Thanks, > SJ Thanks Barry