From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81891C54FB3 for ; Thu, 29 May 2025 07:21:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D01606B00E0; Thu, 29 May 2025 03:21:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB2786B0112; Thu, 29 May 2025 03:21:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC7CC6B0111; Thu, 29 May 2025 03:21:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9EB266B0088 for ; Thu, 29 May 2025 03:21:47 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3E9AC160876 for ; Thu, 29 May 2025 07:21:47 +0000 (UTC) X-FDA: 83495100654.06.61FB51A Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf11.hostedemail.com (Postfix) with ESMTP id 628C14000A for ; Thu, 29 May 2025 07:21:44 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748503305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gv5aLEatGoBFv6kZ81IwWdPKS29mbXa1Y/8WBSKiImA=; b=VFq6CpWFXLLmCxbZTUNUwg2oospUZr++95JH71ZE+fOVdjmsepssC/6fug2iPPP2dzuUR1 OI3UFx6T6PKf/70szoagdkp2Yov2YTWBakuO0mz2kRKwUPQLvs4DjREn/3Rzvf0c5yxrFj foGkYbEMzaGmiNaR89VRjd18iYbs10E= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748503305; a=rsa-sha256; cv=none; b=0jkm1WWrRP0AN+xke0BAmqlobpGrTV1cNQzH6cyS+iX+01hnB8XWdWxzLmuxKRcgPRRyYW 1PJO9d591pRswnrooYthAhPahpA5m3seCoZFjD9s4L3Vo0PKng44DK5xiuV0gma2I7gEko UAN2GuE9oR9XdJQJ4cOnWX9zKOzQ3bU= X-AuditID: a67dfc5b-681ff7000002311f-6a-68380b04bd67 Message-ID: <82ea9158-c71e-4c11-a46d-397e9ec9349b@sk.com> Date: Thu, 29 May 2025 16:21:39 +0900 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Cc: kernel_team@skhynix.com, Jonathan Corbet , damon@lists.linux.dev, kernel-team@meta.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 0/4] mm/damon: introduce DAMON_STAT for simple and practical access monitoring Content-Language: ko To: SeongJae Park , Andrew Morton References: <20250526210936.2744-1-sj@kernel.org> From: Honggyu Kim In-Reply-To: <20250526210936.2744-1-sj@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrDLMWRmVeSWpSXmKPExsXC9ZZnkS4Lt0WGwYlzUhZz1q9hs3hyoJ3R 4sn/36wW+y4CuQvblrBYXN41h83i3pr/rBaHv75hcuDw2LSqk81j06dJ7B4nZvxm8XixeSaj x+K+yawe5y5WeHzeJBfAHsVlk5Kak1mWWqRvl8CVMf/obuaCv2oVyxf/ZWlgbJbvYuTkkBAw kZiw5DgzjH2+9y47iM0rYCmx7PZV1i5GDg4WAVWJdztcIcKCEidnPmEBsUUF5CXu35oBVs4s sJtRYmKrLogtLJAi8Wz5DUaIuIjE7M42ZpAxIgI+Ei3LE0HCQgKGErN+7mEFsdkE1CSuvJzE BGJzChhJfN7XygTRaibRtbULaoy8xPa3c4DGcAFdeYJN4uWbVhaIkyUlDq64wTKBUXAWkvNm IVk9C8msWUhmLWBkWcUolJlXlpuYmWOil1GZl1mhl5yfu4kRGCfLav9E72D8dCH4EKMAB6MS D+8JRvMMIdbEsuLK3EOMEhzMSiK8TfZmGUK8KYmVValF+fFFpTmpxYcYpTlYlMR5jb6VpwgJ pCeWpGanphakFsFkmTg4pRoY09NsPCpSuB9NnCEj0c8T6Jz83Uvpx9710gt7Yp7KvFg4v7zA esqfGX/7f6w9HcBffqz0ynxB7QlX1TrDXoVzLrNSN+UUbxHgXVAWu/1v+N+MV65BV5SXdLzJ Dm1v1TSc/Mrl57xPNf7HDrEf4v8yWe8R/7EUM8ntt9S5ppafyeLS7VhRt2+uEktxRqKhFnNR cSIALASJrI8CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrMLMWRmVeSWpSXmKPExsXCNUNLT5eF2yLDYP9yPos569ewWTw50M5o 8eT/b1aLfReB3MNzT7JaLGxbwmJxedccNot7a/6zWhz++obJgdNj06pONo9Nnyaxe5yY8ZvF 48XmmYwei/sms3qcu1jhsfjFByaPz5vkAjiiuGxSUnMyy1KL9O0SuDLmH93NXPBXrWL54r8s DYzN8l2MnBwSAiYS53vvsoPYvAKWEstuX2XtYuTgYBFQlXi3wxUiLChxcuYTFhBbVEBe4v6t GWDlzAK7GSUmtuqC2MICKRLPlt9ghIiLSMzubGMGGSMi4CPRsjwRJCwkYCgx6+ceVhCbTUBN 4srLSUwgNqeAkcTnfa1MEK1mEl1bu6DGyEtsfzuHeQIj3ywkV8xCsmEWkpZZSFoWMLKsYhTJ zCvLTczMMdUrzs6ozMus0EvOz93ECAz6ZbV/Ju5g/HLZ/RCjAAejEg/vCUbzDCHWxLLiytxD jBIczEoivE32ZhlCvCmJlVWpRfnxRaU5qcWHGKU5WJTEeb3CUxOEBNITS1KzU1MLUotgskwc nFINjPvnbQ6vu8EvJ2WUyu1fkL0iqULqfKA8X9RRB3PR7mbeu8cEjwQrnanwYQgLD92b+3Xt 9VPfqmMNrUJmcLmKGVrdP1efyDJtNXvD2hS11euMEhbExTK9ub++LV+nUGFL/5e3Ruu3iX7Z 1HD3yf1ekfl3hcK3Fb798i++evJKrsMf3a8sUHMLVmIpzkg01GIuKk4EAEBzpEJ2AgAA X-CFilter-Loop: Reflected X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 628C14000A X-Stat-Signature: s416agprjrb7k8z97jsy58oknzgbakqy X-Rspam-User: X-HE-Tag: 1748503304-827012 X-HE-Meta: U2FsdGVkX18atNAykHQY/jRK/Telpxv39omNiPe6LIzXkiSuTvmAB+10L5gSTev+rGJ7+l6ai45SKkBJrvvfH3tqh08jjm01qhJejP+ZQEwHkunfXAR3+aFvAHBFNme0rNbkwhUHP2hMtjKU75PfvsVLCDq3t4QEuYRhM76xA3ARGeMckJu7MDftfWozU87Hw2US0SzFJS0Cg6MtHjc7RTWsmVCXGnbkj+nbjpczZXco9kuMPS/kIO7UGE3fc5eYT7q8z35W9Vi/+dz4mbrDQ+UbB0wilK/tsYNrOyXMOlV84iA4jKj7MioaDl3dU7FEIVh+h29oOb2sW8ydM8krDrwoDKWP4Ii93oMINd6W0m9PEOozHg5gkct4a6BUnuIGh/Z8/0NW8oAwvcxgy9xU8V99NghOLPZ2gdadHM3XsU8yJowDKKDC/HpvwlPdQx/eXwJg/G5RQhm4+J3ZT0EY0Uyj6jZJO6O/qD7ySZa6hGXT3KfxNDTEwNgjdhIh9UAbQ+UP2vpAZ5VSwtJwRNYT9C54HIYe4cX4uqMF5qx+UZElDZjI+JhZZDf0KEqGgXTes/0J/MuwW25vQbl8Sb93T/nGjCraCXSQ1kvO2iUa4GozgYU+yCw37MHrpQtoSWN3yNXmo/u0avJ646Oyo4Msna7+nE+ZfpyCpsimesXV3mzQJGOH5gRU2TvpbH6NZYWdEsI93H4zuEAILoDcC0k5Z5S/7CQm72mnG0Mkk2+95n/dtVwquyTk35MrofbH4az11Kl5P/15TvWbsUBPOFnBDYBTRpMt9l8B8ST5IKiGeQc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, Thanks for your work. On 5/27/2025 6:09 AM, SeongJae Park wrote: > DAMON-based access monitoring is not simple due to required DAMON > control and results visualizations. Introduce a static kernel module > for making it simple. The module can be enabled without manual setup > and provides access pattern metrics that easy to fetch and understand > the practical access pattern information, namely estimated memory > bandwidth and memory idle time percentiles. > > Background and Problems > ======================= > > DAMON can be used for monitoring data access patterns of the system and > workloads. Specifically, users can start DAMON to monitor access events > on specific address space with fine controls including address ranges to > monitor and time intervals between samplings and aggregations. The > resulting access information snapshot contains access frequency > (nr_accesses) and how long the frequency was kept (age) for each byte. > > The monitoring usage is not simple and practical enough for production > usage. Users should first start DAMON with a number of parameters, and > wait until DAMON's monitoring results capture a reasonable amount of the > time data (age). In production, such manual start and wait is > impractical to capture useful information from a high number of machines > in a timely manner. > > The monitoring result is also too detailed to be used on production > environments. The raw results are hard to be aggregated and/or compared > for production environments having a large scale of time, space and > machines fleet. > > Users have to implement and use their own automation of DAMON control > and results processing. It is repetitive and challenging since there is > no good reference or guideline for such automation. > > Solution: DAMON_STAT > ==================== > > Implement such automation in kernel space as a static kernel module, > namely DAMON_STAT. It can be enabled at build, boot, or run time via > its build configuration or module parameter. It monitors the entire > physical address space with monitoring intervals that auto-tuned for a > reasonable amount of access observations and minimum overhead. It > converts the raw monitoring results into simpler metrics that can easily > be aggregated and compared, namely estimated memory bandwidth and idle > time percentiles. Refer to the commit messages of the second and the > third patches of this patch series for more details about the metrics. I see the description looks good but it'd be useful if you could share some execution commands and expected output examples that some newbies can get better ideas. I honestly do not have a clear idea how I can use this kind of static kernel modules as general users although I have developed some features of DAMON. So could you please help? Thanks, Honggyu > > Discussions > =========== > > The module aims to be useful on production environments constructed with > a large number of machines that run a long time. The auto-tuned > monitoring intervals ensure a reasonable quality of the outputs. The > auto-tuning also ensures its overhead be reasonable and low enough to be > enabled always on the production. The simplified monitoring results > metrics can be useful for showing both coldness (idle time percentiles) > and hotness (memory bandwidth) of the system's access pattern. We > expect the information can be useful for assessing system memory > utilization and inspiring optimizations or investigations on both kernel > and user space memory management logics for large scale fleets. > > We hence expect the module is good enough to be just used in most > environments. For special cases that require a custom access monitoring > automation, users will still benefit by using DAMON_STAT as a reference > or a guideline for their specialized automation. > > Revision History > ================ > > Changes from RFC > (https://lore.kernel.org/20250519164415.43935-1-sj@kernel.org) > - Add an admin-guide documentation > - Wordsmith commit messages > - Rebase to latest mm-new > > SeongJae Park (4): > mm/damon: introduce DAMON_STAT module > mm/damon/stat: calculate and expose estimated memory bandwidth > mm/damon/stat: calculate and expose idle time percentiles > Docs/admin-guide/mm/damon: add DAMON_STAT usage document > > Documentation/admin-guide/mm/damon/index.rst | 1 + > Documentation/admin-guide/mm/damon/stat.rst | 69 ++++++ > mm/damon/Kconfig | 16 ++ > mm/damon/Makefile | 1 + > mm/damon/stat.c | 245 +++++++++++++++++++ > 5 files changed, 332 insertions(+) > create mode 100644 Documentation/admin-guide/mm/damon/stat.rst > create mode 100644 mm/damon/stat.c > > > base-commit: 90887f57d7a67917136e7c70d26fb3f2fcdc6f53