From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 638F7C2D0CD for ; Mon, 19 May 2025 16:44:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 36BE46B008A; Mon, 19 May 2025 12:44:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 31D606B00BD; Mon, 19 May 2025 12:44:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E4AA6B00D5; Mon, 19 May 2025 12:44:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 001FB6B008A for ; Mon, 19 May 2025 12:44:19 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 59EFD120ACF for ; Mon, 19 May 2025 16:44:22 +0000 (UTC) X-FDA: 83460230364.26.2CCDE77 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf20.hostedemail.com (Postfix) with ESMTP id C8A231C000B for ; Mon, 19 May 2025 16:44:20 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="tu/PZ/R7"; spf=pass (imf20.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747673060; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=0uZJiDgxnCT+hMz4fQQskn3lgRHmPFOQvuVStsivMfY=; b=XMESnut/TcI6GmzBGjBnpG755TNx0GZELJKpzwD/KL9lz3KIgMR6A97DhYUcQO/rqb1tXZ xrlN8h+iblhA5Pg7okQI+AKtwYf3azng472J/i3lR1F5Wg9SEsL3PmuHy8K+5x0uMhxreP Iq6Yh6tX8THhBisQ+CwPYEH/DE2owXE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="tu/PZ/R7"; spf=pass (imf20.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747673060; a=rsa-sha256; cv=none; b=bpyCcZm4F/XDKb1p4/uccQyIU2lvsFVDjJbM3ojyGBEOyYs2e1kNZOLafZEZtDrrcbIKc8 Utpxw/MvHNWaB04FHggpF7T7UvnWaocxQIo2EdmryxegDLHR1JGEcIe59jx21s0m8QSSSw A+KcDiw0E1mh2UVCbBkmn8eQDtgAnwg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 1A0F0A4E041; Mon, 19 May 2025 16:44:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BA7AC4CEE4; Mon, 19 May 2025 16:44:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747673059; bh=tNwc3KlCa+UsKLQNf/TF+YhpZdTxy9+n5nN6fjl4mIs=; h=From:To:Cc:Subject:Date:From; b=tu/PZ/R7xjoRbosdEr05GmX51LuS5dRnQIBbAZGIX21QL8XPHXCUIAr7RPvHIIehu QWkAwHxCRdTCYNrmZtFte8S1zhfJTt4BDmMliyhWX7wQ/RGIao89Mbobx1dmk6wPV8 Jcjj/l50sygfZyS40m3V3NGtRbOi1A9VnyhBGYRGeDkBzgvoOkot4I4rkpAf/4t4YH 4JKw9SJ6s5ZX2ZWL3HRJDu5b6qJbWou7YEepiIsJPTLz6EKZODlafgIHjNAAtJuRSx OQ7zWgrD8xofPA6uiFF3YAr1D9fAHjVntHhcXO8pLucpMpikG8XeJX3ivAs5fCjNwv yeLIY1AuwFOgA== From: SeongJae Park To: Cc: SeongJae Park , Andrew Morton , damon@lists.linux.dev, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 0/3] mm/damon: introduce DAMON_STAT for access monitoring that just works Date: Mon, 19 May 2025 09:44:12 -0700 Message-Id: <20250519164415.43935-1-sj@kernel.org> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C8A231C000B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: zkjpduxq49qx9qsureuxzed9acbtw8iw X-HE-Tag: 1747673060-413148 X-HE-Meta: U2FsdGVkX19PfXAvT9VUsYvGjNxy9vAGCKX/MHJW62lXixB7F9M5yEmWi+7xHjlKmAkbpmxsyDxfJzjw/Ot8ctbd0dCj9fCx6udJD3qsuSpKu7Khra731MFB8MR6MRVOK0IxNLVG0YrNpoJ5pJx/um0jqlFpHughyj8HPtdakPrCu1J71m1/xEibfsNJROLmVpaS6U0WUw95jVH6EjLPAqXE35fy+oZdK6lF5bRiAcUTxTjklHbG2PpFkqBMR0m5v7C260xmEzls49EPZSw/b3PSbDpl99XdYi6IA4b7vV6jBuuy9C6dPSXWi8ymh3H1idgsVHjBRJDwhcqgBxlEFBsLYoWkBdXPdhwkJPY5B0F8+MOVqJsiHfDSex/GwrN5b6bDq5HgIeLHwnVl7zUJKI2qZRNCiqb2hwyiNPPrpOuAXL+N3LYp7/uvlX94WQIFx2qZpU7MOF+gzPISE5ARhpiFzvdszjmOvJK0NlFPgJ1ducze+uckbo6rcup/Va/oyxu4hX2Ap/VfhV81ProkeTOFPiqqjq4axBXJK6BUcPtkxOEYjw/sdUxy2ZyWT4UA54HxUmjihC6fejHVR9vkEfjNqbUMRRCdS5NEK5mkk17dXZxE+gfMWx0jMxlaWs3YZKTR9flpEvAEf9x0lfupCuDvTEDqJ26C/CCd2WqDvE0AmlWO9DNbls5cxwD+bDu6K8NO3e9qWhfzmvv9YcBvO6ENbY8+to+6JbVyY7tA3QZE2HbmHBXvxRmyDMrI2mlRK80CpTdf/g+E5djqyfs5a+01Ezd2vi8eTP5dER92He4kIdw+DJjwi2biIvE1+Jpqqix79MrjRNl7vXmn8z7bNdGNk19hj1yfc8mgGiJrshXTOHYXjWqG908fG8kFtMkOk19gVXZ7WEChLUrHlQtKq5r8tYQWntkN/PQ0LXNA1Vu/z1q6z5HSEOOzDNgxnNt9oesehMILaqaDR/cGc0u aJcYW5Eh R8dihn5b/myTXKXlrDXQwHcaljmu7I4AjCLt5RWCVsX0aMjVoBmxCdmdSofxhEY6kgSJX3f2lT5gDCS5zvWaaK0F9AZQdALUH1RrIT8S4/mm4jZ2ttwsTi2tx2wO7ZrMojiXtRmg/qjBmPbpNu4G6p0IU9BqqVIiPXKtfO33EYTFz2hc4T9rU7n+WsrN7vmgb/FFK8ykDQ2lZT02FDbWK9i2UTqBibl7YthjCOZGIcIvle/cG9/HW0DIGjgdlf7NrUGTI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: DAMON-based access monitoring requires manual DAMON control and results parsing. Introduce a static kernel module for making it simple and intuitive. The module can be enabled without manual setup and provides simple but practical access pattern metrics, namely memory idle time percentiles and estimatd memory bandwidth. Background and Problems ======================= DAMON can be used for monitoring data access pattern of the system and workloads. For that, users can start DAMON to monitor access events on specific address space with fine controls including address ranges to do monitor and time intervals between samplings and aggregations. The resulting access information snapshot contains access frequency (nr_accesses) and how long the frequency was maintained (age) for each byte. The monitoring usage is not simple and practical enough for production usage. Users should first start DAMON with a number of parameters, and wait until DAMON's monitoring results capture reasonable amount of the time data (age). In production, such manual start and wait is impractical to capture useful information from a high number of machines in a timely manner. The monitoring result is also too detailed to be used on production environments. The raw results are hard to be aggregated and/or compared for production environments having large scale of time, space and machines fleet. Users have to implement and use their own automation of DAMON control and results processing. It is repetitive and challenging since there is no good reference or guideline for such automation. Solution: DAMON_STAT ==================== Implement a such automation in kernel space as a static kernel module, namely DAMON_STAT. It can be enabled at build, boot, or run time via its build configuration or module parameter. It monitors entire physical address space with monitoring intervals that auto-tuned for reasonable amount of access observations and minimum overhead. It converts the raw monitoring results into simpler metrics that can easily aggregated and compared, namely idle time percentiles and memory bandwidth. Discussions =========== The module aims to be useful on production environments constructed with large number of machines that runs ling time. The auto-tuned monitoring intervals ensures a reasonable quality of the outputs. The auto-tuning also ensures its overhead be reasonable and low enough to be enabled always on the production. The simplified monitoring results metrics can be useful for showing both coldness (idle time percentiles) and hotness (memory bandwidth) of the system's access pattern. We expect the information can be useful for assessing system memory utilization and inspiring optimizations or investigations on both kernel and user space memory management logics for large scale fleets. We hence expect the module is good enough to be just used in most environments. For special cases that require a custom access monitoring automation, users will still get benefit by using DAMON_STAT as a reference or a guideline for their automation. SeongJae Park (3): mm/damon: introduce DAMON_STAT module mm/damon/stat: calculate and expose estimated memory bandwidth mm/damon/stat: calculate and expose idle time percentiles mm/damon/Kconfig | 16 +++ mm/damon/Makefile | 1 + mm/damon/stat.c | 245 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 262 insertions(+) create mode 100644 mm/damon/stat.c base-commit: 251509f2949105d3d0e3cdcd3921670a6aee3a0e -- 2.39.5