From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2DDAC30653 for ; Sun, 7 Jul 2024 15:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C73EB6B0089; Sun, 7 Jul 2024 11:44:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C23966B008A; Sun, 7 Jul 2024 11:44:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B12566B008C; Sun, 7 Jul 2024 11:44:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 92C826B0089 for ; Sun, 7 Jul 2024 11:44:44 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3B7631A0E06 for ; Sun, 7 Jul 2024 15:44:43 +0000 (UTC) X-FDA: 82313379246.30.A68C80B Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf18.hostedemail.com (Postfix) with ESMTP id 33D1A1C0002 for ; Sun, 7 Jul 2024 15:44:40 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aNXb+dtK; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720367049; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jzALJo6TMAZf4s8azIfcjmEHSjLWPURwlmFGrCYnna4=; b=ayOjre0b/35EVQxri1130QAWX5knQ0EH5wviWPvdK89ZWPgFcDc2lpGdOPjsjFPqyU0faV bS7x/khunweYueetV9SVwfSi44h54lqQN/4t0fp7OWOKXOgZ6vrXL4lAFCRzk7wdSAgzbi AwVzHYIZXXkwHsHsojm1+CMVJFDihnQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720367049; a=rsa-sha256; cv=none; b=EfzUHvBIEn0iUFOIuedKVEvHtEcHIYWbtTXsIiaME0BuEI8dKFKf0T/B/G9F/xG2l6ZW+z ihJa+XqeU+QjrROdOTRDmSBAkG6mGIxPr7Y5KZchtuuxnAXd/h7H3upHOwkOypkpVZJE8r 2jJLnVbpcAkJ9hHDd9twqiEniniQSAA= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aNXb+dtK; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id DA9A5CE09AC; Sun, 7 Jul 2024 15:44:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87C38C3277B; Sun, 7 Jul 2024 15:44:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720367075; bh=rfh5ejnUKDUF97vTImVmbiYlu6frF3yu68+qEVv2Y7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aNXb+dtKH7s7ItamI+cgdR3vS5phwKiTIUoaLlYDufil6gSSYNHkZcoY64xCCSnJh yynhJKwXi5BKPyNL+kKa6iQtEg6nKgD61KfXlOnxrJztjyKQ9UKK6eIU9WPVLxTZ6h jY/MjHaabCXCVspGZV5N0Mr8ptrewz6cidJ+FfBN7FWW8FgMCKI9x1Afi2/c4z9qeS dwodGPqaH/oJu+A2iYyeQ8mEnVuDt8RzrNmqQfTdd0UoeRoCWcDw5bYqMck1Hb5ciw ErR72cJGo2B8yMpkbAqd5Oxrxl+o9Z+6LYkmEoOUK4Ef4La1FpMbiclyizOSlL2Ipj jPWlbwPfTGhkQ== From: SeongJae Park To: David Rientjes Cc: SeongJae Park , Michal Hocko , Andrew Morton , Mel Gorman , Balbir Singh , Peter Zijlstra , linux-mm@kvack.org, damon@lists.linux.dev Subject: Re: Tools for explaining memory mappings/usage/pressure Date: Sun, 7 Jul 2024 08:44:27 -0700 Message-Id: <20240707154428.77697-1-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <29c27dab-a590-5df2-c840-279bf9dff090@google.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 33D1A1C0002 X-Stat-Signature: bcbzr99fshsirfspb4ty6ahgbp99nfod X-Rspam-User: X-HE-Tag: 1720367080-982995 X-HE-Meta: U2FsdGVkX18bVLuxro9A4jaSZDzrLcu5gjb6+c2RSK1G2fCy9OPmTYYU5WbJrBd6cXQhNhR5P8nH0ku/IAPgUlaaXN14hYShmRhxlfpldszNGSSVrhu7q2DnKliJrEkm4Dudj6tn0QIU4Z8y3x7Za4vH4UrAEYZkZqF0XLDJausn6sb0UZ9cZHE8UNLvuR8bEVGfBeDd/gJzStcOCsKcEYO4zuAfKfV919FjQPWTLSMC/DbX7cVCh7i5PAN5tR6bhRMrGg0G017q+7xxTWkXSwnowvTifEyUG3AU5xw4hMQSqqm3FXF4iLlLu+w8A4Gmx9v8F6yhGF/nQySJ4GJCt7+sa8kwffNe+PCnURm/DdKia3TWBc+V0HR6E9hHKZTikaH6WepFQHSoDhmc2V7HmVEMufpkTgdLIZVSwbqugoxpdjI2OvNEGd0rVf+SPpBrN/kmSOBIOP6UjuvDl215ROD5tpNTEyhP2H4SZFaQ5bO8pPDv7Ft/0Xb/ZsTU9b3IEIF2SWFYv3DhpYto/11u1OLDmwcYjNnEb6Xon1gCbkK7QZ2RGlsQ/AQV2IyeBgZ9lZM4fMKrCCXzCGsSjnsxaK6Nk9JSSkur914qtKsEfQVR+UKKfB6+4FAmoH426sbLTXAliurFWtUt2Sda1wIGaxmuVnEmCMl7AfO49tzeJkP8/mvxfWolj5ZEor6orcw800//N11BxvLSmlk7eom9Ia6kMB60cqQa3M5BLDXjbxBiK1bvD5JFgiS3w6dVFJ/NYN57s0zy3oQfZthYIioMeoadgaSHp9ijMqeaZcG58O3o8FhBGWolAi1HRh/qLtvwnbfY8UM2Io2FGkXWrj5QQ+SH49Cyu18Y+a6vNVkD/ho25pPzGhvN688xUkGNXjioYskeEt0slHliRYX2aPc9qigJjEFDLGF44MvLPnwzAPLpD1r4YTckBn6eM3EzRJnhMgy5c4yCopu24h2JMuB M2GOmSHg HrvJ3pb1fnxjbEQY3oTy6Smx2P0WQFDFHDz87GtYbmrzzIuWYXnoXzGUD4pUZtzlFvR4gjy019HPJU2oteTEIgmFcQe0Lcnh8NXAdS6F1iRGQ17CI9vcEVtSOXv+r47nFgaS6wB7kcsiBYGSo2o/xrOe6W8IkW3iFrf6PaKiiA36boV9AZo+0B80swNO5lTbDrdPvG+rxtQXL8NlTydgpkeh0Wk8mh0/fyMZN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi David, On Sat, 6 Jul 2024 13:55:11 -0700 (PDT) David Rientjes wrote: [...] > Rather than hacky scripts that collect things like vmstat, memory.stat, > buddyinfo, etc, at regular intervals, it would be preferable to hand off > something more complete. Idea is an open source tool that can be run in > the background to collect metrics for the system, NUMA nodes, and memcg > hierarchies, as well as potentially from subsystems in the kernel like > delay accounting. IOW, I want to be able to say "install ${tool} and send > over the log file." > > Are thre any open source tools that do a good job of this today that I can > latch onto? DAMON user-space tool, damo[1], provides background recording and reporting of memory access information including size of memory showing specific access pattern (e.g., working set size). Nowadays we're extending the tool to capture and provide more information for holistic and intuitive system investigations. Currently basic memory footprints and CPU usage of functions are provided. The current status of the tool would be far from what you're looking for, though. I'm also not sure if current future plan of the tool would perfectly meet your requirements. We're open to any contributions to damo, though. Hopefully others may know better tools for this. I'm looking forward to a chance to learn from those. > If not, sounds like I'll be writing one from scratch. Let me > know if there's interest in this as well. We're open at not only receiving contributions for damo, but also providing contributions to other projects (and using it). So, yes, I'm interested in this :) [1] https://github.com/awslabs/damo Thanks, SJ > > Thanks!