From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F92EC02180 for ; Mon, 13 Jan 2025 21:48:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A31056B0093; Mon, 13 Jan 2025 16:48:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E13D6B0095; Mon, 13 Jan 2025 16:48:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85AEB6B0098; Mon, 13 Jan 2025 16:48:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 64A9C6B0093 for ; Mon, 13 Jan 2025 16:48:07 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 15FBBA0345 for ; Mon, 13 Jan 2025 21:48:07 +0000 (UTC) X-FDA: 83003767014.06.9B0DA98 Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) by imf06.hostedemail.com (Postfix) with ESMTP id 2F4E0180006 for ; Mon, 13 Jan 2025 21:48:04 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=u7BgVvb7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of surenb@google.com designates 209.85.160.174 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736804885; a=rsa-sha256; cv=none; b=TyuHJtGzwnbyzIzsEivzWU0jokFqS+tPIsWEAZ/VEmL5wAl2apBEDcYHtKAWVoGLRBi4Fy 8+ygeWy3TXftCVDGu15s9MBqbXEo3XISUaRaocqHwYlZwVoSukjYQ/gJj46LqXj4QMGTul XxTJC3LN7iBlLuLiB9orD5uxearz7Fc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=u7BgVvb7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of surenb@google.com designates 209.85.160.174 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736804885; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EJccBiXtE6/AoSCV4ZShoVvMfacW/cyosuY4AfqBk+s=; b=ewHuXHfswT5nkHo10G5aj0VrKer0TFC0MKGtBFkHhserRU7rfav+krtnyoveRhY14nw+ot se/CoTyKUSaGumSw5aHMw+7FZ9FLDZyF3j1fiXoi3LKEpQnodrLSP2vT8dB3lGdRsaCEuD dZOjrCL0thwTE/2BEg3E77pZKirVUfI= Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-467abce2ef9so66221cf.0 for ; Mon, 13 Jan 2025 13:48:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736804884; x=1737409684; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EJccBiXtE6/AoSCV4ZShoVvMfacW/cyosuY4AfqBk+s=; b=u7BgVvb7Ya5a5yFrG1VT8QdAfc2v1OeNd6Mya/8NQqocHUWkgfuQVb0S7kWDrrcHtl Az7iiMxyxyBHVjH62FVv9T1EB9XUddsFlv4ox8hG/ymai0CnlnRThtsT4b3qzhoP++8M tTJiG+nZxJs6PHcf1BKFNqW4USOhZEiwCbnVl8jjpDUUUxh93UHy12tm/Mptqb13P7NI kl+J3N28GolT+Xy1I6jguN2Z5ttA45VSDDBeDXs99H1JWbxg8Xqn/deZGGS/RU3vZ419 yNhrnWPaB5DHfy+lkVtiV451F5nZ4IoKtO7tPGD7UvIJbroZRZuW0EYcAlnk416f+LgG jcvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736804884; x=1737409684; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EJccBiXtE6/AoSCV4ZShoVvMfacW/cyosuY4AfqBk+s=; b=eoLsaVdhY9fRCqUflStaTkjdZoPNVLCkoKcbPL2yZuqOiQyNjXQBKyPuAcSMUj6I6G 1hjTJVAujCEDElQdoHudWsq2sVhHMsTerYfoiqofmq4HzXYzVZCHaMesQyT/U5aXNOCG hDCH1X2FyCX76LQhk0rtIHh1iJhERqiAHrWQBizkaoMID1blIZ5SCHQtTHXInZE5G+sy go7flYYPfmA/qo6v0qVrO0vcFsRz+xq+c97rRaj1TXmAJK71fIAnPAa+XNwvtm4ossYf xB+GCB9O8qnMyzPJucHwdNHpaFGKiSVQz+zkyu/ALC8CJpfYRwy/w/O98DUnZXYlM15P 1tww== X-Forwarded-Encrypted: i=1; AJvYcCUgvgb5OFkOW63xjbZMjxn5E8qVXsjmX1/PXINVRchWSoriltATBde0Csi4Q8s2s/Zm10yW1FnduQ==@kvack.org X-Gm-Message-State: AOJu0YwENNh/2OjOAH4w/sHbocGDjAPWiv24gHYvncFj8OXgv7EfIUIO fseIM8fpyJt93EPYuxrgDpkT+kyFop2LA5WkU5r45OpUcbwtkV1iyZA0dKCpPIWShbAf8P9llVk L5C6OSG2KnkNwFJjV1Paf8UF+u+UVP2yZ9HxY X-Gm-Gg: ASbGncuM/ThQ0LEHucXUIlj58w3KEwzRAEIZn03zjQCGl3vJiX+Q5W9uHhNEmK7Yjyq 2ompqHEpCmqrE8jZqPOo6LAbUOsj7p7m5PJaICw== X-Google-Smtp-Source: AGHT+IFZGREkbVva1Py3JXCJlVUziiF/hneXr0Am6CLXSH+/7n7a1Mu1ijqOOOasNJsTnOrRADLe2yj43mISl3QxAwY= X-Received: by 2002:ac8:5e4c:0:b0:46d:d452:a1a0 with SMTP id d75a77b69052e-46de96c545emr1001081cf.2.1736804882487; Mon, 13 Jan 2025 13:48:02 -0800 (PST) MIME-Version: 1.0 References: <20250106112103.25401-1-hao.ge@linux.dev> <48f208b6.32ab.19455c70dbe.Coremail.00107082@163.com> In-Reply-To: <48f208b6.32ab.19455c70dbe.Coremail.00107082@163.com> From: Suren Baghdasaryan Date: Mon, 13 Jan 2025 13:47:50 -0800 X-Gm-Features: AbW1kvY_qR92Y21e-WGgo7EiR8YzhSt4rAMueBCjnYoT8jUyKfHqgJjk4FlegqU Message-ID: Subject: Re: [PATCH] tools/mm: Introduce a tool to handle entries in allocinfo To: David Wang <00107082@163.com> Cc: kent.overstreet@linux.dev, Hao Ge , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Hao Ge , Alessio Balsini , Pasha Tatashin , Sourav Panda Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2F4E0180006 X-Stat-Signature: sfr3qbgwshrztr38hn7d8unwe8e4ztz7 X-HE-Tag: 1736804884-113479 X-HE-Meta: U2FsdGVkX19gwsfzPp9CJONjAhGzyKKGLfLC6GJePJ7c1K2Ry9xdP4tHwPppFo2JpP1Oto3cnr76NYNKpD7H0n82FMKc2cCNpZQ+wJfP46eQ8yl9zfmvMlumMxb+bVosV8FJLceSCFhWiCmhfupjEGFX3VkqPPwXOfvH3QGXS1VW6TjTz688cgby9i4JqNJihNi7iPeog2R27/8wx6Ke+EpLgTkEe5/huwzBj+qtkHhvIAByMlE2u9qV6f3KPSbtXM78JHsMyf92yTLeTDNU3Z4vmJtnYqfBBUT0nbV/qZfos9Wi66v4LhDtmM5Pss1yOHlz3EcBPsBwrPL2fZOHJoY3NlLutR4MwQTqDjH4fOzAK/y7gyxSB6/Q+bI/HQ1SNsESnOLisZQM9HfNG5CqL7XSjBL+lvEkxjIU1gEs5HGfXWL3P+Z26o5GKh0Mrsrs41gDza39+um7JyE093e+F6dwQyA7bWvDjbxVtHWj5IFutvk9W/KF8XuaERc3c0wWTRL7FtWGj72zNBt+Hq4qNdqDbOQjeSUQTp3ksXDuoZ/Kd+0ZmjzfiqJ05+NcRra9i9jWXKNu9WFBT77WxFzgp3qRcAb+aiE6PLt0XUe5painfjUsGd+gs/2qDddkZ+Kg6dLZaIpCQ/XaAx3w1DNDPPF+1uur8gYRuIOpyoe7G+yJ2I5RJn47FRqoh4lH6e+jiY/KWXV3iGa/GyiKJQyqZuxR3jLxIcukG+sE5FAVDOB+6mlfzchNKzhCDyQJSyZ+EqWLos3Rtn+nOVg1fGQ3fYey+p/hzReTHTAmnjf6EJof73HPGUWg9XOIS53YXKCKJRbMcP/jntT7+SpnpYnFyqrJ8HTLe6aS0wHPoxwus9oA6hF8hAaLTQqrcX/KLQRKAhj0wTuKTPcZN3SymVRGZuu2C2xWbq2XVlm7TDAPupIAR0C3cMpekeHxLysifRS/woby3H75I0hJmjfQzB8 WznKa2vz W37jChSDRifkKZ4hB3MbA3QBi2M5anoMA8kQYIJfVAj4Zq1e/IqhH9pEgjVAgifZ+H3F5vp4rlx91y/rEXNuZg4GVabCjb6lVCgdHmACUBH3zn/QxT8owsCNXSG9oCQddGqWsJsPBGAWL806zzYlpwHYtOj9EitU+0TWxCOOieu2b16whLLxoaadv3RBwjJZp88Hm8XMixAargg0O9IodSCpuS4OSSYR3fOC4aVR+OvWHEW1+GN5i/N7KriRd2HvaGk/6EEQaQInBWvpouXypU3L7hnuCEYNuPv+DgREQWLzgHfBURSrEXZzT4hSzvcAkvO4uRzWi+gjfbsppM/V7k/hhJEmwbEroEc6u X-Bogosity: Ham, tests=bogofilter, spamicity=0.080494, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Jan 11, 2025 at 6:32=E2=80=AFAM David Wang <00107082@163.com> wrote= : > > Hi, Hi David, Sorry for the delay. I'm not ignoring your input, I'm just a bit busy and didn't have time to properly reply to your questions. > > I have using this feature for a long while, and I believe this memory all= oc profiling feature > is quite powerful. > > But, I have been wondering how to use this data, specifically: > how anomaly could be detected, what pattern should be defined as anomaly? > > So far, I have tools collecting those data (via prometheus), make basic a= nalysis, i.e. top-k, group-by or rate. > Those analysis help me understand my system, but I cannot tell whether it= is abnormal or not. > > And sometimes I would just read through /proc/allocinfo, trying to pickup= something. > (Sometimes get lucky, actually only once, find the underflow problem week= s ago.) > > A tool would be more helpful if it can identify anomalies, and we can add= more pattern as develop along. You are absolutely correct. An automatic detection of problematic patterns would be the ultimate goal. We are analyzing the data we collect and trying to come up with strategies for identifying such patterns. Simple and obvious pattern for a leak would be constant growth but there might be others like sawtooth pattern or spikes which could point to opportunities to optimize the usage by employing object pools/caches. Categorizing allocations into hierarchical groups and measuring per-group consumption might be another useful technique we are considering. All this is in quite early stages, so ideas and suggestions from people using this API would be very valuable. > > A pattern may be hard to define, especially when it involves context. For= example, > I happened to notice following strange things recently: > > 896 14 kernel/sched/topology.c:2275 func:__sdt_alloc 1025 > 896 14 kernel/sched/topology.c:2266 func:__sdt_alloc 1025 > 96 6 kernel/sched/topology.c:2259 func:__sdt_alloc 1025 > 12288 24 kernel/sched/topology.c:2252 func:__sdt_alloc 1025 = <----- B > 0 0 kernel/sched/topology.c:2242 func:__sdt_alloc 210 > 0 0 kernel/sched/topology.c:2238 func:__sdt_alloc 210 > 0 0 kernel/sched/topology.c:2234 func:__sdt_alloc 210 > 0 0 kernel/sched/topology.c:2230 func:__sdt_alloc 210 = <----- A > Code A > 2230 sdd->sd =3D alloc_percpu(struct sched_domain *); > 2231 if (!sdd->sd) > 2232 return -ENOMEM; > 2233 > > Code B > 2246 for_each_cpu(j, cpu_map) { > ... > > 2251 > 2252 sd =3D kzalloc_node(sizeof(struct sched_doma= in) + cpumask_size(), > 2253 GFP_KERNEL, cpu_to_node(j)); > 2254 if (!sd) > 2255 return -ENOMEM; > 2256 > 2257 *per_cpu_ptr(sdd->sd, j) =3D sd; > > > The address of memory alloced by 'Code B', is stored in memory "Code A', = the allocation counter for 'Code A' > is *0*, while 'Code B' is not *0*. Something odd happens here, either it= is expected and some ownership changes happened somewhere > , or it is a leak, or it is an accounting problem. > > If a tool can help identify this kind of pattern, that would be great!~ Hmm. I don't see an easy way to identify such code dependencies from allocinfo data alone. I think that would involve some sophisticated code analysis tooling. > > > Any suggestions about how to proceed with the memory problem of kernel/sc= hed/topology.c mentioneded > above?, or is it a problem at all? >From your follow-up email, it looks like you already found the answer :) Thanks, Suren. > > > Thanks > David > > > > > At 2025-01-07 05:11:47, "Suren Baghdasaryan" wrote: > >On Mon, Jan 6, 2025 at 3:22=E2=80=AFAM Hao Ge wrote: > >> > >> From: Hao Ge > >> > >> Some users always say that the information provided by /proc/allocinfo > >> is too extensive or bulky. > >> > > > >CC'ing Alessio along with Pasha and Sourav who were interested in such a= tool. > > > >Hi Hao, > >Thanks for the tool! Actually Alessio just developed a tool called > >alloctop (similar to slabtop) which I think will do what you want and > >more. It supports sorting, filtering, continuous update, etc. It's > >written in Rust and we are planning to upstream it once we finish > >testing and evaluating it on Android. Please take a look and see if it > >fits your usecase. Please also note that this tool has been > >implemented just last week, so hot off the press and might have some > >early bugs. > >Thanks, > >Suren. > > > >[1] https://cs.android.com/android/platform/superproject/main/+/main:sys= tem/memory/libmeminfo/tools/alloctop/src/ > > > >> >