From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75E38E77197 for ; Tue, 7 Jan 2025 15:11:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEA876B0088; Tue, 7 Jan 2025 10:11:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D993F6B00B3; Tue, 7 Jan 2025 10:11:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C12FE6B00C2; Tue, 7 Jan 2025 10:11:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9B34E6B0088 for ; Tue, 7 Jan 2025 10:11:19 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 08C551A0A51 for ; Tue, 7 Jan 2025 15:11:19 +0000 (UTC) X-FDA: 82980994278.18.4F9DE53 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf05.hostedemail.com (Postfix) with ESMTP id A029F100017 for ; Tue, 7 Jan 2025 15:11:16 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=android.com header.s=20230601 header.b=hmEJ3NNi; spf=pass (imf05.hostedemail.com: domain of balsini@android.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=balsini@android.com; dmarc=pass (policy=reject) header.from=android.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736262676; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rthohaTiXaUEMXyOIF/3gdLlOjHPZ2+wmeo71X27bAQ=; b=Ebb2Wf7ime3YOoeKVssA5Mlm9JA+sPtt1/G08VUXRzpjriji/KgAi6/k4YFj9sBnS+L/T9 FRi+sKY0V5EvJf3hYW3OLC6BAjkyWi2O+1Nq6lx5U8JYx4Ytq7+qVuNw2I2xW4hB9+ivba 189sxOzgS4Nm9X39oq+Z5pgcfkFC8e8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=android.com header.s=20230601 header.b=hmEJ3NNi; spf=pass (imf05.hostedemail.com: domain of balsini@android.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=balsini@android.com; dmarc=pass (policy=reject) header.from=android.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736262676; a=rsa-sha256; cv=none; b=8BF+fYcjByJwGpdtI87k5ZnHDug/ieOlwQqmS/1axm/gsHpSSpq2ZoB6/9NfakgllLIrYW Xd9JoQGyQgzlh3sczzUARXsCNBpQQPlhtv+vVejIgXVZvGhlvqAw5q1sz54lj+InHfwT3u FIv1/IRg/cPMxv2xx0Dknl8Bj7D6zjY= Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5d3e6274015so27721142a12.0 for ; Tue, 07 Jan 2025 07:11:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20230601; t=1736262675; x=1736867475; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=rthohaTiXaUEMXyOIF/3gdLlOjHPZ2+wmeo71X27bAQ=; b=hmEJ3NNiv9yjoar1XWeusDxZq0UPj5F8IZr0CPt/J52tVg+pM+T4nc/P/Fb1xgm1Tu mTj7PKtBfBImapiawuYWi4JKq/sdtXXe6k6SrqFKMKsrAIa1Uqa8je9XqrGlywqoxym2 nnXn+hUKsN/1+bzUePzR0SnrIG6pSuHsB1lSgy+1Qu+92hyoW6PL6rWVXyXmwqmaH3WE wskYHxZkaGGJuL4z/e++YC4/DeCHdKf7mYptk4NiGpF+YTg9cY+AGcnkctlHEQkJOIuO 5G0ErxlPa9Cyb1sMxio9KQ0OA7Mx+X7+6Q1rb9Jw3rTRLwDcu048ya642RkpbMaxt07q shAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736262675; x=1736867475; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rthohaTiXaUEMXyOIF/3gdLlOjHPZ2+wmeo71X27bAQ=; b=iiDt4suCP1BQXkZGEpwsjKitY0+nHKXitdzfHyGvoT/86XdTaK7Fl6ukKVmJ2s2VP9 jQDFzSo0T7ABY8nsEpBB5EYOYiSiapgPrPQI4XkMnDERKFZaHTulBRFemZcYA6I9c5rF xSt80nz9KCTY17CHb4EWMYGbUe0QUHLBxr4A+oJpHGEE4VsYCjcTDGzsknNuk7yOTtZR oB29X0qMi+6xtV1jQpb3o7YH6VKX7aDZMy/gLmg4DWLi27wOUT9mMYUrX1aGPXhEQS9o 1GEAxUb9CTwBYS1/dOfN36sR+7I8Y3pdm14lYGOLRcN9p+cq2z/XEy0tVsGSFtxrR0Ds 8I7w== X-Forwarded-Encrypted: i=1; AJvYcCVMHhyd8/Icy4jtM/uHLGwGH6RTaEvrqrTTmOuOwriOVIIVXcJL5ZCjnDWlPkYPCfKO0oEvLGZgvw==@kvack.org X-Gm-Message-State: AOJu0YzzzpdzWwFmprPHNxKcsKUcOq3fORdSUy+Cc8/x1IZJJjdCOlZb d8hvzZbpD1mqf1Uy236gZPd7QWI+47darMH20YLWcwEmUbHFRVl3YunBYJ5aslE919bL6ani8RT XPDphe3xi0V0t56mmhaWaZhjqyJYRV06R3hd1 X-Gm-Gg: ASbGncszhZAM3yFyZWxX/ZAanQnQc4CfymjeV2uVAMZEHB4OfIWGnq6q+gEmAUnxgJd 3L9iDkUycFJZ9fQPJr0jy5xKOExEV54kywPRa6ifsH3j/SnB2HJEf4u7VIO0GV+Rerw== X-Google-Smtp-Source: AGHT+IFbtE3QBoOYQ0mMCF3j89UXsF0hIdif5UfePei0EPcvBluBMkfJaSpaiKa+EY23zkTvDeWWgtJDSyMXtEPA/hg= X-Received: by 2002:a05:6402:5241:b0:5d0:e826:f0dc with SMTP id 4fb4d7f45d1cf-5d81dd9cc74mr53212592a12.12.1736262674460; Tue, 07 Jan 2025 07:11:14 -0800 (PST) MIME-Version: 1.0 References: <20250106112103.25401-1-hao.ge@linux.dev> In-Reply-To: From: Alessio Balsini Date: Tue, 7 Jan 2025 15:11:03 +0000 X-Gm-Features: AbW1kvZOQTAnGw5anrCJtUOkQt4yMLh10HaUbOK_S7_Yu8jigCP5E07FT1gjugc Message-ID: Subject: Re: [PATCH] tools/mm: Introduce a tool to handle entries in allocinfo To: Suren Baghdasaryan Cc: Hao Ge , akpm@linux-foundation.org, kent.overstreet@linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Hao Ge , Alessio Balsini , Pasha Tatashin , Sourav Panda , David Wang <00107082@163.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A029F100017 X-Rspamd-Server: rspam12 X-Stat-Signature: 3j3e1ohimdnwauy8t9acjygmmup35nw7 X-Rspam-User: X-HE-Tag: 1736262676-339180 X-HE-Meta: U2FsdGVkX1/MlWHWYmjRIVccv4hhzUmzOB+DzIdF+edt0/QRejHlVyklgxHLFGEWhfhY+ne7/BQkQvT6u3YeZcpNjFzpRLSf943O+L56eVVACTgZuSt+DQ+Oa3Q4bUmybhOwq8GDk1Svrp+M6LyawxQ7XT0i3nfHlPEFi0ATY/Wt+1NSYIspAoByWGGKLqWgf3vUwP23J61Gkv0a0ypFkmj1bhaDbI0JVA5ekabLxATTJntUvNPaTKZTtBYYOm/d+1pqmKnKTfFDoARbKh+9bU3zNOM8+sB5YjltNNUp7NXAWiROBQigaF94XUbIKw4eTL6CCxfyfHpRyUKpqPLPEfzMEJznnJODG0j4nQ0qtlIKWxi714+TipEoaXpSAn3j3k5kEyfYT0t2hfCxxDeIuE4dcXmr/4R7eT/s3Oyp+mwt73zFR0Y0HG/FaLi4B/0iQmwqhNo+XemEqk9ZWs30WgEDo0/l13lH/sjwzJp07OrQ4udLRb/RP430R+7JWBv0qIVypKPopaHAmCGu+ASiRWS1c70KtrnAdAHO7zxasK7+2vSMhY76MPsXHQM6et/gGHvFrLoUCPTUpFwqU/uB5/4LEKRLD1cj5C2NeT/vJwS0kpa7rSH+x2NtLygKrZzlE6qqcBf8LkJUpoCJX/d+lzTMxifWUSfdf7MQYklmYqJpT4cTPwEbglKiuvTjBGiXd5esbRBjTQZ5smwbf+iyC+P05i2lrkzRTm+jtcsxov70/rEXHUFL/Lj7Ca4+C+SQNqI3t9GI596BgB0P422gTnYIQAbGbh8KrBjSfEwi1QMZGCbyPEGUCSdG6eAW6o8loi9s3iOGy6BCI+LxYhTDwFYzMCdc03vAISnWqflD84zwOSFiApf3s9wpY2P3X0DJ5zxwkAyr8IkzaqHk++FTytzWxkQMtMbWxUekp0Vo6am0vXW8cJTEFtR+trm9HyBInzye9IgeoS6m0dAbdk0 AgErEB2T M2x2OJYxtrBZNP3SGt+vjuOv4qRMvnAKq1wMMsm3a8YyG69BY1nk3g/dlHi6p0kiHFaSaHuTUQIhW3uvLhl4LuGFAKDfVdYs+OJHzT6KhUOZwDXe8O1NhbALzvKFAE6qsEmo9Rw5ko1kLUCdIebWNqreSVJszdme1GPjSBPYKndlJbm5C0058g46TeCvkKeUZ/hetjxDo+yVX6IOCxvnJLxvGC12W/qIRrkmem9PzGPcW9g4233/zI06ItL1mh2l2q3UvW9nVcRWDDpA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Thank you Suren for looping me in and Hao for the tool! As Suren mentioned, we recently developed `alloctop` for the same reason as yours: easy and fast manipulation of `/proc/allocinfo` instead of having lo= ng chains of piped sorts, greps, awks... `alloctop` takes `slabtop` as a reference for command line parameters and output format. It currently supports sorting and filtering by minimum allocation size, as = well as some kind of aggregation based on tag (similar to what `du` does). As Suren mentioned, the tool is currently being tested on Android, but we w= ould be happy to collaborate on adding new features and making it part of tools/= mm. Thanks, Alessio On Mon, Jan 6, 2025 at 9:12=E2=80=AFPM Suren Baghdasaryan wrote: > > On Mon, Jan 6, 2025 at 3:22=E2=80=AFAM Hao Ge wrote: > > > > From: Hao Ge > > > > Some users always say that the information provided by /proc/allocinfo > > is too extensive or bulky. > > > > However, from allocinfo's own perspective, it is not at fault as it > > needs to provide comprehensive information. Users can then select the > > information that is useful to them based on their own needs. > > > > For commonly used scenarios, we can develop a tool to facilitate users = in > > filtering and utilizing relevant information from allocinfo. > > > > Currently, there is only one filtering feature available, which is to > > filter out entries where the 'bytes' field is 0 (since it is often used > > to understand the current memory allocation status, as previously > > mentioned by David Wang, who noticed that 2/3 of the lines have an > > accumulative counter of 0, indicating no memory activities,it will fill > > up the entire screen,by using the "-s" parameter, a lot of unnecessary > > information for this scenario can be filtered out.) > > > > In subsequent phases, we will continue to add more features to this too= l, > > with the goal of making it convenient for most people to use the memory > > allocation profiling tool. > > CC'ing Alessio along with Pasha and Sourav who were interested in such a = tool. > > Hi Hao, > Thanks for the tool! Actually Alessio just developed a tool called > alloctop (similar to slabtop) which I think will do what you want and > more. It supports sorting, filtering, continuous update, etc. It's > written in Rust and we are planning to upstream it once we finish > testing and evaluating it on Android. Please take a look and see if it > fits your usecase. Please also note that this tool has been > implemented just last week, so hot off the press and might have some > early bugs. > Thanks, > Suren. > > [1] https://cs.android.com/android/platform/superproject/main/+/main:syst= em/memory/libmeminfo/tools/alloctop/src/ > > > > > Signed-off-by: Hao Ge > > --- > > MAINTAINERS | 1 + > > tools/mm/Makefile | 4 +- > > tools/mm/allocinfo_tool.c | 150 ++++++++++++++++++++++++++++++++++++++ > > 3 files changed, 153 insertions(+), 2 deletions(-) > > create mode 100644 tools/mm/allocinfo_tool.c > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 910305c11e8a..cfc3f9f0c046 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -15000,6 +15000,7 @@ F: Documentation/mm/allocation-profiling.r= st > > F: include/linux/alloc_tag.h > > F: include/linux/pgalloc_tag.h > > F: lib/alloc_tag.c > > +F: tools/mm/allocinfo_tool.c > > > > MEMORY CONTROLLER DRIVERS > > M: Krzysztof Kozlowski > > diff --git a/tools/mm/Makefile b/tools/mm/Makefile > > index f5725b5c23aa..f669d534a82b 100644 > > --- a/tools/mm/Makefile > > +++ b/tools/mm/Makefile > > @@ -3,7 +3,7 @@ > > # > > include ../scripts/Makefile.include > > > > -BUILD_TARGETS=3Dpage-types slabinfo page_owner_sort thp_swap_allocator= _test > > +BUILD_TARGETS=3Dpage-types slabinfo page_owner_sort thp_swap_allocator= _test allocinfo_tool > > INSTALL_TARGETS =3D $(BUILD_TARGETS) thpmaps > > > > LIB_DIR =3D ../lib/api > > @@ -23,7 +23,7 @@ $(LIBS): > > $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) > > > > clean: > > - $(RM) page-types slabinfo page_owner_sort thp_swap_allocator_te= st > > + $(RM) page-types slabinfo page_owner_sort thp_swap_allocator_te= st allocinfo_tool > > make -C $(LIB_DIR) clean > > > > sbindir ?=3D /usr/sbin > > diff --git a/tools/mm/allocinfo_tool.c b/tools/mm/allocinfo_tool.c > > new file mode 100644 > > index 000000000000..817f46d07a50 > > --- /dev/null > > +++ b/tools/mm/allocinfo_tool.c > > @@ -0,0 +1,150 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > +/* > > + * allocinfo_tool: Tool to parse allocinfo > > + * > > + * Authors: Hao Ge > > + * > > + * Compile with: > > + * gcc -o allocinfo_tool allocinfo_tool.c > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#define ALLOCINFO_FILE "/proc/allocinfo" > > +#define BUF_SIZE 1024 > > +#define NAME_MAX_LENTH 64 > > + > > +struct alloc_tag_counters { > > + signed long long bytes; > > + unsigned long long calls; > > +}; > > + > > +struct codetag { > > + unsigned int lineno; > > + char modname[NAME_MAX_LENTH]; > > + char function[NAME_MAX_LENTH]; > > + char filename[NAME_MAX_LENTH]; > > +}; > > + > > +struct alloc_info { > > + struct alloc_tag_counters counters; > > + struct codetag tag; > > + int has_modname; > > +}; > > + > > +static int arg_opt; > > + > > +enum OPT_BIT { > > + SKIP_ZERO =3D 1 << 0, > > +}; > > + > > +void usage(void) > > +{ > > + printf("Usage: ./allocinfo_tool [OPTIONS]\n" > > + "-s\t\t\tskip bytes for 0 allocinfo entries\n" > > + ); > > +} > > + > > +int parse_alloc_info(char *line, struct alloc_info *info) > > +{ > > + if (sscanf(line, "%12lli %8llu %[^:]:%d [%[^]]] func:%s", > > + &info->counters.bytes, &info->counters.calls, > > + info->tag.filename, &info->tag.lineno, > > + info->tag.modname, info->tag.function) =3D=3D 6){ > > + info->has_modname =3D 1; > > + return 1; > > + }; > > + > > + if (sscanf(line, "%12llu %8llu %[^:]:%u func:%s", > > + &info->counters.bytes, &info->counters.calls, > > + info->tag.filename, &info->tag.lineno, > > + info->tag.function) =3D=3D 5){ > > + info->has_modname =3D 0; > > + return 1; > > + } > > + > > + return 0; > > +} > > + > > +int read_alloc_info(void) > > +{ > > + FILE *file =3D fopen(ALLOCINFO_FILE, "r"); > > + > > + if (!file) { > > + perror("Failed to open /proc/allocinfo"); > > + return EXIT_FAILURE; > > + } > > + > > + int line =3D 0, i =3D 0; > > + char *buffer =3D malloc(BUF_SIZE); > > + struct alloc_info *info; > > + > > + while (fgets(buffer, BUF_SIZE, file)) { > > + > > + /* > > + * allocinfo - version: 1.0 > > + * # > > + */ > > + if (line < 2) { > > + printf("%s", buffer); > > + line++; > > + continue; > > + } > > + > > + info =3D realloc(info, sizeof(struct alloc_info) * (i += 1)); > > + > > + if (parse_alloc_info(buffer, info + i) =3D=3D 0) { > > + printf("Mismatch with the format of /proc/alloc= info"); > > + return 0; > > + } > > + > > + if ((arg_opt & SKIP_ZERO) && (info[i].counters.bytes = =3D=3D 0)) > > + continue; > > + > > + printf("%12lli %8llu ", info[i].counters.bytes, info[i]= .counters.calls); > > + > > + if (info[i].has_modname) > > + printf("%s:%u [%s] func:%s", > > + info[i].tag.filename, info[i].tag.lineno= , > > + info[i].tag.modname, info[i].tag.functio= n); > > + else > > + printf("%s:%u func:%s", > > + info[i].tag.filename, info[i].tag.lineno= , > > + info[i].tag.function); > > + printf(" "); > > + printf("\n"); > > + i++; > > + } > > + > > + free(info); > > + free(buffer); > > + fclose(file); > > +} > > + > > +int main(int argc, char *argv[]) > > +{ > > + > > + int opt; > > + struct option longopts[] =3D { > > + { "s", 0, NULL, 1}, > > + { 0, 0, 0, 0}, > > + }; > > + > > + while ((opt =3D getopt_long(argc, argv, "s", longopts, NULL)) != =3D -1) { > > + switch (opt) { > > + case 's': > > + arg_opt =3D arg_opt | SKIP_ZERO; > > + break; > > + default: > > + usage(); > > + exit(1); > > + } > > + } > > + > > + read_alloc_info(); > > + > > +} > > -- > > 2.25.1 > > >