Re: [PATCH] tools/mm: Introduce a tool to handle entries in allocinfo

[Hao Ge <hao.ge@linux.dev>]

[-- Attachment #1: Type: text/plain, Size: 9603 bytes --]

Hi Suren and Alessio


Thanks for your information.


On 2025/1/7 23:11, Alessio Balsini wrote:
> Thank you Suren for looping me in and Hao for the tool!
>
> As Suren mentioned, we recently developed `alloctop` for the same reason as
> yours: easy and fast manipulation of `/proc/allocinfo` instead of having long
> chains of piped sorts, greps, awks...
> `alloctop` takes `slabtop` as a reference for command line parameters and
> output format.
> It currently supports sorting and filtering by minimum allocation size, as well
> as some kind of aggregation based on tag (similar to what `du` does).


Yes, I originally intended to gradually add such functionalities as well 
as those

discovered during subsequent usage.


> As Suren mentioned, the tool is currently being tested on Android, but we would
> be happy to collaborate on adding new features and making it part of tools/mm.


I'm also happy to do this.

I salute everyone who has put in the effort for this.

Make this tool increasingly user-friendly.

Thanks Best Regards Hao

>
> Thanks,
> Alessio
>
> On Mon, Jan 6, 2025 at 9:12 PM Suren Baghdasaryan<surenb@google.com>  wrote:
>> On Mon, Jan 6, 2025 at 3:22 AM Hao Ge<hao.ge@linux.dev>  wrote:
>>> From: Hao Ge<gehao@kylinos.cn>
>>>
>>> Some users always say that the information provided by /proc/allocinfo
>>> is too extensive or bulky.
>>>
>>> However, from allocinfo's own perspective, it is not at fault as it
>>> needs to provide comprehensive information. Users can then select the
>>> information that is useful to them based on their own needs.
>>>
>>> For commonly used scenarios, we can develop a tool to facilitate users in
>>> filtering and utilizing relevant information from allocinfo.
>>>
>>> Currently, there is only one filtering feature available, which is to
>>> filter out entries where the 'bytes' field is 0 (since it is often used
>>> to understand the current memory allocation status, as previously
>>> mentioned by David Wang, who noticed that 2/3 of the lines have an
>>> accumulative counter of 0, indicating no memory activities,it will fill
>>> up the entire screen,by using the "-s" parameter, a lot of unnecessary
>>> information for this scenario can be filtered out.)
>>>
>>> In subsequent phases, we will continue to add more features to this tool,
>>> with the goal of making it convenient for most people to use the memory
>>> allocation profiling tool.
>> CC'ing Alessio along with Pasha and Sourav who were interested in such a tool.
>>
>> Hi Hao,
>> Thanks for the tool! Actually Alessio just developed a tool called
>> alloctop (similar to slabtop) which I think will do what you want and
>> more. It supports sorting, filtering, continuous update, etc. It's
>> written in Rust and we are planning to upstream it once we finish
>> testing and evaluating it on Android. Please take a look and see if it
>> fits your usecase. Please also note that this tool has been
>> implemented just last week, so hot off the press and might have some
>> early bugs.
>> Thanks,
>> Suren.
>>
>> [1]https://cs.android.com/android/platform/superproject/main/+/main:system/memory/libmeminfo/tools/alloctop/src/
>>
>>> Signed-off-by: Hao Ge<gehao@kylinos.cn>
>>> ---
>>>   MAINTAINERS               |   1 +
>>>   tools/mm/Makefile         |   4 +-
>>>   tools/mm/allocinfo_tool.c | 150 ++++++++++++++++++++++++++++++++++++++
>>>   3 files changed, 153 insertions(+), 2 deletions(-)
>>>   create mode 100644 tools/mm/allocinfo_tool.c
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 910305c11e8a..cfc3f9f0c046 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -15000,6 +15000,7 @@ F:      Documentation/mm/allocation-profiling.rst
>>>   F:     include/linux/alloc_tag.h
>>>   F:     include/linux/pgalloc_tag.h
>>>   F:     lib/alloc_tag.c
>>> +F:     tools/mm/allocinfo_tool.c
>>>
>>>   MEMORY CONTROLLER DRIVERS
>>>   M:     Krzysztof Kozlowski<krzk@kernel.org>
>>> diff --git a/tools/mm/Makefile b/tools/mm/Makefile
>>> index f5725b5c23aa..f669d534a82b 100644
>>> --- a/tools/mm/Makefile
>>> +++ b/tools/mm/Makefile
>>> @@ -3,7 +3,7 @@
>>>   #
>>>   include ../scripts/Makefile.include
>>>
>>> -BUILD_TARGETS=page-types slabinfo page_owner_sort thp_swap_allocator_test
>>> +BUILD_TARGETS=page-types slabinfo page_owner_sort thp_swap_allocator_test allocinfo_tool
>>>   INSTALL_TARGETS = $(BUILD_TARGETS) thpmaps
>>>
>>>   LIB_DIR = ../lib/api
>>> @@ -23,7 +23,7 @@ $(LIBS):
>>>          $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
>>>
>>>   clean:
>>> -       $(RM) page-types slabinfo page_owner_sort thp_swap_allocator_test
>>> +       $(RM) page-types slabinfo page_owner_sort thp_swap_allocator_test allocinfo_tool
>>>          make -C $(LIB_DIR) clean
>>>
>>>   sbindir ?= /usr/sbin
>>> diff --git a/tools/mm/allocinfo_tool.c b/tools/mm/allocinfo_tool.c
>>> new file mode 100644
>>> index 000000000000..817f46d07a50
>>> --- /dev/null
>>> +++ b/tools/mm/allocinfo_tool.c
>>> @@ -0,0 +1,150 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * allocinfo_tool: Tool to parse allocinfo
>>> + *
>>> + * Authors: Hao Ge<hao.ge@linux.dev>
>>> + *
>>> + * Compile with:
>>> + * gcc -o allocinfo_tool allocinfo_tool.c
>>> + */
>>> +
>>> +#include<stdio.h>
>>> +#include<stdlib.h>
>>> +#include<sys/stat.h>
>>> +#include <string.h>
>>> +#include <getopt.h>
>>> +
>>> +#define ALLOCINFO_FILE "/proc/allocinfo"
>>> +#define BUF_SIZE 1024
>>> +#define NAME_MAX_LENTH 64
>>> +
>>> +struct alloc_tag_counters {
>>> +       signed long long bytes;
>>> +       unsigned long long calls;
>>> +};
>>> +
>>> +struct codetag {
>>> +       unsigned int lineno;
>>> +       char modname[NAME_MAX_LENTH];
>>> +       char function[NAME_MAX_LENTH];
>>> +       char filename[NAME_MAX_LENTH];
>>> +};
>>> +
>>> +struct alloc_info {
>>> +       struct alloc_tag_counters counters;
>>> +       struct codetag tag;
>>> +       int has_modname;
>>> +};
>>> +
>>> +static int arg_opt;
>>> +
>>> +enum OPT_BIT {
>>> +       SKIP_ZERO = 1 << 0,
>>> +};
>>> +
>>> +void usage(void)
>>> +{
>>> +       printf("Usage: ./allocinfo_tool [OPTIONS]\n"
>>> +              "-s\t\t\tskip bytes for 0 allocinfo entries\n"
>>> +       );
>>> +}
>>> +
>>> +int parse_alloc_info(char *line, struct alloc_info *info)
>>> +{
>>> +       if (sscanf(line, "%12lli %8llu %[^:]:%d [%[^]]] func:%s",
>>> +                     &info->counters.bytes, &info->counters.calls,
>>> +                     info->tag.filename, &info->tag.lineno,
>>> +                     info->tag.modname, info->tag.function) == 6){
>>> +               info->has_modname = 1;
>>> +               return 1;
>>> +       };
>>> +
>>> +       if (sscanf(line, "%12llu %8llu %[^:]:%u func:%s",
>>> +                  &info->counters.bytes, &info->counters.calls,
>>> +                  info->tag.filename, &info->tag.lineno,
>>> +                  info->tag.function) == 5){
>>> +               info->has_modname = 0;
>>> +               return 1;
>>> +       }
>>> +
>>> +       return 0;
>>> +}
>>> +
>>> +int read_alloc_info(void)
>>> +{
>>> +       FILE *file = fopen(ALLOCINFO_FILE, "r");
>>> +
>>> +       if (!file) {
>>> +               perror("Failed to open /proc/allocinfo");
>>> +               return EXIT_FAILURE;
>>> +       }
>>> +
>>> +       int line = 0, i = 0;
>>> +       char *buffer = malloc(BUF_SIZE);
>>> +       struct alloc_info *info;
>>> +
>>> +       while (fgets(buffer, BUF_SIZE, file)) {
>>> +
>>> +               /*
>>> +                * allocinfo - version: 1.0
>>> +                * #     <size>  <calls> <tag info>
>>> +                */
>>> +               if (line < 2) {
>>> +                       printf("%s", buffer);
>>> +                       line++;
>>> +                       continue;
>>> +               }
>>> +
>>> +               info = realloc(info, sizeof(struct alloc_info) * (i + 1));
>>> +
>>> +               if (parse_alloc_info(buffer, info + i) == 0) {
>>> +                       printf("Mismatch with the format of /proc/allocinfo");
>>> +                       return 0;
>>> +               }
>>> +
>>> +               if ((arg_opt & SKIP_ZERO) && (info[i].counters.bytes == 0))
>>> +                       continue;
>>> +
>>> +               printf("%12lli %8llu ", info[i].counters.bytes, info[i].counters.calls);
>>> +
>>> +               if (info[i].has_modname)
>>> +                       printf("%s:%u [%s] func:%s",
>>> +                              info[i].tag.filename, info[i].tag.lineno,
>>> +                              info[i].tag.modname, info[i].tag.function);
>>> +               else
>>> +                       printf("%s:%u func:%s",
>>> +                              info[i].tag.filename, info[i].tag.lineno,
>>> +                              info[i].tag.function);
>>> +               printf(" ");
>>> +               printf("\n");
>>> +               i++;
>>> +       }
>>> +
>>> +       free(info);
>>> +       free(buffer);
>>> +       fclose(file);
>>> +}
>>> +
>>> +int main(int argc, char *argv[])
>>> +{
>>> +
>>> +       int opt;
>>> +       struct option longopts[] = {
>>> +               { "s", 0, NULL, 1},
>>> +               { 0, 0, 0, 0},
>>> +       };
>>> +
>>> +       while ((opt = getopt_long(argc, argv, "s", longopts, NULL)) != -1) {
>>> +               switch (opt) {
>>> +               case 's':
>>> +                       arg_opt = arg_opt | SKIP_ZERO;
>>> +                       break;
>>> +               default:
>>> +                       usage();
>>> +                       exit(1);
>>> +               }
>>> +       }
>>> +
>>> +       read_alloc_info();
>>> +
>>> +}
>>> --
>>> 2.25.1
>>>

[-- Attachment #2: Type: text/html, Size: 17117 bytes --]