From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 525D1E77187 for ; Wed, 18 Dec 2024 18:23:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD22E6B007B; Wed, 18 Dec 2024 13:23:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A82B56B0082; Wed, 18 Dec 2024 13:23:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 970F16B0083; Wed, 18 Dec 2024 13:23:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 79E186B007B for ; Wed, 18 Dec 2024 13:23:07 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 043E380C46 for ; Wed, 18 Dec 2024 18:23:06 +0000 (UTC) X-FDA: 82908901278.14.406A96E Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf17.hostedemail.com (Postfix) with ESMTP id C0E3240018 for ; Wed, 18 Dec 2024 18:22:40 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TeOD2kOa; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of surenb@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734546170; a=rsa-sha256; cv=none; b=bn6zZ/6Af7JigDTW8KWxO+PtWW+07c7jBXMXvdAV+7qO+DhAazZ4pTkJdJ2Yfjfp1BlMhQ LC+m+RqpJief45gbdjh5P5w8hMW6KYHpLxgOZnfCa+TswV1lRg1lXoW9sSjXTWWyFJ+sZL rzLe5F/Xjzr5UlGA6GFvRciHJhRaNpA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TeOD2kOa; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of surenb@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734546170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W0A+dpFjHjp2zWph7U8eNrtq5+sexdo+5EisfkZYoyQ=; b=gqrsGqJbYOVCwyX7QBEUqyF6n6JsODc4mULEvb4HaAgT3iYGAo1RokYFUQnScneJVPbUw+ SLtDZ6lAihb1CNWmjALWkchYcu/k6Xm8AJGzxL4v5CBCJWDtEh2xwRXXwRdQCPlIIsaBIT NQjMcSZiro2CXOZ2ENU09S7WSN3tbf0= Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-467896541e1so11481cf.0 for ; Wed, 18 Dec 2024 10:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1734546184; x=1735150984; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=W0A+dpFjHjp2zWph7U8eNrtq5+sexdo+5EisfkZYoyQ=; b=TeOD2kOaTtq2sJgotbGc4tgtrlQo1p62LmzCswqVKsCS5ffP6ClUnNoVmtOEjUPkzc 45Hj52Jstej7CwQchvtfnNaBRqe5InXBiz5jqG18+eWeSF0Vfug0Nvmiyodbmmfoh65w z3xIbe0VcpCqUMpmsz/DQzzPL2LV8z606qVFueLceZsBAKXTK0x+Gnjq5hnS90taUqqg 3xCjMRwz2dfWWwaN7yxFlhU3eQEIObTCOvuo1ZJwrGbXS81XtXxnCbHcsg6fjIudvEwu 2HdBxbZa+xr2ItkkPO3Q1rfauoQ8D1UCvEovIA2LzXddVySfE9NbDEQtUuUEIvk5Mnts 9COQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734546184; x=1735150984; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W0A+dpFjHjp2zWph7U8eNrtq5+sexdo+5EisfkZYoyQ=; b=fRcQu33TYm1t2EWiFqyq3BWqrUlFKf4myVXaTd4My4xH8NdfAdC+xZ41OxfsOGVM/6 m3Cr7kkUGpVSAUG4vcDnsE5kq8ZzoiVHu73CGwvfP20gujXmGyzaOKG4XjAP1YGb8Rtb 8+BkWdiwiB5GtQXTHhdqWFvLPiH+yXSQQWuH8I2gV9vd9Cm2Gu5BL//UlzEYMPSXFrcA VtfVXhjgoL9dnK/lRwgX2GIFwQiHPZVzmmTexpMvvufOIHA1KvkiMqoGJZv8bmk+zA2K fs61r/LLeybMV9QNnmyIhJ2QzcIF8emVBhbtLibs9J88yO0NgdXoU5VGoWN9PEWhfcjS AqKg== X-Forwarded-Encrypted: i=1; AJvYcCVuZsDNZRAGkckLABKq3Sfc87rbaLBtc99cEMvGwCjxPP6kdpLaf1oANsjpWmJYR19wyM/qGgxK6g==@kvack.org X-Gm-Message-State: AOJu0Yynjx7PczLPU+R/AsquaoWisgYbohg1Lp9TO0xJgKqOtqQcZh/G Hpghe8DFrgHHmYZpOXu21Z7gwDfHdPf62ad7JzYON3Iv071LL99CAtDq2P7MwWtd96YBWFoVA4P a83LJ5T1IX3VtgkkKQl/BwTb7LjZ97Jl4xBDj X-Gm-Gg: ASbGncsJMqiOxQ16HqQ2BDNSADrOKmZS8+B3fRlx5pI9BOXe4n3Sq7HMMPTa/ax1l2Z o9acwzAuqty3dQCHmgmjeMYnIaTMlqnPkFM3FFEKSyJ2XWxLiWlp541IKTM/0dbeF28v5 X-Google-Smtp-Source: AGHT+IHPZKSwx7QRa5wtSQCvyk5nuu54w3JKnzfJJFuw+5pCi8ZgohU/2vi1K5O3rhOjLF48x4btPwIcDJfaa69UAUo= X-Received: by 2002:ac8:7c46:0:b0:466:90dd:19f3 with SMTP id d75a77b69052e-46935520de4mr3420281cf.7.1734546184036; Wed, 18 Dec 2024 10:23:04 -0800 (PST) MIME-Version: 1.0 References: <20240913055729.7208-1-00107082@163.com> <710f85e7.a802.193d9d08d8a.Coremail.00107082@163.com> In-Reply-To: <710f85e7.a802.193d9d08d8a.Coremail.00107082@163.com> From: Suren Baghdasaryan Date: Wed, 18 Dec 2024 10:22:53 -0800 Message-ID: Subject: Re: [PATCH v2] lib/alloc_tag: Add accumulative call counter for memory allocation profiling To: David Wang <00107082@163.com> Cc: kent.overstreet@linux.dev, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C0E3240018 X-Stat-Signature: 6odnzp7iupdm6hwkmjecfrz9s5fzo13t X-HE-Tag: 1734546160-410995 X-HE-Meta: U2FsdGVkX1+F/34XWFCyU1Ib0qFC2gYTCFPu5+taQIgZKZnpIIPBvJ3wnllJCBiUoyl6EX9ZpEUtlPnvPtl5tmNgmGgrX7JWBcYCHGmwtkqXl7BLp13X/UDEufR2prDDTshjeHbrH1jvRAQ3vqpNr0ZtXF1T2gmdg0bMnVGymGlprQc/awD319VZgMnXjP8Ag1DyytJyvpSIGa+6xg5TiyNaeVYtDN1lHipEPAUqHms5MuBEX9cz+IQ4bdvdFrw39Jhujur4eVT7P/AVeZ0YmVDJ+J7OL1O2VA2kUDPDgm4noaaziTFCij2WbItPca+kZ2zGBrvw4y5EQbCVgf+DeNzs+NtptX+C8tbn56yr4Jf4vAvUGTYpHxe7j1C70ISikKU1D4wkFUrYapc7CVWPD2rFqE6K/B/zZh3Xkj7w2NEPBf9uh77PvlG/Si0lvjCINNdHc18Qh4KFGvgl38SmpGCKG/45bq/n2pjzju5iQt4ULiLuYlIG6HBQgqa5USs9Zcngmjny0r367ejoWI+J2KXBq7CPQhOMpCY3Qd3Z4vqnRGoZZq23hbTEcbnH0UZtk4NQNw5rU1689EhiMVE8zJEnbC70AAAdDz559AaJFpvh+skSSlQxr3rrqBgVe0AzdxwAWfPS/Vjy/Hh1keOB+Cqe1EeVzoxKFswk/m8bS7wuTPL7VYG+9HMKCS3zA6YuC1YhiD3erwpYKaSFYOjd0O4EI/1ULHv+0kCu7iixAXm0YTYnKUre5QU5jZrpf481HDEOQRKw4IWR8K1zBQ42rQsBqz+gO/gR0mR1PBHkXLK3zphJHfxqYH0BvDVcbOFuw2g5/qjkAwOGr2nKgBFXr2gGm3D4h+Hphln+yPtYAuUg6fb5jSfg10jdV94GIcnsJKiZbGP0aB6cEIJcimyKrNs5Xvp+rRRJQkGKEbjqfW5bn1CNHnD5B+PJ/pS008musUSY5ZM+0ER5iNbeOzM 85FSR4R2 M482N5LnlhYl2hnQjxKvSjIBf7lsi6ZqWoxpr3SSmH9YKhxli0drUH8+tSd4bSErFVgExf0lnDEVviU8AxgbgG+A9Cn3MKUeOBdSl+kxaj4v6rua2GFeuu5s458yWBnKE6+W032g/zpID0SeDMWJJ/yhBdJlh/IKgYn7qN0KHJr7mWi26yMseRaNU9POtGAll3l1K2B3kTLSjvrns7KhOYjhgW5/T0yqPDkVj5SvLCR11z9hDH4twtickEAfFTfML1BiY7LSLAlh01XVY5jGJh2X3uK5xizRLaHPQ/k9EXV6growx/2xTvPXw30HOJHYJPnSNOUf4ZS53VNFTM1kNbrboFMRHCWK0MUL21puoayvJz+gn5vSrJT6VLsv5QHbu2TtITJdcmVkqO3+LKulRc0/gwuXEepwvYA+fcaUhyiNZga0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.244373, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Dec 18, 2024 at 4:49=E2=80=AFAM David Wang <00107082@163.com> wrote= : > > Hi, > > I found another usage/benefit for accumulative counters: > > On my system, /proc/allocinfo yields about 5065 lines, of which 2/3 line= s have accumulative counter *0*. > meaning no memory activities. (right?) > It is quite a waste to keep those items which are *not alive yet*. > With additional changes, only 1684 lines in /proc/allocinfo on my system: > > --- a/lib/alloc_tag.c > +++ b/lib/alloc_tag.c > @@ -95,8 +95,11 @@ static void alloc_tag_to_text(struct seq_buf *out, str= uct codetag *ct) > struct alloc_tag_counters counter =3D alloc_tag_read(tag); > s64 bytes =3D counter.bytes; > > + if (counter.accu_calls =3D=3D 0) > + return; > seq_buf_printf(out, "%12lli %8llu ", bytes, counter.calls); > > > I think this is quite an improvement worth pursuing. > (counter.calls could also be used to filter out "inactive" items, but > lines keep disappearing/reappearing can confuse monitoring systems.) Please see discussion at https://lore.kernel.org/all/20241211085616.2471901-1-quic_zhenhuah@quicinc.= com/ > > > Thanks~ > David > > > At 2024-09-13 13:57:29, "David Wang" <00107082@163.com> wrote: > >Accumulative allocation counter can be used to evaluate > >memory allocation behavior/rate via delta(counters)/delta(time). > >It can help analysis performance issues, identify top modules > >with high rate of memory allocation activity. > >Considering this would incur extra performance and memory impact, > >introduce kconfig MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER. > > > >Signed-off-by: David Wang <00107082@163.com> > >--- > >Changes in v2: > >- Add kconfig MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER as > >suggested by "Kent Overstreet " > > > >--- > > include/linux/alloc_tag.h | 18 +++++++++++++++--- > > lib/Kconfig.debug | 10 ++++++++++ > > lib/alloc_tag.c | 10 +++++++++- > > 3 files changed, 34 insertions(+), 4 deletions(-) > > > >diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h > >index 8c61ccd161ba..5a94d61205b5 100644 > >--- a/include/linux/alloc_tag.h > >+++ b/include/linux/alloc_tag.h > >@@ -18,6 +18,9 @@ > > struct alloc_tag_counters { > > u64 bytes; > > u64 calls; > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ u64 accu_calls; > >+#endif > > }; > > > > /* > >@@ -103,13 +106,19 @@ static inline bool mem_alloc_profiling_enabled(voi= d) > > static inline struct alloc_tag_counters alloc_tag_read(struct alloc_tag= *tag) > > { > > struct alloc_tag_counters v =3D { 0, 0 }; > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ v.accu_calls =3D 0; > >+#endif > > struct alloc_tag_counters *counter; > > int cpu; > > > > for_each_possible_cpu(cpu) { > >- counter =3D per_cpu_ptr(tag->counters, cpu); > >- v.bytes +=3D counter->bytes; > >- v.calls +=3D counter->calls; > >+ counter =3D per_cpu_ptr(tag->counters, cpu); > >+ v.bytes +=3D counter->bytes; > >+ v.calls +=3D counter->calls; > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ v.accu_calls +=3D counter->accu_calls; > >+#endif > > } > > > > return v; > >@@ -145,6 +154,9 @@ static inline void __alloc_tag_ref_set(union codetag= _ref *ref, struct alloc_tag > > * counter because when we free each part the counter will be dec= remented. > > */ > > this_cpu_inc(tag->counters->calls); > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ this_cpu_inc(tag->counters->accu_calls); > >+#endif > > } > > > > static inline void alloc_tag_ref_set(union codetag_ref *ref, struct all= oc_tag *tag) > >diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > >index a30c03a66172..1e9974d28510 100644 > >--- a/lib/Kconfig.debug > >+++ b/lib/Kconfig.debug > >@@ -1000,6 +1000,16 @@ config MEM_ALLOC_PROFILING_DEBUG > > Adds warnings with helpful error messages for memory allocation > > profiling. > > > >+config MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ bool "Enable accumulative allocation counters" > >+ default n > >+ depends on MEM_ALLOC_PROFILING > >+ help > >+ Record accumulative call counters for memory allocation. This m= ay have > >+ extra performance and memory impact, but the impact is small. > >+ The stat can be used to evaluate allocation activity/rate > >+ via delta(counter)/delta(time). > >+ > > source "lib/Kconfig.kasan" > > source "lib/Kconfig.kfence" > > source "lib/Kconfig.kmsan" > >diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c > >index 81e5f9a70f22..6b03edb04e7d 100644 > >--- a/lib/alloc_tag.c > >+++ b/lib/alloc_tag.c > >@@ -66,8 +66,12 @@ static void allocinfo_stop(struct seq_file *m, void *= arg) > > static void print_allocinfo_header(struct seq_buf *buf) > > { > > /* Output format version, so we can change it. */ > >- seq_buf_printf(buf, "allocinfo - version: 1.0\n"); > >+ seq_buf_printf(buf, "allocinfo - version: 1.1\n"); > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ seq_buf_printf(buf, "# \n"); > >+#else > > seq_buf_printf(buf, "# \n"); > >+#endif > > } > > > > static void alloc_tag_to_text(struct seq_buf *out, struct codetag *ct) > >@@ -78,8 +82,12 @@ static void alloc_tag_to_text(struct seq_buf *out, st= ruct codetag *ct) > > > > seq_buf_printf(out, "%12lli %8llu ", bytes, counter.calls); > > codetag_to_text(out, ct); > >+#ifdef CONFIG_MEM_ALLOC_PROFILING_ACCUMULATIVE_CALL_COUNTER > >+ seq_buf_printf(out, " %llu\n", counter.accu_calls); > >+#else > > seq_buf_putc(out, ' '); > > seq_buf_putc(out, '\n'); > >+#endif > > } > > > > static int allocinfo_show(struct seq_file *m, void *arg) > >-- > >2.39.2