From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAD4AC5AD49 for ; Fri, 30 May 2025 21:46:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 501FA6B0200; Fri, 30 May 2025 17:46:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D89C6B0203; Fri, 30 May 2025 17:46:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 416996B0204; Fri, 30 May 2025 17:46:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 226C36B0200 for ; Fri, 30 May 2025 17:46:14 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 84FFE802CD for ; Fri, 30 May 2025 21:46:13 +0000 (UTC) X-FDA: 83500907826.01.663340A Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by imf03.hostedemail.com (Postfix) with ESMTP id 7176F2000A for ; Fri, 30 May 2025 21:46:11 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b=OD6l+dC4; dmarc=pass (policy=reject) header.from=purestorage.com; spf=pass (imf03.hostedemail.com: domain of cachen@purestorage.com designates 209.85.215.169 as permitted sender) smtp.mailfrom=cachen@purestorage.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748641571; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aVxH2UjMZI6eUoDQvYUv6mT3dx/2JioBgFmwVVyhlFU=; b=NyYevSzWrkWk2Pmf2n5rdIja3TTUvdN098wA27oMhkowHcD3pMoHXHC+TRfjXxZvmJFOJy T7jnyu4dlSR0ZqNOl0wOVcBf4GvogIXCix33muykAUcdYjx7z1bYhHshugVvSmopqM8AlG u53cwwWTFB80yBJczk7pBq1B3xkiPPs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748641571; a=rsa-sha256; cv=none; b=jGh03sGwpfvthg+rSw2bXuyvn3fg2mP/RcPzQmvjHqyu47gnGxbIqBGoJH5C+U198E7Iz5 t9JE2mTs8naGsswrpTz90mlF9uNzTpMB+XSWtoQD9iljYnqi2itXiV6giwaB/pTo8YNnzW 9FBQQ0FSF1HlXeGIooD4q6fFf4LlfO0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b=OD6l+dC4; dmarc=pass (policy=reject) header.from=purestorage.com; spf=pass (imf03.hostedemail.com: domain of cachen@purestorage.com designates 209.85.215.169 as permitted sender) smtp.mailfrom=cachen@purestorage.com Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-b2ed1bc24fcso133410a12.0 for ; Fri, 30 May 2025 14:46:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1748641569; x=1749246369; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=aVxH2UjMZI6eUoDQvYUv6mT3dx/2JioBgFmwVVyhlFU=; b=OD6l+dC4PP9Y5g/+MeRjjfxT1D8yJOODFoIm3ykAz97YTNDJYjxAI73JflHD3e9hQg hfYm4fZMYaTgN/uMpXYKOxld5X7ebuP6sBddYB6QLrHfQD8eQmH1cjVTvdzlUyH4LwUH DdBIbaZGTvXR6+mzfeDUAxCNlId9SXmfWuBbjFetNXAqEmCbX+Xcu9E+PmrpTlQkWVzk u9p/l3gSWpXJMdFtUDdpv1vD8wI5+oFTVz/8TgdGcZAajxY0shceb4UN0rBHgfGDuyKW WjcQoeRM1XiQQzrlufFDJc7BxbyOMrAYYFjwVaj/RX6JN/ex04kgaw4P7tjm7JdoIcky GLXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748641569; x=1749246369; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aVxH2UjMZI6eUoDQvYUv6mT3dx/2JioBgFmwVVyhlFU=; b=EOLvegEzPNZZn7jCvVoX8AhZlbFm1tvwajHh+QclTPbrr5kWgZKNT4bi0GortKskiV 9c73dcFG+JxjKeJBnBT+j4AWCbDhm0/c8qlcU9U/JYtJN6gxRkclNqW14vLSpddSGiqU pGyzUF8Jv/ao+Xb4PNKJWJccs5RNUJHHKAkHJOH+F8YPtBJ4DXzZy4MCX85T/1EE5tgE PDRRk9uMmReaPlhKcJbCmvY8DhfJV9yc7fy33GCuAnRKn6+Ab66JmQsxnlgNdcs65T8B TwU2leDdvqJiGtP1WjTSZ5fM5Suh5t0JdHyksCyVzl1bvnOGO+tlkZUCEAsspaJdoXa6 VB9Q== X-Gm-Message-State: AOJu0Yyo4nk4M4Fky/1svA/WKmcf/Zbr9bQPhyusLuCHgoRQJhAwa9bt 0xHE85bvH+EhefGYNIGB2TkqlsRbco15OKMEd5TqSW5glx7xGW8Nd3ceN7eazt/nwpYvfrFAJLh chA+LOvC2uxiKlb+rRxYgl+DXW+ITFcom3prY7Z+gdw== X-Gm-Gg: ASbGncsd6Cm4nAU8oiqqdUZqVDLMTiyDSi0Rqcav7L7p67mj7Q1P8q0JWGJ/qaYOxLh PYsFmkY/P8QlHFezkOsl5gAFvgFcSAFwAtwOi/yS0yk/tdLqiD7O8OGqN3Q1lkbOFqWmnnfylqF Vuk/0+ZjjGFGjb1lBscheeAP1RT9nvk62xKgPW5IPtfihNbb7XklMWDQ== X-Google-Smtp-Source: AGHT+IHQC0TFt1tgmLdebNMPYlmceD8rc186Gqn/zhQgnl7dQuEs5VA5WNuicVs2XhNOJ2AwdFGdbDHnmoM5B66wUdw= X-Received: by 2002:a17:903:41d1:b0:234:a734:4abe with SMTP id d9443c01a7336-235351e0972mr20493035ad.1.1748641569119; Fri, 30 May 2025 14:46:09 -0700 (PDT) MIME-Version: 1.0 References: <20250530003944.2929392-1-cachen@purestorage.com> <5iiwnofmnx565g3xv3zdt35b7qkuwylzedkidnav72t24asswj@omjgyjnauulg> In-Reply-To: <5iiwnofmnx565g3xv3zdt35b7qkuwylzedkidnav72t24asswj@omjgyjnauulg> From: Casey Chen Date: Fri, 30 May 2025 14:45:57 -0700 X-Gm-Features: AX0GCFu0oi4IrIUfuaOU5jZ4_InQNJetoMuX-3oxj61nK6f53hSh76wxxFzlDo8 Message-ID: Subject: Re: [PATCH 0/1] alloc_tag: add per-numa node stats To: Kent Overstreet Cc: linux-mm@kvack.org, surenb@google.com, yzhong@purestorage.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7176F2000A X-Stat-Signature: 1u6a6hday9sr6ykft3ysm4sbyz9jqqqt X-Rspam-User: X-HE-Tag: 1748641571-2567 X-HE-Meta: U2FsdGVkX19Xdvp39VQje9XazlmShGcEO+91emHaS/rGGNeko4Eepa5LWbYn7lfrRn6PyEF+dW9HJKD67P+mdKJyN1tJQ5Rqkg6C1wdQu0mpa0mW0FctIjYoqzR0OaO06oRFNYwjG16PnYvzJxJIVGPG6sHZ4DU5E/J0kxj4ZMi5SDjMMo2q8F+kMShK8UorLEHvBWBYkwjUhzi1DXjM/J7/pLk3ZWmkuZmVHkGaYq8IEBg/WHK6TbxkqFbsI5T11C/GSbc+bzU2Baqp6fQTdrDcN7ex51BZebmlxvYTxPZPkejiHi9QJWvIhtyMWT5ZiEb/xZbHM5d1geLCHJpG0ro3XocUyv6xIxpq6euBqW5EsSobcsGEOcp3Wn7cXcvk+WcspHiLZUJGoLbgRqt7TwVT4i7RTnCJx5668uoCJGADZoh4ChOdfs+qoDiq1oK5pP9pS/40WuU6vA55bDhH5bHSBj2NMJh1mDLr2+ylGoDo9kBAwDuINxae9huRCjeNEAiQP+qLWSp/Ur6UeYLGbZE6QwbtecIJeTaOhbxluATe2wJPT1/BE/MgFTaBwzIwr6q1xmP2WrC4IXdoKu20t7E76Yqn6X6wO4tsgfnXurixCwBY1x/6Ee3LRb8k0v7R8ggpTUFV+VNgjzwBx/HbI9bxF44idnBpLgeeCBE8ZQWvwBqnsXzxQ03QjaCoSZaYIZScCvDkHdMZbZavE+xi1iQIiDcxaFVe8UB814CwC4u1b5WRTdPEuHm8DLjtEqSVtP4yIYwTq114oNlT86oqK8A1dc4vzy5Kk+qPeMOBiNQuzG9wD21CiYtaBMhT//+eZNYBWOXZCelxtbLAqKplagxMlViIQ9eEqekHbtP9YN3sgpx2iD+D8pu8ZGdwzJhN7xLrRboH5qajpIRHI8B6fypP4odr7Yn8B0csWEGWxDYSYBjOGrBYU+RpoZZKOfiWWV0U9unTVaPiF8C5ajJ +syVTtBB l3Rm+7OAAK3fRbzQw7y9y8sYzVomAUCs26mEAwlqg1CvcVp+a6zKh4gweH37nSiCcj7h0SnvGXWSAmiBVTE56EecbiiI23N+h5Jsulx6UHeTObTk2VK0XwDAxxmsDHLAV6hqcPca88nPDYJp2wWPOUXKB+/yfb7cPPzWgUgXX3MXBqJXZn1u2tCKPPm1+3GE+Gc6S52kKljL6f7Ty73Bte8j7Q+P6CMerUfwGPeOYQjf63kwlA6/bxOpbtV12md1CePTSm6RsSeUeTB5KXMRXKLxs/C9qOllBN1C8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 29, 2025 at 6:11=E2=80=AFPM Kent Overstreet wrote: > > On Thu, May 29, 2025 at 06:39:43PM -0600, Casey Chen wrote: > > The patch is based 4aab42ee1e4e ("mm/zblock: make active_list rcu_list"= ) > > from branch mm-new of git://git.kernel.org/pub/scm/linux/kernel/git/akp= m/mm > > > > The patch adds per-NUMA alloc_tag stats. Bytes/calls in total and per-N= UMA > > nodes are displayed in a single row for each alloc_tag in /proc/allocin= fo. > > Also percpu allocation is marked and its stats is stored on NUMA node 0= . > > For example, the resulting file looks like below. > > > > percpu y total 8588 2147 numa0 8588 2147 numa1 = 0 0 kernel/irq/irqdesc.c:425 func:alloc_desc > > percpu n total 447232 1747 numa0 269568 1053 numa1 = 177664 694 lib/maple_tree.c:165 func:mt_alloc_bulk > > percpu n total 83200 325 numa0 30976 121 numa1 = 52224 204 lib/maple_tree.c:160 func:mt_alloc_one > > ... > > percpu n total 364800 5700 numa0 109440 1710 numa1 = 255360 3990 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:1410 [m= lx5_core] func:mlx5_alloc_cmd_msg > > percpu n total 1249280 39040 numa0 374784 11712 numa1 = 874496 27328 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:1376 [m= lx5_core] func:alloc_cmd_box > > Err, what is 'percpu y/n'? > Mark percpu allocation with 'percpu y/n' because for percpu allocation stats, 'bytes' is per-cpu, we have to multiply it by the number of CPUs to get the total bytes. Mark it so we know the exact amount of memory used. Any /proc/allocinfo parser can understand it and make correct calculations. > > > > To save memory, we dynamically allocate per-NUMA node stats counter onc= e the > > system boots up and knows how many NUMA nodes available. percpu allocat= ors > > are used for memory allocation hence increase PERCPU_DYNAMIC_RESERVE. > > > > For in-kernel alloc_tags, pcpu_alloc_noprof() is called so the memory f= or > > these counters are not accounted in profiling stats. > > > > For loadable modules, __alloc_percpu_gfp() is called and memory is acco= unted. > > Intruiging, but I'd make it a kconfig option, AFAIK this would mainly be > of interest to people looking at optimizing allocations to make sure > they're on the right numa node? Yes, to help us know if there is an NUMA imbalance issue and make some optimizations. I can make it a kconfig. Does anybody else have any opinion about this feature ? Thanks!