From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71640C5B543 for ; Sat, 31 May 2025 00:05:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 939C96B0161; Fri, 30 May 2025 20:05:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 910756B0163; Fri, 30 May 2025 20:05:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84E346B0164; Fri, 30 May 2025 20:05:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 65B5C6B0161 for ; Fri, 30 May 2025 20:05:36 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 07317E7E37 for ; Sat, 31 May 2025 00:05:36 +0000 (UTC) X-FDA: 83501259072.28.E18EDCF Received: from out-172.mta1.migadu.com (out-172.mta1.migadu.com [95.215.58.172]) by imf21.hostedemail.com (Postfix) with ESMTP id 405681C000B for ; Sat, 31 May 2025 00:05:32 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=f4DlkDEo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of kent.overstreet@linux.dev designates 95.215.58.172 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748649934; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3htrUDjmeVpFzO8+cFlSoMEzGBljzIb+pDxNBTGCk5Q=; b=d5rrXHlxVwTFb/3/9Z4RL/GgfOZJLI5+aX2EBD0Gue6MX5FyiozD2+s38wGiLOGXJXr9Yr K/TREBR4BAITVSiuUKktvu+IE61ei1S3yXromdNJhcueUTascCm5e0ulz9LJzhq1pdpR3F FbBMWhtOVhOY5SoPT03c9x6xys2h03E= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=f4DlkDEo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of kent.overstreet@linux.dev designates 95.215.58.172 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748649934; a=rsa-sha256; cv=none; b=fg0UqEZIJuVIVpUaK73F0VyBLrkrBATSYG6qOj8Ls+Aj0NRdZIdFSOz9wtD4t/xJrkN9Kp FFVoHRuQ6hL6cLnDZbR3hip+vTZh4n9Z2UTR8kq7I1FsRwqKdIgClL6oPBUme13B+jagIe Aiqwbe31IMAvOC6ZTk93oGlDbcZYA6Q= Date: Fri, 30 May 2025 20:05:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1748649930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3htrUDjmeVpFzO8+cFlSoMEzGBljzIb+pDxNBTGCk5Q=; b=f4DlkDEo4SnBUNtZ0CxcXUEpEcuCzr+fN+xpu5PUjnIlkUHlbkIcX60oMagsRAdYQYG1nk QdTtJq9pJZz+lEq1ZkgkIigu9UyU9ZrnJDlJe1CfSIu1ptYJRvJelYXtwjz+9r0qEbwRp6 4myWoAvKLOswzT6d1cBhe3ERgV85ddo= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Casey Chen Cc: linux-mm@kvack.org, surenb@google.com, yzhong@purestorage.com Subject: Re: [PATCH 0/1] alloc_tag: add per-numa node stats Message-ID: References: <20250530003944.2929392-1-cachen@purestorage.com> <5iiwnofmnx565g3xv3zdt35b7qkuwylzedkidnav72t24asswj@omjgyjnauulg> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 405681C000B X-Stat-Signature: 6xzydfnaky6zdmqowtoe783o3jggdcg1 X-Rspam-User: X-HE-Tag: 1748649932-650922 X-HE-Meta: U2FsdGVkX18ZYMQ6TN8N7wlx+6WVLVCzSebXlXaR62A9NYaaDHiMTbJdajFpzY3lF/kHtC+IGZkpzGDz73RcRmm7G9FyTWQTRZsF7IIxESSoAYq4NXWWH9VBsTrNBUhvZ14YKw42e5tQOpxVK9e+oAFlACKLpxh8DhDAGW7qHrQ1B1RYb729cWaEDSdmCouJIQAYXWOtH252nPlWDopA83jl+hJexigh2d52fiRC/gbmZvzVuWafP1jMwSnkSLZ59+/T68TZ9GMWFThaZJI5QuQEhXPVsvaLgbAnrT/ONYQeOVKBD0hfuZ7GDz1rR6HcVRq1WQ6tf+E7sLCTSebt7XGabbPYt9Uwyyz07yfjZ8a4Sk5Y5QlMk2qLhEzt1cqMYuNDSZFgKGh/unve2jHuByPd61Wmo0B9adov/WsjzntaQOnU3W1XjjKq+YkdwckNoh/U7k9FERclgqnINwUQZRt0iwxfN9mOUz8IXO7I/8VOLvrKq1tgxVQq8BujNPLQpueOR7+t8RuWW+gWi6G45Byb0OKPJirt4tmCErbvB/yxXFhzEOzRD8tXiUHrsSO4afAZK40lmk4WyoOAbpZVg+7mfDxnx1u6UaS9MtRekcgV2795aZ8vllv1yfvPm6tFnmHZxyL0erJ/gZ0wC8rV5tlpbQ29JW2VrXvBKgj2O13UpuTHzPEDgr5KEqVbDSkc0I3Tw4+fzou/dUSDPNBHkd6w3mivdTI35Juo7d+5XBvGlXQEa7sOHIJZ38M6swBDCn05Q+QgyBfuMIc44Ln+/HSAMQKL34MGNOYrg5/yiOUVgoTmWS3KxId8MkC4tg5LgQ2Lx9PGKJQ90OEGDa31r0zVDmaglPyJCz1qq/kFLXbrQCI6BFsDxFdvILbNpK7WMJZHBX8PMOpUK7rv7MbPArio1+wg6q9AgMCIMa6EWv4wHs9X3J2bVQ/+Wx7hHoPWjHURx5QrHE5e59Flp/i 611butlK 8IMLHsEmxtPYkkSDTQqgSzNAU5TatJBiz+m5R3ncxs019PrBM7vCExwUbfVnK8JNQkbR6NC176MlM6ty0c+e7I4qLLFwwAspYBKlQSxCjfdyOLMdaDmk76mo6rpUwzaROwK7R7BHGGUTbug3aMFuZ2eyCZ5d62EOjCr7DHmwCn2Qyz47cAHesZs+6WQHysvVvOw7Ytj5KBisx/JV6Q4P8NkwJX3b7YEUP0GxJWFIQTHvV+BGnviEqQ6LAaOuhbZ+A3ZJgcOqFyyPFa+5Llz2/FPLMg/sqbKFYBnuNlb2FwiEORXjctXe5bx3HCQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 30, 2025 at 02:45:57PM -0700, Casey Chen wrote: > On Thu, May 29, 2025 at 6:11 PM Kent Overstreet > wrote: > > > > On Thu, May 29, 2025 at 06:39:43PM -0600, Casey Chen wrote: > > > The patch is based 4aab42ee1e4e ("mm/zblock: make active_list rcu_list") > > > from branch mm-new of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm > > > > > > The patch adds per-NUMA alloc_tag stats. Bytes/calls in total and per-NUMA > > > nodes are displayed in a single row for each alloc_tag in /proc/allocinfo. > > > Also percpu allocation is marked and its stats is stored on NUMA node 0. > > > For example, the resulting file looks like below. > > > > > > percpu y total 8588 2147 numa0 8588 2147 numa1 0 0 kernel/irq/irqdesc.c:425 func:alloc_desc > > > percpu n total 447232 1747 numa0 269568 1053 numa1 177664 694 lib/maple_tree.c:165 func:mt_alloc_bulk > > > percpu n total 83200 325 numa0 30976 121 numa1 52224 204 lib/maple_tree.c:160 func:mt_alloc_one > > > ... > > > percpu n total 364800 5700 numa0 109440 1710 numa1 255360 3990 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:1410 [mlx5_core] func:mlx5_alloc_cmd_msg > > > percpu n total 1249280 39040 numa0 374784 11712 numa1 874496 27328 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:1376 [mlx5_core] func:alloc_cmd_box > > > > Err, what is 'percpu y/n'? > > > > Mark percpu allocation with 'percpu y/n' because for percpu allocation > stats, 'bytes' is per-cpu, we have to multiply it by the number of > CPUs to get the total bytes. Mark it so we know the exact amount of > memory used. Any /proc/allocinfo parser can understand it and make > correct calculations. Ok, just wanted to be sure it wasn't something else. Let's shorten that though, a single character should suffice (we already have a header that can explain what it is) - if you're growing the width we don't want to overflow. > > > > > > > To save memory, we dynamically allocate per-NUMA node stats counter once the > > > system boots up and knows how many NUMA nodes available. percpu allocators > > > are used for memory allocation hence increase PERCPU_DYNAMIC_RESERVE. > > > > > > For in-kernel alloc_tags, pcpu_alloc_noprof() is called so the memory for > > > these counters are not accounted in profiling stats. > > > > > > For loadable modules, __alloc_percpu_gfp() is called and memory is accounted. > > > > Intruiging, but I'd make it a kconfig option, AFAIK this would mainly be > > of interest to people looking at optimizing allocations to make sure > > they're on the right numa node? > > Yes, to help us know if there is an NUMA imbalance issue and make some > optimizations. I can make it a kconfig. Does anybody else have any > opinion about this feature ? Thanks! I would like to see some other opinions from potential users, have you been circulating it?