From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22AE9C5B555 for ; Mon, 2 Jun 2025 20:48:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9674E6B0339; Mon, 2 Jun 2025 16:48:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 918036B033A; Mon, 2 Jun 2025 16:48:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82E726B033B; Mon, 2 Jun 2025 16:48:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6A4FA6B0339 for ; Mon, 2 Jun 2025 16:48:58 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 209FC1A13C1 for ; Mon, 2 Jun 2025 20:48:58 +0000 (UTC) X-FDA: 83511649956.03.E190E5A Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf18.hostedemail.com (Postfix) with ESMTP id 163371C000D for ; Mon, 2 Jun 2025 20:48:55 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b="bWKzG/P9"; spf=pass (imf18.hostedemail.com: domain of cachen@purestorage.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=cachen@purestorage.com; dmarc=pass (policy=reject) header.from=purestorage.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748897336; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4du05IZSI5/ABABnMZ/NfR2Oh39wOq4x43RDq/tSHLw=; b=rEf/EE35leZe81jkQHKcb2f4S3KxszBg6TnCx6slMJqIIIjSjBG3xVum3Rrk8H4pgerWBQ SrGHUSc9rvbfMkxWmUeZAIeRHHi/4mlN8DHCyi51t+fMiJFAOkRMiUucTc5BbIi14zoxki leFJajxbFMB9MPbvx24rjo5s/bJHF4w= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b="bWKzG/P9"; spf=pass (imf18.hostedemail.com: domain of cachen@purestorage.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=cachen@purestorage.com; dmarc=pass (policy=reject) header.from=purestorage.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748897336; a=rsa-sha256; cv=none; b=JIp5UQOPu64J/X1APmOC0ZVmuMvZS1w3gGDjWgdVkKILtCr3y12XMsdRIa4P0YSLbRIKo8 sNEq4U/covVd2QcyvKA4TFIcwvhnVEqcw/PHdOCLqOO8dGGbpA9ZZR5eOVs+HTqBprcJZI 9IBHfjNlSYa79rehPNrdcX1yir9h4ec= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-235248ba788so4757945ad.0 for ; Mon, 02 Jun 2025 13:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1748897335; x=1749502135; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4du05IZSI5/ABABnMZ/NfR2Oh39wOq4x43RDq/tSHLw=; b=bWKzG/P9wSCp3nCOdg9G5LxO/SQqLBze5ISzEgI3f/5qP0gsV9bmd+NOdJYOQ3dqr4 13h94d79dcs/yRJFY9Gx+ern1ufm2SmO5iN5Fq6koAMIfBVEHVIm7dnaZ6sraMZ5KQPD 9j9BWKzjKs9FR+54NLM29NWsWOUr0pj25V9nuUcpdjlcjH4/UxH1Cda9Zn8bZPFYLLao L/k4wdK5oOeI/PXdnuhy+Y1c5HNJRrgewdcFI27L2jQ3c+Jo8Y81PLmE98oTTOJbxqA1 zQGYThIqufUlSX/8GINZMWenfzIcPSSrlJVgcP98Io4HXGKV9D+pm6x9feqXir5jMuWE krdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748897335; x=1749502135; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4du05IZSI5/ABABnMZ/NfR2Oh39wOq4x43RDq/tSHLw=; b=jKLhJoTcZX1X4Vfaj0/Oxv/woBhD/YAwMSYv3ykEaxwuprqPhGXfDvm4QL3NWbNdE7 l/HP/MpYNsLiY7H3jXAZWhl4CnYnJwvnKIkM2H8YSiWPKA4IjK5ZfxIytMRuHoiE8yji b5XSSd6H4I9hSGZOtnRy02jPKjfg2VX/6/2rH4Ryq3hQEWVruUC7/meKOb8idcFMf8mp Gl5DlHXWTD/tKu6u3HM1HhXzHyP90QO3WgOKarR3xsaioJn8Xusv1ifaVbqGwRpdfOEs svv+VwuCWccR/UoQvOf33/vxk9/tJnenK7KSCGdf52eFFhjptnSphBVZmPnjQqfw7KK4 aWwQ== X-Gm-Message-State: AOJu0Yz9Hi55srbM8OJUBdZkzDlX7xs91Wc2wVbFxGWBoNl+K2Q5Zxfs DYiN/bXVV3DAKGekblRcoLE8fD+AZgxHMTl5Mt7+pqI2Nq9kAyXRDTYeOhqVg25/JSoeClaiq8X CMS0TSmJVFfXmRcd+n/yPkr/3L1H3BPMxVY1U/qhxkQ== X-Gm-Gg: ASbGncsDNw7HBO/Z6GMafxfKHWHItrR5M/9LohWKdKu6kh76FlnRWvVdfptCY4hgfMt DbA0BFYb+8sJCVJqpw19EfSzHm8LU7e0L9/soZvOAjo6q3V/eOnY+HLu1nx4No/A9PorGxmjtI7 4n9OEdwuT+t+RCQUxllcDM98vcUA1qBrfS X-Google-Smtp-Source: AGHT+IFRh+Hkd+3NUz8EL+FmCAG/gFahBMOBeNjQeVhN6cszbjQpuJ0Hj+2dXff9JvecL7iLYiMKqWlE4tpmShyGKiU= X-Received: by 2002:a17:902:ecc7:b0:234:11cb:6f95 with SMTP id d9443c01a7336-235c6cf31dfmr4680095ad.6.1748897334670; Mon, 02 Jun 2025 13:48:54 -0700 (PDT) MIME-Version: 1.0 References: <20250530003944.2929392-1-cachen@purestorage.com> <5iiwnofmnx565g3xv3zdt35b7qkuwylzedkidnav72t24asswj@omjgyjnauulg> In-Reply-To: From: Casey Chen Date: Mon, 2 Jun 2025 13:48:43 -0700 X-Gm-Features: AX0GCFvqREo9TByYZCuukGVD35RRwYPn7F_d2U5WXQZYXHULIdbVaItV3LfZ4vU Message-ID: Subject: Re: [PATCH 0/1] alloc_tag: add per-numa node stats To: Kent Overstreet Cc: linux-mm@kvack.org, surenb@google.com, yzhong@purestorage.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: c8pjhewpcqmsgpdsaiket7ijp7fm6y1t X-Rspamd-Queue-Id: 163371C000D X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1748897335-160786 X-HE-Meta: U2FsdGVkX188D1on9qBDxsUmyfVIIMu0F0OC05SmErlkljK66y+O0LgvElNLuQD5g0O36A5wW7m3yks1J/nDaq8twvJWlWAz7W8tGnxClfrjL/fs1GmMW73O1hoc0IgWTZTL4+rF4u/BJLqAGw+nBarPYA2DrHVRezelfuKO/6nd8nhZ0F3M3IxPn076QjYWUfdEYUVE6bMC9Kha+sRcAn4LuB2QMCjNWkcw5HZo2OHVLNChzyoVOQSgyBhGq8CfTJxwv80sFMMdYhixcWdgSxMaDGevvuTKQ4z59LMJpYHTnfEUczrUyTQZ03hcxyZUrGXJqlbVRp9hmzAtycKLg/PUbfKGn0CXDonMuLbRI4zbhNnD9XX2WdtAH1PAF62Qf20omWij9UN97RUZNJeZ2Hsiq3eDKUWE0wLI/wUjo6e44uTLxOQv64dZOk0xoU03zVR7CVSuoS+QxaK00hJxJS38urCARHP3Ww81zTjXn0KXx++bVkcZ2m7gXW/xlROSWGyKTPiBSiTcP+xxMRGMXdaYlm/OcOgrKf6xLpHCXc2WdHOb5OjhpVtFmYES2p0AbklfmVMay2UsknkDh3Xk6x4Rx9QM/RI6oB1aXnmUmQEijc4OPU4HfUK3JMXD4PEbNQSXlSM35a43GcG0ZYyGREire/MPsisk88ZjpoouSBpuoUbS8glNa+N0mW4nwdKbYuz5nh2OIT3on4djnY+w8cK3umE9Td7fxp1Ab7Dt1UXqURfP4sEXawUILtlK+ud1LbQ22DOp56qoPvPmjJuUyZo24qXFkXkcAok9iHJr2z1FWLLhlXNTIQitMcCH/a/oXIjEMaMPCxKy+vM0jD5NO6XO989OKBssQm/f3Zb9igr8Qwibzafa7hiU+FbBJP43bSLWoLMS06SMrzjc82Oo6PIj/JbmSS+vMAUvNubegYMbXJp0QZFjj90OS5eZr850d6W1lGoW/aIPfzSFIi8 54tK6GXv VjnIrS1TxlZjQ3/m0W8YIC88V6D75+okfh7VEyIMj86/gwACfpkNt/QmTE7SR00pYC3IQ5oiI/kObI+SnGg9zEktx1w5yOQmiC5WjK5GntQb1wRiOAyMNfIkD/dMYmedVY6conUUvMcSQL8MBmaIRzUHOVS03Szj6JFpevaGsYVldmjg8kthibVJDcCbmCCz3R4bg5mtGF/qIwzmkCSDkmhXBDM/ujvYD0WgIWZnTXlo5LcnniLIpJL9tHzXSA2HkpFh2ic0nKY1HMU0RKMhKzPxZDTHlCoI8H57o X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 30, 2025 at 5:05=E2=80=AFPM Kent Overstreet wrote: > > On Fri, May 30, 2025 at 02:45:57PM -0700, Casey Chen wrote: > > On Thu, May 29, 2025 at 6:11=E2=80=AFPM Kent Overstreet > > wrote: > > > > > > On Thu, May 29, 2025 at 06:39:43PM -0600, Casey Chen wrote: > > > > The patch is based 4aab42ee1e4e ("mm/zblock: make active_list rcu_l= ist") > > > > from branch mm-new of git://git.kernel.org/pub/scm/linux/kernel/git= /akpm/mm > > > > > > > > The patch adds per-NUMA alloc_tag stats. Bytes/calls in total and p= er-NUMA > > > > nodes are displayed in a single row for each alloc_tag in /proc/all= ocinfo. > > > > Also percpu allocation is marked and its stats is stored on NUMA no= de 0. > > > > For example, the resulting file looks like below. > > > > > > > > percpu y total 8588 2147 numa0 8588 2147 nu= ma1 0 0 kernel/irq/irqdesc.c:425 func:alloc_desc > > > > percpu n total 447232 1747 numa0 269568 1053 nu= ma1 177664 694 lib/maple_tree.c:165 func:mt_alloc_bulk > > > > percpu n total 83200 325 numa0 30976 121 nu= ma1 52224 204 lib/maple_tree.c:160 func:mt_alloc_one > > > > ... > > > > percpu n total 364800 5700 numa0 109440 1710 nu= ma1 255360 3990 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:141= 0 [mlx5_core] func:mlx5_alloc_cmd_msg > > > > percpu n total 1249280 39040 numa0 374784 11712 nu= ma1 874496 27328 drivers/net/ethernet/mellanox/mlx5/core/cmd.c:137= 6 [mlx5_core] func:alloc_cmd_box > > > > > > Err, what is 'percpu y/n'? > > > > > > > Mark percpu allocation with 'percpu y/n' because for percpu allocation > > stats, 'bytes' is per-cpu, we have to multiply it by the number of > > CPUs to get the total bytes. Mark it so we know the exact amount of > > memory used. Any /proc/allocinfo parser can understand it and make > > correct calculations. > > Ok, just wanted to be sure it wasn't something else. Let's shorten that > though, a single character should suffice (we already have a header that > can explain what it is) - if you're growing the width we don't want to > overflow. > Does it have a header ? > > > > > > > > > > To save memory, we dynamically allocate per-NUMA node stats counter= once the > > > > system boots up and knows how many NUMA nodes available. percpu all= ocators > > > > are used for memory allocation hence increase PERCPU_DYNAMIC_RESERV= E. > > > > > > > > For in-kernel alloc_tags, pcpu_alloc_noprof() is called so the memo= ry for > > > > these counters are not accounted in profiling stats. > > > > > > > > For loadable modules, __alloc_percpu_gfp() is called and memory is = accounted. > > > > > > Intruiging, but I'd make it a kconfig option, AFAIK this would mainly= be > > > of interest to people looking at optimizing allocations to make sure > > > they're on the right numa node? > > > > Yes, to help us know if there is an NUMA imbalance issue and make some > > optimizations. I can make it a kconfig. Does anybody else have any > > opinion about this feature ? Thanks! > > I would like to see some other opinions from potential users, have you > been circulating it? We have been using it internally for a while. I don't know who the potential users are and how to reach them so I am sharing it here to collect opinions from others.