From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAB7DC83F09 for ; Tue, 8 Jul 2025 23:07:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 431F26B0095; Tue, 8 Jul 2025 19:07:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 409D86B0092; Tue, 8 Jul 2025 19:07:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 346A36B0095; Tue, 8 Jul 2025 19:07:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 238136B008A for ; Tue, 8 Jul 2025 19:07:23 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 99C3257998 for ; Tue, 8 Jul 2025 23:07:22 +0000 (UTC) X-FDA: 83642635524.04.CAB441A Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf19.hostedemail.com (Postfix) with ESMTP id A80101A0003 for ; Tue, 8 Jul 2025 23:07:20 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b=M0TQryd2; spf=pass (imf19.hostedemail.com: domain of cachen@purestorage.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=cachen@purestorage.com; dmarc=pass (policy=reject) header.from=purestorage.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752016040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ORrNhGhx1xAy0Po/oCOkd4206JtidBzGrSaq8D7iH54=; b=PkvHX1BGFymMXYZnrmPyuAYSFQzNR+3UFFib+iI64bMDn0KO6eHU1UOHn9bq7Ydqz0JjKd j7KNiHkEmd5OArgA0gN2wBrQI1gcS/Q0nnfe9DnpJ54/9E1OJx2Lqoc5PwF5RTlk4otyef mvpEsc3MqKcmQzBd7X//xuHwe6/kJvg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=purestorage.com header.s=google2022 header.b=M0TQryd2; spf=pass (imf19.hostedemail.com: domain of cachen@purestorage.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=cachen@purestorage.com; dmarc=pass (policy=reject) header.from=purestorage.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752016040; a=rsa-sha256; cv=none; b=1yJ9hE040XWyqOnmSGRzvoQ3/UEZy6iGL+WevLbylnwmkaGe0aCH5Vl8W+fnuWxezZuGSo gJMyjfGEdFOY7kppUpGJL+Sd2O/crFmL9fIeVynyG1aZ9taem6FIfpHqn0DtelWn1SpUMj FP1fqeehwAR0aJyaXGR3LRZnJaKfLpM= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-234b122f2feso3887375ad.0 for ; Tue, 08 Jul 2025 16:07:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1752016039; x=1752620839; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ORrNhGhx1xAy0Po/oCOkd4206JtidBzGrSaq8D7iH54=; b=M0TQryd2qDfmKagdtcVf9U60bdZMB8pb6n4y9mWm/g+2N3D4/2F9N69vcJU5g7t2K9 NqS0BGmBWrUVJ7N2RMMTeFUIbghHFwcaBGrFxJoxjX/LO5xz+ckQU490A/SWyBtNNCox VepOx1YhHfFwvxleAJcvJAESmMkSCsfhwYpkK692SD9EvvFkgMV0imLeQwHM1o6ZSC4R mUUh3CFRUuX6A2XunpigLWi7oIeU9Ep9vOgoyVGYvPtzzBMfkgdohj7JsiWi53OlS0f1 kWSm0NJvAd9lBnCRob9K6r7oHB85aTWRn85GSZhQn+t50IFYU1S1Ojc4FRFhovELpPYr SQHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752016039; x=1752620839; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ORrNhGhx1xAy0Po/oCOkd4206JtidBzGrSaq8D7iH54=; b=GDoRQn0kbJUOeruNdxmFCSQVLCTUlMlg/0rFQ3gXTCO2PnUAdH+kU9Ut2UfDkTuAcS 2WHrXGaxKMJALOZWoNNJ88xSFeOIN7bm1F3/P1duR1nEPl0D2Lyi6kl2S67JUY5PIlyO X/9OMoz5r5TZLg6Cz/wnLWSTft5TdymeP/TBxz6xPqMblfY3dS1GWPwHtjBxev3gU9o8 B6s4laWKUqeWfpkj3o9fdXIj5LnRMDpFoqTjmft4p9vAw3IWb7enKweJaJSlC9VdGPla pLOeiqd6EFAHXeZPJwVvhTZNIUz2kAtU21uPtaAre6wLjNSbzBa0mRLyh8r64J0tBjPJ YSqw== X-Forwarded-Encrypted: i=1; AJvYcCX4NvMhvk7jujKzSOz4UAXRjCXzBpF4coPxeKtiYGDRvFKkjD5cbeuDFcRLUeaHL6tE40tWw+93vA==@kvack.org X-Gm-Message-State: AOJu0YzE193JGa88/v9B9PayZvavNmW9sCB3bzYnFpxc78ObK1oNPUaj fD/8uA32oKDFSzOYUQY24Ypprer8mSFhaEUgADse8WYhk8s6T75/s3CNPblBH7KJHwR6MUXiDGU P4c3wEcT4b2JVFo+D3h9N9KA1G1uuNilAhgkJSj/36z8KHQtQ/fHy0TZONQ== X-Gm-Gg: ASbGncv1B6WHnT3RyaFcZoFr7JdAYtpAtw9EmXWqTPVD5yjxavAz3GAvbLWVwzSgT8F seX77K3R2v/tEROt3vNX97XVWVyzuS6WvsuOdyVDIuvTgWi8LKjFt8zxV325IyVddxH+zRJQtu0 UI4h5sWFFfIGnho3RIgvAQHHugtYq/FMpkIWrKYtbgYxE= X-Google-Smtp-Source: AGHT+IHafJZSNlc3LwJHoPCVexsbbzG5eLYra9gLQmPwSGpxSqUN1/USBkhF30c0AMmJuh3uHbnCi6A+WUCshzmTOdI= X-Received: by 2002:a17:903:2f8a:b0:234:db06:ab5 with SMTP id d9443c01a7336-23ddb31356cmr1508005ad.12.1752016039406; Tue, 08 Jul 2025 16:07:19 -0700 (PDT) MIME-Version: 1.0 References: <20250610233053.973796-1-cachen@purestorage.com> <3c9b5773-83ed-4f13-11a8-fcc162c8c483@google.com> In-Reply-To: <3c9b5773-83ed-4f13-11a8-fcc162c8c483@google.com> From: Casey Chen Date: Tue, 8 Jul 2025 16:07:08 -0700 X-Gm-Features: Ac12FXwrCH883qf6GxaQmge34hOSXFNsa0Bf0fqsJXpu5vn54JG0qYzehNGJr6s Message-ID: Subject: Re: [PATCH] alloc_tag: add per-NUMA node stats To: David Rientjes Cc: Kent Overstreet , Andrew Morton , surenb@google.com, corbet@lwn.net, dennis@kernel.org, tj@kernel.org, cl@gentwo.org, Vlastimil Babka , mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, roman.gushchin@linux.dev, harry.yoo@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, yzhong@purestorage.com, Sourav Panda Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: A80101A0003 X-Rspamd-Server: rspam09 X-Stat-Signature: bge3ho7njqhjrhim5w1xrm1c9kch3x5g X-HE-Tag: 1752016040-914963 X-HE-Meta: U2FsdGVkX18ibon2DL86iHNt7K6IhBTKhnwF3xUZphrgKrbJPkjlHmeeEFPvKb9wWu6sa9V+qYVZS4D1UFC+pPYX6lMQDTbag2EBOVezYajmcKcAFjT6+iIHNJbOQcecYebOfaZFAPjhVA7M0ZubVVvtGh3nwC2qXE+u1yI1FQES/gRfr1G+Z51G0ARi0aS2GaZq0l64i1w+Wo9opubK2lUderTuo2ibKqCUCD2jJPRxqWdGE1kbouzazD16rzXLd6uSnV0NiB7L7LcXBdPsnD8zYas32h3OlQhdiQImz3XeH77vCgCEmqpZtixED6UmsYekfr8hSH5ZqFQiaE2/Jzc5PMljq9qrSkfLYUXY0ZyDqqg+t5ZWvQ+e+e1pO4Ft7GXqoqW6ExlfcgJF2mo10Yr9XO1EiJ2+ju2WfRHJ9cSMJU8BpWpAwpIRQHyT67pEZrQi9u6rAA1WpMZjrpMxBXR5UbBkGFtbGEb7nleqt46mscDVPFX653AUxvoGODCSeVYSy1iHWuI8BS2Kmvt48Lns0F5a3RxpXR/RXJmTfRRj1yTJvuTDwe3cuG7oT38Hr00CYb8AZD7mYt0ry1yJNyqWLgYQqhHtHhgL5FzyWzgqTpR+A9OvXfnuiDkgGds3Kto5g4i7r/o/5R8rf+PajGbu2j/Ubu+pls6U859niNxDcLluyiz/96O7tslncxr5fiXKBykEJ3j5rh0V2vdjC7wdYmCorCq98ghULEvgQfA2GSqaAJOIUaMRGDTjwG049+bHCcXigpPvtToq7lZrOsTQzkpSNPZinQMucUa3IP2m4Rvx+30NIXIaSvOoIcCcCFAK2AatONWPGy7/OgjnueVY7vrrbC9/JNtyoaGwl8ktNM4OGq7UrC8AcqdCXUav7qiQFiZp2CTB36VHCGRu0BsbAOtrYZytEggkzvYG3ap4Stvdss0KmaF5hHUB2xy0S4glripKR7c5+ZFVyjK XbfWqX0o 18Qj9RBRv7m2I9SCt66ryuHuNJ0L0DnkGJpMKKit2hdAe8Oy4IO4ODdRCiyTiGmMSFRRrHpW4cYESM3ytOOmj1tcGNET4QHbqSihdNTFcLfY9wUce1ggBnCKzPq1PRhJ/tJC4X7Wt5ABI3IlhXNy/R2Of03qnYsUkzNUcSaLo5EKHTTYX1CU++buA5sIoWtTL6/qjXXxH+LpgdnvxRgYC15iMx3ETNpDzWlRYS5w/TV+epmzE/9i8NZpW/O2WmbYbodheU/h36PFv0vYarx8FjSt7kG3aJ+9PUCJp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 8, 2025 at 2:52=E2=80=AFPM David Rientjes = wrote: > > On Wed, 18 Jun 2025, Kent Overstreet wrote: > > > On Tue, Jun 10, 2025 at 05:30:53PM -0600, Casey Chen wrote: > > > Add support for tracking per-NUMA node statistics in /proc/allocinfo. > > > Previously, each alloc_tag had a single set of counters (bytes and > > > calls), aggregated across all CPUs. With this change, each CPU can > > > maintain separate counters for each NUMA node, allowing finer-grained > > > memory allocation profiling. > > > > > > This feature is controlled by the new > > > CONFIG_MEM_ALLOC_PROFILING_PER_NUMA_STATS option: > > > > > > * When enabled (=3Dy), the output includes per-node statistics follow= ing > > > the total bytes/calls: > > > > > > > > > ... > > > 315456 9858 mm/dmapool.c:338 func:pool_alloc_page > > > nid0 94912 2966 > > > nid1 220544 6892 > > > 7680 60 mm/dmapool.c:254 func:dma_pool_create > > > nid0 4224 33 > > > nid1 3456 27 > > > > I just received a report of memory reclaim issues where it seems DMA32 > > is stuffed full. > > > > So naturally, instrumenting to see what's consuming DMA32 is going to b= e > > the first thing to do, which made me think of your patchset. > > > > I wonder if we should think about something a bit more general, so it's > > easy to break out accounting different ways depending on what we want t= o > > debug. > > > > Right, per-node memory attribution, or per zone, is very useful. > > Casey, what's the latest status of your patch? Using alloc_tag for > attributing memory overheads has been exceedingly useful for Google Cloud > and adding better insight it for per-node breakdown would be even better. > > Our use case is quite simple: we sell guest memory to the customer as > persistent hugetlb and keep some memory on the host for ourselves (VMM, > host userspace, host kernel). We track every page of that overhead memor= y > because memory pressure here can cause all sorts of issues like userspace > unresponsiveness. We also want to sell as much guest memory as possible > to avoid stranding cpus. > > To do that, per-node breakdown of memory allocations would be a tremendou= s > help. We have memory that is asymmetric for NUMA, even for memory that > has affinity to the NIC. Being able to inspect the origins of memory for > a specific NUMA node that is under memory pressure where other NUMA nodes > are not under memory pressure would be excellent. > > Adding Sourav Panda as well as he may have additional thoughts on this. Thanks for your interest. I have been busy with other team projects and will get back on this soon. I will address comments and send out a new code review.