From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 280F6CD1297 for ; Wed, 10 Apr 2024 23:58:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FA276B0083; Wed, 10 Apr 2024 19:58:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AA4C6B0085; Wed, 10 Apr 2024 19:58:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24B4B6B0088; Wed, 10 Apr 2024 19:58:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 03E9E6B0083 for ; Wed, 10 Apr 2024 19:58:31 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 90F99A0993 for ; Wed, 10 Apr 2024 23:58:31 +0000 (UTC) X-FDA: 81995289222.21.A78476A Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf14.hostedemail.com (Postfix) with ESMTP id C2849100002 for ; Wed, 10 Apr 2024 23:58:29 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fMCp0WBt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of rientjes@google.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712793509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1ZXB+0PtaF+vG2Eqe2ku2kHYSPF78NFZHwzDIf6uFVo=; b=gmHVFJz8uyI4znadfzfg/+ntBaCZe5onyzc3kXwG2cfcWHvzeb3FuM5DrgdgFXTH9A3qOP 3xNOme2+/SwiNilmtAbnDW/Ufnw4gb+i04qjbylt5hFIz8R7pf9/fpz2VwRjgaR+nqdimn qCdRHA+jJPj1MPFo7o+7U/EDMbWyMv8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fMCp0WBt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of rientjes@google.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712793509; a=rsa-sha256; cv=none; b=Ly0Y7WTE+4UvrBCGGkcrXji8f+gYq7fTE+JVSO/6tGOyIi4KZLhObZcX4RsGFe/dkRRkke xYsyr87qd2ysMAlTBEqJVlnBazLiTHdJfFfaZL5XbjCuZ7hR/A8qXVG4SSGa84Pym74zbZ 9VxVlvCrkBblDJsBMY2vZaBroF7hbjA= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1e42a6158d5so36875ad.0 for ; Wed, 10 Apr 2024 16:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712793508; x=1713398308; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=1ZXB+0PtaF+vG2Eqe2ku2kHYSPF78NFZHwzDIf6uFVo=; b=fMCp0WBtyoAHm0HxjYb80R3NZi83wmPu0lao/OUvFTs0Ris8GnerVCpqi12DjaImXj LC4bK3J7i1Jq6hL5jZjWdiAlc5DdTv45em2GUS0MLc71TR4AW2/Jl8xxqZzGFle1I6BN CZPSPEGn33j4VvoaH/5W3VMxSD0KReRIjGO4rklHHfpqnwLP3ZP4v9AOErkQIDlh2W7o CHHt6E4vjCdLDx43989copRx66vbFHjeJU+SxtmHHr52dnmf85xzJs4xb7FrpXlWzAFN oVlOooXcn8uVlYbYw4sBEs4N6nyFFGU7/9qjBGLixtesbu6IVB5OucMF5jj51IbUZAGG mXGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712793508; x=1713398308; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1ZXB+0PtaF+vG2Eqe2ku2kHYSPF78NFZHwzDIf6uFVo=; b=IFGh9KfCwskryzDbj1HuC/aE+yCxAfR+Bx4GdlkKTSMJMUPTCWiOPS6Dy/Saj7m16V 1AQi4BTWOmtqAenFkDyBhUTV/6TRfWsxe9dD0ZovYF2N4c6SgsPNG74i1Pk9SqiM78zg zbGrdoJqEALslVxP/jIImiKjbOman9a0CY2L4Jus0agBZVQYKWi2qJIbDj1nfGmufmYe 7gAT7p07+SXnpm1WmDCn3cFpDZEteUP0kChymcLi0cLECJVQLB8peUPnBJJGjsrNs9aT hnZxLFHWFIf38U9xV31wZ54DFoY4jTonRHEwh4CUscMcPx4K4rMkz792GfnSCNwYOEwo PWyQ== X-Forwarded-Encrypted: i=1; AJvYcCVdEidoZzSXjLC0NDIbOWMQuTx6nnMkYhRwp2PnsC1M4VDfndQHWxyjPx001BIynVQavlvDv9Q3ELufmXL09qaH3+Q= X-Gm-Message-State: AOJu0YwOvwZTAZOaWSnEzKN0T/77UVMrp9rX2AaKSPTs7YbUFzjd2xFv vAalrV8MCnxgf/de332VsvhIR7+8w0A7+HdrU0V7albzcPYY13XEPWgrAAMZDA== X-Google-Smtp-Source: AGHT+IGd+QI2KOJ5c/8/pTyKY+Y+N0EZgS87fvMttED3Ds2Pyg5DEehZ91bfZhC4ZUFBKIIxCloFUA== X-Received: by 2002:a17:902:e743:b0:1e3:c01d:fb17 with SMTP id p3-20020a170902e74300b001e3c01dfb17mr55118plf.11.1712793508162; Wed, 10 Apr 2024 16:58:28 -0700 (PDT) Received: from [2620:0:1008:15:2d0b:3c67:e0d1:ea8] ([2620:0:1008:15:2d0b:3c67:e0d1:ea8]) by smtp.gmail.com with ESMTPSA id ga18-20020a056a00621200b006edcf5533cesm214863pfb.79.2024.04.10.16.58.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 16:58:27 -0700 (PDT) Date: Wed, 10 Apr 2024 16:58:26 -0700 (PDT) From: David Rientjes To: Andrew Morton cc: Sourav Panda , corbet@lwn.net, gregkh@linuxfoundation.org, rafael@kernel.org, mike.kravetz@oracle.com, muchun.song@linux.dev, rppt@kernel.org, david@redhat.com, rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, pasha.tatashin@soleen.com, yosryahmed@google.com, hannes@cmpxchg.org, shakeelb@google.com, kirill.shutemov@linux.intel.com, wangkefeng.wang@huawei.com, adobriyan@gmail.com, Vlastimil Babka , "Liam R. Howlett" , surenb@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, Matthew Wilcox , weixugc@google.com Subject: Re: [PATCH v9 1/1] mm: report per-page metadata information In-Reply-To: <20240319143320.d1b1ef7f6fa77b748579ba59@linux-foundation.org> Message-ID: <65b77d3e-d683-1e90-ebb0-5c7758143048@google.com> References: <20240220214558.3377482-1-souravpanda@google.com> <20240220214558.3377482-2-souravpanda@google.com> <20240319143320.d1b1ef7f6fa77b748579ba59@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: C2849100002 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: e4e8wzhtf7x1n8z5duygsn8z4pyewz7d X-HE-Tag: 1712793509-864544 X-HE-Meta: U2FsdGVkX1/obAKgnm9RYY7Hx1myMdU7HLi+q7eKwCGYNjDstJyzasSrXVaS+FMu3xpMXaWz4pxUT93ImIfO8rOVknubFjKy8KGIPKUqwDflQG+edu/ae26mtcFA4QGMIgfo6pk8ip75PWI0x76S0TyjhKxc66FC9IqrRTqyQM8COaeDXYN72NNogV7ofwssKKQa4PSKg+AgKeS9JfzQj2euqb3WQkVK1xaAjvCHWZy0wHKsUSEgF/Rdfbocb5MG604+pDGoPNQsGOBgYNd5zomhoifDzz5g8CtQfeSlsCC+c2QYk0bG4MEyqajz63oN2XMqFbqp/9BQC2q/467/YkbuSadbAMPC2k7Vt0ix8OjvDN2DRIH8wyySRYYIVy20ycyv3Tje0aGAoHN01MxgFZKU36rjzZ5sGh8KGpqeleIo2g9JHjzWj9Pr/9A/SEzWtLlF2CA35tsBRNVClOgJtP/22dqNHJqnTMWfoLNVic683y/jvhytr8fm1C6JDCo3iXOs1t/7VW20pa8ljuZs82NLnimDxXGKHbe4RlUVvRQpIGwN2c8k+mxdjHLF8CGxt9+cc1/qISwGetNmqoDlSR4FHOWwuFHNaBiJsi4p5mbL8DFc0hMAEwjKJs3JQ8KuQ3K7i7y+gsblIr9hpXVn53rl0y76ljjk0Jx2Kk4srBGV8M9h7ndJF9KzA8UlvJc4DUo19DxKjR/97EbC/mKppByUVaCzZWtnGokTr7VShl/h8/ekpUxjaYc+aki+JyXVx/ITw992Dh9B1kKbbCFLSxEdH9s+wi2gR6LbdmFNpyHR+RHs2w6lnPZryANvpWrDUpDOTNXrHbLsBaz7woJ/OMdImb+aYuraNtrHb8F/AvFi4hgOZ9cgD9pHd/uJyk9tb4V7Ik411FvalGiAbBYjTBdGGsl+y/vnD4QSKCWcrPcI0W4Sl/j3JQaJ5sIEpEr1m+R2QF4KtQ6X5B/hoYj mJWl0N0+ EWsvO3E0QqfFXtPJNboSaK2Q0xPUZXgub2eBjltoVw5CYcjlsPeu6hlD1BOTOUVmvBN9Xht4UYN2Q8/qH5Rkm42m8Iqg7NXbq0f2jW1senSjN4nUIlJnWTmnQ7dG8xOkU2eVNTdxw8yDPjAQxPcAcXNz5S1rlImmMXFT5Q/101z/rV4F592rRszt+8rL/dmvOxmjrJP0okZDI86CpxeKOo/QSetN43tU3PmPj/hkoa8XWSJ9fH4R2in6ukIqycbOr15ODNUUqDMw5eZPu18UWYD9ETOEwy2YwiJbxoExJtJo/K1jBv0+H/tw3jCUXimrrwSjFn2rtXcWEVfoOm8LHIbWgnCTUTKTYALQQ2DKngqBVPmJ8pyDZHX/RQylx4MUxQYMB+Xq4mToeTO1WL+0CeVgMP32IB6Aa0JzEkezS///xAIGtEythiPJWTxnWVqGzOaE51L96aggrBkB1gVI0DR7GSInzjMsK1+102xhkIcNCMj0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 19 Mar 2024, Andrew Morton wrote: > On Tue, 20 Feb 2024 13:45:58 -0800 Sourav Panda wrote: > > > Adds two new per-node fields, namely nr_memmap and nr_memmap_boot, > > to /sys/devices/system/node/nodeN/vmstat and a global Memmap field > > to /proc/meminfo. This information can be used by users to see how > > much memory is being used by per-page metadata, which can vary > > depending on build configuration, machine architecture, and system > > use. > > I yield to no man in my admiration of changelogging but boy, that's a > lot of changelogging. Would it be possible to consolidate the [0/N] > coverletter and the [1/N] changelog into a single thing please? > > > Documentation/filesystems/proc.rst | 3 +++ > > fs/proc/meminfo.c | 4 ++++ > > include/linux/mmzone.h | 4 ++++ > > include/linux/vmstat.h | 4 ++++ > > mm/hugetlb_vmemmap.c | 17 ++++++++++++---- > > mm/mm_init.c | 3 +++ > > mm/page_alloc.c | 1 + > > mm/page_ext.c | 32 +++++++++++++++++++++--------- > > mm/sparse-vmemmap.c | 8 ++++++++ > > mm/sparse.c | 7 ++++++- > > mm/vmstat.c | 26 +++++++++++++++++++++++- > > 11 files changed, 94 insertions(+), 15 deletions(-) > > And yet we offer the users basically no documentation. The new sysfs > file should be documented under Documentation/ABI somewhere and > perhaps we could prepare some more expansive user-facing documentation > elsewhere? > Sourav, is it possible to refresh this series into a v10 on top of the latest upstream kernel with a single condensed changelog that details the current behavior, what extension this is adding, and how it is generally useful? As noted here, the cover letter has great material that discusses the rationale for this change but would be lost if only this patch is merged. So typically the cover letter material gets concatenated into the changelog, but in this case there's a lot of overlap. A single patch that includes a succinct changelog would be awesome. And then the requested documentation in Documentation/ABI either included in the same patch or as a second patch in the series? I don't think the resulting patch series will actually need a cover letter after that, it will be able to stand on its own. > I'd like to hear others' views on the overall usefulness/utility of this > change, please? > Likely true for all hyperscalers, the immediate use case that this could be applied to is to track boot memory overhead and any regression over time (across kernel upgrades, firmware upgrades, etc) that may change the amount of total memory available. We'd want to subtract out the boot overhead that we know about (like struct page here) and then alert on any regression where we're losing memory from reboot to reboot for any reason. This increased visibility into boot memory overhead allows us to create a mechanism to track changes over time when otherwise that attribution of that memory is not available.