From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A815C25B79 for ; Fri, 24 May 2024 20:00:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1D43B6B0085; Fri, 24 May 2024 16:00:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 15D246B008A; Fri, 24 May 2024 16:00:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3F4C6B008C; Fri, 24 May 2024 16:00:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D39D36B0085 for ; Fri, 24 May 2024 16:00:14 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 56042A0568 for ; Fri, 24 May 2024 20:00:14 +0000 (UTC) X-FDA: 82154355948.25.0A83219 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf30.hostedemail.com (Postfix) with ESMTP id 9272F80019 for ; Fri, 24 May 2024 20:00:11 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kh6v3cSV; spf=pass (imf30.hostedemail.com: domain of rientjes@google.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716580811; a=rsa-sha256; cv=none; b=NbMkU6S8iZfkp68263mHVPth6ZXCC0IisSJcFV3CWxzlv7mvGdPs4ofUmaIQ4MVhhnQsQH BYPTFMc3Qvnpgp1K50sNQaNbk6j5fQoDPHp7jnGTjmtAfxh8kRogWQ93I3jhhBXcO1DAv3 b/+MwmN98tJaeUeDTgd07zqYtWWn3VA= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kh6v3cSV; spf=pass (imf30.hostedemail.com: domain of rientjes@google.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716580811; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J03u/azUiuyJ/7gPFey8suUIJZ5rc1p0eW52hXcyf1w=; b=NkneK/2xmm9HF9VxL2a9XylQONwqamPc3/pwJwrpmKrTr8gIVIIZI5lcwyWiH802c7yKhF /ZANZF0x3m/MMNvMTLgkbsrSH4KoFqZR0OFWQbtBbITHxDUolamPi3P5F/HaaR9r9bW362 S4P8Yak2oaUCDhuWGFTp2BS1L0Y7Nhc= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1f337d0e382so33195ad.0 for ; Fri, 24 May 2024 13:00:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716580810; x=1717185610; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=J03u/azUiuyJ/7gPFey8suUIJZ5rc1p0eW52hXcyf1w=; b=Kh6v3cSVUdgetdigLPVUVPG1L4gYhnpJTX7G7zUxokJd+5mxvbvS49KsqAgFIS3k8X bgVxsV5EjT1nN9fPaSruhKGUN+IxvJCQv/V+kJXzyIQyqpux3qzZho1r7s/wjss1bL0i 63GYvSJNq9XqgXu0NP9dN1It7EL/ebWwScopOxYn7eLUJzXmCbBuFIN//3Rew3x+h4F2 cLqa2oPFRjrC924OeKta/pv/kAW2W6RlAd8HkmjWJFdt1/Vh5FoIkLXb64IvU+8No3K/ BNbaWcLwPbqPuU9bC4P1R3btUhznDWbwujUo64QRPgLdfAJDLDVZU9Lq+skWW1GXDJLu pHnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716580810; x=1717185610; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J03u/azUiuyJ/7gPFey8suUIJZ5rc1p0eW52hXcyf1w=; b=CNaJGdv/EstAAEFcOy0/bS9qTzeD5pW99KrSM384j2KowEV496WjuzCcb2Ry/0SQTJ HU4NdUp9qN2oUc9YpxiJSqnotYJMHMeRCwFCSjfC6ripL9Z2HqEW0SxHtl2yEHEDJvHr o4ilDwCjaT5y1THXhkTM3SyzAdfxW4Nt8df3y3HwQlGxucdvGFzSHTyjj1zVGjT0UqoQ lIGfs7pVdgNI1sZ5tqaPIoEKZoSoIn1R8VlKMVCkdQIxJ68WFyPtrewpVKACLff8pOUY F7Bo2PKYnwPsNj+bLST1XgvGEfaVr4IAIHtCC2F+AidDu2RfJuN5nLBWHm0HG7tzrnts fALQ== X-Forwarded-Encrypted: i=1; AJvYcCUKu9yXvgYchDN6Hxx7q0S2qt18G47wuleRcoMF1THqbuQsmUcIlYIiZcdBrmI3TZhjA8yfo9w4boOS1uGST7lk/io= X-Gm-Message-State: AOJu0YynnF/fEUfIV/BR9St68Wka1XxOGd/cYMwSRDC+Ko5V5r+bD1KT g2npu1hSKSMFrntrbd/jVRbg/lKkRKUXKsI/qL473qrzJ/GUGoQnln4UMRtmaw== X-Google-Smtp-Source: AGHT+IGCIXQDdrO6BGUEBPI0rbMRLEFS1n4FDhL/LvduVdqztKRsVCv6w1yspdPe5J3ICBr0hoDTAg== X-Received: by 2002:a17:902:d902:b0:1f4:50b4:a50b with SMTP id d9443c01a7336-1f466ce2270mr381615ad.18.1716580808226; Fri, 24 May 2024 13:00:08 -0700 (PDT) Received: from [2620:0:1008:15:c92f:57d0:1ea4:5439] ([2620:0:1008:15:c92f:57d0:1ea4:5439]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-6f8fc05e08bsm1456208b3a.52.2024.05.24.13.00.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 May 2024 13:00:07 -0700 (PDT) Date: Fri, 24 May 2024 13:00:06 -0700 (PDT) From: David Rientjes To: Sourav Panda cc: corbet@lwn.net, gregkh@linuxfoundation.org, rafael@kernel.org, Andrew Morton , mike.kravetz@oracle.com, muchun.song@linux.dev, rppt@kernel.org, david@redhat.com, rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, pasha.tatashin@soleen.com, yosryahmed@google.com, hannes@cmpxchg.org, shakeelb@google.com, kirill.shutemov@linux.intel.com, wangkefeng.wang@huawei.com, adobriyan@gmail.com, Vlastimil Babka , "Liam R. Howlett" , surenb@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, Matthew Wilcox , weixugc@google.com Subject: Re: [PATCH v12] mm: report per-page metadata information In-Reply-To: <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com> Message-ID: <1e1b89af-3b7a-7400-cfd7-d22a101955de@google.com> References: <20240512010611.290464-1-souravpanda@google.com> <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9272F80019 X-Stat-Signature: r8kd6u7r549b7ncbsh5y3j5mphdft6wc X-HE-Tag: 1716580811-972101 X-HE-Meta: U2FsdGVkX1/PqTMVsBUEpa18C44Np40/zFmBbuI3FBzcC/0qjbjOM/wvDXJ1YHlZIXjPSwwHzqiWoCpEUmRYwS5wB0bExnPwz5TyZE8+OyvzhN61Q9ZH1WNZ1+Vr0zwDl8hdNqwRPzokjxv+qJSa0I0zR81w5lA0g0M09KeINC7rk13Dt0axh+L4VMneOLn/GN2Ri//44I4t6YobP1J/AfQ1tH6LXl/rDHh3zBcxfI7dEG1Qo7aVL+81zpcG1wjn/DjJ3iXj8l5Mn5P2Qn4kP3hZwDRKYUohsPzBb4eFKUU68e6oEZro0sRFNfW3OvE/WJkrjuoAw1JbvyaRcF+70tnC5RSlZNnjrkT/EQpPgpETqViv23EtbB/Pq6GE6NTcGUBDb6aZintUNt5PKN7zTYbAGVKwcAsPwB+GzlCEZwkvfkbnPv67dTw68gTOQLcYNzZoHM5NDQtyXL156UQ7auvRQPSGRH+LcWuNxYFJuofekJdskrA5v54a/XWRZoe6GMQER8EKS+pajgNRCfUfvs547ndlG5XOPDlGVvJ17nExOMiVLy02Fxj1nGDr9u93al67xMYPE40fXX36kj1EJvFolQ0gH3tsO/8hGS9mCmzp3V861Qmv03D/NvRLlXW3Y11NGOTFTytpkaAvNDW7UiAZaXrHVtgKaf/afYtZdmiaTg8U7DyB23xKzMX9IQbJwJpX4Z7iIjsQ4Cm248cdI0xkjTkEHZIwaMEuhNzzh4k/BX/9+vBtGy5lCEU+2+iKIY5QkmxohEeKDbB/3pFtmZVEyQq7Z/9xdX29lrOPHU36hdNB5fhPtVOVnLTTmYkZ9WLNqumUKF2BvqSBGOEK81Vsl0JathuKmFztzZuuz4Chzj2EyDx1rOmWnNQMji82gDte8i/e+F9rj5KZZGzNrXABKLu9rHl+a/c5JD2E9VASbGwWqAVvrRqPDXrr2alEgtJkXSr9EMFjLNTb9aC qyvUEA1j QH71rpkUl8PNspc3cxHIhdFXnW2NScRvv3oHe+XmSHBd1xxBZ4dYxRPFPjKw+dDAbCFC6aDyvoWrM0cbTzSmc0YhkXPT7Ud1P7jLJ1BWoemwJsS45gwpVh/TrTxJIcEHQNvT8kRWfg2mAfF0eTb8i7zAzIsWdWSflj6HAPsMEbgCs3X6Mm8tuw4gUPNwSrsTwWo09A+dEvjr6kRwRyJt2IrTWRpgDIfxO0oyAqFyFujLUlZi2wP9BKucUyNI3I4AusFaRKzZoRlGzVB9205Ayz+wtEh/JhE2B7Wbgp/JOmfD7DXLT2SpjMb7UwPokJQ7tYUn2Qtn2FAy+RjvsdkQ68SS/AYQ+SdJanUfDulg/QbPLD/sfkMQ745eozIqoZMOkhuz5qABfeBWURoPh0LriFB0Olznfi6sT4zUksGqLaJbEaNJc8YrjRuRLir+5v+TwKufls2zp5tmjOF8tqGlk2DPT+BqkqK2iJ2Zp359fWjCAzNtPaHY95WT4P3HB2pzucOQlYgTxFt01jRonpxnUWq+KpqcLoPNJ53O04ItP25voYGv2RgiP0og0TvfAOWUzejR8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, 18 May 2024, David Rientjes wrote: > On Sun, 12 May 2024, Sourav Panda wrote: > > > Today, we do not have any observability of per-page metadata > > and how much it takes away from the machine capacity. Thus, > > we want to describe the amount of memory that is going towards > > per-page metadata, which can vary depending on build > > configuration, machine architecture, and system use. > > > > This patch adds 2 fields to /proc/vmstat that can used as shown > > below: > > > > Accounting per-page metadata allocated by boot-allocator: > > /proc/vmstat:nr_memmap_boot * PAGE_SIZE > > > > Accounting per-page metadata allocated by buddy-allocator: > > /proc/vmstat:nr_memmap * PAGE_SIZE > > > > Accounting total Perpage metadata allocated on the machine: > > (/proc/vmstat:nr_memmap_boot + > > /proc/vmstat:nr_memmap) * PAGE_SIZE > > > > Utility for userspace: > > > > Observability: Describe the amount of memory overhead that is > > going to per-page metadata on the system at any given time since > > this overhead is not currently observable. > > > > Debugging: Tracking the changes or absolute value in struct pages > > can help detect anomalies as they can be correlated with other > > metrics in the machine (e.g., memtotal, number of huge pages, > > etc). > > > > page_ext overheads: Some kernel features such as page_owner > > page_table_check that use page_ext can be optionally enabled via > > kernel parameters. Having the total per-page metadata information > > helps users precisely measure impact. Furthermore, page-metadata > > metrics will reflect the amount of struct pages reliquished > > (or overhead reduced) when hugetlbfs pages are reserved which > > will vary depending on whether hugetlb vmemmap optimization is > > enabled or not. > > > > For background and results see: > > lore.kernel.org/all/20240220214558.3377482-1-souravpanda@google.com > > > > Signed-off-by: Sourav Panda > > Reviewed-by: Pasha Tatashin > > Acked-by: David Rientjes > This would be a very useful extension to be able to provide observability of per-page metadata overhead and the impact of things like HVO on the overall footprint. Today, we don't have observability for this memory overhead. Andrew, anything else that can be addressed before this is eligible for staging in MM unstable?