From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C218C4332F for ; Thu, 2 Nov 2023 18:06:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C73778D00A1; Thu, 2 Nov 2023 14:06:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C23D88D000F; Thu, 2 Nov 2023 14:06:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC5038D00A1; Thu, 2 Nov 2023 14:06:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 95A7E8D000F for ; Thu, 2 Nov 2023 14:06:26 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6C52F1A0DD7 for ; Thu, 2 Nov 2023 18:06:26 +0000 (UTC) X-FDA: 81413793972.24.E8F529F Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by imf11.hostedemail.com (Postfix) with ESMTP id 6F4774000E for ; Thu, 2 Nov 2023 18:06:24 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0pJkrgLC; spf=pass (imf11.hostedemail.com: domain of weixugc@google.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698948384; a=rsa-sha256; cv=none; b=c47uVCPW/UzAtLQg/RotMCC5s8dAXvO6PN5whZOgkXlT31V/iVDyu0YuyvjlMMbCKhZfm7 YO5YCD2tPql3zSNUSrYGVzhHZSG4eud8eFlcTEdiJ4nHZTPyenSQyaMv6eXGug2XP9/41H WdNPfJg/zdz7s3PxLvXLtTbm2EgfQcg= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0pJkrgLC; spf=pass (imf11.hostedemail.com: domain of weixugc@google.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698948384; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8Yc0Vs7ENia0QB6Jekhs2eJe0iOFbFKiK50kmBF20/E=; b=Lz1SsWbFmf8FkbHu6gnpxVdLzMy6sgwImJzaafIsSBbGFlDQ1+wyvTk1jF30dumP0q2orP 8X69T3zQ2ufMoG9it7yFTJmVNPD64hYuKcZ4Np2yNsnYTu3J2NtP0jeNnAH4sxTolhGTU9 gctn8eQ+gJ/DQcsENK3t+7s1arhBO+I= Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-32d895584f1so677327f8f.1 for ; Thu, 02 Nov 2023 11:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1698948383; x=1699553183; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8Yc0Vs7ENia0QB6Jekhs2eJe0iOFbFKiK50kmBF20/E=; b=0pJkrgLCpXPKlf7DTJebwjcNbSBC4auuDMUBxteMuSVdviHvAiQPbAhixBQbC4Y15/ ts11LbK0ImMAKHbgQMV2LVhMP2GEzsilFXgXe6jURHxxMY0DR2jRXe+3/nd3nA8OZ91I WFN9iPyqBXcqC2vlxFq0umlflnE05/2WGpgwrsuYiMVII9WJDMUcKWse7z10fU2pNSex Xz4ag322LLNwQKACQOMP6I4AvWD1qnWPGnS1/evWtpYqLbQKN1e1eNiJGxrUNk6o8Oq/ LxxKnnYPr5xln4phnCre4Pw/UaHz7McREn5cH8kC9ZI9jWzb+dMFDuKCGUtypT0CmKXg CkUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698948383; x=1699553183; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8Yc0Vs7ENia0QB6Jekhs2eJe0iOFbFKiK50kmBF20/E=; b=Wv2+AqWVKXP1fsvsL3fST1s6ujb0EkAkO17NRsBOfgnPk+v997Q9LTFNutOaqCe+Lf VEw9Z+eVgAJ75eHVfx3cAMKPSv3CIRjbYs/zQX+RSHNxCTw+Kr0PeL5fLvsB4lY+Dnn6 cMgj11dsSIlm4A696pQ4IQoVhgYvm+3phNz6p/LOtv7L07aoNiJ493nuN/G2prtAIWip KnT879cbKemAO6kPKUu8Bn6mIqZOJzLEYbCbuYC1MALUvQyZAaP5ucmCCHD93+Ld/jTV U9go21NM+TimEzk39fLQuGP1TCh9ecACDaOFgHTcoJ/s7LVXQnC+NeyMrHgih0rQyPNo 609Q== X-Gm-Message-State: AOJu0YwGKb0A3MXjklepOBq5r1CJIWSRq6iEM3Oehtoac0kn5rmTvgK2 62D93odVT2WGr+4HKiYWSgFG3OA9F8xK+Xt3UWDWBQ== X-Google-Smtp-Source: AGHT+IHxgMVDIc1myhEwVwiOiXyS7PCrW3Q4+lNTE2vtJGBS0emSiUxvpgM+3Thyp+4mZnvr/GaOhiSSJdvkhfObMOw= X-Received: by 2002:a5d:68c1:0:b0:32d:a022:8559 with SMTP id p1-20020a5d68c1000000b0032da0228559mr15540985wrw.47.1698948382660; Thu, 02 Nov 2023 11:06:22 -0700 (PDT) MIME-Version: 1.0 References: <20231101230816.1459373-1-souravpanda@google.com> <20231101230816.1459373-2-souravpanda@google.com> <1e99ff39-b1cf-48b8-8b6d-ba5391e00db5@redhat.com> <025ef794-91a9-4f0c-9eb6-b0a4856fa10a@redhat.com> <99113dee-6d4d-4494-9eda-62b1faafdbae@redhat.com> In-Reply-To: From: Wei Xu Date: Thu, 2 Nov 2023 11:06:10 -0700 Message-ID: Subject: Re: [PATCH v5 1/1] mm: report per-page metadata information To: Pasha Tatashin Cc: David Hildenbrand , Sourav Panda , corbet@lwn.net, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, mike.kravetz@oracle.com, muchun.song@linux.dev, rppt@kernel.org, rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, yosryahmed@google.com, hannes@cmpxchg.org, shakeelb@google.com, kirill.shutemov@linux.intel.com, wangkefeng.wang@huawei.com, adobriyan@gmail.com, vbabka@suse.cz, Liam.Howlett@oracle.com, surenb@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, willy@infradead.org, Greg Thelen Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 6F4774000E X-Stat-Signature: j8jxd9ur47jmgk8uia71chq889p6xwrg X-Rspam-User: X-HE-Tag: 1698948384-787284 X-HE-Meta: U2FsdGVkX1+LxbfWX+kWTW3nF+RSsr1BoQHXyD295VC6mwEnGSMJvrHn76xiGU1wYvirZceW0LvTV5bzBd7CCST3X4gUlKXXRYMB/tRRn8nNYxGrPihbQ3EOMtM80ZaUZ9n57BQJ9zqpEDNQ9ly9jbukaNwgh/sZKXNClEXx/KAOc90gLVq5OMT2RuDmhtxzl7fRvJTD1PlI3/HvyBxMYfc2x8znYO+8EIMwCm8/ffrClHtObaTO/rYlKCklPnzz7aiD2mch+TNDTBDeGuqczk1QcyKiLL7HaNeUQ/9SKyXCgTqrG2zJt4YgjPjUmQ8yjK8bx5WiErBguZSKRkz5Ro+dOVN26GJK2k8GeKvIl81MD6WZT4b2M4zxqwAWQmJCDAearxUouZyN43Q/h7p2o2PwqDIQkfR36Mkdz5kKabb0qjVzgTY3p/ETJGLrAdEZnkIxvoPtpyP2U65zvmkWxETwCGa1CtjjWk/YFCi46gTJ9/JgOLqxOf3oMUnO/Z72dGYp+T9r4Wfi3AA8ii4djqBmimYtRd6rrx6I1rqdnGerQamRxq8t7MlSTiU4gSGxVWqRN7NkVAyzG6QHImL1ERgvt9+C6u82uKqOlMeRzPjEqigBEtXSObflzpx1c1GmAUUFuZF+ebSDbADu0yyjmNTZewvfhGUTVVAZ24UCP0C6dDfyYD3EpS1yLI9JIYDDivqY94Y08rDiALPyK/MYzSqyHtTXwtt7ur1DhsnleZbiblylE2kzBJL10DcDtFMyL39chke/kwhBkTAu2VaJZH5ZM11/j5UFL2A/EXt83NXlwH9KSbWsRoKmx0eUCU+SuH38ob/EfCoFXTkiiH96x3Yb5WQ7xb+4yfE0ieKInTLe36vHpjiRTZQe4C3RasDic/gcFlTPhmQnFgCF4ht1zhJJYb2mbkN+lEIU4x7DhRD4GwpqlU50pXePVBbnwH14wBjMhmmstrlgHF0kfL+ RHdXu9bh eWjWyToTFk4YXHfsXxDi4a+Zmgpm6M+saENAULLb3z+eYf2NQZJFrA35L6qe2huiQ1tEL0cwWJ1L6fs0bJCch847Ooi3l1xCNF34LGGXdsoOXwD3SB51mzTktvLl/uxcrkcncSBXzRrtZgM5aURJkcdpovgpyauiiMRh3EWM+yAx8ljHD62FJpSeLIdCOYQeKUFGOJfwS4d/l/W3Oqe7a8gWqgn08JPu2VD2V1bqz+ea5bMggHuuGggP9Dj4EffHtPrEXTGNE5wX4cMeCAfgwvK7FsXx4os8C/NlBXY8kZyOWNdrRufWjIghfOGsGgcSc1SkjfBiAhGDqxkC14E3imnke0peJmP47Fgm0buJXJNz5dSU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 2, 2023 at 10:12=E2=80=AFAM Pasha Tatashin wrote: > > > > Wei, noticed that all other fields in /proc/meminfo are part of > > > MemTotal, but this new field may be not (depending where struct pages > > > > I could have sworn that I pointed that out in a previous version and > > requested to document that special case in the patch description. :) > > Sounds, good we will document that parts of per-page may not be part > of MemTotal. But this still doesn't answer how we can use the new PageMetadata field to help break down the runtime kernel overhead within MemUsed (MemTotal - MemFree). > > > are allocated), so what would be the best way to export page metadata > > > without redefining MemTotal? Keep the new field in /proc/meminfo but > > > be ok that it is not part of MemTotal or do two counters? If we do tw= o > > > counters, we will still need to keep one that is a buddy allocator in > > > /proc/meminfo and the other one somewhere outside? > > I think the simplest thing to do now is to only report the buddy allocations of per-page metadata in meminfo. The meaning of the new counter is easier to understand and consistent with MemTotal and other fields in meminfo. Its implementation can also be greatly simplified and we don't need to handle the other special cases, either, e.g. pagemeta allocated from DAX devices. > > IMHO, we should just leave MemTotal alone ("memory managed by the buddy > > that could actually mostly get freed up and reused -- although that's > > not completely true") and have a new counter that includes any system > > memory (MemSystem? but as we learned, as separate files), including mos= t > > memblock allocations/reservations as well (metadata, early pagetables, > > initrd, kernel, ...). > > > > The you would actually know how much memory the system is using > > (exclusing things like crashmem, mem=3D, ...). > > > > That part is tricky, though -- I recall there are memblock reservations > > that are similar to the crashkernel -- which is why the current state i= s > > to account memory when it's handed to the buddy under MemTotal -- which > > is straight forward and simply. > > It may be simplified if we define MemSystem as all the usable memory > provided by firmware to Linux kernel. > For BIOS it would be the "usable" ranges in the original e820 memory > list before it's been modified by the kernel based on the parameters. > > For device-tree architectures, it would be the memory binding provided > by the original device tree from the firmware. > > Pasha