From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6117FC3DA6E for ; Wed, 10 Jan 2024 03:51:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C58A36B008A; Tue, 9 Jan 2024 22:51:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C09156B0092; Tue, 9 Jan 2024 22:51:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACFC46B0093; Tue, 9 Jan 2024 22:51:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9977E6B008A for ; Tue, 9 Jan 2024 22:51:53 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4D974A0995 for ; Wed, 10 Jan 2024 03:51:53 +0000 (UTC) X-FDA: 81662027706.18.C622C93 Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) by imf10.hostedemail.com (Postfix) with ESMTP id 82045C0015 for ; Wed, 10 Jan 2024 03:51:51 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FfLghxlg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.46 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704858711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BYH/9c1/9yMK0K3/b+ahOYYkUh2b/IWjshK8NDQIJo4=; b=rSPxuGKzX/io4OHmYEhmX/BGFsyyUOgqaply3ug5dZCj3dWAY3L+JdQVME0W/kmTi0htEL wxdPsKMpPutPgZ73T5Kz/RNnswWG4xo7vlT3jkH6CtC7+Ocwv5Ej5WvssgAwf7CVGcQwCB LMqXn0qOKziSZGLiUTZ8IFwBY7OM40c= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FfLghxlg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.46 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704858711; a=rsa-sha256; cv=none; b=bWr6xCHKikVzRt9MtF3ytvksKX8HbOFHxTEDcneMgjeFhrud4B7QqDj7EjbWu7VWArlPFk o2wT/NgofuQ3+PBlvoO0x35FPsxuKpRPL+f06erzZQmvkUY3tTGLaTPIgZTcALjgLBWPOE +t1CnLIanmuzkfowSkkxgNe14abCFgE= Received: by mail-ua1-f46.google.com with SMTP id a1e0cc1a2514c-7cd5ab5d5bbso1082488241.3 for ; Tue, 09 Jan 2024 19:51:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704858710; x=1705463510; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=BYH/9c1/9yMK0K3/b+ahOYYkUh2b/IWjshK8NDQIJo4=; b=FfLghxlgsk5OqZdwwkXxLsTlCPYGSo7ScD+084PqI41btndiO8wnd+9KwoVckc8DPe Z3G2EXbxK/NqcI6RxWOd0IZBpdrOcHg3/NndYALmnZT+3C3wCMihNchXfvYCu0P+MWab KDy5wokKX/z3SzZSw4asQWqceY+M//fqAUOvRdkp27g7N+D12txhYkMA4Zka1wDbCy+E pJnxBOiaFEyLUX3Z8TkWzhMB17Bwbm9qvwPRmwW0aDo+S9Sp+lyT6kX3BrMJo4yh0+3G 0hPKGaAsLwpUgk0geqHpwN4gmRUyS00IFfKVkTT8chccDIK9LyhZqFGbSnsunnEb+biQ ImaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704858710; x=1705463510; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BYH/9c1/9yMK0K3/b+ahOYYkUh2b/IWjshK8NDQIJo4=; b=Dvsdcg11P5xwbumnHwQwrkAYnRK7WGJ23LBVgub26cB0PokTrEJ6MQuK1UsEw2S7FN LK9HYtlQcUUBeuATnwdQgfBmj3/QPYdX2ofw1MXSy6h6fKpcys43qA1dSP5xToANkQ8V 4u1uxnTlVuJyDwIZKgBjhKh7CF3FR4ANYxLlSV3zQYa31gs1DpDLilA/XyW18sOPihRF 0D34QfxGMhdrUqtvZ2SKqf27onJabJZ+EwyDm9mLia0ylVHgoOwd2+2CLbmh6Sg+7M5s ULBiBJj2ZaS+u7Jf0I3dBi9FRRqLPEKBkr5uB7zo7zrG1YwhA/1kHfQZ4igfx3MUzZQJ 0bzg== X-Gm-Message-State: AOJu0Ywt3/qvYU6UafwydNlJAr69e18C6f+IS6Y2QVZQeugIjf0wJ83c 0uWxqrF3/KdIv8s59Ukf6UTRIMYGhfTebfFmBoo= X-Google-Smtp-Source: AGHT+IEXuLm/aL2xqpgcy/6Ev2k+3Z/Oiq1KTGW0rYu5D3z4VgV/Ua4kRHzl22STtvPqVNCmEYzT79uA8Twby5rBN+A= X-Received: by 2002:a05:6102:5091:b0:467:a40b:715a with SMTP id bl17-20020a056102509100b00467a40b715amr279594vsb.21.1704858710459; Tue, 09 Jan 2024 19:51:50 -0800 (PST) MIME-Version: 1.0 References: <20240102153828.1002295-1-ryan.roberts@arm.com> <4e7445a0-acc9-487f-999f-a2b6d03d265e@nvidia.com> In-Reply-To: <4e7445a0-acc9-487f-999f-a2b6d03d265e@nvidia.com> From: Barry Song <21cnbao@gmail.com> Date: Wed, 10 Jan 2024 11:51:36 +0800 Message-ID: Subject: Re: [RFC PATCH v1] tools/mm: Add thpmaps script to dump THP usage info To: John Hubbard Cc: Ryan Roberts , Andrew Morton , Zenghui Yu , Matthew Wilcox , David Hildenbrand , Kefeng Wang , Zi Yan , Alistair Popple , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 82045C0015 X-Stat-Signature: 84h41i9otdceutkr1yah8a6iwajpypfo X-HE-Tag: 1704858711-22023 X-HE-Meta: U2FsdGVkX19sMwfEApGtmsWVDWCuHL0ysORArX4i5D4Y3tiKyj7hFg3ra6dHzM47lg9ey1bwVHr/6NvoZpEwl+32xkfwM4ugFy30IswJOPmvUu5LsSrH/88NDvJ6gc9aGI55+6DrGizzxtmU7fLnhHvk8gpG04Fxag0xUInUUSXc5jdoHb+tADHCQkYZPP9InH2zruWzvBy/LywNzo6AM+e60VtO+AilAgfYH/xRSoQZ37+XStwAcSTW7ww8FyqCX4ugLzV8jz1fc/jne3jfoDc62fYYAz/qhN9QoichK8UEcMSf63csVAyUg5Xznmopcc+neTpLOreGZcsgn6j44dBfsYU2a7rS5ZNojISyYlhRp5irLfiU8ns5vgX1rVJrmo4jwblyXJ7kG+BjDzN8vr0B6V5nBN7+dryQQusVG0M+rmR2xYTcPBPxQptpZVTFF/WLvea4M931BQuOIHYvpk8dB1lWwLg3IxBEDmfpQzBnMAvrwsZFviU/rvY6ffkWlB6WbcSN1v4i6SDLCrUMsOIFF1iP1R3lAK1ow8D/YkHpTyY0CAjS3Bgt4dUF8itrSrgpv8CT8gIoaCyloOo4grU76rPAffMcLyrdqrEJqpqDel6oBX6y7lapRL0RP8ugC3QjTiGgooODy4kyq7Qq0qwubU7dLN4D8MG84CBANVG3KB2Qu4ZgwX18vFgMFd/PLDnYdjbP+XqpdWHUzz48RCqoz9YZr7b7rIoEDrL2OR4K6bIMKX7mICdisKHOxw1LhY4XdJyionv9681aaUmmobHs8e8HdUktbRZawZHsAK/rqCZnQNzq0vNiUnmJd8GnWR0B/8irksqXSRgIWbnZa2e4qxDYarpLRSiylAz93RALkBqNzOTITltHjPqt6D0EwC73bEFn9rdDHMLRHwFJv3oJm9rA2Bmf0piLz4iffhpcsyb1o3zJFf32pTBtei55nD5VWEpqiMle/jZq89n uhoc7mnd 1EaflXy6ishdXF6PGVzMIJQf7QC2H7RfFlEKmeqA1CEaCRCeeIVxy7a0YFlVd6kmgsiJ/cil2vPpPD+rZuGCsY8JuuIhBYFfJNNbDgmeGQTlgMPy5b/kxmqVrHTrfihmmJMrGe0HFjGAKu+qzhoQLpIs5z9lZZVg4PczRU2XoZllKXG+1Fi3YmaetcbxDTNKp2/sUqSykazqS2uHB8llF9Km/ckkBEHP8NQAsidYk+Uzl8LoRmfRZagZxzkE7WzLFn6X6INwe380fSgUDDGuS0E87tEuch236X/ozdBBvWK5A25b+3/oYnv9xyHKmGCyPY2zg6OEGWIoUAnjAilfiCH5u0nJAky75GsJbrBgFgLbiLe9ob4ECAuw/WA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 10, 2024 at 11:35=E2=80=AFAM John Hubbard = wrote: > > On 1/2/24 07:38, Ryan Roberts wrote: > > With the proliferation of large folios for file-backed memory, and more > > recently the introduction of multi-size THP for anonymous memory, it is > > becoming useful to be able to see exactly how large folios are mapped > > into processes. For some architectures (e.g. arm64), if most memory is > > mapped using contpte-sized and -aligned blocks, TLB usage can be > > optimized so it's useful to see where these requirements are and are no= t > > being met. > > > > thpmaps is a Python utility that reads /proc//smaps, > > /proc//pagemap and /proc/kpageflags to print information about how > > transparent huge pages (both file and anon) are mapped to a specified > > process or cgroup. It aims to help users debug and optimize their > > workloads. In future we may wish to introduce stats directly into the > > kernel (e.g. smaps or similar), but for now this provides a short term > > solution without the need to introduce any new ABI. > > > ... > > I've found this very useful for debugging, and I know others have reque= sted a > > way to check if mTHP and contpte is working, so thought this might a go= od short > > term solution until we figure out how best to add stats in the kernel? > > > > Hi Ryan, > > One thing that immediately came up during some recent testing of mTHP > on arm64: the pid requirement is sometimes a little awkward. I'm running > tests on a machine at a time for now, inside various containers and > such, and it would be nice if there were an easy way to get some numbers > for the mTHPs across the whole machine. > > I'm not sure if that changes anything about thpmaps here. Probably > this is fine as-is. But I wanted to give some initial reactions from > just some quick runs: the global state would be convenient. +1. but this seems to be impossible by scanning pagemap? so may we add this statistics information in kernel just like /proc/meminfo or a separate /proc/mthp_info? > > thanks, > -- > John Hubbard > NVIDIA Thanks barry