From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F228C3DA6E for ; Wed, 10 Jan 2024 03:35:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B0566B0087; Tue, 9 Jan 2024 22:35:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 939666B008A; Tue, 9 Jan 2024 22:35:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B27B6B008C; Tue, 9 Jan 2024 22:35:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 664E06B0087 for ; Tue, 9 Jan 2024 22:35:20 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 370E840320 for ; Wed, 10 Jan 2024 03:35:20 +0000 (UTC) X-FDA: 81661986000.26.8400E9E Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2084.outbound.protection.outlook.com [40.107.92.84]) by imf14.hostedemail.com (Postfix) with ESMTP id 183A8100011 for ; Wed, 10 Jan 2024 03:35:16 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=jVT8tfuj; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf14.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704857717; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dtkp9JZ3A08khNWSaZ38FFtChZTDnv6coQvRF5LDWOc=; b=yNi72oNXwfNHqLKIKtq8Xxz5XzDGmywL4Ib21zc/CzAVJI2+IUJM8T3bX0TFU0qNQUFRI4 G9/3Uu+wSMerzT9iPJ0MJ7wXZ8ltsw4FkDRWpbSYvRo2xQ7QIwk8tRnO0mht1I2S+kx3ZH SpT8f+BOOpQWsx9Z3IKQRmwfzJV0Waw= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1704857717; a=rsa-sha256; cv=pass; b=a+hyqsm8+D58ZpY/MkhreU+JFPKoycR0YL4UCKNTwuGHsQ2yqOmx3P6BlCji+4nzpukgCW b8e3QYW33/76sXPLLhhanMXGhR4nIlcqWRo5pZM6DQFufmeKIQ8M+crX1J9BTxDOLuKVJk ya71At2pLyZOAcUJmdFZRTk9FNUBVGQ= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=jVT8tfuj; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf14.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J35ijLOElmJN2/qf6m+aRcY8iD1iCCm/wnkIVGFCWNPNnf/WSUvf2k5Q1zM6Jik7nftA4hZqwcVeI1+VrbMwQzJYfIQaRGgdk5BRCubqtWsusFn0QgCp1qRWerfzp9mjSKuE3WEJz7DtGX7RA0TCgNwNAFPhGJZQqg+gw11j6sW4M7WAE+UdPHAhC6zzIh3zc0EKWHjqKYKaVmypcKpxzY+8JVr0rIIMDkUDaTe2UbJTeMsVqNr3dISbjqEpqz6y2dm0Te+ylXHhkhxnkzK2vCtJviBoVCyG/M4p/SnU7tuBTWxaDjma02wEOcij7rPurqiHInrvj7VMBDlCU19H3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dtkp9JZ3A08khNWSaZ38FFtChZTDnv6coQvRF5LDWOc=; b=ArSQzdkI1q+iWb18McRQrVOIgxLNXyY2rCcOBmfKQLR9YJzlYe3Xf1pIpo6ZatVZ1jFI9fKclqgou/j+hNbQF+72atvzUaWdA01w73ZQpXXNEeKGo5W7y1/2CVLoXDgUXQ4R3PRD1bJfnGXXDYVA2gjSTyVEDewVl1D7bxmew1LiIi1bPWx1/JA7X0MNRI/Or0dYtaeyK2OPyjdWlbUcoaQ0SE1VvOPURievTV5otqA2yrVIRiNjnKSDecyR/j3SRkIHVV1NFRWFrpzzdhjuqR/GDXVHQPeRVJXx4zACxfmr/zHPO+h8VvV+sSl4sfu532qmXRtzEeXyxt88W2KLaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dtkp9JZ3A08khNWSaZ38FFtChZTDnv6coQvRF5LDWOc=; b=jVT8tfujmODu20N69kE+VTg+JGv01YrK2xdfbm59tkokJ/A3DIPxmMBe9A69iQYwFMyDA0Q90mOriojURxyzbB6+jYYOUl+GzaUITUx+TQqL2kgRfHw+oMeLQmS2LA5BXVlhBhOYs4gzy6PW6mBQtyRTy/5qiqqZ4BGHM6JY6iggMTPYD5+yn0xiBq/QmSEiyhmmY5OusDmVWPWdq3qVWKXRH/zm+bDU4globP42ab3FK0B+8ZRdt+jFwN9BAIZ/NZo1cLW/lxAYJ6IOXYoeePjq67vFgwiSz3forOXGDqzWlaRvY0451+6A5PRdxP791v+Vqx3n8u0e00ENbQYg8A== Received: from MN2PR01CA0008.prod.exchangelabs.com (2603:10b6:208:10c::21) by CY8PR12MB8214.namprd12.prod.outlook.com (2603:10b6:930:76::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7159.23; Wed, 10 Jan 2024 03:35:13 +0000 Received: from BL6PEPF0001AB72.namprd02.prod.outlook.com (2603:10b6:208:10c:cafe::5c) by MN2PR01CA0008.outlook.office365.com (2603:10b6:208:10c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7159.23 via Frontend Transport; Wed, 10 Jan 2024 03:35:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL6PEPF0001AB72.mail.protection.outlook.com (10.167.242.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.14 via Frontend Transport; Wed, 10 Jan 2024 03:35:13 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 9 Jan 2024 19:34:58 -0800 Received: from [10.110.48.28] (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 9 Jan 2024 19:34:57 -0800 Message-ID: <4e7445a0-acc9-487f-999f-a2b6d03d265e@nvidia.com> Date: Tue, 9 Jan 2024 19:34:57 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v1] tools/mm: Add thpmaps script to dump THP usage info Content-Language: en-US To: Ryan Roberts , Andrew Morton , Zenghui Yu , "Matthew Wilcox" , David Hildenbrand , "Kefeng Wang" , Zi Yan , Barry Song <21cnbao@gmail.com>, Alistair Popple CC: References: <20240102153828.1002295-1-ryan.roberts@arm.com> From: John Hubbard In-Reply-To: <20240102153828.1002295-1-ryan.roberts@arm.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB72:EE_|CY8PR12MB8214:EE_ X-MS-Office365-Filtering-Correlation-Id: 47e4e6da-52d6-4c32-286b-08dc118d27b9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I8LD85TWr8SRoxcHIy31FhZ0v5lIvZ6NSJO3QY3yjjTTpGUj5JNL7XMfXP70EMnaMAWBumVBRFvXI+pXQ5ZTYCBUuCWLpnEqK0pgclVlA54+mTFiiRL2lheq/uWYh+RB4C/QVfra7MR5bEUoABzkec8k42XWz7gnT/4Qw+1TtSFbmHSPOVc/eCdJBrheDj8jjABNGtQQduT/o5xbp+GI0B/QcbH8v52iLGTS5IhnXQvzsj44yWs9ppFpQQNMI/Ka9/0W2p8D1mdK14LfbiPNQEGiNEVFXc1ITcVzDkoZb7t+erBsy6xSorhjoj9c1k0CyekSEYYDdfGFZoAZ8kNrYcHFdCr2CIgua1uPrlamHiE0mymQHBsSdlNpI0NR7I8mwIchuuP8lxiniml0heVac+MMWeBsIrzm66K/5qAS8T9NlhTFoUFHhldINzWEN3bj6ZKprhG73qJC9khFtdmIPY4PjftW6epes4kbivj97NGcX6MoEJlYf9wJOlbyZMa7o7UuNdT67uSKB4p/QUDXhUsfcMCthGolQjzTxa/E0MBtKNMrjfa4vK/gFYCeMp0AYgWX0QaqcJG0luWkJf3WCT4KL8kj4t/RKwVWs84KJUX+PGyyOg5fdz5uZFr4EQOOJ6E6W9BmXnLRzIPiHNen6gOXL1MyUt0eN2/1lEbXlQKnTnkbZ05mCli7vlm5km1GRqE/uuN4Rg1d1uH/B/8++lj+AejtBfG0tnAQKYyIsCTrdboxv9gYEyTAut0LgMIA5Z8pcVy2JcStWhG7nhgHVA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(396003)(136003)(346002)(230922051799003)(451199024)(1800799012)(82310400011)(186009)(64100799003)(46966006)(40470700004)(36840700001)(5660300002)(2906002)(86362001)(31696002)(7636003)(478600001)(356005)(53546011)(36756003)(2616005)(83380400001)(336012)(36860700001)(16526019)(82740400003)(26005)(426003)(316002)(16576012)(8936002)(8676002)(70586007)(47076005)(41300700001)(4326008)(110136005)(6636002)(70206006)(40480700001)(40460700003)(31686004)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2024 03:35:13.2243 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 47e4e6da-52d6-4c32-286b-08dc118d27b9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB72.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8214 X-Stat-Signature: pb4zzftkarug9uk1buwnp95ph9spdrn4 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 183A8100011 X-Rspam-User: X-HE-Tag: 1704857716-798298 X-HE-Meta: U2FsdGVkX195i8Rn/V2/JfuRl0quZHhSTrcNR6Fbx1Gnhg5kG0oRwAAwlAFFFJoPhsXxla7jOFm52jcGc/oA63ApyfVyARxDC6Yw+h/wBjJ9LEXGezknfqVEQk9boQwjD8IWCzimk1Rt3BM3+5v0bGc5RQMWtabF/aHv0JzomBJFBOMs11rPyP7+5QN6yfpj1xGoy9wgSptDloPUNnGRd4bApoP79v7ZnFkRE+1nU3Sc624JDjqmlM2oFl+6Kx6mltwkeWbGf+JHzK/YcycbFgA2EgYCqNiFJ8WX/ngw76ljpRODCIk8V9gyPZBwjCelPv2/DMM9Up4fryeSboIar+IhJRH1YkcIidT/fl8Y4HlL3RzOUn5kIxDFvVc2EXrIz3ToLihcHGGFszcKcKnSMMpIhLHf8Ms949/HytLawui88Q08YHw/869WGf0Kb/CD5tu/38XR/bA+DVg1/2yIRmtMG3PwhI9USyz/RDXwgsF5tH5cl+/4okCR7zoZtxHtSmqvIqckLOFVFz5mPpeu3ufN7m9LWa+0DIowfcAUpi9o0D58H3DdBJHOZeNzD6JnnKzD/S/VDZ4gFGXqPvBsYpgLgdsm1vup/cVGVg0HQ4uLcXpsfhEPCj8RLD6uGuyKkCGNbty/mA6TiL7BCdzsNY+ADdRHTskTmb/1Uxeys1nQi+z6LkOReA1sP+fgVzxjlsPBzrRlAJTELgAt6p5q96rlr+HQgjFf17JtZZ22DncyLO+D1vCODDGHMyiJCvDMFCbqrnAj8ejgTvA6gywoX7NMafZi0drlPGcUqHpa/Ncv1VEKVwlHs2OpEc5FYLYxM+UkQEU4fEKx4PPs81uNrgP8Z7BWjgFGqUOaYKj4/0siNzhLYk3fgkzGB8FY1Q8swcgjEz58POyHzKXtQfdpE07tp8toWIUuyzXCRzB3xF1ILoWr8lf7ILf+QSQVaXWiFolxdVWWZFT2rIR7FD+ EqF2YIgU IRHNTzVEg5H/j8HmAY1MWG+lICmnhGb6cUbo7fRDFL9uVbuBiCCPdgiSWDQx2UMMVVkcZFJAYDOERlGtC0DEDzyuHxUZeLpkG18Kpx+C+OdauYdiiWqBOHSbkU6wNo3WoRZTt259QaG5dnSmWRL0X86xxUF9TPh3twHmdFCHGXmSqaOKX60mpRdP6/cfdRV+Lj/gPxLaUfuUpoOt/d1ZlZY4unLGh8soHfS2SkM4cSfl8hppzuNKnTSmeG/R/IeUNLB8PiuyLpY6TUmQ0+XeTroi6I8pweynAoaU6GeI1oZPCHiBsZT55MfN6MYptPZuYbysfyRCnTFQo6jOMJ7v7kNbIz+O4eS5PVfMfTr9uqopu56xwLvRHheQpAGir20kkc1iJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/2/24 07:38, Ryan Roberts wrote: > With the proliferation of large folios for file-backed memory, and more > recently the introduction of multi-size THP for anonymous memory, it is > becoming useful to be able to see exactly how large folios are mapped > into processes. For some architectures (e.g. arm64), if most memory is > mapped using contpte-sized and -aligned blocks, TLB usage can be > optimized so it's useful to see where these requirements are and are not > being met. > > thpmaps is a Python utility that reads /proc//smaps, > /proc//pagemap and /proc/kpageflags to print information about how > transparent huge pages (both file and anon) are mapped to a specified > process or cgroup. It aims to help users debug and optimize their > workloads. In future we may wish to introduce stats directly into the > kernel (e.g. smaps or similar), but for now this provides a short term > solution without the need to introduce any new ABI. > ... > I've found this very useful for debugging, and I know others have requested a > way to check if mTHP and contpte is working, so thought this might a good short > term solution until we figure out how best to add stats in the kernel? > Hi Ryan, One thing that immediately came up during some recent testing of mTHP on arm64: the pid requirement is sometimes a little awkward. I'm running tests on a machine at a time for now, inside various containers and such, and it would be nice if there were an easy way to get some numbers for the mTHPs across the whole machine. I'm not sure if that changes anything about thpmaps here. Probably this is fine as-is. But I wanted to give some initial reactions from just some quick runs: the global state would be convenient. thanks, -- John Hubbard NVIDIA