From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB441C4707C for ; Fri, 12 Jan 2024 19:15:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E71C6B0081; Fri, 12 Jan 2024 14:15:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06FE86B0085; Fri, 12 Jan 2024 14:15:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDDD76B0087; Fri, 12 Jan 2024 14:15:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C6D2E6B0081 for ; Fri, 12 Jan 2024 14:15:45 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9B2201A0E2D for ; Fri, 12 Jan 2024 19:15:45 +0000 (UTC) X-FDA: 81671613450.05.FC0FD13 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2068.outbound.protection.outlook.com [40.107.93.68]) by imf07.hostedemail.com (Postfix) with ESMTP id C9D054000C for ; Fri, 12 Jan 2024 19:15:42 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=uSziR0ov; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf07.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.93.68 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705086942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sLr8fvOyAgPYSUVlF4aTOtNXGiqHKuehRnpor6/7p6k=; b=joXX8xJpI3ImyA86HyvH+YYpypjpnc08jVdRQcqPHPevl6Jg8g/guaqUhfhsrVyEptNVue vLlg1+cWQcPCVl/gE3uTjsnhmEk2/3Zx/5ytVSMfv1INN9TxMuM0fMHS/4mC2H0caOtLCg U42imJ7RXd4LXGdzAEA3UntObubXK1Y= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=uSziR0ov; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf07.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.93.68 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1705086942; a=rsa-sha256; cv=pass; b=bOkbSX54rhVVdFTwVxFbqMrnSICGaZkeNEo0OLFdluhTKBFLV8dJXaRw2Nv4MqyR688X9/ eDs9+PTpwiB+FFYsENmGiu41h76+WtPEI/94Kz4tyLIUh2aaCSaXBMBEQcutYWgenz32nq MB+Gfyo4k8D9lJi3J8SjQa3WC8yQ4Y0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Dths1SRnm5TBGnqgLPGDYOQWLgpi42/UMk0b1p7WskCDjQHXnZgY9XJFyH046PatNbFd248GEblSa/4RlX4LM0wL3/6MbsJ0/mLMeFJ9oyuYKW8eG9oE/RHcMVxQpnf/D65dBphF/vqGOhzXQdtTiE94vZjYYClyWMqgL4I/SPTx9OmTjr94nsXuqmCIOIdlp2yKg/ibYxHzAWjHiomLtUbrldhC+feJO+VxnVwTvzN3fJG0uAKRpM1wpBwORLCwvaiR46arhWrvN+jWJwTkmmhROUQOHpEjGaO7N1+ZscY3uqhTSm4TMRDoP9B9ggjrHrlds4owNcMPSn83ZYvHgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sLr8fvOyAgPYSUVlF4aTOtNXGiqHKuehRnpor6/7p6k=; b=Jw8NG0UjE0pBxk1U4W3PQUq9dfs/zWzwXvhBy8uviSzxYcGNHAWMf+Wpgk/vPomEzcTAxsblP1opKJkUCZZxPaz+POiFJJ+rsvV50XqOEGVfMs0dhdfjXjqVczJu0tCG2UdJ21T9HZJ/lqfiKJc1ChZBomPp6bagqLgKYwNhARfh9k4TJ812/XISEaJEwprfiIReSPuyM/pPxoEz/YBP5FdsJBY2SkWMBXogC3uSb4TlaOYTAnM2PFZc4/Zcp0lCkBQptWcGge5pEozefj5XMpQJHOAs10XbTKaMy/y0fWHy4qCem+rfdbx4fl9aSbeMA5p+h1qEOFpQbFIXSPUaAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sLr8fvOyAgPYSUVlF4aTOtNXGiqHKuehRnpor6/7p6k=; b=uSziR0ovDWLp8gMX++wSNZHe/YCDdcMzNFxCnQryR358hWyhB2cTCxvUJVar0Iq3JAyAV24LVZRkXrhnj+FSRLjumLbazjVRKIn/wAS5s3HBY7yNdFgGJq+x3Prn0KsLJmPDtx+pr05MwhD77uw0Q/kSlkPhfaJWhup6NS49Cq+nP+PDc7cRJsl73wk6xrDgY7/sl2Yeoz0n0XFRsP03Rgrp3bch+jEYgU5Yo5b0RfsFKVj5hSFebBMdMIF1Wxp+MgeqNm5qJlu8IH6vPTzowfUtrvHzqDLVy63mQxtknfqvBWSfTtaLmGeu89zJUmywZ0eQ+0xZ8Nx/iyx1TWl4Mg== Received: from BY5PR12MB4130.namprd12.prod.outlook.com (2603:10b6:a03:20b::16) by DS7PR12MB6045.namprd12.prod.outlook.com (2603:10b6:8:86::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.23; Fri, 12 Jan 2024 19:15:39 +0000 Received: from BY5PR12MB4130.namprd12.prod.outlook.com ([fe80::90bb:c583:cc57:aa1a]) by BY5PR12MB4130.namprd12.prod.outlook.com ([fe80::90bb:c583:cc57:aa1a%4]) with mapi id 15.20.7181.020; Fri, 12 Jan 2024 19:15:39 +0000 Message-ID: <0f5b9444-fd79-49f0-b9d8-f5e04c044696@nvidia.com> Date: Fri, 12 Jan 2024 11:14:52 -0800 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] tools/mm: Add thpmaps script to dump THP usage info Content-Language: en-US To: Ryan Roberts , Andrew Morton , Zenghui Yu , Matthew Wilcox , David Hildenbrand , Kefeng Wang , Zi Yan , Barry Song <21cnbao@gmail.com>, Alistair Popple , William Kucharski Cc: linux-mm@kvack.org, Barry Song References: <20240110173203.3419437-1-ryan.roberts@arm.com> <33341ca8-1354-4f3f-b377-0b7d04da48d0@nvidia.com> <43230798-af22-4f59-b37c-8257bae32af8@arm.com> <22905bf7-570f-41a9-8dd0-b8a250c97de3@arm.com> From: John Hubbard In-Reply-To: <22905bf7-570f-41a9-8dd0-b8a250c97de3@arm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SJ0PR05CA0009.namprd05.prod.outlook.com (2603:10b6:a03:33b::14) To BY5PR12MB4130.namprd12.prod.outlook.com (2603:10b6:a03:20b::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR12MB4130:EE_|DS7PR12MB6045:EE_ X-MS-Office365-Filtering-Correlation-Id: 6c1b1133-3bd5-4bfb-b4eb-08dc13a2dcca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kAEQZ+tXTV62jx6N8gfrWHe16lffjBR2bbSgZ2+iwTx8tG3DwOpbaggRGaBM52jAViLQ2XvQzhtmaZP+tlve8rZHeeaYvYT3vp03MhwNDEoMSEXy5Du7S4lwa6rn0dsH1Bu7aFVfrbAfEqS/58N5SucQsgpQnULfhiVqMD0gzOsvR8hq0alVW8othPLhPe707NMooOCrOr7TsQBHo4OWnQ5efVb0UBzzsG32UQaQlXYg0huBXxn5YoBA0p93JpLYsDdbBkFrUY5aZITwGwGhIiiCLG0xyKThfcLcB1sOg7Ere48fatE/uNi5ft/UNEy5MnK6CW5XB9YG5ZJs3pqJFOOBv/OYW6pEzBVxB6e4nJYCAE+6CRjsyUp5ljYJ9rSfnhU6WB3WUdURbtkZz/DH3d+LbtbEz4kSs6nCLlHir62cBXqdXq1dJlD8KWEx/ACBq36MBetr7dxZzmAECACT+Y/9ajcovWyaiGXViGxqXrd07ZONG3+vom7bjos25ZwT6kW167lwTxmoeuyN21HIsTtZFvrQovILCAAS612kH0XM3nOgYxHGufyzMgJR6eafdhn9oLNLH7a8pjVaHT0QKwKe7yDyzzKWIpI9tT5qaJc8N6/75AcGckP6jtrvKvkk1acWukn77SmkgvM/gYqrtpr5aIWUUwTQE1t6QreFPlA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR12MB4130.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(376002)(396003)(346002)(39860400002)(366004)(230922051799003)(186009)(64100799003)(1800799012)(451199024)(31696002)(86362001)(38100700002)(2616005)(83380400001)(36756003)(31686004)(921011)(478600001)(5660300002)(66556008)(66476007)(66946007)(6486002)(6506007)(6666004)(6512007)(8676002)(8936002)(4326008)(53546011)(2906002)(7416002)(316002)(110136005)(41300700001)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZDZEakVwTmVqeEJQWUtuUC9Sb2s1QUk2U0tRaVpQWHhLd1FjazY3U2lnM3ly?= =?utf-8?B?ZC9QaDFMWEpxV3Q1UklzTnMrSVhhK2hQd1hXM2ZSeUpkaGhVSXF3T1RSSXdK?= =?utf-8?B?L200QXBsckJKSyt1VklnM3l4emhXS1ZYald4WjdaOUlRcUoyWHlMald6eGI1?= =?utf-8?B?UW0ycHl0N1ZPSWYwT1JzWkw0T25xaFA3U240T3l0QUVydHZqZC90c3p1bUNV?= =?utf-8?B?VFpGR0JGeXRpTGE1elMwTGFoeTNManZmSDJudXNhZGVnMlJQWGZ6NE83UVJD?= =?utf-8?B?amhYdWdwN01VWEQxbGNJdjRER3pDQktjSDlrcitPeGtXcjNOSnlYbEwreldC?= =?utf-8?B?QnlWREVxT1ZneVIwNkM4VWR5UStBVGY3eWJPOC9SK2Q3Y0l4UWVNUkkvcG40?= =?utf-8?B?LzBYRCtJalF1WWNxQWNKdXpBd2taTzlLZDNkMDdHMmNQMG16ZlhkUVprcGtW?= =?utf-8?B?bzJ6SkFIRVluajZEWU9MeEhiSThHdkg1emE0bGRpSHZmMjYxUUdlUTQvdkRi?= =?utf-8?B?bDFRZ1BOL0x6K3hJTElKTXJlZENOY0tkV3J1RWlBTmxqNW9KUmY5T0p5RVkv?= =?utf-8?B?WTF1STVHSW1EVGtjUEM0NTNFNlgwcW82SDNua25nUG5sL3pwQ1hha3VVSktE?= =?utf-8?B?N2xSTnB2VjA3N2FiVEFJMWVUb3pFN2t6QkllK0RtaXk3aEsva1FtR1BPQWda?= =?utf-8?B?aGd2dU5hY1NMdm1EdUdIWFAybFZBYnBPRkdRY3A0emdCMmJxYUxRckczVVBv?= =?utf-8?B?WS91RGRJQlRVUWFodkpxQXFPZ09GUVhSNXNXajBqNmMrSTlhOWFMRUk4elVW?= =?utf-8?B?NS9WSlFiK3I3MENYSWRhakVWZkpsNmRmUXVyWkozZzk4UlNaMEJXSkoveHl3?= =?utf-8?B?TEh3aE9CcTZOSXhIME1hd05yUGRTR3VPNWwwZXBLR3EyRlYrQzRyVERGcExB?= =?utf-8?B?aElrNktNYy82VjcrSXl5SklVUFlhd1B0Y3hwYXFibVk3UFhiVVJGQzRieGN1?= =?utf-8?B?VGdMakhFdU5EeExwMk1WU1BWbjMzN2JHVWRwQnVOVmp0QThOcHkxSnhVRVlW?= =?utf-8?B?czlFa0xXMjZnNDU2L0NGajZzNWtvcXEzVFUvcUJuZ0xNUTZETk1uN1N4SUdj?= =?utf-8?B?eU5OYjhyeWtiMDRPRThJMXdCTzgrREl3T3ZCUHVpTjJvazFBYitBeU9TS3Qz?= =?utf-8?B?a3p6alF5Rk9GWlUwSktTUEQvRUx2aDRJV2t1R3JsM01lRUYzMEkrZGMwOXJ0?= =?utf-8?B?UllpN21oNUM5dEZMU2VtQW92V3lyelV5enFKZnFWYitVS3kxMXdOcFk0d3VJ?= =?utf-8?B?MkFLVFpJSDJMeWJsMEd0V0N6N21mTDcxSnViTDJacjAzQnp4RGZ4Rk04RTYw?= =?utf-8?B?aHpKU1JyS2pxazNQei94Uy95SFl4RHc3ZHZVSWljaVpZM3hFd05mVlNPUnk3?= =?utf-8?B?Q3ZDMG9zTGFGcm9GaTRldlp5dTl0Ujg5WkhYU0ZBWDFNUUVZUnNleEpaczFy?= =?utf-8?B?c2V3emw5MGgrVnZ2MUQ4a1NvenZ1dkpLbWt3THphTTJqS293b1IwZkhXV2JI?= =?utf-8?B?N29sWFJ4RkVkUnkwakkxdTRYR1AyaGhRb2ZyTGV6ZVgzNTVOT01zb0xFSGRD?= =?utf-8?B?VVNBQTFsZk81OUIvS3FicUk5bDBNWFVHV29NUGN2MXdWK09Gb1pJemJFSExz?= =?utf-8?B?c1YrREY5TkVNTGRMa2hSekRUdEdld1Byci9ScGUvU3d3NEpDY29BN29BT3hY?= =?utf-8?B?WXFseUxkU05oaHkzMDRXTHRpKyt4amFjSUovVXhPSVJyd3pQd3VnRGFDRW1j?= =?utf-8?B?c0hDcXJhOGlhZXJkMU5HaEkrWjUzbFJ2UW9vbm1ud2Vqa0dLbEs2dWxXQkcr?= =?utf-8?B?dTVpcDVSOUNGdUJaL0tvZXlkdTlLQkFTem1RSkNWZXI3Z2VKQmh5V2krbTVy?= =?utf-8?B?S2dzUjJ6VGtJSVczd1RjUjBuU1JBbmlTL0N3N2VOQmpTRUthWDlBWXVrc2h6?= =?utf-8?B?V2hWZ0NzYmd5NnMrUVRXVnErZEZEOHJBU2lQOStzQVZWVitxcExTTkczc2Ew?= =?utf-8?B?bGZFVjVzbmhUUWgxNDhQRlVkZENUNzNiZ2VYVUNlSzZZTWx2VzdSY09PZkRj?= =?utf-8?B?ZElOQzFjUnFUVFd3MXhsMnNNd0l3TnhQc1RZeklLTG1RNlBGS2lrZ2VGc3N3?= =?utf-8?B?YVE9PQ==?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6c1b1133-3bd5-4bfb-b4eb-08dc13a2dcca X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4130.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2024 19:15:39.2141 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RDYtKd5aZu+XZ2ZHODqmM089bJXwflzxcSJo2LCYVA/Hq4qh+1K5LazDix1oub1LS/PV4S7hSrZ1kkhU23wXAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6045 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C9D054000C X-Stat-Signature: 7jppzftayaate97rpmxhoox48siwzcnm X-Rspam-User: X-HE-Tag: 1705086942-160425 X-HE-Meta: U2FsdGVkX19AktpWrI+ENY9kLuGEDkbvXOdlQbjhpJJPrKoLEfW6w0403sWyCczy3xlmVbRTzZnDKSJ6Ay80TdGgnzKl2ZrRmMqsDn8NeEpLgsghBk5Cx3en1JuaXbz8cgplAzn7bvCNsjeyVJNaTDcap19dOpwh8mXwV4pFrk3LwPhcoiMuX2hnDLpO7hAY3lAgyDzf7EK0L/RsbvRfvhSAdlgpJJZdhtVuxE/QWuAT0l4kXzqsjeVV+JRRbBDVPNII9FVwGMI4FZadvkaGU2EN7xnZoXpg+cuhDSDCBb8HRdJWN4Ep9YXJyDI30qS1mv+X0F9Fx0zrsgdiQr7oPB+6oDgq0Ta4wID8OJYsmPeFkk/poVlYGqWBo+ThgnIUx2m1oqE/wXmVaZzQwZw5HiK0JKQn/v56RNUzm722HHIWQJRvZa3OIZHISycojhXz2JYJD6lX2ffSRiFlZ5DEYCeuawyTceS6HCd/+zQS0QaTCQSrcjCxasRLUIRtyjpdi4XknZoMdTC9M3DZtFYhTXFgIQ2xSGA30R0rBR8+qeCEQ/umMOz4pqZhEatsXQljCmr1668FAQ32O8Hoj+Ig31qVEQZiL6IRUYe+UM8t1vGCOUWGoGEL9R1v+Pp3iThNRfFRnGodhGrc7dUR+HO+tNF4Y+2m77k4lg6mqMzqKDCAeQ8OcgzvM7xDH1kbFTgaQBMhXS1EIoL/OrNKfJEfnDV1vqlNywv2w9J0KSOYbaF7j+FwuHiUnZ2GG69tU5inUWTobEZ6woiK1ahfJjumnLYP5qx5P0v1pzZ58japyiDOmcWgl3FTn/VQBKvwcYamReHQg4Kr0v5fLufLhwwcpDbE3CG6B8DW0VtVRpJpQ4PlBtj2R86tLEsFb7SvJqlwlRh6YehwlP7sMdt9fqGvKAXXbbRhTrTff82Wuf6KxcXsZRFmL62MePO1nfY9hJZA7QOWozFMVMvExh6dWRo ZG/S6Xgb 7svIE9OFZmpjWanqXfCp0VZu+GNltICBZNjZvZ/HweC1mrH7DX1n/Dp1T+nLM4nnOj5++9kcxtkk+LkWSP9wg1+ATp8dwlc3QBMuJj1bOa+WeI6dcaPknP8pJn/jyE9cEa5Ku+mE/ThN3oW08pcdtSjRS05fc+P2+dWN0cE1ELGsv0qWDWo3uKYJ/Ut8iHYpvGOrnNgaGk26MErkSIQbBgG0UM139lJObemOw7BhO3dpb2ZtBSvYXRDgXL7t2FgZTGd+c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/12/24 02:00, Ryan Roberts wrote: >> ... >> After spending a day or two exploring running systems with this, I'd >> like to suggest: >> >> 1) measure "native PMD THPs" vs. pte-mapped mTHPs. This provides a lot >> of information: mTHP is configured as expected, and is helping or not, >> etc. > > There is a difference between how a THP is mapped (PTE vs PMD) and its size. A > PMD-sized THP can still be mapped with PTEs. So I'd rather not completely filter > out PMD-sized THPs, if that's your suggestion. But we could make a distinction It's not... > between THPs mapped by PTE and those mapped by PMD; the kernel interface doesn't > directly give us this, but we can infer it from the AnonHugePages and *PmdMapped > stats in smaps. Yes, that would be excellent! > >> >> 2) Not having to list out all the mTHP sizes would be nice. Instead, >> just use the possible sizes from /sys/kernel/mm/transparent_hugepage/* , >> unless the user specifies sizes. > > This is exactly what the tool already does. Perhaps you haven't fully understood > the counters that it outputs? Oh yes, we are in perfect agreement about my not understanding these counters. :) I'd even expound upon that a bit: despite having a fairly good working understanding the mTHP implementation in the kernel; despite reading and re-reading the thpmaps documentation and peeking a number of times at the thpmaps script; and despite poring over the thpmaps output, I am still having a rough time with these counters. Mainly because there is a set of hidden assumptions, many of which are revealed below. But it's actually just a few key points that were missing from the documentation, plus the ability to clearly see the pte-mapped parts. And your proposed changes below look great; I've got a few more to add and that should finish the job. > > You *always* get the following counters (although note the tool *hides* all Good. It was not clear that these counters were always active. The --cont documentation misleads the reader a bit on that matter. > counters whose value is 0 by default - show them with --inc-empty). This example > is for a system with 4K base pages: > > # thpmaps --pid 1 --summary --inc-empty > > anon-thp-aligned-16kB: > anon-thp-aligned-32kB: > anon-thp-aligned-64kB: > anon-thp-aligned-128kB: > anon-thp-aligned-256kB: > anon-thp-aligned-512kB: > anon-thp-aligned-1024kB: > anon-thp-aligned-2048kB: > anon-thp-unaligned-16kB: > anon-thp-unaligned-32kB: > anon-thp-unaligned-64kB: > anon-thp-unaligned-128kB: > anon-thp-unaligned-256kB: > anon-thp-unaligned-512kB: > anon-thp-unaligned-1024kB: > anon-thp-unaligned-2048kB: > anon-thp-partial: > file-thp-aligned-16kB: > file-thp-aligned-32kB: > file-thp-aligned-64kB: > file-thp-aligned-128kB: > file-thp-aligned-256kB: > file-thp-aligned-512kB: > file-thp-aligned-1024kB: > file-thp-aligned-2048kB: > file-thp-unaligned-16kB: > file-thp-unaligned-32kB: > file-thp-unaligned-64kB: > file-thp-unaligned-128kB: > file-thp-unaligned-256kB: > file-thp-unaligned-512kB: > file-thp-unaligned-1024kB: > file-thp-unaligned-2048kB: > file-thp-partial: > > So you have counters for every supported THP size in the system - they will be > different for a 64K base page system. > > anon vs file: hopefully obvious > > aligned vs unaligned: In both cases the THP is mapped fully and contiguously. In > the aligned cases it is mapped so that it is naturally aligned. So a 16K THP is I think we should use "aligned" or "aligned to ", and stop saying "naturally aligned", throughout. "Natural" adds no additional information, and it makes the reader wonder if there is some other aspect to the alignment (does natural imply PMD-mapped? etc) that they are unaware of. > mapped into VA space on a 16K boundary, a 32K THP on a 32K boundary, etc. > > partial: Parts of THPs that are partially mapped into VA space. > > Note this does not draw a distinction between PMD-mapped and PTE-mapped THPs. > But a THP can only be PMD-mapped if it is both PMD-aligned and PMD-sized. So > only 2 counters can include PMD-mappings; anon-thp-aligned-2048kB and > file-thp-aligned-2048kB. We can filter that out by subtracting the relevant > smaps counters from them. I could add a --ignore-pmd-mapped flag to do that? Or That would work but is relatively awkward, but...1 > I could rename all the existing counters to include "pte" and introduce 2 new > counters: anon-thp-aligned-pmd-2048kB and file-thp-aligned-pmd-2048kB? ...this would be perfect, I think. The "pte" would help self-document, and separately things out allows for a clearer view into the behavior. > > The --cont option will add *additional* special counters, if specified. The idea > here is to provide a view on what percentage of memory is getting > contpte-mapped. So if you provide "--cont 64K" it will give you a counter > showing how much memory is in 64K, naturally aligned blocks (actually 2 > counters; file and anon). Those blocks can come from fully mapped and aligned > 64K THPs. But they can also come from bigger THPs - for example, if a 128K THP > is aligned on a 64K boundary (but not a 128K boundary), then it will provide 2 > 64K cont blocks, but it will be counted as unaligned in > anon-thp-unaligned-128kB. Or if a 2M THP is partially mapped so that only it's > first 1M is mapped and aligned on a 64K boundary, then it will be counted in the > *-thp-partial counter and would add 1M to the *-cont-aligned-64kB counter. > Interesting, and completely undocumented until now. Let's add this to the tool's help output! In fact, all of the above. > > Sorry if I've labored the point here. But I think the only thing the tool > doesn't already do that you are asking for is to differentiate PTE- vs PMD- > mappings? That, plus explain itself, yes. :) > >> >> ... >>                          (e.g. /sys/fs/cgroup for cgroup-v2 or >>>>>                          /sys/fs/cgroup/pids for cgroup-v1). Exactly one >>>>>                          of --pid and --cgroup must be provided. >>>> >>>> Maybe we could add "--global" to that list. That would look, in order, >>>> inside cgroups2 and cgroups, for a list of pids, and then run as if >>>> --cgroup /sys/fs/cgroup or --cgroup /sys/fs/cgroup/pids were specified. >>> >>> I think actually it might be better just to make global the default when neither >>> --pid nor --cgroup are provided? And in this case, I'll just grab all the pids >>> from /proc rather than traverse the cgroup hierachy, that way it will work on >>> systems without cgroups. Does that work for you? >> >> Yes! That was my initial idea, in fact, and after over-thinking it for >> a while, it turned into the above. haha :) > > OK great - implemented for v3. > Sweet! thanks, -- John Hubbard NVIDIA