From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF24FE7717F for ; Mon, 16 Dec 2024 16:03:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 469D76B00B0; Mon, 16 Dec 2024 11:03:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 419DB6B00B1; Mon, 16 Dec 2024 11:03:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B9C16B00B2; Mon, 16 Dec 2024 11:03:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0BD936B00B0 for ; Mon, 16 Dec 2024 11:03:13 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BAD3D16115F for ; Mon, 16 Dec 2024 16:03:12 +0000 (UTC) X-FDA: 82901291088.02.A9BB61C Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf14.hostedemail.com (Postfix) with ESMTP id 28959100011 for ; Mon, 16 Dec 2024 16:02:37 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WNmD1baj; spf=pass (imf14.hostedemail.com: domain of haowenchao22@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=haowenchao22@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734364976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uf9egybKI/MKwFIGN0jlMtv9pzalLYkTVpIwCM8w0nA=; b=0Gugb7GJ4DExA+MABpzhsFRY/6cQH6yihPREecf/yZkD+L6H7smIwyZ+XgMdh2WN0h81cw y2qv17IMt9dX8TASyqOdQEDDssKbwL1UwS1wNrj//OfVnsX+Ry2jg6hcYFUags0ZAr7OL5 18Sm1h9Rbp7c3TEvQI/h/QHqqpuJ3GA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734364976; a=rsa-sha256; cv=none; b=E+EZxBOfNC3UUo0/R6b9ypz6ilErev66goViU9AUGuDkqf3UtzsmFlySz8kdF0x5EyVlXV Q8ig2aiVuL9/VKB2HLnV9NVKNjzdluLVTJvNHspeZdBGPkZ+mysPMQVP1/1QcSwDyauuK5 lzkWrSmh/QIA3bMc5WE9bAw04NrGuCk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WNmD1baj; spf=pass (imf14.hostedemail.com: domain of haowenchao22@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=haowenchao22@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2161eb94cceso28682375ad.2 for ; Mon, 16 Dec 2024 08:03:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734364989; x=1734969789; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=uf9egybKI/MKwFIGN0jlMtv9pzalLYkTVpIwCM8w0nA=; b=WNmD1bajPngPfugasMrfpCoT14bOwquIWmnGlQ8VT6++d6A0gNqqolhl2hGoylshy7 +MwBHI2R4br5Qz6Ixd+lH8BBLxAloN12FK23kVNbFG7Ejpq9ovPCFEGK3Bri2hxD5mRh m2C6SYsjNhJAp/x4FSjR1wZI3Vpc+EFYAv5Ecm7wyD4xkz5HMLx1WT0BfHC7cTcH6ODN XT1hct1gtxdI9Qbv/MQuLXSkpCC78zSL/mNyM5vO4czcTrrWZ0Cmbkyhr+x5/uHUqVAr zbRmdn5trccqViH+MPXfamtSfXcxAdQLykAkfrSSQns8i6wxyixa4h0YvFpJ+eBTPd6E 0vTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734364989; x=1734969789; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uf9egybKI/MKwFIGN0jlMtv9pzalLYkTVpIwCM8w0nA=; b=NVL3RZnaKenERXtV6tnLFUWQ+aJGsS2Dnj6qj5bEhcUWyDR2i/IP/fBmdKu9RSVpvN JfJf/HE0caCrJsOnkZq6n3kP06hXqBTpDAEGcYzY8N5tDjjmHo81Fun1Pdr6zDBvLQ1F 3z4sZ97MOc2wPlsmUgVm4YjmgQDljuYzP1L0b7REs6cg8VtQHhrdQEeFpT4nFi26i0+H fyR1Ix0cSa85yzVXUe9m+GixSmJXKQw0m0vB1DRbmicuyENNhPNntNm3okUqlpKRuO7k 1v/IVXU12b4hgYcQOpp+vuGqEwHPqt20ZLXrlLRoHGC/+i5tr3P9QG0mnEay0ajcw/q5 nlIA== X-Forwarded-Encrypted: i=1; AJvYcCUavBzlUJt6e4inIE+fN2j25RVcFAg2b/7ozRIwSixMa85E0XnXKtR0ungmd59fT1qZcFPS2BWT4Q==@kvack.org X-Gm-Message-State: AOJu0Ywo9MYHe5Mt4avb+bkUap61Hu80ONNLJAYyzfb2/1fasBz7pn9Z ZW3Fq8kIdWjLkkLHBiHXbFcJCLDuetZRcgMDHRq2FAYOb0neytokxyzLVvueQF0= X-Gm-Gg: ASbGncvUPunFAsq+t7AYXs6v0gRWvkcqEmPqiUzGKrttk0Ja6VPk8GfGsnckEjS8IV1 GzlQ3xHCsA7Lv9aV8J4eKw3BpqwGwM+77KSqly++3TANgPwVMaa2IreuqSgNZiUaftvMLrN1S7c A5QP2ch0aeCtdJ9EWlaVJdf55zkoN5RfaYAYxyLHqinQVuVWat5Omdc9aJKFKwyLE0Yz28zyRfl 06QDTJ/DCyOu53HrxhubhaXdNqEK6yHcOpCJ3/FVZnjXLBGcnsMLnWNF3Dw6UZfbqj5ggAn8Q== X-Google-Smtp-Source: AGHT+IHhNdOmIfe4WNpSFZkqbTMp/z+SfeJeZ8rB9+dD41oswN6PiNS5ukRlgiyh6NBjpGwjucyF1w== X-Received: by 2002:a17:902:e808:b0:216:4c88:d939 with SMTP id d9443c01a7336-21892a41d6fmr170574275ad.38.1734364989268; Mon, 16 Dec 2024 08:03:09 -0800 (PST) Received: from [10.234.7.32] ([43.224.245.237]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-218a1e7227asm44452215ad.272.2024.12.16.08.03.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 16 Dec 2024 08:03:08 -0800 (PST) Message-ID: Date: Tue, 17 Dec 2024 00:03:03 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] smaps: count large pages smaller than PMD size to anonymous_thp Content-Language: en-US To: Barry Song <21cnbao@gmail.com>, Lance Yang Cc: David Hildenbrand , Andrew Morton , Matthew Wilcox , Oscar Salvador , Muhammad Usama Anjum , Andrii Nakryiko , Ryan Roberts , Peter Xu , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org References: <20241203134949.2588947-1-haowenchao22@gmail.com> <926c6f86-82c6-41bb-a24d-5418163d5c5e@redhat.com> From: Wenchao Hao In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 28959100011 X-Stat-Signature: wcgcsectbek55r9gz7xn4rt358wiho4t X-Rspam-User: X-HE-Tag: 1734364957-882143 X-HE-Meta: U2FsdGVkX1/kRAQd9n3EUbBPMm/p7391IvJGqt/crEBUMeJPRH6ecwJ3h7m4YFVf4xqiPa7c/QH47fx30k9qVtO26NL9TK+GEyaP7MqDdGFyPGSa24pq3PtEIHnj/ralSsjwmD6lGcWuRGsVpBgzaI4bg5P5erEZt5/6B8+KgKfuF/BpempFRYluXXCtj93HfOCukgFA2Qv0+19DpTJPevOaftz3WboiFeOJjYn+HAxhSe5qwiIOApHOLfDCbEbdOEwa7TWkOkOrW2PBo+K+G7UW+/CIj7W1p9nq3QmlEXiGXJ51pLcPBdjT+gS+x4odSayj2UvevAQhXM3SdFy+vs1cowzjqJoMvaW834grV0fQSF9rau+CdKWjmkZVqDWhwy58S1krOkdHd5EIonK3H2AY2LZJsDPSQOeYpxfuBZ3/wM0D7ywDFDWGnrzrrPIGzibQqScCtj4dMTt6yqRGt+fzIbtSr5vMrGHbX2R1wv/na6eYTC6vhfvsX/6bQhXfj/5kF/PyAlWpP/o9JoaXSkTB7OYLSiSCrklkra2VG3iNYLADstlarvZ4fUs+/X7Qj4JDqJl8VWGFS5P5byIXNb9GFOkgEnB+gBpz93auBcmhc0Xs5I5+7t/aHPCkroLqgXkcuJXbtW4nVv09zy63yn2LK9wvxyRNuy2M3EGm4IYWilfDviVBYhuWeEFj+7LOS8erkWTGexKqzSJjNbtsQX0ey/y1Fo1CJ8tW/qTKYORMaJ0xW2cBXImqw1yp+r4apQxhhAnvI/eTfvgbx4uvAPGv7DHYO+qVQoojlIwyzxQ7IAX1QiQv1pCU04Bb3qCrwKfvneCl9e3yN3WM3Uh1Gpuaz3kiGuHHp8JVTRRcnE6eWRBXNSj9DvQDvGirk8b8MD4CsiK/5uY1gkDJn1ogoOxzvU69ml1kjGq7JdjqoXhzedlCnazw5h5A1+7JvyTml6a/J3hT32B1UlimDAL Q+ybhLF9 DS4/AAJ2o5UZ7JLO1idq7Yc4hbmew+6zjSonWkJoIiz1VY9QZWGlJYUSouD6meQAF6cdNkQGfeUvwec8QRXowzeGev16+RJPQob5Koe4/U6OcT06uQ+Jx+6F2HrZahYemqMVvRfB9tFrJWGjP/yGIfHsx5vUHlogG3zv54VghDgCbjwjXQ+4enh6yFy8Bt6fM4ltrqtz/VbvtYP9WzE1jBv7XVNKnYgkvB2WLjtF4Xnp6cb3qA8JISd30UQai3lOjFr6oA+QuBDCkxh/AcXGUZwzU3FOSCOhGogpj9vA+V984G8fTEvOBShzEte1hzaG6pg5O1JTWnUmfriJiXETSCN0zgE8UVEn2VE36gJwfcXot5jBTFthv0aOk31p9egGwYxG2TJ/xXIya5rjsD3ax/jbl1jU3NInvVTfE1U1Jy4TXuSh8ziVv/nVlw//ajuQctOTi9CmzeneakfasgWkPzcn70roVbkXt1pQI63KcmHZ/6qTPtdIeNQs86sreSXmDaM7M1+blVDB3G3qFQ3i0U0E2lA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.102987, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/12/8 14:06, Barry Song wrote: > On Fri, Dec 6, 2024 at 7:16 PM Lance Yang wrote: >> >> On Tue, Dec 3, 2024 at 10:17 PM David Hildenbrand wrote: >>> >>> On 03.12.24 14:49, Wenchao Hao wrote: >>>> Currently, /proc/xxx/smaps reports the size of anonymous huge pages for >>>> each VMA, but it does not include large pages smaller than PMD size. >>>> >>>> This patch adds the statistics of anonymous huge pages allocated by >>>> mTHP which is smaller than PMD size to AnonHugePages field in smaps. >>>> >>>> Signed-off-by: Wenchao Hao >>>> --- >>>> fs/proc/task_mmu.c | 6 ++++++ >>>> 1 file changed, 6 insertions(+) >>>> >>>> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c >>>> index 38a5a3e9cba2..b655011627d8 100644 >>>> --- a/fs/proc/task_mmu.c >>>> +++ b/fs/proc/task_mmu.c >>>> @@ -717,6 +717,12 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page, >>>> if (!folio_test_swapbacked(folio) && !dirty && >>>> !folio_test_dirty(folio)) >>>> mss->lazyfree += size; >>>> + >>>> + /* >>>> + * Count large pages smaller than PMD size to anonymous_thp >>>> + */ >>>> + if (!compound && PageHead(page) && folio_order(folio)) >>>> + mss->anonymous_thp += folio_size(folio); >>>> } >>>> >>>> if (folio_test_ksm(folio)) >>> >>> >>> I think we decided to leave this (and /proc/meminfo) be one of the last >>> interfaces where this is only concerned with PMD-sized ones: >>> >>> Documentation/admin-guide/mm/transhuge.rst: >>> >>> The number of PMD-sized anonymous transparent huge pages currently used by the >>> system is available by reading the AnonHugePages field in ``/proc/meminfo``. >>> To identify what applications are using PMD-sized anonymous transparent huge >>> pages, it is necessary to read ``/proc/PID/smaps`` and count the AnonHugePages >>> fields for each mapping. (Note that AnonHugePages only applies to traditional >>> PMD-sized THP for historical reasons and should have been called >>> AnonHugePmdMapped). >> >> Yeah, I think we need to keep AnonHugePages unchanged within these interfaces >> due to historical reasons ;) >> >> Perhaps, there might be another way to count all THP allocated for each process. > > My point is that counting the THP allocations per process doesn't seem > as important > when compared to the overall system's status. We already have > interfaces to track > the following: > > * The number of mTHPs allocated or fallback events; > * The total number of anonymous mTHP folios in the system. > * The total number of partially unmapped mTHP folios in the system. > > To me, knowing the details for each process doesn’t seem particularly > critical for > profiling. To be honest, I don't see a need for this at all, except perhaps for > debugging to verify if mTHP is present. > > If feasible, we could explore converting Ryan's Python script into a native > C program. I believe this would be more than sufficient for embedded systems > and Android. > Hi Barry, Yes, the reason I want to use smap to collect this data is that I wasn’t familiar with this tool before. When analyzing the performance impact of enabling mTHP, I want to understand the actual memory usage of the process being analyzed, including the proportions of anonymous pages, swap pages, large pages and so on. This helps determine whether the test results align with expectations. Indeed, the main purpose of adding this is to make debugging more convenient. For now, I’ll perform the analysis and testing on the Fedora distribution, so I can use the pyrhon tool directly. If it becomes unavoidable to run this tool on embedded devices in the future, I may take the time to create a simplified version of the analysis tool in C based on this script. >> >> Thanks, >> Lance >> >> >>> >>> >>> >>> -- >>> Cheers, >>> >>> David / dhildenb > > Thanks > Barry