From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D23EE77170 for ; Wed, 4 Dec 2024 17:05:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B94F6B0083; Wed, 4 Dec 2024 12:05:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 169FA6B0089; Wed, 4 Dec 2024 12:05:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 031266B008A; Wed, 4 Dec 2024 12:05:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DC6956B0083 for ; Wed, 4 Dec 2024 12:05:56 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 418031611EF for ; Wed, 4 Dec 2024 17:05:56 +0000 (UTC) X-FDA: 82857903576.18.93CA741 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 7CDDC18001C for ; Wed, 4 Dec 2024 17:05:38 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733331940; a=rsa-sha256; cv=none; b=KtSrZw0nLhsvadSc5FJMzOCBkTFaYgRn+QdStJRRuFR8g7zpI0hzTVTkMFWcmR2b6ypzL4 /zqFfXI93steCOytjun04cb34P9256xyhRzkX/P3nr1Gl5BfMjxaRGBEoDLEPUBDj3Ezwy ekTqSDJKnVffr3bb93KYX/HAyra4U6A= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733331940; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2e82/+uISYtrsDYmvvt/L9iIvMpmDW7pxNjUFME2rG4=; b=aQUwlca4mLhSk2QRycZndntb1/VVxPf6AerLb8klIsH0jMTNabfporiRBNcosEP/nTBp9Y auUCsl2n6uggsr0epVtp+esO+h9zM7htAjJE2nZ9SPuJaIrQskD63+rDfb3nE/ZyzDjrfP y8frdhG9w85lw0Nt5QZo5D6aNkJjjWs= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E2DE41063; Wed, 4 Dec 2024 09:06:16 -0800 (PST) Received: from [10.1.31.170] (XHFQ2J9959.cambridge.arm.com [10.1.31.170]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2440C3F71E; Wed, 4 Dec 2024 09:05:47 -0800 (PST) Message-ID: <500d1007-56f5-43cb-be9d-4a39fccc6e53@arm.com> Date: Wed, 4 Dec 2024 17:05:45 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] smaps: count large pages smaller than PMD size to anonymous_thp To: Wenchao Hao , David Hildenbrand , Andrew Morton , Matthew Wilcox , Oscar Salvador , Muhammad Usama Anjum , Andrii Nakryiko , Peter Xu , Barry Song <21cnbao@gmail.com>, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org References: <20241203134949.2588947-1-haowenchao22@gmail.com> <926c6f86-82c6-41bb-a24d-5418163d5c5e@redhat.com> <31158c3e-bf1b-47e8-a41b-0417c538b62e@gmail.com> Content-Language: en-GB From: Ryan Roberts In-Reply-To: <31158c3e-bf1b-47e8-a41b-0417c538b62e@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7CDDC18001C X-Stat-Signature: faiwum4dsfchg7w8mehhrr9tmd694115 X-Rspam-User: X-HE-Tag: 1733331938-718784 X-HE-Meta: U2FsdGVkX1/qKnjoWRCzIu3xHtmLiAIbIfV4bHWmW0Ca4r/JPnVoDIzouA6LJ0fu7/kXdTlDBep7U13MRawHKTv0SJfLJGDBkq+phfTdlxOQIOqq8nHAfZonYUkMTpE2bnsD2WUr+DtcP2VWGzUEQWS7lIpfYOHZ1M7ctQoVQhB8lTu1OJhT4kZSJ+eAJZV86+NaO5DmVRTB/Bhr6K32UDAMGvAANqcEb7HG46K9E7FjSvAugo0JXJF/yKBYs7Wco4AVUKSLeTmfGZo78Fsvh7ZwIl8k7BByXwtSHGfcA3mtV0cwqpXNIJ7h1o9DnhMbUWotTqFxITuILSOHqe6PF2ATCgX65rzf5Fd+QVumDLfPT9mmG1OJxwjhbdLw0n/jotcL8rkz+dvqISmbQ2T40XymiyMlMFdr2NH6OP95DRROQ/dMNkVA/1+kUwEupzB+OPbYKpp2jwaCGOe8HQhP/WfxjtEpXKQSfRq3XpE3WKYKPXDRMI2WeUIHsrqwVpKOTlG/7MtDfKC3B/sTbJ0Ti72/zq1OFPkD5p8kFryjXxYLCHormHAENRTT5MeR9fipewmIj7jPvw7ZA0UGhLqpUH5CF2NjQXmxWupmLcbmTtAfZQvdPy2PqpD6yc57tEfHToU5YlrhaFWcX1sEKQ+qxNraD85OUzyMQXxXgi5cNCGQEhbCt9OeP0XhHwwLR3mq3eUibOMzSY2gwnGUHnAHclvkqFx77PvWokAjzcZ0vI26xNSM/1kZTDasjx+Ya1oUA7XrGyiVm9eUEoYe8yhIa9lXNLTQATNq/HcBldbpZwJe2FUPoNkBzgId63nOvVGu52olOuOXQeckequpI+/xZaWT1mCZx/IPVpMmzfGStdCfJKwypMoCKhsoKA5vNC5HpwP7qVJnds1hVTMb7PAjsVEYa9mtr7752Gb7yyhNkXOTcg5CyAlDGevI+dSUGCNJ9VSvLP5eTNU4xc+7Fx/ 9HYcx3Oo POviNhPKD1+Na5Y6LIZZEkqx1EOCZDiRdM00yUACwlKEzoxLRriSj00m/I0BX+9187HNS7QEoRtShUzfC5/9Le5+bx7uIUr4YwPX5fC+5TXaz+XcWpayLX3q9qLv6bfiNvi3b4A4C4ohpvx51LFqtp/VDIwCAOTMahInMFbpgC6pWnq44dy83j1+nqY7YDnzlJbeiHefxYsQei26AAcuZoeJ6Mdm3k7iRen4W8jvAqkkAvjtqOvocO0Rl4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 04/12/2024 14:40, Wenchao Hao wrote: > On 2024/12/3 22:42, Ryan Roberts wrote: >> On 03/12/2024 14:17, David Hildenbrand wrote: >>> On 03.12.24 14:49, Wenchao Hao wrote: >>>> Currently, /proc/xxx/smaps reports the size of anonymous huge pages for >>>> each VMA, but it does not include large pages smaller than PMD size. >>>> >>>> This patch adds the statistics of anonymous huge pages allocated by >>>> mTHP which is smaller than PMD size to AnonHugePages field in smaps. >>>> >>>> Signed-off-by: Wenchao Hao >>>> --- >>>>   fs/proc/task_mmu.c | 6 ++++++ >>>>   1 file changed, 6 insertions(+) >>>> >>>> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c >>>> index 38a5a3e9cba2..b655011627d8 100644 >>>> --- a/fs/proc/task_mmu.c >>>> +++ b/fs/proc/task_mmu.c >>>> @@ -717,6 +717,12 @@ static void smaps_account(struct mem_size_stats *mss, >>>> struct page *page, >>>>           if (!folio_test_swapbacked(folio) && !dirty && >>>>               !folio_test_dirty(folio)) >>>>               mss->lazyfree += size; >>>> + >>>> +        /* >>>> +         * Count large pages smaller than PMD size to anonymous_thp >>>> +         */ >>>> +        if (!compound && PageHead(page) && folio_order(folio)) >>>> +            mss->anonymous_thp += folio_size(folio); >>>>       } >>>>         if (folio_test_ksm(folio)) >>> >>> >>> I think we decided to leave this (and /proc/meminfo) be one of the last >>> interfaces where this is only concerned with PMD-sized ones: >>> >>> Documentation/admin-guide/mm/transhuge.rst: >>> >>> The number of PMD-sized anonymous transparent huge pages currently used by the >>> system is available by reading the AnonHugePages field in ``/proc/meminfo``. >>> To identify what applications are using PMD-sized anonymous transparent huge >>> pages, it is necessary to read ``/proc/PID/smaps`` and count the AnonHugePages >>> fields for each mapping. (Note that AnonHugePages only applies to traditional >>> PMD-sized THP for historical reasons and should have been called >>> AnonHugePmdMapped). >>> >> >> Agreed. If you need per-process metrics for mTHP, we have a python script at >> tools/mm/thpmaps which does a fairly good job of parsing pagemap. --help gives >> you all the options. >> > > I tried this tool, and it is very powerful and practical IMO. > However, thereare two disadvantages: > > - This tool is heavily dependent on Python and Python libraries. > After installing several libraries with the pip command, I was able to > get it running. I think numpy is the only package it uses which is not in the standard library? What other libraries did you need to install? > In practice, the environment we need to analyze may be a mobile or > embedded environment, where it is very difficult to deploy these > libraries. Yes, I agree that's a problem, especially for Android. The script has proven useful to me for debugging in a traditional Linux distro environment though. > - It seems that this tool only counts file-backed large pages? During No; the tool counts file-backed and anon memory. But it reports it in separate counters. See `thpmaps --help` for full details. > the actual test, I mapped a region of anonymous pages and mapped it > as large pages, but the tool did not display those large pages. > Below is my test file(mTHP related sysfs interface is set to "always" > to make sure using large pages): Which mTHP sizes did you enable? Depending on your value of SIZE and which mTHP sizes are enabled, you may not have a correctly aligned region in p. So mTHP would not be allocated. Best to over-allocate then explicitly align p to the mTHP size, then fault it in. > > int main() > { > int i; > char *c; > unsigned long *p; > > p = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); What is SIZE here? > if (!p) { > perror("fail to get memory"); > exit(-1); > } > > c = (unsigned char *)p; > > for (i = 0; i < SIZE / 8; i += 8) > *(p + i) = 0xffff + i; Err... what's your intent here? I think you're writting to 1 in every 8 longs? Probably just write to the first byte of every page. Thanks, Ryan > > while (1) > sleep(10); > > return 0; > } > > Thanks, > wenchao >