From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61FC6EE4993 for ; Tue, 22 Aug 2023 18:04:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAB44280055; Tue, 22 Aug 2023 14:04:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A5B41280040; Tue, 22 Aug 2023 14:04:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FBBF280055; Tue, 22 Aug 2023 14:04:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 81241280040 for ; Tue, 22 Aug 2023 14:04:03 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 476911C9348 for ; Tue, 22 Aug 2023 18:04:03 +0000 (UTC) X-FDA: 81152514366.21.B66DC56 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by imf29.hostedemail.com (Postfix) with ESMTP id 01BD312002C for ; Tue, 22 Aug 2023 18:04:00 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=nZsyCCnn; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=EHkCF1bJ; dmarc=none; spf=pass (imf29.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.19 as permitted sender) smtp.mailfrom=shr@devkernel.io ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692727441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1HtTCgJc0ZUa0k/Dq5F5KRBm+gk3KHx/7qQeQ8kY1jM=; b=MZLTXUuVOdx9YjYX2A6pvBY51HY2ZAMVavyYX9NUkmcosTCgxbeWb26xOfj1zQ9QJGQJ0m 8/H55F6jnHDpsKBbAeZZtkSIFGn1y/vWA3bxTjBenVDU6nABNYnIhcA4VMGtdb0G6zCTat DzLc7qTKT92OkV5fVAoNxo7JHbVTmFQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=nZsyCCnn; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=EHkCF1bJ; dmarc=none; spf=pass (imf29.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.19 as permitted sender) smtp.mailfrom=shr@devkernel.io ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692727441; a=rsa-sha256; cv=none; b=m0Jz9ZmiwI/TVyfeopFGc9AgzDrBz2Ld9u2Nxt9cTqYuynXSOe8cE0xFvOYY5eY/oFaygz mVcs7sbLviVLwlK3TeZP6tmGwPe9i1lX0FXGV5Ne9fe/yjTVi3XSOLCJoWNBS5ZKBdm7k+ 07q0buTq9LsHbrLQ1rBbTgO3hko6yLE= Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 562A7320082A; Tue, 22 Aug 2023 14:03:54 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Tue, 22 Aug 2023 14:03:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devkernel.io; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1692727433; x=1692813833; bh=1H tTCgJc0ZUa0k/Dq5F5KRBm+gk3KHx/7qQeQ8kY1jM=; b=nZsyCCnnMgdO1jntrd hhIoQVgbVRN4XBA7ltAGNkUrqy6b+VhnySBUbnekQiw9pBHQacvqDvnyI96uTRsZ ThRwMZFMkZexR0EQAjEVuylVFrJp+w7hF+g9edswmi9ftBUEwp4gySC/qtGW4SeS 4sn0Pm5frk/7Zr5UMU7waweSB66nhckmHvi4/QHolythKXnRResiQV7TlZuW3Z4k vLoGn7AaeFG0pPhIjGKfb/MuRF5UaAPXjnAWWOo4pHzemA22bEM3JiaQHhLz0vJP IRq9iWBPlh2xBxfUOPn3FpcIgHOpZr2ceF4iuA6LYMkNKMOfa+yCsiAyTWFJFn/j LTNA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1692727433; x=1692813833; bh=1HtTCgJc0ZUa0 k/Dq5F5KRBm+gk3KHx/7qQeQ8kY1jM=; b=EHkCF1bJhw7orL/GQThs6FpRSO6Pd rimPKj40oFxjzbXvEw1SN+vifDkMOqwTenLz+s0YaT8wzBfJjc+VYP2IoHMPlLYH AuTDJQJipFeH9UCUy0BPs5Ypm3u5bf2xTOk7+h4xhp/cDkwklMHyEbjdigHUiSN4 IW2ycjD7ZYWSEFDG1BfsEePcy82S5GnNVQpVEuqJy+DG585Hfrk8bxDfRz4TrsCr qp8Cnj1ZMBm2Neq6c/EH9rSlLX5CH+FZXdi9zdwlhx9CCxesvw/cDO47h9HMb4A5 tI8lxHjsLbq0cGp4ubhF8egANB7fzh9XGuA5rjHP8Kg4Q86/CAGccJAtA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedruddvuddguddulecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpehffgfhvfevufffjgfkgggtsehttdertddtredtnecuhfhrohhmpefuthgv fhgrnhcutfhovghstghhuceoshhhrhesuggvvhhkvghrnhgvlhdrihhoqeenucggtffrrg htthgvrhhnpeevlefggffhheduiedtheejveehtdfhtedvhfeludetvdegieekgeeggfdu geeutdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hshhhrseguvghvkhgvrhhnvghlrdhioh X-ME-Proxy: Feedback-ID: i84614614:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 22 Aug 2023 14:03:52 -0400 (EDT) References: <20230817162301.3472457-1-shr@devkernel.io> <886b6a56-8acb-e975-b5f3-d8098a2285ab@redhat.com> User-agent: mu4e 1.10.1; emacs 28.2.50 From: Stefan Roesch To: David Hildenbrand Cc: kernel-team@fb.com, akpm@linux-foundation.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v3] proc/ksm: add ksm stats to /proc/pid/smaps Date: Tue, 22 Aug 2023 11:03:16 -0700 In-reply-to: <886b6a56-8acb-e975-b5f3-d8098a2285ab@redhat.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Queue-Id: 01BD312002C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: egpkzawd3z64xzist7p7o9sajpyzbh5w X-HE-Tag: 1692727440-55996 X-HE-Meta: U2FsdGVkX1/zhEVIAZf7h0I//Pv4BH1UkZLLSeCQHZoCtf49lQb1Pr6fEQwb+LeK9lCzk+5Crm0OqkrdGgdo3BNoNrfFhpE4n05vd5HPVHFXoz+gLrAymaxULFn5JCHXv4qcWDG5N3sjpP0TobrFQ102ea1wceRsBDucji/EY4AIgDg+fEMpO1DgEe87nH64kqst8o0Nr72/FKqesqkIGbIIDGdNa4QA4X7Hc0hg0pSkGOsEm0V3us4u1Hj/7OHi+B/SslCv7n3TOtrL+mueXvEDZiisvhCTtoQOMT/i8cycDFie27WgDw8V6OYPrVPFZE2nQVQbYhJjXFjYqVt/0GKVp+hcaEsNs9SxSChX+CSTy7dVoOK1QqDuiVyklhyRLbZzR896ACc+iTMJUKYstxi0ovSLop4ieR3jCVwxmouS14yb373CZFnUPmnjMRauosg7GdJVTewqQ4vIQ8uc7br1TbsbGNSBnAIEgTtUaQuvwaN8lEvFOGbpAXbyNjBxwhcXnbeffy0S7m3Jos4RtZNsmK1n6M7FFn/WDLdjsfBGYLGyW7Dw9FFhYRCT1Ld4/Kwup7S+5J+0A04sE5iSUr6K6CSrSuC6CGxFtxqRg9wx9iVQlJZOqZI0BfSBw/jsajtje+11cuYXy0xgvIJxNeUHwhk10RdwpR5pq/xgYQz/+Xz31rNd1++iWN4CDqVDRojcFBUGD2ula+Qt/II8ACfbD2UZtR2Wu5K6WH8NZQPpnu/1l8kq8+vevoAALj6uTY3yXSqoJY+/H8nZq9Jv9cdN5Ww1KCf7n4slQT2j0z2CwPW1RZwwC6sIHMtp+Y11FgJJVZsIhisk76Naa9HI/5qVV9PlRcWvAbveEQqBwPjDyWi0CH1cZu1P/F7fYHZpanqmhseQ55J+Xi3fCOOn+UMt7gstDC5cTRJkXPU7MlZBU6FRmvV70jjMUiVbi50E++v545a30kETR6RQ/DO hRKd7edW Bkj0uo9n/dReUtu2yDjj9gVHUKNxI1nLqCjHf6uk/J4s5lBc4WWtneAEV+pTCIkx/RC6WYk/r3uUvbaliRf5bE0/l+mV95h4KaJtvptbhpr7xgaesxnfsA+x7dVNYnSNunPDdro1/uKzVjllA3qZE/RFUeg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > On 17.08.23 18:23, Stefan Roesch wrote: >> With madvise and prctl KSM can be enabled for different VMA's. Once it >> is enabled we can query how effective KSM is overall. However we cannot >> easily query if an individual VMA benefits from KSM. >> This commit adds a KSM section to the /prod//smaps file. It reports >> how many of the pages are KSM pages. The returned value for KSM is >> independent of the use of the shared zeropage. >> Here is a typical output: >> 7f420a000000-7f421a000000 rw-p 00000000 00:00 0 >> Size: 262144 kB >> KernelPageSize: 4 kB >> MMUPageSize: 4 kB >> Rss: 51212 kB >> Pss: 8276 kB >> Shared_Clean: 172 kB >> Shared_Dirty: 42996 kB >> Private_Clean: 196 kB >> Private_Dirty: 7848 kB >> Referenced: 15388 kB >> Anonymous: 51212 kB >> KSM: 41376 kB >> LazyFree: 0 kB >> AnonHugePages: 0 kB >> ShmemPmdMapped: 0 kB >> FilePmdMapped: 0 kB >> Shared_Hugetlb: 0 kB >> Private_Hugetlb: 0 kB >> Swap: 202016 kB >> SwapPss: 3882 kB >> Locked: 0 kB >> THPeligible: 0 >> ProtectionKey: 0 >> ksm_state: 0 >> ksm_skip_base: 0 >> ksm_skip_count: 0 >> VmFlags: rd wr mr mw me nr mg anon >> This information also helps with the following workflow: >> - First enable KSM for all the VMA's of a process with prctl. >> - Then analyze with the above smaps report which VMA's benefit the most >> - Change the application (if possible) to add the corresponding madvise >> calls for the VMA's that benefit the most >> Signed-off-by: Stefan Roesch >> --- >> Documentation/filesystems/proc.rst | 4 ++++ >> fs/proc/task_mmu.c | 5 +++++ >> 2 files changed, 9 insertions(+) >> diff --git a/Documentation/filesystems/proc.rst >> b/Documentation/filesystems/proc.rst >> index 7897a7dafcbc..d5bdfd59f5b0 100644 >> --- a/Documentation/filesystems/proc.rst >> +++ b/Documentation/filesystems/proc.rst >> @@ -461,6 +461,7 @@ Memory Area, or VMA) there is a series of lines such as the following:: >> Private_Dirty: 0 kB >> Referenced: 892 kB >> Anonymous: 0 kB >> + KSM: 0 kB >> LazyFree: 0 kB >> AnonHugePages: 0 kB >> ShmemPmdMapped: 0 kB >> @@ -501,6 +502,9 @@ accessed. >> a mapping associated with a file may contain anonymous pages: when MAP_PRIVATE >> and a page is modified, the file page is replaced by a private anonymous copy. >> +"KSM" shows the amount of anonymous memory that has been de-duplicated. The >> +value is independent of the use of shared zeropage. >> + >> "LazyFree" shows the amount of memory which is marked by madvise(MADV_FREE). >> The memory isn't freed immediately with madvise(). It's freed in memory >> pressure if the memory is clean. Please note that the printed value might >> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c >> index 51315133cdc2..f591c750ffda 100644 >> --- a/fs/proc/task_mmu.c >> +++ b/fs/proc/task_mmu.c >> @@ -396,6 +396,7 @@ struct mem_size_stats { >> unsigned long swap; >> unsigned long shared_hugetlb; >> unsigned long private_hugetlb; >> + unsigned long ksm; >> u64 pss; >> u64 pss_anon; >> u64 pss_file; >> @@ -452,6 +453,9 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page, >> mss->lazyfree += size; >> } >> + if (PageKsm(page)) >> + mss->ksm += size; >> + > > Did you accidentally not include handling of the KSM-shared zeropage? > > Or did I misinterpret "independent of the use of the shared zeropage." I think I misunderstood your review comment. I'll address it in the next version.