From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB53AC001E0 for ; Wed, 16 Aug 2023 16:22:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F28B8E0003; Wed, 16 Aug 2023 12:22:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A3448D0001; Wed, 16 Aug 2023 12:22:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 444C48E0003; Wed, 16 Aug 2023 12:22:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 386A38D0001 for ; Wed, 16 Aug 2023 12:22:33 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0F26EC0744 for ; Wed, 16 Aug 2023 16:22:33 +0000 (UTC) X-FDA: 81130485786.15.AB890E8 Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by imf10.hostedemail.com (Postfix) with ESMTP id 59CFDC002C for ; Wed, 16 Aug 2023 16:22:29 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=LweUZTQi; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="fVPFI/k3"; spf=pass (imf10.hostedemail.com: domain of shr@devkernel.io designates 66.111.4.29 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692202949; a=rsa-sha256; cv=none; b=4+5Y6jPHDMy17di2t41qR49sl/rhZUH4dgit+vGNzJ70+OFZhq537MPI7ef13eppJ8zRrH P+ay8+tTSSfcOpHndsPWuTK1J7VRPE7m/k9NZG204Yg8WhFrsMhG2FsRJwmGAkwfc4KnWH Du/jCYdNyeHkaQOAwdYk1rsaGKBq1HM= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=LweUZTQi; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="fVPFI/k3"; spf=pass (imf10.hostedemail.com: domain of shr@devkernel.io designates 66.111.4.29 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692202949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lsRtaTNVobWQ1QYQuf1PXHLidC8c/K56+tUwFDf+W1s=; b=2qWPWD62PHG/F/PaKy0w8jQbkNfJYPEVjziDAsYzySfyOuWZDqNT8D+jrX48S924ZbM6nU GRfHYvISHfTWCXeS9qyV7zkhmQfyl2twQ+JGXWMPjX3FHxOOm6RZuEoXDSETon/D9vf+SE WnXC/rOar9valvBDzaiWcOQjDediZYo= Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 9B0EF5C00A9; Wed, 16 Aug 2023 12:22:28 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 16 Aug 2023 12:22:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devkernel.io; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1692202948; x=1692289348; bh=ls RtaTNVobWQ1QYQuf1PXHLidC8c/K56+tUwFDf+W1s=; b=LweUZTQitbkPkyPv+P AUoeEO8q8DAWpXFs39CnFzS0yPT6yEdBYwSfaj2UFCwQVPDLNAA14DA5Nlk6Il7m FvLQawh57P75apVu662fOa9r3u29VKtDuJVLA26KOzH2icptGh1VTG5suki6yKUJ SH6bWLRYlMKAzOT8s0xdTZjDOJ7s/hUZcZVBG46GNdIHjCpMwmUsYIUUUkkYVwwT cvbSVdYTFdAXZXK3fUffKaiRF1R2XBJe5qfEp4NwYgWEAL5ZiZkPf6/ArNd/cIPu kDV3zujGlvKY2tVFLeOWroVlW6hV2sePmB7dSRFhYQjOGBkpD82b/XgS8LNuk9+z 33bg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1692202948; x=1692289348; bh=lsRtaTNVobWQ1 QYQuf1PXHLidC8c/K56+tUwFDf+W1s=; b=fVPFI/k3HxbxqvvKOssjyYK/I8/tY y7cXXizRSmm4/xoqgsHRIIoGrxUk+jDfy3gycjySI7yyd3CVSc+eFzk42d2AdeDp xzPrx7cCqfsf4LPCuRNqDCGKXzFvI5QrDgfPMfhIBdjgQyd76KuqDLkM0b0sZWWj xYV5jPfoDBcJ/VvABK8yO7OUqtR3LEcOOTX8Liwm2NecBoYma5sNYdMlvPat+oIs PinrrWQcuJf8IoNC8w3GZVzuYMA1G7y0xgxtp+ozaI5t3WutIGInXJGFeAR28mHG dMEVYHelmeSQQxVKGL/xS6r2NetrcG9tO34VcP9bcxhIy7RnsZoWV1+Aw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedruddtledguddtudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpehffgfhvfevufffjgfkgggtsehttdfotddtredtnecuhfhrohhmpefuthgv fhgrnhcutfhovghstghhuceoshhhrhesuggvvhhkvghrnhgvlhdrihhoqeenucggtffrrg htthgvrhhnpeefledufefhgedvffeiuefhvddutedvteefgfeigeffudfhgeffueduveek ieeuudenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hshhhrseguvghvkhgvrhhnvghlrdhioh X-ME-Proxy: Feedback-ID: i84614614:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 16 Aug 2023 12:22:27 -0400 (EDT) References: <20230811162803.1361989-1-shr@devkernel.io> <43d64aee-4bd9-bba0-9434-55cec26bd9dc@redhat.com> User-agent: mu4e 1.10.1; emacs 28.2.50 From: Stefan Roesch To: David Hildenbrand Cc: kernel-team@fb.com, akpm@linux-foundation.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2] proc/ksm: add ksm stats to /proc/pid/smaps Date: Wed, 16 Aug 2023 09:22:03 -0700 In-reply-to: Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 59CFDC002C X-Stat-Signature: qkf3ikik17txnpgys51mkhocx8ftr7pk X-Rspam-User: X-HE-Tag: 1692202949-20782 X-HE-Meta: U2FsdGVkX1+x2HgJrNxEa/8uEgx8WExcUN0CD5LX7a6Od6LyMh0yqjb13lyNVY5tPfAiwa3ISJr1L1qx6dLai5nOqVDkmZ8OMyN4v7bnv/CoxdL6lBbFXDEOWkV00qzZax/9YChBnM+WkHYErPJ4JG+d6OVupuVHsytTkgplnrGyie0nBRwjf2TRazpt2T253VgU7qUfdFeyU/0hwRRda/+TvE3p9QfyF6YcKL3VrURRLPWCYq3b33AGjg40jzhD1cYInwkaHPVOlbjtOBKeMhjF+bhl9sVgEGX5pfiZg+TbGM4DXUCrndFcB5AG7gmH9zoBzkrezLO8eVXmyw4IA1WBW9MekcqoDZjpwiF7R5tDI79YyU2AgkqL0brdx4jXUZhoMIkZBIkUv8IjOBkfqI4113sYLOKB9PwW1Z96IhlStriASrPdR1mkoOo6eVv4PIKtziQ8l2APTFTpAEvbkq0ySLzEn5WqibmBZ1xmzb9R6WUJC4vt9N8JcsC20U/q7Sh7nUhETBnI34w+GsuFDbwHP6aAl/32z5hHAtblBALM1IZogBZMSY00g39qaSCNcnDxit6lKv87TD4NU1DHeORAyhzYe3BVa32Lav7Q7Sjj4jclnCtno6FW4Astczqe6Wf8WWFt72N/CS5TzVEfA6Hv+4zhuEifSfkFuu12bd6mhRVkbb/5sem9jCYIEHhF7jtnfi8XW5EKRD8A4Sk7jcpkuwNkqIm+GPrtEgQufKtXPXa32hVIHFAR7lyGAHfLkzF8ROEDOWUtu4PDeLvsg6W7GBoFWOJBSFV2ZXLHA2LYbd3rhS2uImrtYt/tXixmvssWhnD1kAn4XcKXSP+zSTBxVIcT33zPoFND8LVgeDChL49A0NVoj0s5IRDozbRo/3YvDguS/um6YJR4UqvHirrNBIkjYDXwodHcjKoCWls8a+cfGK4up41/nTjhckZIAviQRKWzigdVdnl2/jE lgCGxNgZ 99FbAUcVDfwqZCxltbsDVF7M0O0G1vFH07rYB+07MEsTAnSmoy8Ew57gRoD2Ib/SwNextLDO6j0sEd7cDTqDEaMDUcdUAftRlIMRqr0XURGabL6kjv6juIlj/PmN9fDjELHpt98sFFi7PrZ1GzBaODJ8y8edbu7AIqgUJ1rLdRHExpQ7jnMTNDnMOa6IhZca3nWDgQHBx5XE61FgCWvjQugNFcY3s8di5dBgIOmcBdAlER90JA7vCPU7c4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > On 15.08.23 19:10, Stefan Roesch wrote: >> David Hildenbrand writes: >> >>> Sorry for the late reply, Gmail once again decided to classify your mails as >>> spam (for whatever reason). >>> >>> On 11.08.23 18:28, Stefan Roesch wrote: >>>> With madvise and prctl KSM can be enabled for different VMA's. Once it >>>> is enabled we can query how effective KSM is overall. However we cannot >>>> easily query if an individual VMA benefits from KSM. >>>> This commit adds a KSM section to the /prod//smaps file. It reports >>>> how many of the pages are KSM pages. >>>> Here is a typical output: >>>> 7f420a000000-7f421a000000 rw-p 00000000 00:00 0 >>>> Size: 262144 kB >>>> KernelPageSize: 4 kB >>>> MMUPageSize: 4 kB >>>> Rss: 51212 kB >>>> Pss: 8276 kB >>>> Shared_Clean: 172 kB >>>> Shared_Dirty: 42996 kB >>>> Private_Clean: 196 kB >>>> Private_Dirty: 7848 kB >>>> Referenced: 15388 kB >>>> Anonymous: 51212 kB >>>> KSM: 41376 kB >>>> LazyFree: 0 kB >>>> AnonHugePages: 0 kB >>>> ShmemPmdMapped: 0 kB >>>> FilePmdMapped: 0 kB >>>> Shared_Hugetlb: 0 kB >>>> Private_Hugetlb: 0 kB >>>> Swap: 202016 kB >>>> SwapPss: 3882 kB >>>> Locked: 0 kB >>>> THPeligible: 0 >>>> ProtectionKey: 0 >>>> ksm_state: 0 >>>> ksm_skip_base: 0 >>>> ksm_skip_count: 0 >>>> VmFlags: rd wr mr mw me nr mg anon >>>> This information also helps with the following workflow: >>>> - First enable KSM for all the VMA's of a process with prctl. >>>> - Then analyze with the above smaps report which VMA's benefit the most >>>> - Change the application (if possible) to add the corresponding madvise >>>> calls for the VMA's that benefit the most >>>> Signed-off-by: Stefan Roesch >>>> --- >>>> Documentation/filesystems/proc.rst | 3 +++ >>>> fs/proc/task_mmu.c | 5 +++++ >>>> 2 files changed, 8 insertions(+) >>>> diff --git a/Documentation/filesystems/proc.rst >>>> b/Documentation/filesystems/proc.rst >>>> index 7897a7dafcbc..4ef3c0bbf16a 100644 >>>> --- a/Documentation/filesystems/proc.rst >>>> +++ b/Documentation/filesystems/proc.rst >>>> @@ -461,6 +461,7 @@ Memory Area, or VMA) there is a series of lines such as the following:: >>>> Private_Dirty: 0 kB >>>> Referenced: 892 kB >>>> Anonymous: 0 kB >>>> + KSM: 0 kB >>>> LazyFree: 0 kB >>>> AnonHugePages: 0 kB >>>> ShmemPmdMapped: 0 kB >>>> @@ -501,6 +502,8 @@ accessed. >>>> a mapping associated with a file may contain anonymous pages: when MAP_PRIVATE >>>> and a page is modified, the file page is replaced by a private anonymous copy. >>>> +"KSM" shows the amount of anonymous memory that has been de-duplicated. >>> >>> >>> How do we want to treat memory that has been deduplicated into the shared >>> zeropage? >>> >>> It would also match this description. >>> >>> See in mm-stable: >>> >>> commit 30ff6ed9a65c7e73545319fc15f7bcf9c52457eb >>> Author: xu xin >>> Date: Tue Jun 13 11:09:28 2023 +0800 >>> >>> ksm: support unsharing KSM-placed zero pages >>> >>> Patch series "ksm: support tracking KSM-placed zero-pages", v10. >> I see two approaches how to deal with zero page: >> - If zero page is not enabled, it works as is >> - If enabled >> - Document that zero page is accounted for the current vma or >> - Pass in the pte from smaps_pte_entry() to smaps_account() so we can >> determine if this is a zero page. > > That's probably the right thing to do: make the stat return the same value > independent of the usage of the shared zeropage. > I'll update the documentation accordingly. >> I'm not sure what to do about smaps_pmd_entry in that case. We >> probably don't care about compund pages. > > No, KSM only places the shared zeropage for PTEs, no need to handle PMDs.