From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 431B9C0015E for ; Tue, 15 Aug 2023 17:17:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C3CC8D0011; Tue, 15 Aug 2023 13:17:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 873A68D0001; Tue, 15 Aug 2023 13:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 714A18D0011; Tue, 15 Aug 2023 13:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 622908D0001 for ; Tue, 15 Aug 2023 13:17:27 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 095C7140DD9 for ; Tue, 15 Aug 2023 17:17:27 +0000 (UTC) X-FDA: 81126995334.03.CA74F8F Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) by imf03.hostedemail.com (Postfix) with ESMTP id 6484A20029 for ; Tue, 15 Aug 2023 17:17:22 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=YUIGkNI6; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=UnNWA4GG; dmarc=none; spf=pass (imf03.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.24 as permitted sender) smtp.mailfrom=shr@devkernel.io ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692119842; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3YVa9SLafbAppolvaU0gOB/8rD3tfIXlrgSkLKnfB8Y=; b=WlUZO9vKQ7z7m0uDTq2kuecz9jXDjcn//+vm9w3/uGHLtxtisJHkHFLYShSDYtu+R4AaC6 1ofTAVm981KrwOI8MRGLyoZxJTq4VRIqCQVn57yBq6NMdEJSQV9tqQIEfiiU0uaVR3z0bB I/2mlkJ2mVC3mKN/6Sri4qYTKsEL6Fk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=YUIGkNI6; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=UnNWA4GG; dmarc=none; spf=pass (imf03.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.24 as permitted sender) smtp.mailfrom=shr@devkernel.io ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692119842; a=rsa-sha256; cv=none; b=ayspmbzUpBpuj0ernBb3N2CGxDoBSVdhnrkek4fItT2oUTp7KxpBbke01HV+Mx6sKJjJAM +dpfxOMztvNi2cUGaSPGEz5+LepL4rEjLf2J0vhHKBkuerRU8TUP8oMJztAtaZTx2pjpYr Lv9jQU/6AYfaxckPgCuV1lmngZ0vYFs= Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 276F93200927; Tue, 15 Aug 2023 13:17:20 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 15 Aug 2023 13:17:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devkernel.io; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1692119839; x=1692206239; bh=3Y Va9SLafbAppolvaU0gOB/8rD3tfIXlrgSkLKnfB8Y=; b=YUIGkNI61be5H3tfus iHULantyHkns1KFWmB7QbSj2jlBI/43ex2LjoimptigzKEA2091tP4nHePhDlRbc McqGkeqlOgR/i8+DLk/7mKsgcC0UrVyUkcaEVknLEP8hMCHbYW5fBsJ+0roI8RzO D48ayd/NjyLDrXJqzjK1qEtWY7LsQ3PKVhp//JLdl/mqQB4ZXd95lrgOL5GWUQSc V69XU9IiiKFdbwdnEIy7KXrXRjCvYc5IogbaE450SjQPBcik/uqtVcc15b5fock1 8wdiG0RHFQZbSo1NxnOIqUMqhxQrJBz4wQPBMWitbLIM6yIrhUiHCtsVM52El3iH VNmQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1692119839; x=1692206239; bh=3YVa9SLafbApp olvaU0gOB/8rD3tfIXlrgSkLKnfB8Y=; b=UnNWA4GGhDaLfvAUjXw17Uy0Ld52N 6gCfwIZ1B75jBvJjh3OWEAlXFutO16XUyDPjpQAguXsrJMfgbmWm8PXuVTBLglFT J7vugOmDvKeDrENQG2jSJkMeTonnx6CJ4RlvgthnsRvxaIEQIGPYxAMR5vbCBQZ6 /YYij/Te5zSF7i+wG0mkiy2A5nXrwLmfASOP+GVrxhcTyxWGtWHalQZSGRZ5ZcvW fFAMiRJ2UX2vzD2XVFKW5vI0Tsn36pwuobN2DyvIriLlQlaXXLG6WH+hJiYumwu0 sHFHNj451bMDdEh+vj3OaSaitgNvwu55lk22yoh0BVuSVNKfDxJjk0L0Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedruddtjedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpehffgfhvfevufffjgfkgggtsehttdertddtredtnecuhfhrohhmpefuthgv fhgrnhcutfhovghstghhuceoshhhrhesuggvvhhkvghrnhgvlhdrihhoqeenucggtffrrg htthgvrhhnpeevlefggffhheduiedtheejveehtdfhtedvhfeludetvdegieekgeeggfdu geeutdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hshhhrseguvghvkhgvrhhnvghlrdhioh X-ME-Proxy: Feedback-ID: i84614614:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 15 Aug 2023 13:17:18 -0400 (EDT) References: <20230811162803.1361989-1-shr@devkernel.io> <43d64aee-4bd9-bba0-9434-55cec26bd9dc@redhat.com> User-agent: mu4e 1.10.1; emacs 28.2.50 From: Stefan Roesch To: David Hildenbrand Cc: kernel-team@fb.com, akpm@linux-foundation.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2] proc/ksm: add ksm stats to /proc/pid/smaps Date: Tue, 15 Aug 2023 10:10:55 -0700 In-reply-to: <43d64aee-4bd9-bba0-9434-55cec26bd9dc@redhat.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Queue-Id: 6484A20029 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4onw5sfiosc5rzwxkchcog694hqx6x37 X-HE-Tag: 1692119842-27008 X-HE-Meta: U2FsdGVkX1+RQMEEQ7m931Tbd2OT676XxYalJ34uhuQ0ud3AbKhtOucqYYZeSUBXonrNG2WxCnlyKbrT9e8fatsbpBuuZ2FFpvQRs9IgGaR8GHaZQaGWJABOGFmGQt5Ha7A9i+enyLyOS5vUSPGaFd6zziKj+s1AK/ucCygjDXDNxySsSRJDKEFRXHGqQFphKXoPbEQ6WRWe9WFWEFh5RXVF46MCb0ykP8WAfFHcl93LA53pmSTgyJy05Ld1etDo85Ybjfo4umqF3SnPOZcddxfeaiev1lLz/PFcXCv4/9Se++O4XnoKGQibvch9wg9LBBAd6GvQ8SjF2HI4TxB4wgQjMdYCxw0I/5NNQGID0HpS9EgD2BYRUs2AIx6HuHN1axO578zJo0rMGvEMfF1oHf/jSsGuzd2s2V324QF9v/3eiQ3F7HU9IbYttmVtXDRYRXgvQ9o42oSgP77v8cJBTkMujHD/HbxbtSFGQrqq/Tx/oXWLN6UaQ7QKYQHCcLjrFbb58X81JNhoybgGPDi3M79UFu8Y8lqjpY9LvWWV73VB08PBnQLlzRlZ8cGwjmEAS4PkWNdzyrtoq7+xCVX9xRJzcxAUy3FxNFLRDpeNLR1EVTV6TuQyy+GV3LREep2QIQZ1g304WveAjcOEoU1GhtzY2RFMsfwHy+fiNmeA15SkXEnPpMqJtkTmLAMlz+nmi6LXt+hPVpUtU/tszIW3CagT6Pa01q7ip5KwFjDD1Jz1rdZ48Zh6uFV7cBFakoLDbIf1Atg6+z6KLurVGTstC6KijNwPnEj6ickUHit6WQ2D7Fkty/1dkoRw2Wn+nO6IUbnREk3zlX+Q47kIk1W3mm80MiaCiiGQ73b9zsZ/JQUSepIZ0qXLiMsmBM4NqNJ9ppM3qPsCWjF2xtCt7peoactSy3pPDQRjahOFy5xjLBAdRWt1yhRoqf+H9XYXvpaWzEkN9P4V7LJt7Bwgbk1 w5R66ZCN 6ATFZ4cJxhIVq9A1EIbfqCtdDPq2J+OeBx+8Cwwl2w1ZxJgmgHChWEwU8bH0p8V0wDtwA1Vr+ZT39B7tW2yLFYKsaQh0lCUTRPPGCmvk1B8WvCSHTVicJJzlpse2zhfprD+1vmsj4rekEUr5seHpUtV2zmYQQEnqf3wiKeXJvtTwEXv61MkaO9B7zFQY64XEkK1JoTtrkSjmnaouX6+Icvnu99me7fyQXUDdcgSZztZlLPAA5XZBxi5O9KQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > Sorry for the late reply, Gmail once again decided to classify your mails as > spam (for whatever reason). > > On 11.08.23 18:28, Stefan Roesch wrote: >> With madvise and prctl KSM can be enabled for different VMA's. Once it >> is enabled we can query how effective KSM is overall. However we cannot >> easily query if an individual VMA benefits from KSM. >> This commit adds a KSM section to the /prod//smaps file. It reports >> how many of the pages are KSM pages. >> Here is a typical output: >> 7f420a000000-7f421a000000 rw-p 00000000 00:00 0 >> Size: 262144 kB >> KernelPageSize: 4 kB >> MMUPageSize: 4 kB >> Rss: 51212 kB >> Pss: 8276 kB >> Shared_Clean: 172 kB >> Shared_Dirty: 42996 kB >> Private_Clean: 196 kB >> Private_Dirty: 7848 kB >> Referenced: 15388 kB >> Anonymous: 51212 kB >> KSM: 41376 kB >> LazyFree: 0 kB >> AnonHugePages: 0 kB >> ShmemPmdMapped: 0 kB >> FilePmdMapped: 0 kB >> Shared_Hugetlb: 0 kB >> Private_Hugetlb: 0 kB >> Swap: 202016 kB >> SwapPss: 3882 kB >> Locked: 0 kB >> THPeligible: 0 >> ProtectionKey: 0 >> ksm_state: 0 >> ksm_skip_base: 0 >> ksm_skip_count: 0 >> VmFlags: rd wr mr mw me nr mg anon >> This information also helps with the following workflow: >> - First enable KSM for all the VMA's of a process with prctl. >> - Then analyze with the above smaps report which VMA's benefit the most >> - Change the application (if possible) to add the corresponding madvise >> calls for the VMA's that benefit the most >> Signed-off-by: Stefan Roesch >> --- >> Documentation/filesystems/proc.rst | 3 +++ >> fs/proc/task_mmu.c | 5 +++++ >> 2 files changed, 8 insertions(+) >> diff --git a/Documentation/filesystems/proc.rst >> b/Documentation/filesystems/proc.rst >> index 7897a7dafcbc..4ef3c0bbf16a 100644 >> --- a/Documentation/filesystems/proc.rst >> +++ b/Documentation/filesystems/proc.rst >> @@ -461,6 +461,7 @@ Memory Area, or VMA) there is a series of lines such as the following:: >> Private_Dirty: 0 kB >> Referenced: 892 kB >> Anonymous: 0 kB >> + KSM: 0 kB >> LazyFree: 0 kB >> AnonHugePages: 0 kB >> ShmemPmdMapped: 0 kB >> @@ -501,6 +502,8 @@ accessed. >> a mapping associated with a file may contain anonymous pages: when MAP_PRIVATE >> and a page is modified, the file page is replaced by a private anonymous copy. >> +"KSM" shows the amount of anonymous memory that has been de-duplicated. > > > How do we want to treat memory that has been deduplicated into the shared > zeropage? > > It would also match this description. > > See in mm-stable: > > commit 30ff6ed9a65c7e73545319fc15f7bcf9c52457eb > Author: xu xin > Date: Tue Jun 13 11:09:28 2023 +0800 > > ksm: support unsharing KSM-placed zero pages > > Patch series "ksm: support tracking KSM-placed zero-pages", v10. I see two approaches how to deal with zero page: - If zero page is not enabled, it works as is - If enabled - Document that zero page is accounted for the current vma or - Pass in the pte from smaps_pte_entry() to smaps_account() so we can determine if this is a zero page. I'm not sure what to do about smaps_pmd_entry in that case. We probably don't care about compund pages.