From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A65FFC6FD1D for ; Thu, 30 Mar 2023 16:46:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C4D56B007D; Thu, 30 Mar 2023 12:46:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 375286B007E; Thu, 30 Mar 2023 12:46:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23C316B0080; Thu, 30 Mar 2023 12:46:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 14D776B007D for ; Thu, 30 Mar 2023 12:46:05 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D7A96120A2C for ; Thu, 30 Mar 2023 16:46:04 +0000 (UTC) X-FDA: 80626141848.28.64B3499 Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) by imf28.hostedemail.com (Postfix) with ESMTP id A5F89C0011 for ; Thu, 30 Mar 2023 16:46:02 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm3 header.b=qWfDLQAr; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=NVaM3Tni; spf=pass (imf28.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.21 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680194762; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C1ANtzD99ITAlQKFihJs3nRdO+Noal3WKWuXkIP7KiE=; b=rNe7ss3fKi7sULyiGGIJKgqLEFgi8fcpIPg502XIEDHVwEUHoupLFqU8EN2mRmWZhgfphH OLAS+2LSZpw9XHkAMwNICZnj8PGQfBQt2a3ux/6MF3icTppTqyJMUAhrsf/QiO5ViVBIvG Nj3E/AG0wqjSYwGp2CzHPx4A3Mt1/zA= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm3 header.b=qWfDLQAr; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=NVaM3Tni; spf=pass (imf28.hostedemail.com: domain of shr@devkernel.io designates 64.147.123.21 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680194762; a=rsa-sha256; cv=none; b=Ub2PJ3ZavgFVRI+6Xxhzj+Fh7nvG+GHuhl55+sbIVQv0DHa1AiPnEndgHZ1pfCW6hqmSAS Jnl+jCgON17pCmlbDJMiutpQzn1IXcOgvle9te898sgtESui96mBNDHQUzSsSQ9W+4DPSR DZiPrqVNAKDvqenDg6BWqCIOb43Izg8= Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 9EF98320093D; Thu, 30 Mar 2023 12:46:00 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 30 Mar 2023 12:46:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devkernel.io; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1680194760; x=1680281160; bh=C1 ANtzD99ITAlQKFihJs3nRdO+Noal3WKWuXkIP7KiE=; b=qWfDLQArvQmB7aFjev i+7YCgnvM+n5QYFJf4i0ZT1ZNX6HtVJSTQxYz43+2Drn7L7LCxlaHIzkgLEN39qV YIsQPKvCBzhhrFnVeYyXBZPYuX/wpXfUZ9hjq23q9C9Ebb7gvLcJASrQkj+GLlw2 qug7ru5btkiOT4DBF8jZs4YQ9QUYvBG5tIc8dxl2xXpjHHRUokiIJJPXHBZQo3MT On3EPZDQTha7icw1ZQAtZF3OcWb0FQAL/j/EA4kCuY/gpv9PEbMA1kxpQBv6pl2V vLiinfJh3jaYx3AYaiwcb6GBEauiO6W7oNoiAda8gIpVloTAiLQtrLvMHUSz/AUp QiEQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1680194760; x=1680281160; bh=C1ANtzD99ITAl QKFihJs3nRdO+Noal3WKWuXkIP7KiE=; b=NVaM3TniOM/E/HCLGOxRciWTO6yxV XM7LjAUJ3U/xNGvBJs35vqjQ4Hclu2Fe6EdVvW41l6AG7J4HzSjXXKiIomDJpErV +evqY5IF7fjzVYBZoTlwgagH52hwwiDQJU/y9UgJJYoAtl1uBR+LnbiPdt/UN5WC BT6FPgO2I7MNlahwGLf73ZS1vbImKIohAWpeCK0DkMOH1uJ7pHRWWJ2aqis9b9nt qJl4mA9zR214L+Ye1aaIR7c0zn4W+2Hi72DBqeXmJrs5oq3wUYfOQheVZFasrGua athtCv6WXGMtCsur2KVb9bYj9Acfysh4iWF20afel0+fT7Kp6/WVpa2dg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdehledguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfhgfhffvvefuffgjkfggtgesthdtredttdertdenucfhrhhomhepufhtvghf rghnucftohgvshgthhcuoehshhhrseguvghvkhgvrhhnvghlrdhioheqnecuggftrfgrth htvghrnhepveelgffghfehudeitdehjeevhedthfetvdfhledutedvgeeikeeggefgudeg uedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsh hhrhesuggvvhhkvghrnhgvlhdrihho X-ME-Proxy: Feedback-ID: i84614614:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 30 Mar 2023 12:45:58 -0400 (EDT) References: <20230310182851.2579138-1-shr@devkernel.io> <20230328160914.5b6b66e4a5ad39e41fd63710@linux-foundation.org> <37dcd52a-2e32-c01d-b805-45d862721fbc@redhat.com> User-agent: mu4e 1.6.11; emacs 28.2.50 From: Stefan Roesch To: David Hildenbrand Cc: Johannes Weiner , Andrew Morton , kernel-team@fb.com, linux-mm@kvack.org, riel@surriel.com, mhocko@suse.com, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, Hugh Dickins Subject: Re: [PATCH v4 0/3] mm: process/cgroup ksm support Date: Thu, 30 Mar 2023 09:41:36 -0700 In-reply-to: Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A5F89C0011 X-Stat-Signature: a48y5ahmsfooscoqedaeruqj1ukcqnih X-Rspam-User: X-HE-Tag: 1680194762-703127 X-HE-Meta: U2FsdGVkX18dN72JI+eik7mphOrwQqaqnHdavmwUCPRy+hxAmfbQhgW7HLbNmvjcWtvvYn7jjH6YV9B06wg4d6XubA2U8Qq16rEZXke6ChakEEAv/+W0qUnhrXBqxkJfxeMdvFkxu2pYeW7Ie75vxyojfwtMd1q6eYOUghFrxRDkcdKqH/KrQIdQUMRqY+Fv/XtxL+D2FEbq7QJetDJZNAUe/K7lQRntapbP2xt6ArSXdShh4bSCXky889SDl4XJpIsVGoTHZTNGc6U7ZqMq/UiVWq4Zl2O1kivWu/wEpCtI1I+MN//8iFaeaWTjMC+xRaMJW6/Bo/gDKm7saolW4tm0MWkK2SPOFo1jhxpR68bYWm4dHW44zqUOxB7YENI2hNR/yzV09vRJ1lFyLvu3T0D/D2hAQsdz+IB3L4QZJPN+bJqNeT09nSuZy2S7b+BuS7XyW9/mR8DXSeJEbxeOfjX6aUswMkPYet3BbguRdUSNVycGoInbmOY6RPtUP6mjO2iddS7I/V/addqDV7FgNi4UL8gJC/o8/qsx6mezKBPGb/qY3K2jEPaOrdOajbaLDPeaabP870VOEd702tpBGPgqAbKXdPmc6OUA4WZ7d9+SqJN2UkCOg7mruOV/5VPr9S9Ur3KFpcmTNoLV8b4TKfUf18s6J/pqYV3aaGETiadLx+VrUphvj43uBJbnvJeN0gIs7l21tusPCUWXdg2oP17nXYBNGOopKgPdI6gDhWYdMATOsKEz1a68qYhozvZ6s9BJb8mQcFEfBglof4otMKy/ec6IPRdkDmNdQ1nrR0IMfsy4E7gm1aBW/XBkQ3s/Qe+ZIKbNZtoKTZbDhORGHexKLLZ0c2JzTKVxbGVH9FR6+wuPE/DUt2XZePJEW0lZV0CJgdv/fwMBs+3xR/av7hzoOQc+jUCS4Xt/4K4QOoqkf4MYSXcGyA+5v91g+SoXHTXfySMGzgyogLESmKU 3MXZv8TC NzJ5mULWnFMpWLA64douBxqPDWWe1Izb08dLMKTd2RQkGupU2oNqZla4verTnuFcCLMyltVnEigcAoo7hWp5Kgca0XhKJAZCkimamE0SoPtqW3P5CuQlZeFiXNTHAz/C1fxdNYEv6Ts35nYpLiNvGM0yODBHi8ZLMwyS9OeTLti0jzRnmML8dVev9mpauXUTo8E9k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: My mistake I first answered to an older email. David Hildenbrand writes: > On 30.03.23 16:26, Johannes Weiner wrote: >> On Thu, Mar 30, 2023 at 06:55:31AM +0200, David Hildenbrand wrote: >>> On 29.03.23 01:09, Andrew Morton wrote: >>>> On Fri, 10 Mar 2023 10:28:48 -0800 Stefan Roesch wrote: >>>> >>>>> So far KSM can only be enabled by calling madvise for memory regions. To >>>>> be able to use KSM for more workloads, KSM needs to have the ability to be >>>>> enabled / disabled at the process / cgroup level. >>>> >>>> Review on this series has been a bit thin. Are we OK with moving this >>>> into mm-stable for the next merge window? >>> >>> I still want to review (traveling this week), but I also don't want to block >>> this forever. >>> >>> I think I didn't get a reply from Stefan to my question [1] yet (only some >>> comments from Johannes). I would still be interested in the variance of >>> pages we end up de-duplicating for processes. >>> >>> The 20% statement in the cover letter is rather useless and possibly >>> misleading if no details about the actual workload are shared. >> The workload is instagram. It forks off Django runtimes on-demand >> until it saturates whatever hardware it's running on. This benefits >> from merging common heap/stack state between instances. Since that >> runtime is quite large, the 20% number is not surprising, and matches >> our expectations of duplicative memory between instances. > > Thanks for this explanation. It's valuable to get at least a feeling for the > workload because it doesn't seem to apply to other workloads at all. > >> Obviously we could spend months analysing which exact allocations are >> identical, and then more months or years reworking the architecture to >> deduplicate them by hand and in userspace. But this isn't practical, >> and KSM is specifically for cases where this isn't practical. >> Based on your request in the previous thread, we investigated whether >> the boost was coming from the unintended side effects of KSM splitting >> THPs. This wasn't the case. >> If you have other theories on how the results could be bogus, we'd be >> happy to investigate those as well. But you have to let us know what >> you're looking for. >> > > Maybe I'm bad at making such requests but > > "Stefan, can you do me a favor and investigate which pages we end up > deduplicating -- especially if it's mostly only the zeropage and if it's > still that significant when disabling THP?" > > "In any case, it would be nice to get a feeling for how much variety in > these 20% of deduplicated pages are. " > > is pretty clear to me. And shouldn't take months. > /sys/kernel/mm/ksm/pages_shared is over 10000 when we run this on an Instagram workload. The workload consists of 36 processes plus a few sidecar processes. Each of these individual processes has around 500MB in KSM pages. Also to give some idea for individual VMA's 7ef5d5600000-7ef5e5600000 rw-p 00000000 00:00 0 (Size: 262144 KB, KSM: 73160 KB) >> Beyond that, I don't think we need to prove from scratch that KSM can > > I never expected a proof. I was merely trying to understand if it's really KSM > that helps here. Also with the intention to figure out if KSM is really the > right tool to use here or if it simply "helps by luck" as with the shared > zeropage. That end result could have been valuable to your use case as well, > because KSM overhead is real. > >> be a worthwhile optimization. It's been established that it can >> be. This series is about enabling it in scenarios where madvise() >> isn't practical, that's it, and it's yielding the expected results. > > I'm sorry to say, but you sound a bit aggressive and annoyed. I also have no > idea why Stefan isn't replying to me but always you. > > Am I asking the wrong questions? Do you want me to stop looking at KSM code? > Your review is valuable, Johannes was quicker than me.