From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0571C61DA3 for ; Tue, 21 Feb 2023 18:03:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D4F76B0071; Tue, 21 Feb 2023 13:03:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 384106B0073; Tue, 21 Feb 2023 13:03:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24BE46B0074; Tue, 21 Feb 2023 13:03:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 176DA6B0071 for ; Tue, 21 Feb 2023 13:03:33 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DD82E160734 for ; Tue, 21 Feb 2023 18:03:32 +0000 (UTC) X-FDA: 80492071464.25.A222046 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by imf24.hostedemail.com (Postfix) with ESMTP id 976F0180019 for ; Tue, 21 Feb 2023 18:03:29 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=uFm3Ouqi; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=AaTB+z0p; spf=pass (imf24.hostedemail.com: domain of shr@devkernel.io designates 66.111.4.25 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677002609; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZH9OeA2FvCaJR9ZdNyNOKK8xdZTHpoN/UMcS3ejigug=; b=p0kE+IT3CWloqEILfk/hNPOV7+sDCr+j3zbBh/6uuZ2IFzqTXSP1E/Aa8DPeom8kuFicmI EpedmIPb1yTJ+GS3q5FVWCvNTO/6TuQ1UyK/s4qjzLWHesKufHggzgbLglqyUC3EKSIkSm 4G/cWIXmGiIZffW3gHwreT1ZzrNC1Vk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=devkernel.io header.s=fm2 header.b=uFm3Ouqi; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=AaTB+z0p; spf=pass (imf24.hostedemail.com: domain of shr@devkernel.io designates 66.111.4.25 as permitted sender) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677002609; a=rsa-sha256; cv=none; b=Hw+egbaFhGsd5td3ckbqG0WshudR1qtc/eAHMzKLXkRNP07o7gs9Rr/iYzhwUY4C7IjJ4h zQbnTOoWnrT4yotECQwe/hSsAzjA3BgDavEFA9qAUuVlXFkRDo3Rs7ccMrKxzYVF96dbJT UmuFR7WlYnf25aRu4w3I95FeUff855c= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 14BBF5C00EE; Tue, 21 Feb 2023 13:03:29 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 21 Feb 2023 13:03:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devkernel.io; h= cc:cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm2; t=1677002609; x=1677089009; bh=ZH9OeA2FvC aJR9ZdNyNOKK8xdZTHpoN/UMcS3ejigug=; b=uFm3OuqiNjZk1Pp1VcYl+B4GLI LHh0AzdprYYpC0NdWU4lBklhoeaUyj8Ky/y48Su+7draK6ujiDqll6TAHxuTTvVE e58ql/Hott1vljDoBYKN/P+cgMOQsAvatGq9i0/dLBmBCdZIo5IGOrDPNISTITrG Kz01+aqAf4LzyCQKvaqvQFmRYKFhEsfqEfYIM+tBqafPCpGuGNM8lLYKX/fwbd53 8ak3vPCjL4xHJzBgOi4zdvDpJLMTF0z7sK4VhVf2Pu1dA0QpKzgj2Vx948kMk8Mv XRgznh0CTgY3I8BUuNBoCT5T9Ao2ACFibsD7UsDtVyvIUIqv9EsVPMTxv85g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1677002609; x=1677089009; bh=ZH9OeA2FvCaJR9ZdNyNOKK8xdZTH poN/UMcS3ejigug=; b=AaTB+z0pCAZE5fWJg+tXQ9XobiP2hdb4zrZw2akHB+Oq /qN7mH5X2STy4YUeJKdrmAVcWwKMBS3Sa8GuvfgjwhNSOAenm+aLhc7bmzGwK6Tp BA1ZycvGfZc5z/r9Q2D1JXAitRXeKOWlVYi7RNVLBki4+c3c8fy28zWF1K6ev4L8 fsfVrceQ3CHhYx+6fta5ib+QtTtwyNct0wCVGoIyNHJJLR/wky9b410Oy6SRH2Ou 67cUxgssYFu/RWi7E/7yieo3p0/iaD3kKriy1VD1GvbhDEBuB1e9njUg2amd8XaC woBK0CSqBTzVKLeGaUTTvgalkjFDHf21CTqq9a6fEA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudejjedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfhgfhffvvefuffgjkfggtgesth dtofdttdertdenucfhrhhomhepufhtvghfrghnucftohgvshgthhcuoehshhhrseguvghv khgvrhhnvghlrdhioheqnecuggftrfgrthhtvghrnhepfeeludefhfegvdffieeuhfdvud etvdetfefgieegffduhfegffeuudevkeeiuedunecuvehluhhsthgvrhfuihiivgeptden ucfrrghrrghmpehmrghilhhfrhhomhepshhhrhesuggvvhhkvghrnhgvlhdrihho X-ME-Proxy: Feedback-ID: i84614614:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 21 Feb 2023 13:03:27 -0500 (EST) References: <20230210215023.2740545-1-shr@devkernel.io> User-agent: mu4e 1.6.11; emacs 28.2.50 From: Stefan Roesch To: Johannes Weiner Cc: kernel-team@fb.com, linux-mm@kvack.org, riel@surriel.com, mhocko@suse.com, david@redhat.com, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, akpm@linux-foundation.org Subject: Re: [RFC PATCH v2 00/19] mm: process/cgroup ksm support Date: Tue, 21 Feb 2023 09:59:59 -0800 In-reply-to: Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: yamom8ratexq4kzp3uwoyh3yj1snjb1m X-Rspamd-Queue-Id: 976F0180019 X-HE-Tag: 1677002609-262734 X-HE-Meta: U2FsdGVkX19k4LtsrJXF6Fw3QXsDzRZPEAk9ADNPhDUQt+V5xQBbZzdnOwamDR3PSmCPrE2A2Ft6RGClnvrU/yl/5c9lDVVZlA2cMelqXCHYmyRBteXRV5MR9XbyIFWJMC7arcfy4mNw/Em0l/Su16DkQW+7Jg8pA4A49FjZnkrOv5N4N9UsYxj+HIozmV6K7U0yZyOYVGn75yIrYL9mhXKIMPhHl+jAdt1zjA1qn8LNvuBZZKywCquM/u+WfCY0k5dZZjrQnGHFUMdpgIrsqIOo+G7hdtxcmljHSYVPGv6BhMGOzRdBpNCaFiH49JuWhEarviPDEqjX1JWKz7hOHHWDty+d+b6eHH8C14IB66pe/ebozLZuO4povp/PHnnKyoEP61CqU+Q7ENVshuyKkUlgMzhQbc8XmUPZ7peMxK66/MfMj1M+fRGk9kqhdvgGgExEO9AfL0a9kUpMn5M3vY9cHShgh/BOdBa4yeSaO/D61zlnSN1cIQMdV6BDoL/m5aHCyjfWMmkX3Ki1996ny2sZ2C7mpFbnBihkgCAJ08k/HZ1gBinwg/mTPHAcZ9cKWyTl+xI2za7PUIE6Pn/4fueazGq+6nu/fVK4YBqJ//U2/oXyQ+1478OXPJFvm3C4EO5k/CIUW5h1KV0STgYfBkqTKnoYWNjJ6VZUyNsKKfZYWAhUFilV1YzA1CyBgNFzADRnEbFGb1eEceRWOH5pmWJvztVlS6ipU2pfoJKIukdy8EhqHjim2WsqF/QGImzt29I/mtWZLfHcCAosjBSg8LIS3hrJObahUQ25FwmNTCvzmyIwnpqfC5uKppxhMy+gK+NkK9diS2ML8ClKVBhElsgbIsNi3yfjHsE+v4c3Ghomt+0zkmDfHzwgMjTcxWAPPRZNbghsnZZEPho9EMp2ah7SsIDajONllueMCv/RpTSK5ajVMWiwb8rFaAcpwToaxpo4vozeCBYoQx902fp Yguk0TkY pHzA9NT4XRJktHwN9lGmOr8+S2Do0dI19hKbPz8l+lWVQCqNxSlYp3jDs04GCml2BQDBeQ77jeCq4VMmzBmQhk7cuKcuKfbgVOEW0YUl+TIoDV/pyLDNANqqC+suo8h8/t+PxHChE/LjY6dYyFC+HySpuOSt5l4gf+JQEtfASTjoqCWnK9Vf8CsC4DsxzCHUEL3++ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Johannes Weiner writes: > Hi Stefan, > > On Fri, Feb 10, 2023 at 01:50:04PM -0800, Stefan Roesch wrote: >> So far KSM can only be enabled by calling madvise for memory regions. What is >> required to enable KSM for more workloads is to enable / disable it at the >> process / cgroup level. >> >> Use case: >> The madvise call is not available in the programming language. An example for >> this are programs with forked workloads using a garbage collected language without >> pointers. In such a language madvise cannot be made available. >> >> In addition the addresses of objects get moved around as they are garbage >> collected. KSM sharing needs to be enabled "from the outside" for these type of >> workloads. > > It would be good to expand on the argument that Rik made about the > interpreter being used for things were there are no merging > opportunities, and the KSM scanning overhead isn't amortized. > > There is a fundamental mismatch in scopes. madvise() is a > workload-local decision, whereas sizable sharing opportunities may or > may not exist across multiple workloads. Only a higher-level entity > like a job scheduler can know for certain whether it's running one or > more instances of a job. That job scheduler in turn doesn't have the > necessary knowledge of the workload's internals to make targeted and > well-timed advise calls with, say, process_madvise(). > > This also applies to the security concerns brought up in previous > threads. An individual workload doesn't know what else is running on > the machine, so it needs to be highly conservative about what it can > give up for system-wide merging. However, if the system is dedicated > to running multiple jobs within the same security domain, it's the job > scheduler that knows that sharing isn't a problem, and even desirable. > > So I think this series makes sense, but it would be good to expand a > bit on the reasoning and address the security aspect in the cover/doc. > These are good points Johannes, I'll elaborate on them with the next version of the patch. >> Stefan Roesch (19): >> mm: add new flag to enable ksm per process >> mm: add flag to __ksm_enter >> mm: add flag to __ksm_exit call >> mm: invoke madvise for all vmas in scan_get_next_rmap_item >> mm: support disabling of ksm for a process >> mm: add new prctl option to get and set ksm for a process > > The implementation looks sound to me as well. > > I think it would be a bit easier to review if you folded these ^^^ > patches, the tools patch below, and the prctl selftests, all into one > single commit. It's one logical change. This way the new flags and > helper functions can be reviewed against the new users and callsites > without having to jump back and forth between emails. > I'll fold them in the next version. >> mm: split off pages_volatile function >> mm: expose general_profit metric >> docs: document general_profit sysfs knob >> mm: calculate ksm process profit metric >> mm: add ksm_merge_type() function >> mm: expose ksm process profit metric in ksm_stat >> mm: expose ksm merge type in ksm_stat >> docs: document new procfs ksm knobs > > Same with the new knobs/stats and their documentation. > I'll fold them in the next version. > Logical splitting is easier to follow than geographical splitting. > > Thanks!