From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C06CC433EF for ; Mon, 4 Jul 2022 06:48:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D52F08E0001; Mon, 4 Jul 2022 02:48:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D03666B0073; Mon, 4 Jul 2022 02:48:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF1D58E0001; Mon, 4 Jul 2022 02:48:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B11CB6B0072 for ; Mon, 4 Jul 2022 02:48:10 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 899FF353B9 for ; Mon, 4 Jul 2022 06:48:10 +0000 (UTC) X-FDA: 79648487940.10.80C6DDD Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf03.hostedemail.com (Postfix) with ESMTP id 0B82220047 for ; Mon, 4 Jul 2022 06:48:09 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 74718227D8; Mon, 4 Jul 2022 06:48:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1656917288; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9u7T45ClUffv42dPmfx3ngMwVaZJ6r1BiFszAAErmvI=; b=es4urU8VLsurTkIH16OTZYKTzyFmjLUfe7wkeSaUJcpgX2SYHJ7TF6X/luADMNeyNAYiIH fsi/Lolns0/6MfRveLTOrHoTzbSjfPyIYTeMDOffHmWzq7htPi86VGtSkloC07y3pJ909n 9urZLvuOKK6MsAa/eOz78MfjHi10itQ= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id C01B42C142; Mon, 4 Jul 2022 06:48:06 +0000 (UTC) Date: Mon, 4 Jul 2022 08:48:06 +0200 From: Michal Hocko To: David Hildenbrand Cc: cgel.zte@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, vbabka@suse.cz, minchan@kernel.org, oleksandr@redhat.com, xu xin , Jann Horn , Andrew Morton Subject: Re: [PATCH linux-next] mm/madvise: allow KSM hints for process_madvise Message-ID: References: <20220701084323.1261361-1-xu.xin16@zte.com.cn> <93e1e19a-deff-2dad-0b3c-ef411309ec58@redhat.com> <203548a6-cf70-30ce-6756-f6c909e7ef21@redhat.com> <54b67d6b-f600-1b9b-3d3f-e91b13d04c91@redhat.com> <11d28e6d-edb0-7d11-b476-c5808f3b7c5d@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <11d28e6d-edb0-7d11-b476-c5808f3b7c5d@redhat.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656917290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9u7T45ClUffv42dPmfx3ngMwVaZJ6r1BiFszAAErmvI=; b=dq6wRTv4GXRcM3+6UhTKqlT9+rEOFPx99rnAMlxyGCU+qAVkij38DMX2EztrKMgZJLqsKc kI4dg1w0dqqv/cDxHvp7UYNo32YSvSvZ9kyWb4Ksd8swGQwgNZHuK0bmcxQhmiqqj1Y99b qCjOkGB0oc44OiJsppZEamN7VBwRlfA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656917290; a=rsa-sha256; cv=none; b=LAiLFS9JOa/mjk+3qGHQxHxkeBuGHmlQHvSzWQYRoGsF+NsF19xkLqwlGF3fIO08xSdSkG GFHGmOTHNCXE0QhNwELukrI8sa+s0xxmjRokpWM/LClt+E/EPSzXOU6tX6kR6td9zhWP/q 2ZgabDVRVAWTgK0a8ZiMvqNhPomVUh4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=es4urU8V; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf03.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com X-Stat-Signature: ui98kzkhzbtxe6cpzq3b7kyua3b1atdo X-Rspamd-Queue-Id: 0B82220047 X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=es4urU8V; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf03.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com X-Rspamd-Server: rspam06 X-HE-Tag: 1656917289-475258 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 01-07-22 21:12:56, David Hildenbrand wrote: > On 01.07.22 15:19, Michal Hocko wrote: > > On Fri 01-07-22 14:39:24, David Hildenbrand wrote: > >>> I am not sure about exact details of the KSM implementation but if that > >>> is not a desirable behavior then it should be handled on the KSM level. > >>> The very sam thing can easily happen in a multithreaded (or in general > >>> multi-process with shared mm) environment as well. > >> > >> I don't quite get what you mean. > > > > I meant to say that if KSM needs to be aware of a special CoW semantic > > then it should be handled on the KSM layer regardless whether the KSM > > has been set by the process itself or any other process that has acccess > > to the MM. process_madvise is just another way to access a remote MM > > other than sharing the full MM. > > Okay. > > KSM has been a corner case feature that was restricted to well-defined > and well-tested environments. Until recently, R/O pins of any KSM pages > was essentially completely unreliably. And applications don't expect > such surprises. The shared zeropage is most probably the last > problematic piece. > > Yes, we're getting there that it's a real feature that can see more > (forced) wide-spread use. However, until the known issues in KSM have > been fixed (e.g., below -- there is a whole list of papers regarding > attacks on memory deduplication), it should be limited to well defined > environments and applications only -- IMHO. Very much agreed on all this! To be completely honest I am not really sure that all those consequences are widely understood and optmizing solely on memory savings is a very short sighted strategy IMO. But, it seems that there is a demand for this feature and previous attempts for APIs were much worse both from the semantic and maintainability POV. I am not sure we can get anything more sane than madvise. I also very much agree that current shortcomings have to be adressed first before we open this can of worms to 3rd party actors. I was not aware of those so thank for bringing them up. Maybe I was overly optimistic here. So I guess we have following questions to answer: 1) Do we really want to support KSM triggered by 3rd party? Does it impose new challenges other than existing ones in multi "threaded" environemnts? 2) If yes, is the process_madvise the most appropriate existing API? Or do we need a new one? 3) Should this be a highly privileged operation or we want to allow userspace to shoot its feet because consequences are subtle and not very well understood? > So what I want to express here is that if we're adding an interface that > can be used to just enable KSM on the whole system easily, it might be a > bit to soon for that. No matter what you document, people will ignore it. Agreed. > OTOH, if this is a real debug feature that will only be available in > specific debug/test scenarios (kernel config? toggle? whatsoever?), then > it's "better". If that is already the case, good. No, I think this is aimed to real production deployments. Thanks! -- Michal Hocko SUSE Labs