From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37B08C433FE for ; Fri, 13 May 2022 20:32:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DDDB8D0005; Fri, 13 May 2022 16:32:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78D1D8D0002; Fri, 13 May 2022 16:32:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 654438D0005; Fri, 13 May 2022 16:32:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 51E638D0002 for ; Fri, 13 May 2022 16:32:13 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 24686321A4 for ; Fri, 13 May 2022 20:32:13 +0000 (UTC) X-FDA: 79461866946.08.F07861C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id 109EB1C00BC for ; Fri, 13 May 2022 20:32:03 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6F258622CE; Fri, 13 May 2022 20:32:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DA1DAC34100; Fri, 13 May 2022 20:32:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1652473931; bh=gmC5AwVMGEU2yYbKfPpZCKlEafrDstnH7NSBfxHTE8s=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=GnVFoZiTL3uqj38AiZl1Vz7AxQAjGSbWngWz20DyGxSP0DgtxFcsxMN9FvY1MqChm 1P+o4f+nAG2ZqVKZEYsQSnUTWeH2EC3JvmBHa4LQ0TD4oxpD7+6Za6gESm2+KpQLN/ MxbE+y77lmCFjQXNV3L9gXUYmz/10kBHSZ3Gd8d4= Date: Fri, 13 May 2022 13:32:10 -0700 From: Andrew Morton To: Oleksandr Natalenko Cc: cgel.zte@gmail.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, corbet@lwn.net, xu xin , Yang Yang , Ran Xiaokai , wangyong , Yunkai Zhang , Matthew Wilcox Subject: Re: [PATCH v6] mm/ksm: introduce ksm_force for each process Message-Id: <20220513133210.9dd0a4216bd8baaa1047562c@linux-foundation.org> In-Reply-To: <1835064.A2aMcgg3dW@natalenko.name> References: <20220510122242.1380536-1-xu.xin16@zte.com.cn> <5820954.lOV4Wx5bFT@natalenko.name> <20220512153753.6f999fa8f5519753d43b8fd5@linux-foundation.org> <1835064.A2aMcgg3dW@natalenko.name> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 109EB1C00BC X-Stat-Signature: thdfk6ew1mosqzons5m1aqgr3wwnrfx6 X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=GnVFoZiT; spf=pass (imf21.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam09 X-HE-Tag: 1652473923-811225 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 13 May 2022 11:51:53 +0200 Oleksandr Natalenko wrote: > Hello. > > On pátek 13. května 2022 0:37:53 CEST Andrew Morton wrote: > > On Tue, 10 May 2022 15:30:36 +0200 Oleksandr Natalenko wrote: > > > > > > If ksm_force is set to 1, force all anonymous and 'qualified' VMAs > > > > of this mm to be involved in KSM scanning without explicitly calling > > > > madvise to mark VMA as MADV_MERGEABLE. But It is effective only when > > > > the klob of /sys/kernel/mm/ksm/run is set as 1. > > > > > > > > If ksm_force is set to 0, cancel the feature of ksm_force of this > > > > process (fallback to the default state) and unmerge those merged pages > > > > belonging to VMAs which is not madvised as MADV_MERGEABLE of this process, > > > > but still leave MADV_MERGEABLE areas merged. > > > > > > To my best knowledge, last time a forcible KSM was discussed (see threads [1], [2], [3] and probably others) it was concluded that a) procfs was a horrible interface for things like this one; and b) process_madvise() syscall was among the best suggested places to implement this (which would require a more tricky handling from userspace, but still). > > > > > > So, what changed since that discussion? > > > > > > P.S. For now I do it via dedicated syscall, but I'm not trying to upstream this approach. > > > > Why are you patching the kernel with a new syscall rather than using > > process_madvise()? > > Because I'm not sure how to use `process_madvise()` to achieve $subj properly. > > The objective is to mark all the eligible VMAs of the target task for KSM to consider them for merging. > > For that, all the eligible VMAs have to be traversed. > > Given `process_madvise()` has got an iovec API, this means the process that will call `process_madvise()` has to know the list of VMAs of the target process. In order to traverse them in a race-free manner the target task has to be SIGSTOP'ped or frozen, then the list of VMAs has to be obtained, then `process_madvise()` has to be called, and the the target task can continue. This is: > > a) superfluous (the kernel already knows the list of VMAs of the target tasks, why proxy it through the userspace then?); and > b) may induce more latency than needed because the target task has to be stopped to avoid races. OK. And what happens to new vmas that the target process creates after the process_madvise()? > OTOH, IIUC, even if `MADV_MERGEABLE` is allowed for `process_madvise()`, Is it not? > I cannot just call it like this: > > ``` > iovec.iov_base = 0; > iovec.iov_len = ~0ULL; > process_madvise(pidfd, &iovec, 1, MADV_MERGEABLE, 0); > ``` > > to cover the whole address space because iovec expects total size to be under ssize_t. > > Or maybe there's no need to cover the whole address space, only the lower half of it? Call process_madvise() twice, once for each half? > Or maybe there's another way of doing things, and I just look stupid and do not understand how this is supposed to work?.. > > I'm more than happy to read your comments on this. > I see the problem. I do like the simplicity of the ksm_force concept. Are there alternative ideas?