From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49C89C32793 for ; Wed, 18 Jan 2023 21:13:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E7226B0071; Wed, 18 Jan 2023 16:13:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 797B06B0073; Wed, 18 Jan 2023 16:13:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65F646B0074; Wed, 18 Jan 2023 16:13:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 57DC56B0071 for ; Wed, 18 Jan 2023 16:13:43 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2A0491C60E0 for ; Wed, 18 Jan 2023 21:13:43 +0000 (UTC) X-FDA: 80369171526.01.72DD86B Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf23.hostedemail.com (Postfix) with ESMTP id 57760140008 for ; Wed, 18 Jan 2023 21:13:40 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=XI5WJWm7; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674076420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=89f0PyHMTcU1wP4oLEjhhdUSmrRlYKAe9wYRnXUS3o8=; b=65LG2gSQAPjC0lwalUodIdRhwmmT/j99+5cNMTBEliBtVSYkvf4hylowtcikSGto7XU7zd kdbMThDCUfIfa4EDW7r1HZqJvKA/aU/uyETansguEA0slIkHXL34zRxcRXMYVZPwKdWTuX 0QJEst6e5XRiqbEr2Mdsljme2pQLEMo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=XI5WJWm7; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674076420; a=rsa-sha256; cv=none; b=q+dZ7tbzy2IWcGn7UVgAsTmdVPPBBXr9uG6uVC9AFBURNGDgx5FLepnVvYd+bcDGPHaElU /dYDenarB1Je+aasH6Cq+S3ivWNndeq11HgDs6B24U6DRP3ggwo9O3YolEPB3wrWj32znF x3eHMpzFjCIiOU8+aNfPkWqzJs+pzeY= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C3EBD5C332; Wed, 18 Jan 2023 21:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1674076418; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=89f0PyHMTcU1wP4oLEjhhdUSmrRlYKAe9wYRnXUS3o8=; b=XI5WJWm7SzfxR1z66yjeCcGNGrLMKHRuO2AR/iADQPTSG42FHZpSIgrQBXanf5uhucK3uD EMJFapFzmsgYIJ6B4FvK3t6HZqb6Dg6bKpC9qxHhieWRETt8hbSEn0+goojAVwIuimog9U 4it1d/VWddMXDsvoZ5zNb1APWwJmmBE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A823D139D2; Wed, 18 Jan 2023 21:13:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id jvHDJgJhyGOxMQAAMHmgww (envelope-from ); Wed, 18 Jan 2023 21:13:38 +0000 Date: Wed, 18 Jan 2023 22:13:38 +0100 From: Michal Hocko To: Minchan Kim Cc: Andrew Morton , Suren Baghdasaryan , Matthew Wilcox , linux-mm , LKML , SeongJae Park Subject: Re: [PATCH 3/3] mm: add vmstat statistics for madvise_[cold|pageout] Message-ID: References: <20230117231632.2734737-1-minchan@kernel.org> <20230117231632.2734737-3-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 7i8jt8tz6g7m7o3u4ptkn3ek33jch368 X-Rspamd-Queue-Id: 57760140008 X-HE-Tag: 1674076420-622161 X-HE-Meta: U2FsdGVkX1/OAA3iJCsoTtMvjO+9LhGjKVg3lWhtIYCSm6r8bKQdZRkwE0y4PFhWujAaR4R1KnLnoqc8+LHrKSDfzpFmnK+yLSVi6VlzcO1Nuo3Gn7t9LuJ3uQunW/TKR/dPSoEmoZY28FxHrBIro8ptxlNg8+Lw4ekl3uzRwSmCQJzBKUIUyhOGRJrD2ilT7WP6PgmPHAaPKhBUyZKXKKB9QX22POcN2i/IzZjs37UXEJu+Qqt0X6dQj1yZ3k0XdgrvMcdvAWVmN+smruaxva/mCvj97yes17yZ6JjHtL5lmCWhF8qkoiREeRIgDHa6/9c8JfqIFmdwOX033Lde9f5mH4JhCR1sla5XYDNJso9tG1PtlMoS1hepg+zYqkM34P/8aKIX+s4pP5WM2jktU3Hwcv+x4PmBmpMCFYkg706KINpOFKm2yitKFqLr+EjaIhomDPfA5GNbxflOn9cmxt/oh+vH/b6Fkps5C6AHO0WVQhh/l92AzPaYWPtEZPpdbvwa2DKtopkzDDlDCnBnpVarTmdBDJU2dH1Oj5gq6NwzVg0p8hJm3iSu+hmZXRLqGHi30UUd6B0IPfd+OUZzbrzq9Xu0uIATdc0dGNndKAPiN3vR7BJUyn7ofcR8iJeise/TQMfyl/MDSK8jIHOh/F5M0RRL3H1StoZpIH/ZxFaPPs4jl1Y8BDw6OPEsicB+194YWJpKqgPJh5Y2RHbSwtkcVLtRWe4LsuGzz5aN5uaQNVU3fPOS312lOTKdyhu0uCNpY4qb98qjxwcOVdgREeaB9REHT05GGb/YkFZ+H++eVsPasImCdxVNHqebH/IGA5K/1TqlCi7xLD7PJpbierqnyWEfmGNpicOzMYsLUNNDXKxoNxNM8klynNH5Z4k0ADXhUfZJ/ZzBiw2FwE3cyl55f3K2CTXf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 18-01-23 09:55:38, Minchan Kim wrote: > On Wed, Jan 18, 2023 at 06:27:02PM +0100, Michal Hocko wrote: > > On Wed 18-01-23 09:15:34, Minchan Kim wrote: > > > On Wed, Jan 18, 2023 at 10:11:46AM +0100, Michal Hocko wrote: > > > > On Tue 17-01-23 15:16:32, Minchan Kim wrote: > > > > > madvise LRU manipulation APIs need to scan address ranges to find > > > > > present pages at page table and provides advice hints for them. > > > > > > > > > > Likewise pg[scan/steal] count on vmstat, madvise_pg[scanned/hinted] > > > > > shows the proactive reclaim efficiency so this patch addes those > > > > > two statistics in vmstat. > > > > > > > > Please describe the usecase for those new counters. > > > > > > I wanted to know the proactive reclaim efficieny using MADV_COLD/MDDV_PAGEOUT. > > > Userspace has several policy which when/which vmas need to be hinted by the call > > > and they are evolving. I needed to know how effectively their policy works since > > > the vma ranges are huge(i.e., nr_hinted/nr_scanned). > > > > I can see how that can be an interesting information but is there > > anything actionable about that beyond debugging purposes? In other words > > isn't this something that could be done by tracing instead? > > That's the statictis for telemetry. With those stat, we are collecting > various vmstat fields(i.e., pgsteal/pgscan) from real field devices > and thought those two stats would be good fit along with other reclaim > statistics in vmstat since we can know how much proactive madvise policy > could make system healthier(e.g., less kswapd scan, less allocstall > and so on). > > > > > Also how are you going to identify specific madvise calls when they can > > interleave arbitrarily? > > I guess you are talking about how we could separate MADV_PAGEOUT and > MADV_COLD from vmstat. That's valid question. I thought for the start, > adds just umbrella stat like this and if we want to break down, we need > to introudce sysfs likewise slab. No, not really. MADV_COLD is about aging. There is no actual reclaim going on so pgscan/steal metrics do not make any sense. I am asking about potential different concurrent MADV_PAGEOUT happening. From what you've said earlier (how effectively policy works) I have understood you want to find out how a specific MADV_PAGEOUT effective is. But there maybe different callers of this applied to all sorts of different memory mappings and therefore the efficiency might be really different. As there is no clear way to tell one from the other I am really questioning whether this global stat is actually useful. -- Michal Hocko SUSE Labs