From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC7A0C54E94 for ; Wed, 25 Jan 2023 17:07:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 578EF6B0083; Wed, 25 Jan 2023 12:07:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 529BA6B0087; Wed, 25 Jan 2023 12:07:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 423136B0083; Wed, 25 Jan 2023 12:07:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 330356B0083 for ; Wed, 25 Jan 2023 12:07:06 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 167C6120D50 for ; Wed, 25 Jan 2023 17:07:06 +0000 (UTC) X-FDA: 80393951652.18.316231C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf05.hostedemail.com (Postfix) with ESMTP id 2799C10000A for ; Wed, 25 Jan 2023 17:07:02 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=mERbHiW3; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674666423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X7Ju2TighrFIcVxjh+s1qto4imNLUIwLW2Cvl9ClH/c=; b=lkdekWWe63YEE9Tcb61gWsJW0huZjfuCoBCG9Wy5ZPzd4P6H/wB09hPh0hhkYckSYt/oj4 bMEl9fqOvup3B4Q9RoOCmoaqxtTGP3D8mI0IHoPu7RCjks7TALoAjosb4qcbObTIn9TDLZ PTfpicI5ulfqigztDae6W8tnCtT4BBw= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=mERbHiW3; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674666423; a=rsa-sha256; cv=none; b=IZ36WhBNd48YkRAKUw62h1hj4h1nuRBh+rA43Fxfwcf1RCO7jO1awZAGGSonaJKY772rr/ tHd9025S2LfeS+hO9xOHxv1afpH0R+gv7nYOiyrMiQVsmIihF3K/wvKLuR4rqcnap6S+03 9KGM3KDbRVkjjdwamGz28iO/l4uS+rM= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7C5C721C87; Wed, 25 Jan 2023 17:07:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1674666421; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X7Ju2TighrFIcVxjh+s1qto4imNLUIwLW2Cvl9ClH/c=; b=mERbHiW3P/FkBgADmDFMb3EIyG+9ELTNSjsHkg2ww9hD2EYeRufEt5MgdhgbLBrCPxKiYc QcCRd5KunNq16r6rhn3/xfCygB8tvnR1+nW/vMEooieGQo+mKor0/fGwGD7f3r4Hm4EF+i XiG8HGWhufcJabdDDaEI5WiNWYwc6E8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 57C791358F; Wed, 25 Jan 2023 17:07:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id pVeAErVh0WNuJAAAMHmgww (envelope-from ); Wed, 25 Jan 2023 17:07:01 +0000 Date: Wed, 25 Jan 2023 18:07:00 +0100 From: Michal Hocko To: Minchan Kim Cc: Andrew Morton , Suren Baghdasaryan , Matthew Wilcox , linux-mm , LKML Subject: Re: [PATCH v2] mm/madvise: add vmstat statistics for madvise_[cold|pageout] Message-ID: References: <20230125005457.4139289-1-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: wmuqgm35h4fjufkiwkhcbdtc7a9ufroh X-Rspamd-Queue-Id: 2799C10000A X-HE-Tag: 1674666422-935863 X-HE-Meta: U2FsdGVkX18inPSD/thQKjX+zoLs4Ltzps0hLfj11nYpsyIta/4jGyTB6/dRm2xbmGuYjvZg5njFeUvDljI9tTT9xN7iOsn26Grpiqqdw9FQZhkNbZuIv90xLIfK2tHl/iWwH3maYJjN3+Zj/Yc01uCgUvTGZcPYGWfLKWIV7m36g9IJQ6PDEPL7rcDlXHTqme50m7lPjXbETySWQEedZdURR+JY2V78PS9P5Ewf/4TY4G6N24JGrgAGQEuGpD5AqGcPOY6OzOOgKKDxDrVckTabWR8pMCnilO1TtBd0LM1DrRd5Bt+rrYL9VsT6HndLZyN6nSeHlo+/Fcmy+wgH8dQe1Wk+Sev2IqHDhvPmelX797psNxMP09e9tYQcEc8SDc60h0XxSff8fqGN2KAxG22ymEXZrDae6RLGyLYreI0MJkKIOU4ZV9sS12pO8zzdTAOYb1bQWX+xBcbUlsQLTmD1LjWZv3VyMdUHyv1SMHPxx7mPxJyZ8nFnmoizIhlRBFv0jtKiMQ7ynJsXyt382ZuiMJEJaCItjHTHjIJcpgkA0Xi4Xkrl9J3HBqge0yD5yQib96VwmCV3OviCfD3/uvr8xs2ICs+hlOQk9OiGUQEVhPzuoaKA7v6aBCs9z5lCTgLyZFz/IN5HpRZ4/jMfy659whxL49EMJKifFuIW3k1IvmdHZqzBeNLydDTjY84Sa5Ee2+7WtU/4hVlAQEW8SKBeYc6o0nZnyhOPLzRQdGdbYheFbhRPjeGm8JpSuwd1w/ioo/oO9PfzF21BmMK9M3gs5CDUkxQ9pnyIID+OOT024LwkDPE9CVCSNnhYLBaUSbyMNke8aSG179b7ScAzVkBpaT0Xx3cAhfzEhsL1ixWjoN3pK4WCmNFY1EkdnH3IkPD/rdxi62OtqGE0xEBvKEZUF9XjOM6Io+XUZ0f1q2Y6FqGrrF2JGaVvrA4C34+r5lowrt+4lyZ9udgJ9cH Mfn6muKH C3vWblpIhOsAEM62w1uvT1iVXQtD9AqNsV9fZfjFarVcQKhVKOt6O6wAfjdHPCkgpn/TJhtUkpuUp7P4zbnEV4jQWiIH81+6HEPivXCoNZ+sHgKyYEV/5CA7CGFfWO075dijBatQMtMoWw2dxMTq71Xd88fRVZxt1zmZWH1AcJ07YMo9Te5l+le9lzKXVlAxlgRaUQaMRGF6s8lgyX+jBsQxUVg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 25-01-23 08:36:02, Minchan Kim wrote: > On Wed, Jan 25, 2023 at 09:04:16AM +0100, Michal Hocko wrote: > > On Tue 24-01-23 16:54:57, Minchan Kim wrote: > > > madvise LRU manipulation APIs need to scan address ranges to find > > > present pages at page table and provides advice hints for them. > > > > > > Likewise pg[scan/steal] count on vmstat, madvise_pg[scanned/hinted] > > > shows the proactive reclaim efficiency so this patch adds those > > > two statistics in vmstat. > > > > > > madvise_pgscanned, madvise_pghinted > > > > > > Since proactive reclaim using process_madvise(2) as userland > > > memory policy is popular(e.g,. Android ActivityManagerService), > > > those stats are helpful to know how efficiently the policy works > > > well. > > > > The usecase description is still too vague. What are those values useful > > for? Is there anything actionable based on those numbers? How do you > > deal with multiple parties using madvise resp. process_madvise so that > > their stats are combined? > > The metric helps monitoing system MM health under fleet and experimental > tuning with diffrent policies from the centralized userland memory daemon. That is just too vague for me to imagine anything more specific then, we have numbers and we can show them in a report. What does it actually mean that madvise_pgscanned is high. Or that pghinted / pgscanned is low (that you tend to manually reclaim sparse mappings)? > That's really good fit under vmstat along with other MM metrics. > > > > > In the previous version I have also pointed out that this might be > > easily achieved by tracepoints. Your counterargument was a convenience > > in a large scale monitoring without going much into details. Presumably > > this is because your fleet based monitoring already collects > > /proc/vmstat while tracepoints based monitoring would require additional > > changes. This alone is rather weak argument to be honest because > > deploying tracepoints monitoring is quite trivial and can be done > > outside of the said memory reclaim agent. > > The convenience matters but that's not my argument. > > Ithink using tracepoint for system metric makes no sense even though > the tracepoint could be extended by using bpf or histogram trigger to > get the accumulated counters for system metric. System wide metrics data collection by ftrace is a common use case. I really do not follow your argument here. There are certainly cases where ftrace is suboptimal solution - e.g. when the cumulative data couldn't have been collected early on for one reason or another (e.g. system uptime is already high when you decide to start collecting). But you have stated there is data collection happening so what does prevent collecting this just along with anything else. > The tracepoint is the next step if we want to know further breakdown > once something strange happens. That's why we have separate level metric > system to narrow problem down rather than implementing all the metric > with tracepoint. Please, look at vmstat fields. Almost every fields > would have same question you asked "how do you break down if multiple > processes were invovled to contribute the metric?" Yes, we tended to be much more willing to add counters. Partly because runtime debugging capabilities were not that great in the past as we have these days. > I am fine if you suggest adding tracepoint as well as the vmstat fields > for further breakdown but only relying on tracepoint and frineds for > system global metric doesn't make sense. We have to agree to disagree here. I am not going to nack this but I disagree with this patch because the justification is just too vague and also those numbers cannot really be attributed to anybody performing madvise to actually evaluate that activity. -- Michal Hocko SUSE Labs