From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 970C1C27C76 for ; Wed, 25 Jan 2023 21:38:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A07E06B0071; Wed, 25 Jan 2023 16:38:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B6896B0073; Wed, 25 Jan 2023 16:38:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A5946B0074; Wed, 25 Jan 2023 16:38:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7BF076B0071 for ; Wed, 25 Jan 2023 16:38:05 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 501101C663D for ; Wed, 25 Jan 2023 21:38:05 +0000 (UTC) X-FDA: 80394634530.04.5CB54A3 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf04.hostedemail.com (Postfix) with ESMTP id 7836B40016 for ; Wed, 25 Jan 2023 21:38:02 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Ga8o88o2; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf04.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674682682; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IfU2fRaD+ujd2GlK5fTH5FbMvr0yQ81EIpL8auwwld0=; b=qhrJKaInGiD6Jb2q7mCPjTY2uou4i//nk/lixgBy1xpal6TZ5namsJGbta12yJuNg0m23I 8/3LyYGRJF9CVFSqIhNy9IjxlFpBZPhYWNwQP4lNLA58QaidAOT9hY7DrnaMGaCUB045Ec CtL92Nayysu+/Ap4Nce1zEfRU3qN1OA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Ga8o88o2; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf04.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674682682; a=rsa-sha256; cv=none; b=oKTPSApEqi0D5+VQ20y5hUclYqMcRekNZIy4VZtvphzDcAJpzVAMUuMr6N0GQknEGTz7bf AZ9afV6bKZax46PT4ButQ95VrHAYRX+1oq/lqj0JMhSL48n/cTNgA7aJEMn5Jp5nmcgCDe /mAVVyojIrCLw1aQE2Kr2/vjskHe02Y= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D7E611F8BE; Wed, 25 Jan 2023 21:38:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1674682680; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=IfU2fRaD+ujd2GlK5fTH5FbMvr0yQ81EIpL8auwwld0=; b=Ga8o88o2bV/RDOqBLtubC6S2qs90xKIwq44jGUMIK75wBhw5q2uhRKLBykCbRvpNnM7sEO eYe2zX/bRmvlkNMchYth1XZg5kWL2ECi+aQeA28cTDFStaflpRFOE4PYDcL5NL6OOwNIgh sNMOxV+wetMV9ZMIiUsjwhWYF8TFl4c= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BB6671358F; Wed, 25 Jan 2023 21:38:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id H/ncKjih0WNwIQAAMHmgww (envelope-from ); Wed, 25 Jan 2023 21:38:00 +0000 Date: Wed, 25 Jan 2023 22:37:59 +0100 From: Michal Hocko To: Minchan Kim Cc: Andrew Morton , Suren Baghdasaryan , Matthew Wilcox , linux-mm , LKML Subject: Re: [PATCH v2] mm/madvise: add vmstat statistics for madvise_[cold|pageout] Message-ID: References: <20230125005457.4139289-1-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 7836B40016 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 1e8mpeqrojtzjtwjti55iam5b5wsj6nj X-HE-Tag: 1674682682-800169 X-HE-Meta: U2FsdGVkX1/0aeAX79pQe00DBf4M0nG1BVN94bSw5+KhYVZGEHNVXu5ZQoOJYy3RUzS1ed4w/SgOpznat2fsyvciqlLzErkn2focoURrkeVhh0S1P4SjHjFgdmwjphMYFn8y2Lm9K0jO7HSj9Pxc5AH6liHI5KKRX9Z3vyQnieLT9NvbURxpdj5y61TAhBxdLw21Cd5xeEgSJJ4yDWjl2ivG+liHWxndJHy6zldXssmlMvfxlUSZGrvuNpBggA0cB1l1S9ul2CedMJGgaNm9tweycNoA+gcU6VMvOngood4k3lytUaACyVoiKuj/RrfFIRH7u98Agm9uowLmzXcErz1w2NhzewCWnAm9FveIuuSDoPqelv4Tz6gKRHknDeaFrt4HcHa8y5iCUT4mUpcGKtoYIDt1P4DJNE1bVq/rimJPX0UeOb+RcY35TsPMEzZiI1MHNqZWcz29+JnZO+Z0Lit02vH5YMCgg/k+8UAsa1eXnIv62B6F+IBCrT0xfp+Er+2jCDQHFsBwKRxiqNUKc3fuY5MMHFcW7bqIRb9Y0IaTCFLuqpddxQOc6CM67QoDzUjOi5/9I/DW6pgDL5SAmH8YuFQv3NHeGx+YVdHVaABjddD/5KyO17QY+GK5E0UtuV8CFypKlumzKoiguEtKfqyPWqynlGvp9abtwy+/txV+XMxKrgrUi6++Rn0JBt7/aiBskFvWib9/xcAGOlk/DbmR1MHV+QdRMC7aI3nhfzt7+wLIyWEvbX/EgyWuDWOERwQHKsfig6NyhVz6G5+QpsrJfK5TRbAlacogcKW0Z3/rvhbEKzaiwV8ZmFuvjjUgDyQ4VJ0mkh4Vm2QFTpZcP05BOHd3VjTZhsHeYBtRGjaqSJ5O7dGmDs8ctqV0bj1YEvYMeMnbXo/gfrDB9QLc9Z0xQPI0z8F7lPkeJ6CwW0qHro7LWhWHTChT1WOfHru6aD0LSMeqraSUSiGvy62 /eOSRM8L SYzR0SHzE57W+z4UsPRX4/L30doucOlbDbK0jXTVczBSziQZsTq3y5gZ+ezXj0ZcILtQJWBckuSMtKb+Xo0pIYB997U9hl++Zu66tXXd1ltudIaRpOls9k6J4+p5vriEzxpIf/U0bIgHpTx/d5FI6SbSw0aH4eBrL7VcNXpQlNAZmARXlv4PxwE61uzq7v5GXuH76C2hIYmeahEOeCF4Mr2Fgfg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 25-01-23 10:07:49, Minchan Kim wrote: > On Wed, Jan 25, 2023 at 06:07:00PM +0100, Michal Hocko wrote: > > On Wed 25-01-23 08:36:02, Minchan Kim wrote: > > > On Wed, Jan 25, 2023 at 09:04:16AM +0100, Michal Hocko wrote: > > > > On Tue 24-01-23 16:54:57, Minchan Kim wrote: > > > > > madvise LRU manipulation APIs need to scan address ranges to find > > > > > present pages at page table and provides advice hints for them. > > > > > > > > > > Likewise pg[scan/steal] count on vmstat, madvise_pg[scanned/hinted] > > > > > shows the proactive reclaim efficiency so this patch adds those > > > > > two statistics in vmstat. > > > > > > > > > > madvise_pgscanned, madvise_pghinted > > > > > > > > > > Since proactive reclaim using process_madvise(2) as userland > > > > > memory policy is popular(e.g,. Android ActivityManagerService), > > > > > those stats are helpful to know how efficiently the policy works > > > > > well. > > > > > > > > The usecase description is still too vague. What are those values useful > > > > for? Is there anything actionable based on those numbers? How do you > > > > deal with multiple parties using madvise resp. process_madvise so that > > > > their stats are combined? > > > > > > The metric helps monitoing system MM health under fleet and experimental > > > tuning with diffrent policies from the centralized userland memory daemon. > > > > That is just too vague for me to imagine anything more specific then, we > > have numbers and we can show them in a report. What does it actually > > mean that madvise_pgscanned is high. Or that pghinted / pgscanned is > > low (that you tend to manually reclaim sparse mappings)? > > If that's low, it means the userspace daemon's current tune/policy are > inefficient or too aggressive since it is working on address spacess > of processes which don't have enough memory the hint can work(e.g., > shared addresses, cold address ranges or some special address ranges like > VM_PFNMAP) so sometime, we can detect regression to find culprit or > have a chance to look into better ideas to improve. Are you sure this is really meaningful metric? Just consider a large and sparsely populated mapping. This can be a perfect candidate for user space reclaim target (e.g. consider a mapping covering a large matrix or other similar data structure). pghinted/pgscanned would be really small while the reclaim efficiency could be quite high in that case, wouldn't it? -- Michal Hocko SUSE Labs