From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B9BCC4332F for ; Thu, 2 Nov 2023 06:59:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06FCF80017; Thu, 2 Nov 2023 02:59:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 01F088D0026; Thu, 2 Nov 2023 02:59:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E015480017; Thu, 2 Nov 2023 02:59:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CEAE38D0026 for ; Thu, 2 Nov 2023 02:59:03 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8DF65140E8B for ; Thu, 2 Nov 2023 06:59:03 +0000 (UTC) X-FDA: 81412112166.12.4CDF1B9 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by imf15.hostedemail.com (Postfix) with ESMTP id 56A51A0006 for ; Thu, 2 Nov 2023 06:59:00 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nECHuB2W; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698908341; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Kfr+ZGOnDDtt1rj18uVLyATI/3RK+y1Yeqj+bJJ9Z0U=; b=gD0N7ARAnKaH3f9NSd3s+ONjpsteqhWhnaANSu4UaQvjRC0YmERJ/zunhwHPqb9QT0rkC1 wWCJwOTxs1/pkPmjx7ebrG7XiAwZR3gkBVsz5IGWJhP5JOIZMetsiu7qW1HurycWLfhFXy ZpZ0ow2FbkNG0K3/iwdChtHWVO9TnG8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nECHuB2W; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698908341; a=rsa-sha256; cv=none; b=PMAu8lbxaJkpVqx8G3E6uQLIynrTdOzHImbHPXXqRUnkKhuWtAaprZeFSIPTNd0JxEtYB4 zm7hEh9bCLGhn5vEhHg6xX1+32Vlx4U2oIq4uPmnbT/3whznKiPNCqeowJJlyC7ZLTUXRQ giuyBU0mFj2BeKkoKgXcoeyJQu/bhQg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698908340; x=1730444340; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=MndUEcqNycjEVQtJcezPExD3T4iBwIIUXizsMPBaq/A=; b=nECHuB2W+cKtie30MKmH0oO78+u/xsFHGoiSzJrAkxYrUoHIB9iydDh7 STToW0LmSRiCUboqhRTjOytD7uYfb/n9gCBxjIcxkhWdZ6krPLPJvRF/a hJZucTu69aV18jVP8E2b4XOgTSCRUwRKqYzYwbAYWT8FwClEN6zVaFmiL Igu9Zi/NattDCQSQVuNhHezRKR50aO0k1UAwm6KJ+mXkd7dixlrtAlZrp H2t6Zfu847uRpQ616SPJDzezH2Fx0WCV9P1B2GtOZlmK1MU1TI3ikRbHu RmCqn3axVyXxiz73g+nvl8C6LRieYIqKBdCnb5f4rgq3Pi68u2UdmKhk2 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10881"; a="455130800" X-IronPort-AV: E=Sophos;i="6.03,270,1694761200"; d="scan'208";a="455130800" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2023 23:58:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10881"; a="737654641" X-IronPort-AV: E=Sophos;i="6.03,270,1694761200"; d="scan'208";a="737654641" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2023 23:58:56 -0700 From: "Huang, Ying" To: "Zhijian Li (Fujitsu)" Cc: Andrew Morton , Greg Kroah-Hartman , "rafael@kernel.org" , "linux-mm@kvack.org" , "Yasunori Gotou (Fujitsu)" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH RFC 3/4] mm/vmstat: rename pgdemote_* to pgdemote_dst_* and add pgdemote_src_* In-Reply-To: (Zhijian Li's message of "Thu, 2 Nov 2023 06:34:42 +0000") References: <20231102025648.1285477-1-lizhijian@fujitsu.com> <20231102025648.1285477-4-lizhijian@fujitsu.com> <87r0l81zfd.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Thu, 02 Nov 2023 14:56:54 +0800 Message-ID: <875y2k1w3t.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 56A51A0006 X-Stat-Signature: 85angjspgd9nfney71i9r9e6s6kqrkq3 X-HE-Tag: 1698908340-434017 X-HE-Meta: U2FsdGVkX1+TjMGmE+RpDpzx5VhTRcZ6tSEjQTDZ80ATQTomE+p3Eez70Qz609p+ku2PBF0oZmhOMJNoNyn6C2SxDpaVJBARXXk8/Cvsa+hYNaud/B6Fx9VF0R1pfuNIAaPdw0jqc1fQiIaf6qdRg1+qJ5oR4YA7mXp7rSOQ4EruOi015k8JSTy+l8URbsKEJ1ho5TzxPXkUePI3o01qyWaH2sByGh4NIVQeNf4wAOCs67BSxfg7SMqltWnk09adgckksgnC00u4u95/WMjvKeo6cHnuqq+iSebN8wTae9VPA5NRjq8GA1lObi8imCvxP/i4/o6EKwT+QAqPyyWgcekCw7jZVaMBWfrIbQ2uUYRuSO1bNDEfrRmF4pcguQQsV7ZQoP5u+8IatuD20eywNz/glEsSBkJYpRnx6VMLaS7hoqTYF8sju16gjLrLrTeMGBBFUxF/yzly1TUj8bcffEew487Br5L4rfx10zg9riEI3DQW85s7UOHeBUVWDYYcju1yk/Vy5pMl6yL2ktE44zyNsG5GvSqDBu3WFxApHZk4cPUd5yCk4Vaylh8LBIoqW1bTq8mAdqEyOLP++4AJc+Slw5lQowEOtNO6wUVm21IX13W6ECzRw5of4eYI+NdnadP2XM/RSrR8ICB1ELeREOlHMFi/oNJRiAUtgygPQ7h70eIX876u40CQa3Vw3mXQtkSGnSvaJZM8w/FkfVnrjD3UgXQdY0WcJFwss9YEhgnTKT78q9GQrcCXehRH4IIuSf4k0qXGp94AxcWM3kWfGNwjxv234NtHPBPM5qjdaKxVizvU/IgwK/izc/IHCBVcWoEMLWfx0El+J8OtJO9uRc2c9Ki+KCJxLY9iNX1zBvqcewmkK5X0yGAerBVzWOGkOAG09lrkA3sYwY80f+JsSDeAmuBI1R80xULzuKb22lK6DpNh3Lj8Ev7TNICKSMjW7pdewBLT2hxfwCazAkX 2O9wcU/N IuFy39kma9VqQ4FRKi4S0u3IR81p41h5kOjE5d1AwBtpZgT0ArklsVWFqlDqWYyKgOZSs5f94EB6mwCkOYEuXJb/oL+bFcLgN7+sWgOh36ske93LXxp3jVgsm4o85WP9CLNebu/MgIPDyUlpPB1GA4uifVErGLxBCPWyuWGM/HccZFrJdWQ/EyJ++RoSDO2nxMWCMeAmb6of2wfe+3lMVS7R135k2upG6pjJAVE/AaaSTYRGcb5QNNzRotyKUTLFBTLHMSpYAKUYzovM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: "Zhijian Li (Fujitsu)" writes: > On 02/11/2023 13:45, Huang, Ying wrote: >> Li Zhijian writes: >> >>> pgdemote_src_*: pages demoted from this node. >>> pgdemote_dst_*: pages demoted to this node. >>> >>> So that we are able to know their demotion per-node stats by checking this. >>> >>> In the environment, node0 and node1 are DRAM, node3 is PMEM. >>> >>> Global stats: >>> $ grep -E 'demote' /proc/vmstat >>> pgdemote_src_kswapd 130155 >>> pgdemote_src_direct 113497 >>> pgdemote_src_khugepaged 0 >>> pgdemote_dst_kswapd 130155 >>> pgdemote_dst_direct 113497 >>> pgdemote_dst_khugepaged 0 >>> >>> Per-node stats: >>> $ grep demote /sys/devices/system/node/node0/vmstat >>> pgdemote_src_kswapd 68454 >>> pgdemote_src_direct 83431 >>> pgdemote_src_khugepaged 0 >>> pgdemote_dst_kswapd 0 >>> pgdemote_dst_direct 0 >>> pgdemote_dst_khugepaged 0 >>> >>> $ grep demote /sys/devices/system/node/node1/vmstat >>> pgdemote_src_kswapd 185834 >>> pgdemote_src_direct 30066 >>> pgdemote_src_khugepaged 0 >>> pgdemote_dst_kswapd 0 >>> pgdemote_dst_direct 0 >>> pgdemote_dst_khugepaged 0 >>> >>> $ grep demote /sys/devices/system/node/node3/vmstat >>> pgdemote_src_kswapd 0 >>> pgdemote_src_direct 0 >>> pgdemote_src_khugepaged 0 >>> pgdemote_dst_kswapd 254288 >>> pgdemote_dst_direct 113497 >>> pgdemote_dst_khugepaged 0 >>> >>> From above stats, we know node3 is the demotion destination which one >>> the node0 and node1 will demote to. >> >> Why do we need these information? Do you have some use case? > > I recall our customers have mentioned that they want to know how much the memory is demoted > to the CXL memory device in a specific period. This doesn't sound like a use case. Can you elaborate it? What can only be tuned with the help of the added stats? -- Best Regards, Huang, Ying > >>>> mod_node_page_state(NODE_DATA(target_nid), >>>> - PGDEMOTE_KSWAPD + reclaimer_offset(), nr_succeeded); >>>> + PGDEMOTE_DST_KSWAPD + reclaimer_offset(), nr_succeeded); > > But if the *target_nid* is only indicate the preferred node, this accounting maybe not accurate. > > > Thanks > Zhijian > >> >> -- >> Best Regards, >> Huang, Ying >> >>> Signed-off-by: Li Zhijian >>> --- >>> RFC: their names are open to discussion, maybe pgdemote_from/to_* >>> Another defect of this patch is that, SUM(pgdemote_src_*) is always same >>> as SUM(pgdemote_dst_*) in the global stats, shall we hide one of them. >>> --- >>> include/linux/mmzone.h | 9 ++++++--- >>> mm/vmscan.c | 13 ++++++++++--- >>> mm/vmstat.c | 9 ++++++--- >>> 3 files changed, 22 insertions(+), 9 deletions(-) >>> >>> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h >>> index ad0309eea850..a6140d894bec 100644 >>> --- a/include/linux/mmzone.h >>> +++ b/include/linux/mmzone.h >>> @@ -207,9 +207,12 @@ enum node_stat_item { >>> PGPROMOTE_SUCCESS, /* promote successfully */ >>> PGPROMOTE_CANDIDATE, /* candidate pages to promote */ >>> /* PGDEMOTE_*: pages demoted */ >>> - PGDEMOTE_KSWAPD, >>> - PGDEMOTE_DIRECT, >>> - PGDEMOTE_KHUGEPAGED, >>> + PGDEMOTE_SRC_KSWAPD, >>> + PGDEMOTE_SRC_DIRECT, >>> + PGDEMOTE_SRC_KHUGEPAGED, >>> + PGDEMOTE_DST_KSWAPD, >>> + PGDEMOTE_DST_DIRECT, >>> + PGDEMOTE_DST_KHUGEPAGED, >>> #endif >>> NR_VM_NODE_STAT_ITEMS >>> }; >>> diff --git a/mm/vmscan.c b/mm/vmscan.c >>> index 2f1fb4ec3235..55d2287d7150 100644 >>> --- a/mm/vmscan.c >>> +++ b/mm/vmscan.c >>> @@ -1111,13 +1111,18 @@ void drop_slab(void) >>> static int reclaimer_offset(void) >>> { >>> BUILD_BUG_ON(PGSTEAL_DIRECT - PGSTEAL_KSWAPD != >>> - PGDEMOTE_DIRECT - PGDEMOTE_KSWAPD); >>> + PGDEMOTE_SRC_DIRECT - PGDEMOTE_SRC_KSWAPD); >>> BUILD_BUG_ON(PGSTEAL_DIRECT - PGSTEAL_KSWAPD != >>> PGSCAN_DIRECT - PGSCAN_KSWAPD); >>> BUILD_BUG_ON(PGSTEAL_KHUGEPAGED - PGSTEAL_KSWAPD != >>> - PGDEMOTE_KHUGEPAGED - PGDEMOTE_KSWAPD); >>> + PGDEMOTE_SRC_KHUGEPAGED - PGDEMOTE_SRC_KSWAPD); >>> BUILD_BUG_ON(PGSTEAL_KHUGEPAGED - PGSTEAL_KSWAPD != >>> PGSCAN_KHUGEPAGED - PGSCAN_KSWAPD); >>> + BUILD_BUG_ON(PGDEMOTE_SRC_DIRECT - PGDEMOTE_SRC_KSWAPD != >>> + PGDEMOTE_DST_DIRECT - PGDEMOTE_DST_KSWAPD); >>> + BUILD_BUG_ON(PGDEMOTE_SRC_KHUGEPAGED - PGDEMOTE_SRC_KSWAPD != >>> + PGDEMOTE_DST_KHUGEPAGED - PGDEMOTE_DST_KSWAPD); >>> + >>> >>> if (current_is_kswapd()) >>> return 0; >>> @@ -1678,8 +1683,10 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, >>> (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, >>> &nr_succeeded); >>> >>> + mod_node_page_state(pgdat, >>> + PGDEMOTE_SRC_KSWAPD + reclaimer_offset(), nr_succeeded); >>> mod_node_page_state(NODE_DATA(target_nid), >>> - PGDEMOTE_KSWAPD + reclaimer_offset(), nr_succeeded); >>> + PGDEMOTE_DST_KSWAPD + reclaimer_offset(), nr_succeeded); >>> >>> return nr_succeeded; >>> } >>> diff --git a/mm/vmstat.c b/mm/vmstat.c >>> index f141c48c39e4..63f106a5e008 100644 >>> --- a/mm/vmstat.c >>> +++ b/mm/vmstat.c >>> @@ -1244,9 +1244,12 @@ const char * const vmstat_text[] = { >>> #ifdef CONFIG_NUMA_BALANCING >>> "pgpromote_success", >>> "pgpromote_candidate", >>> - "pgdemote_kswapd", >>> - "pgdemote_direct", >>> - "pgdemote_khugepaged", >>> + "pgdemote_src_kswapd", >>> + "pgdemote_src_direct", >>> + "pgdemote_src_khugepaged", >>> + "pgdemote_dst_kswapd", >>> + "pgdemote_dst_direct", >>> + "pgdemote_dst_khugepaged", >>> #endif >>> >>> /* enum writeback_stat_item counters */