From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E101C3DA4A for ; Wed, 14 Aug 2024 20:52:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8A5A6B00C2; Wed, 14 Aug 2024 16:52:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3C156B00C3; Wed, 14 Aug 2024 16:52:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DAAC6B00C4; Wed, 14 Aug 2024 16:52:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 699DE6B00C2 for ; Wed, 14 Aug 2024 16:52:06 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1BB17A85B2 for ; Wed, 14 Aug 2024 20:52:06 +0000 (UTC) X-FDA: 82452048252.10.0927A88 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id 32DE520015 for ; Wed, 14 Aug 2024 20:52:03 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=u9QNpxPL; dmarc=none; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723668711; a=rsa-sha256; cv=none; b=oqmuaIVV9PYjx7DkrQkbdPAQw1u37dx2o8RYsYTDm6ew7dWE/jNDt6f8V4PziZBeYSXI6g 16xnXgJYDII5UJmgZqEWR0bRa2/GqhN6HNynRHtNYYIgK/fjaOgms5h2VY16rbUR6OobYo B26hHBMDFioesH+X/b/UJQSJZRILqno= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=u9QNpxPL; dmarc=none; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723668711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VPqt5y21VUvQ7phpBwwuxTHi7GThhpUcEh9wSMfM9Lg=; b=XVdcqW3uZrpj2VwJ38aIUdR7Nvc7RKjcNZwDY/Dyuf+yrrjJOL007KiKAs77GFWEpRDEgI gF9xJuChnH2lxY9eby2EA3CsICwM8VMKS1DQEhaIZa9EDa/S4vyEvb8z5uVvdMsPuAAIVL z4c7RhdL8yoCSdQs9tPTnMZD+YJ+PJU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5932C61CC7; Wed, 14 Aug 2024 20:52:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8705FC116B1; Wed, 14 Aug 2024 20:52:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1723668723; bh=+2uyCP79sVA3+SCyRIYroJ/LtfahWV9TyMiSxufHXQE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=u9QNpxPLHLOc2gJi93X48j9NR6/RexgsfDUppFD1aFwaShU/QiC3f2DzCnpyljCGL sjjEQiDw+5mV/rXHWyig2SvkOmw3FnxY8MFGLiPuMkVGWUFHqT0k3Et9Qu+m3fV13o RL4ixLTN1qIL3HTtUVIVKDdSeXTPw55hC2zWosA8= Date: Wed, 14 Aug 2024 13:52:01 -0700 From: Andrew Morton To: kaiyang2@cs.cmu.edu Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, mhocko@kernel.org, nehagholkar@meta.com, abhishekd@meta.com, hannes@cmpxchg.org, weixugc@google.com, rientjes@google.com Subject: Re: [PATCH v3] mm,memcg: provide per-cgroup counters for NUMA balancing operations Message-Id: <20240814135201.58cd0760bbeab13fcea82c4a@linux-foundation.org> In-Reply-To: <20240814174227.30639-1-kaiyang2@cs.cmu.edu> References: <20240814174227.30639-1-kaiyang2@cs.cmu.edu> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Queue-Id: 32DE520015 X-Rspamd-Server: rspam01 X-Stat-Signature: ecksuuiufyepje3p8qkr9be4u5hzm3uz X-HE-Tag: 1723668723-424774 X-HE-Meta: U2FsdGVkX1/BsuZJYC3ePXaVrUltYg24qLNhXqwnJonfPEYQ3Xo12Ak6YEesliIXSUpAki/l32Tdf5tSA39eEPKBNYD6ZdTYcGwEmitVDmY23Nr0/P5CdT/K656m7jVE4o62ItMMEZh95U5t71X9N6vO7OTD6RhQDhS94dVBZgvCDPYHx21dbtGbRnknAS/00s+2aKp7+2AoYUnJhWJqQyZcCMi8dtIjTulAzwlsPpOLPdROvVIyDMvtG00eFbsPO3LfQx8kR7+EdXaoAujCQlNNYmbj0KkyqortXpk9f5btOhIoZklwhU5TEfwDEHIkA2fpw+8PxaKVp78RiFNqt9We7OzTT301/nOEkMFPgYbYlLUY1s5JrWsG/cWVhqs27EpOikibsi2+DbcrhuRB+8zVS57MGT/oumfauhi/uZUF+uZ/B4dNIhJy4FaHMdW7rsy4/lWYw7mYRI8nG3VgvqA8EGBSt1x7MVut5V83SxQ62RoJDvSY2VrTdoL4u519WqQwUZQkITRJfEkD6BaOrhPP/MqhV6lCaBpSCl0TLg7JY+N1PCmZb73pihaa55WtuXfggb8vvG68zJbiaoUGrTYE8wHTHUCefLSlo7D8XY/as/tWW/ONsz8BY2djBRRHvYx7Y8AafRBqgwzV5i6YkoetBWf6HMFyV95/5FWHUd4gEW5xzCdClum4s0QDQi/ruxwHOl9sUrnU7znoBbwagicphVvHee43qPTSz7vbhjxP8uJIB4YYCdhJnlw9h2USCPqgfzaNugL/Hh1paltbU6tNHCWIxcUPLtcfYyHJ3Y+LtAE65GlkVtAUkkmVE4ZaAOk+xUSZtIUZ8JgG6k2tMI/wYk8M2P+FuTs/35NA4ck7QA0EO3a5FqknnU3K8gtBIo4wYYl5A/C1709AO7BUV5EYYzMo97eMeASzbJ40yefzF6GkFHCj/4bc/GHWSMx0bLt1kMV2KtL3abKMuvg 4oB2z75a rOg91S2owzwjTxD0/kpsy6zn2AXUfqnaxpwi3MiMxzFSZEeTLhUOv8d6KWKk8ZA4Kgs/bmpKTBPuUEt2lveatUkhHKcGqd4d4d5f9M2P2XkhGuyUx96AOEhECB5qCCxJaXzdveb7aYQuQyCMtaSHVlhmvMvvMFSXF2J1HiCL4m7+qcOWFU4MQEP3lwoYx6xQ5pcLTymPekqC8OCzTSqeSsWyhzMigcI3kzICp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 14 Aug 2024 17:42:27 +0000 kaiyang2@cs.cmu.edu wrote: > From: Kaiyang Zhao > > The ability to observe the demotion and promotion decisions made by the > kernel on a per-cgroup basis is important for monitoring and tuning > containerized workloads on either NUMA machines or machines > equipped with tiered memory. > > Different containers in the system may experience drastically different > memory tiering actions that cannot be distinguished from the global > counters alone. > > For example, a container running a workload that has a much hotter > memory accesses will likely see more promotions and fewer demotions, > potentially depriving a colocated container of top tier memory to such > an extent that its performance degrades unacceptably. > > For another example, some containers may exhibit longer periods between > data reuse, causing much more numa_hint_faults than numa_pages_migrated. > In this case, tuning hot_threshold_ms may be appropriate, but the signal > can easily be lost if only global counters are available. > > This patch set adds seven counters to memory.stat in a cgroup: > numa_pages_migrated, numa_pte_updates, numa_hint_faults, pgdemote_kswapd, > pgdemote_khugepaged, pgdemote_direct and pgpromote_success. pgdemote_* > and pgpromote_success are also available in memory.numa_stat. > > count_memcg_events_mm() is added to count multiple event occurrences at > once, and get_mem_cgroup_from_folio() is added because we need to get a > reference to the memcg of a folio before it's migrated to track > numa_pages_migrated. The accounting of PGDEMOTE_* is moved to > shrink_inactive_list() before being changed to per-cgroup. > > ... > > @@ -1383,6 +1412,13 @@ static const struct memory_stat memory_stats[] = { > { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, > { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, > { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, > + > + { "pgdemote_kswapd", PGDEMOTE_KSWAPD }, > + { "pgdemote_direct", PGDEMOTE_DIRECT }, > + { "pgdemote_khugepaged", PGDEMOTE_KHUGEPAGED }, > +#ifdef CONFIG_NUMA_BALANCING > + { "pgpromote_success", PGPROMOTE_SUCCESS }, > +#endif > }; Please document these in Documentation/admin-guide/cgroup-v2.rst