From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9636C3DA4A for ; Sat, 10 Aug 2024 00:28:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5AE686B008A; Fri, 9 Aug 2024 20:28:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 55EE16B0092; Fri, 9 Aug 2024 20:28:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 425B06B0095; Fri, 9 Aug 2024 20:28:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 256B96B008A for ; Fri, 9 Aug 2024 20:28:37 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B638E1A1636 for ; Sat, 10 Aug 2024 00:28:36 +0000 (UTC) X-FDA: 82434449832.07.792F212 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf05.hostedemail.com (Postfix) with ESMTP id 9F247100007 for ; Sat, 10 Aug 2024 00:28:34 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PaFTn8rC; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723249626; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yAKpnq9dhMYDITcgJpIyAWsriTJ2oEo63BNC7ioCOWQ=; b=ZbGPk8p4rVaH6XujEWb3V2BpFLTlcBLvBktiAVC0j+/JS1GTADK/KBEiDbo6/a9noRMyOQ Vpe8M5Gw/+iejZFqnRuDqyzDaqTC/4TGqibL7DVsWiUsoFvBTzvtgXr6uW1QbIIT48iKNn LasGPFmM7HoneVAXvVn2/QZVxYKi5i4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PaFTn8rC; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723249626; a=rsa-sha256; cv=none; b=KQaG6bJ/AbYDNgjdYUML3gBAFjLch9FebFhw1izvegT0HsBnDn2T/Xduq/YPwudDUZ2Tz6 m85toGfE3EZnJsBWOY87OwsllowdP9Nno3G0Hj2kLsc7I+uyy1BcLwwvNUILTXcD8wAlU4 XM2keIVs+jEBGnV9+1yHgLGjzVzEU8k= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id D16C1CE13B2; Sat, 10 Aug 2024 00:28:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2CEFC32782; Sat, 10 Aug 2024 00:28:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1723249711; bh=x1GOANqk/UOeN8tsVo1aPaxm7S+bnCcXfZtJh+ysY0E=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PaFTn8rClPV4pCH9y6SKnYBW2LOcS8lLJVlAY4vCNxPZAxG5eqQr+RnltKSb7xGeJ KAWYo4gROt10AoOUiAHVaymM3sLMSgq1HORF05m8tG6ElTyTZqkMyYruMgtPWrskQv hqJo9rpWzBjPIODmZSxm6nce0tXjIBsQ/05OV4h8= Date: Fri, 9 Aug 2024 17:28:30 -0700 From: Andrew Morton To: kaiyang2@cs.cmu.edu Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, mhocko@kernel.org, nehagholkar@meta.com, abhishekd@meta.com, hannes@cmpxchg.org Subject: Re: [PATCH] mm,memcg: provide per-cgroup counters for NUMA balancing operations Message-Id: <20240809172830.58bece856debebd46279fc9d@linux-foundation.org> In-Reply-To: <20240809212115.59291-1-kaiyang2@cs.cmu.edu> References: <20240809212115.59291-1-kaiyang2@cs.cmu.edu> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9F247100007 X-Stat-Signature: m7mwurq8heq3ef7sfrzkxg1mf3mg7dya X-Rspam-User: X-HE-Tag: 1723249714-810194 X-HE-Meta: U2FsdGVkX19UJkjyYhZyQlvyA6+E7pL8HG72zWm3CAKS+VrD5ZEf9J0BZK2RVNNZaKffWDcN3jKNSpslJjHODYmBcqWWw6TlkG2Vza7nulPFEQNKWlBIxdk50h433F7wLKdjukZ53wupol/pmS8WDkVxVROV1FepN/rV70DhY/uBgMPNCxOUkWnUDGzRXLP9HZ0mU5y73cyZQ8P0amTrvslX9/gp8mfM50Hnf7DmLcT/DUc/OJJNTGMvAUuTtGzGlZYg5HUrmUBcTsX5Eu3/SqD32Djsj+tpO+FTXU6TZ1QG9aoGnuJJfRYXJh4lxU7aMN+e/WiZw9xkQPixE/8Bj0SPMqQJOA8k8qtfQem4rgAvT3IaUlipQPKD/RM0NOzQ2K5G7UuZOjE7hf+/cmB928AoX9CsYUDBU1ZKKaVllpVzOjt40qg8Rs2xWreNUkPZ5EgXXRcdApZ2/VpzZ4rlRH9XhuhH82Yzuct++56WupKS8LwFfk6LyLKsZROeD5oJkIK+BXPhiyy3E7+E9MhBEvxSjNkTlsnJwJQ60QKbVEEFpc5a33njc2PdzRPA3o0VNKsEW67pswMkTk5rpZulX5ao07GykfFxGgI6pYQS+hWLkmyxpDex3MRoQY+5lt2UwTkOKKtf3+kQXAZ6QIXcZypJgKdY/omKOKbA9nlEXv4oLyY4WGmWSkpon3W/qseC7612jy8OLxYPUHHtWzhWnKPDQeiXcosO/BkewhoEbSAHNz/7KPjkKynGA6ZNK+YzwG3ixLNY33vBwcGBF3O1hBLJdzcvpmiUVxgv9MPwKEgjELugU5vsvmdigkCP3LNXW6GQac4HF1uYjeUqIHgW8jQAckPKXk2agwo2OM62II/6ik5T+IO4ZY7B/vAHKiLSNctEsQoFWE7i+xxaKGgKKwMiGJr3WKAx6CPuYYbTyfOMp1JsMLR8WmrDTShj7Qyy5g0G9yakpfyAVSScn2k U9VK+2QL Bv4P8fXQm3C59EC1Qb9/1yx/VhemNt3h3100HFx2a3NYPidSxK3bzZ/3vbc+5ca96W6rnMTg0RVp1Avr9BQFmMe3QkmJ0/hMKmRoBZP4Sp6GxJ7+0gFQM3Ks6GCEP1dO16588Nk8RcTesJpDZr8viwfC60OCO0eGlJ59/iPawOA3IBoaH17xx2/RRA/zbItmFWWvjDg7ekCE4FYVP4DoFRlMzfQ/CCHCz4rI/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 9 Aug 2024 21:21:15 +0000 kaiyang2@cs.cmu.edu wrote: > From: Kaiyang Zhao > > The ability to observe the demotion and promotion decisions made by the > kernel on a per-cgroup basis is important for monitoring and tuning > containerized workloads on either NUMA machines or machines > equipped with tiered memory. > > Different containers in the system may experience drastically different > memory tiering actions that cannot be distinguished from the global > counters alone. > > For example, a container running a workload that has a much hotter > memory accesses will likely see more promotions and fewer demotions, > potentially depriving a colocated container of top tier memory to such > an extent that its performance degrades unacceptably. > > For another example, some containers may exhibit longer periods between > data reuse, causing much more numa_hint_faults than numa_pages_migrated. > In this case, tuning hot_threshold_ms may be appropriate, but the signal > can easily be lost if only global counters are available. > > This patch set adds five counters to > memory.stat in a cgroup: numa_pages_migrated, numa_pte_updates, > numa_hint_faults, pgdemote_kswapd and pgdemote_direct. > > count_memcg_events_mm() is added to count multiple event occurrences at > once, and get_mem_cgroup_from_folio() is added because we need to get a > reference to the memcg of a folio before it's migrated to track > numa_pages_migrated. The accounting of PGDEMOTE_* is moved to > shrink_inactive_list() before being changed to per-cgroup. > Thanks. I lack the operational experience to be able to judge the usefulness of this - hopefully others can weigh in. Meanwhile, the patch is simple enough - I'll queue it up for testing.