From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 034EDC54ED1 for ; Tue, 27 May 2025 17:48:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F90D6B007B; Tue, 27 May 2025 13:48:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AA2F6B0082; Tue, 27 May 2025 13:48:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6991A6B0083; Tue, 27 May 2025 13:48:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 506AB6B007B for ; Tue, 27 May 2025 13:48:36 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CD7EF1212E1 for ; Tue, 27 May 2025 17:48:35 +0000 (UTC) X-FDA: 83489422590.29.BA74FEE Received: from out-181.mta1.migadu.com (out-181.mta1.migadu.com [95.215.58.181]) by imf01.hostedemail.com (Postfix) with ESMTP id 0C2FD40003 for ; Tue, 27 May 2025 17:48:33 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ZHBAysI3; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748368114; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eTgeKip1dFEF8TN491NA/pilIthAMBoQOcmCRidVyX8=; b=dqhqDCtqJWZOB8kTa/L598ekn9M/PRjQ4HGz9Ru5KoemN+wYJ2ivjQicW5U2C0lxrMHfyk YeD3TbN3f1YKmS1ejwGuusNNesOvIUpDST01ojqIIf3WOsEyOvlsxtx1WdcLPKrbuSPInG F7rO5j+kJx46Pe9aFXFuvxyqyGHBBeE= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ZHBAysI3; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748368114; a=rsa-sha256; cv=none; b=09D/Jf+dbyZsAiYjs+fm4h3U6H5Q84PmatEbSto1AFRrhlnAIit6owYdwtfaygf8wf2iHX 0wWufljHWp05yMsP+29inttAYD8RNzt+GB+aUrRSPmAtWdbyjGTr2mlC6vMpO6m5TR6s8q yjcdCXKBg7Ll4lrfXmVBvtTOpct5N7M= Date: Tue, 27 May 2025 10:48:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1748368112; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eTgeKip1dFEF8TN491NA/pilIthAMBoQOcmCRidVyX8=; b=ZHBAysI3TGX9PbZFh2Ha984UVBpYhYZH9MEejkZiWAWahrYeFnFSeHrnCLyH4p4/7871H8 68ehRduLxEcj1NthLGOgi80gP6he0VYxbHYjSK95djwgQKhAN6n08wz7dXCxq2XqF58WVK /0uKxEJuzz7+hqp7z79Kggd6trMieEw= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: "Chen, Yu C" Cc: peterz@infradead.org, akpm@linux-foundation.org, mkoutny@suse.com, mingo@redhat.com, tj@kernel.org, hannes@cmpxchg.org, corbet@lwn.net, mgorman@suse.de, mhocko@kernel.org, muchun.song@linux.dev, roman.gushchin@linux.dev, tim.c.chen@intel.com, aubrey.li@intel.com, libo.chen@oracle.com, kprateek.nayak@amd.com, vineethr@linux.ibm.com, venkat88@linux.ibm.com, ayushjai@amd.com, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, yu.chen.surf@foxmail.com Subject: Re: [PATCH v5 2/2] sched/numa: add statistics of numa balance task Message-ID: References: <7ef90a88602ed536be46eba7152ed0d33bad5790.1748002400.git.yu.c.chen@intel.com> <52a1b56b-9598-499d-ac9c-de99479d5166@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52a1b56b-9598-499d-ac9c-de99479d5166@intel.com> X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0C2FD40003 X-Stat-Signature: bngzzjo5oxjcomx8654us54r3e43996a X-Rspam-User: X-HE-Tag: 1748368113-631117 X-HE-Meta: U2FsdGVkX18KY/qlrjN8D8rd3nQnbg4Hqz+DArGoR8HbYCEEKeNSQN3nhb+8zt4ksBs4ScNnaun5nKndeYbi0tUzTKFZ81Y8tB8sY/4/H8MyogMn2d7Oj0UipFtOaC5UtLwZj6zhLNHL9WGCCD4r58TelNLLSW/ujTYvHgeSd3Ws8BWta1loe846cToh8q7y/801ZyMz8TTN3mwNnd+p8ephcrvOCOM5xNWlj6BNFg0JSRucj2KSbMX8LDaJRHLlsS6v/wTIqrdNeD44zDHUBdb6cmP55AuXngID6C5fTq9/sw2bTI3HL8UdrvgM4aNZCteKCv1auiBFXxFhyss4YohwnDCgrLaI0fZclhRo2rK27fT97ssh4G3xcBhQNKZT366xSXqJlEuhkuZ8h6j8Jbs/6Y/V5oWBFwykuGOe/O1pZCox2KMJF9J3HobwPs9OpsR3Rs/wcv9tn0dQJdiAsTy4uxWhV6VKM3hkyioRYSxHJS3Oc7jzEnzd6OnWMnIMDLkpyjP2b+A5BWilAYvjQX6i4AdGGGTJwgPOAOyN79nmNEYZ+gttEMSenL6EKRsygj7yYQV5eJKtaQrmqxhJpkwxCYlKaywBxUhFJrj59JhC7JWGZ0HDU3QNpWScnBrCpl8fLfGuV3KoOL+jSRSfawFFWyeDPjFpAEsUYQG/0FxEZY6RL5NlSMP+8KIcaDZ3wIVsBZ94ey6GGKY+q445gyMu8oNZppOHFSqQ7Xm7BK85TK373SoaCngGjWEM1eHuwH9fo3RRrA/Fi+pINlyGoGbBfV1+D/8Hz4ghsjYDrSTo8SSitMCenRqtGHy2QFgp7/qfRs11Huiv+hFYM9WoDWOZhQHFbXfZLWJsvucbyJjkINuW4TtE6b9kh9RubViUqY4Nxd3x475iJ46frn/9GZienWnh7UwRU1i4T8Oe4WNFBRu3TUOSB9DY9+hKQOS802L203xbTyFpHvIm+Bx jO3MJooH UcAsAMSG89dS/tmaAygKumDc06GqG+23HLRk012wL734RyT+zyMvxmEYiSC1O2DAl4T74nWjm0cQ+TVXndRL7/7bcOpvrr6Z7rQXjKgbgNtF/vsj18TsoQafS/naXlgAIzDFCqgKM4K6SgJnDajz1jxq3RekybpLvrkxNkUOXVRWjywnJdhRrAk5SOew1a4OuK+9PrNZPpBRP/ti8Uh6/7oOX2Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, May 25, 2025 at 08:35:24PM +0800, Chen, Yu C wrote: > On 5/25/2025 1:32 AM, Shakeel Butt wrote: [...] > > can you please give an end-to-end> flow/story of all these events > happening on a timeline. > > > > Yes, sure, let me have a try. > > The goal of NUMA balancing is to co-locate a task and its > memory pages on the same NUMA node. There are two strategies: > migrate the pages to the task's node, or migrate the task to > the node where its pages reside. > > Suppose a task p1 is running on Node 0, but its pages are > located on Node 1. NUMA page fault statistics for p1 reveal > its "page footprint" across nodes. If NUMA balancing detects > that most of p1's pages are on Node 1: > > 1.Page Migration Attempt: > The Numa balance first tries to migrate p1's pages to Node 0. > The numa_page_migrate counter increments. > > 2.Task Migration Strategies: > After the page migration finishes, Numa balance checks every > 1 second to see if p1 can be migrated to Node 1. > > Case 2.1: Idle CPU Available > If Node 1 has an idle CPU, p1 is directly scheduled there. This event is > logged as numa_task_migrated. > Case 2.2: No Idle CPU (Task Swap) > If all CPUs on Node1 are busy, direct migration could cause CPU contention > or load imbalance. Instead: > The Numa balance selects a candidate task p2 on Node 1 that prefers > Node 0 (e.g., due to its own page footprint). > p1 and p2 are swapped. This cross-node swap is recorded as > numa_task_swapped. > Thanks for the explanation, this is really helpful and I would like this to be included in the commit message. > > Beside that, do you think there might be some other scheduling events > > (maybe unrelated to numa balancing) which might be suitable for > > memory.stat? Basically I am trying to find if having sched events in > > memory.stat be an exception for numa balancing or more general. > > If the criterion is a combination of task scheduling strategy and > page-based operations, I cannot find any other existing scheduling > events. For now, NUMA balancing seems to be the only case. Mainly I was looking if in future we need to add more sched events to memory.stat file. Let me reply on the other email chain on what should we do next.