From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3A55C3DA63 for ; Tue, 23 Jul 2024 03:38:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B1326B007B; Mon, 22 Jul 2024 23:38:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 161C36B0083; Mon, 22 Jul 2024 23:38:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04F9D6B0085; Mon, 22 Jul 2024 23:38:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DC7B56B007B for ; Mon, 22 Jul 2024 23:38:53 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 85C4B161BA3 for ; Tue, 23 Jul 2024 03:38:53 +0000 (UTC) X-FDA: 82369610946.19.796E007 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf11.hostedemail.com (Postfix) with ESMTP id 74C4C4000B for ; Tue, 23 Jul 2024 03:38:50 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gJ54VKtM; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721705870; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YROv658EXHYDsrOgp2CZflIBbEscCdGUCPrGxwT/tt4=; b=jxeOZRmZfkY9ys+JgMNH20EcDtc7SeEp7nFCTJjcpp8OKvjJz9GkNoo4iePmITjFeHO5Oj TBtPTgaXttGEVYDnJHyd/ZAZezitfSRdyVRUeVGALtG2kyhvjWh3XuhjXJiKayE/aZDKIV u4vWUMbDhLvy2UY0oAPoBR9G6VmDkXw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gJ54VKtM; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721705870; a=rsa-sha256; cv=none; b=OdvhzDqwjP9WpVrkVquAfswfsXiPfdsOolgXZDXdgj7coQlUOb1GsuhLp7wIRSY46AY+Sg HNf+TVNSgf0QO+IyKHLsuRdT+wl4Vl5qQHitPLkpyUHlkrrRCLqZSfM9TW4C8RKtSkinwF XdVy/HITDv6KXG3Koa8RjY5KFOiM7KA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1721705930; x=1753241930; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=1631P6e/mcs9hwEOVcOrQkreV28dqYK4PniggGBcbso=; b=gJ54VKtMfCMAjBRPvp6X9/UgoKSlR9BiUbmuj6hOkXz8jVAiQ4whJ5XQ nMANMTlt46eSyuQRYyK3Wc2R7a4hnCu6MdEXFPLUdjPAP9bhh2es43UIh Wl09kMJ4NYi4XjmaEgn+7VT9s4NIlYrQiB2b+2Pso3UI8/gDVUX4l4kGk kxyIpUiMtycirXd2B2zKw3ncgx+vWaFmSzqHIWWvwXeWuohagXz+tBIJb 04tJFaUVUKBs7Eoq961jo7qgNWh2jePbBp4NES0l7KpgqvmTh5zyM50+T Fx0QhJReHln17swAeeA7L+yenhq4BRZUxbSfWBIwiUXcY2E6xkQCGAXm6 A==; X-CSE-ConnectionGUID: UbVk8eabShGZSx8jibbDUg== X-CSE-MsgGUID: TC0cWnHzTzeHrHEiV674Pw== X-IronPort-AV: E=McAfee;i="6700,10204,11141"; a="29895397" X-IronPort-AV: E=Sophos;i="6.09,229,1716274800"; d="scan'208";a="29895397" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2024 20:38:49 -0700 X-CSE-ConnectionGUID: gDjPZIziQauDyW62ia0RZQ== X-CSE-MsgGUID: Ybl3aCNrQJKGv0QsZpBNuA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,229,1716274800"; d="scan'208";a="89553417" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2024 20:38:47 -0700 From: "Huang, Ying" To: Zhongkun He Cc: peterz@infradead.org, mgorman@suse.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andi Kleen Subject: Re: [PATCH] mm/numa_balancing: Fix the memory thrashing problem in the single-threaded process In-Reply-To: <20240722123320.2382992-1-hezhongkun.hzk@bytedance.com> (Zhongkun He's message of "Mon, 22 Jul 2024 20:33:20 +0800") References: <20240722123320.2382992-1-hezhongkun.hzk@bytedance.com> Date: Tue, 23 Jul 2024 11:35:14 +0800 Message-ID: <871q3kg4r1.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 74C4C4000B X-Stat-Signature: 3g3htskz6xdpy9n8t365uxd4c88ua6bd X-Rspam-User: X-HE-Tag: 1721705930-766943 X-HE-Meta: U2FsdGVkX1/dsVxiqzGXPRBsTMchaCa51qcwOaFqXMv/4Q7vkIRtEXdCly1MRCQgkxjt3ccS3u5wrtxzxvZkg3X0rnUdmp8CXmrv/es1kUoMhtSym4R4mD7uvkpHbofQj/BEQhm+ouauGaedV479TLXktX59FhTHyImaXrLNavRFrSY1JXGo29gid37ym5N2+SYBqHryFtNK0S+djaZZy0LlqqTNhkeTB/xwKiAd2ejJLGH8/zR6qWZW7Po2aBKi9rcZ5k/++vQ/DtbTI6JHSiZhjPFVCoP0hjlmlbFGP3YPZJoQqf9Y79Rc95Mq2On7CKM7xYlMQgdk6evCHcOlPkDtMYKseMd4nXonACn3nbtpYjDQ5HToKQy0ZH50i/Kl2hMy8I4QLTSPC+PSdidz6/KegQB4vUQQxJIplniIIVnIjd510GH+XugZA2c/xvS4d/wb07F1SmGp0NdEbIPOp/NP/GQU0ixPSFAzWr1pXQtl2wrYrakvpoivgN5ZMaiJufZoxF37HhZdZYTAX5wAAoXsslo4wMM5vrL6ixO8CnRYgTHYJKSGTTCy3QALRMccOK/PhEPv+fOuZ+Mz8ET5DS2g91GNQiKe3LqDlSUdyuCSGYIxa3tNkrWe10M6GqcPcBJi7i/u7GuKLJ3zB3uNAO6kRygvhISF9lA3ykRgQKw1kT5o8/aoCGXRWMGY8HyBtqEV3ID/AQx7eueZ5eHinBFAZrnR/dkxU7SBb/TCt3fE1+mfhQLm/gtwP95ed4RBGXEebZ1p0kO5ieMKLnGlysqcTCIcJnjzioRWPWVAg535ZKu/3CpQZf7cLI15cLYMjGQ+LEY1rQE2CmuDCO4jCSe3zPEsKCW23QnDnpVQ7G8APq5kXiB02EhMRNpMXCq3akRpqcwEA14oorMIJ/7jsEKer75yf80lHM7Ygbowk4l/7/0p22NR7CsYOhw/473KuTcpbBCm6ShNC8DUfVu arsVuE5D mKhM4bzhZfFveWjR1vO3k7a1CScawmY2drEdjwZ7JNmRibG2bRKfTg1HQF3Ku//CjbUeG3S/ap1tapkfCzuYzqtnPgqWFztc6A4UwEUXyTh5I+oEBQYciHMQi5DSWxdJbAzW/46sWkuyFqWLOs8nG2OiVK2/0XUN2nE5p5R3imfJ2GvaVpNeSpPZvMvEIKKAgy4OB0l0xw7N0g5IipGHMfL60KZYAKJhWs7BJy47YqfntkDq/wDv5zaTg5QyNbAsprVARMgxHeymMY+E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Zhongkun He writes: > I found a problem in my test machine that the memory of a process is > repeatedly migrated between two nodes and does not stop. > > 1.Test step and the machines. > ------------ > VM machine: 4 numa nodes and 10GB per node. > > stress --vm 1 --vm-bytes 12g --vm-keep > > The info of numa stat: > while :;do cat memory.numa_stat | grep -w anon;sleep 5;done > anon N0=98304 N1=0 N2=10250747904 N3=2634334208 > anon N0=98304 N1=0 N2=10250747904 N3=2634334208 > anon N0=98304 N1=0 N2=9937256448 N3=2947825664 > anon N0=98304 N1=0 N2=8863514624 N3=4021567488 > anon N0=98304 N1=0 N2=7789772800 N3=5095309312 > anon N0=98304 N1=0 N2=6716030976 N3=6169051136 > anon N0=98304 N1=0 N2=5642289152 N3=7242792960 > anon N0=98304 N1=0 N2=5105442816 N3=7779639296 > anon N0=98304 N1=0 N2=5105442816 N3=7779639296 > anon N0=98304 N1=0 N2=4837007360 N3=8048074752 > anon N0=98304 N1=0 N2=3763265536 N3=9121816576 > anon N0=98304 N1=0 N2=2689523712 N3=10195558400 > anon N0=98304 N1=0 N2=2515148800 N3=10369933312 > anon N0=98304 N1=0 N2=2515148800 N3=10369933312 > anon N0=98304 N1=0 N2=2515148800 N3=10369933312 > anon N0=98304 N1=0 N2=3320455168 N3=9564626944 > anon N0=98304 N1=0 N2=4394196992 N3=8490885120 > anon N0=98304 N1=0 N2=5105442816 N3=7779639296 > anon N0=98304 N1=0 N2=6174195712 N3=6710886400 > anon N0=98304 N1=0 N2=7247937536 N3=5637144576 > anon N0=98304 N1=0 N2=8321679360 N3=4563402752 > anon N0=98304 N1=0 N2=9395421184 N3=3489660928 > anon N0=98304 N1=0 N2=10247872512 N3=2637209600 > anon N0=98304 N1=0 N2=10247872512 N3=2637209600 > > 2. Root cause: > Since commit 3e32158767b0 ("mm/mprotect.c: don't touch single threaded > PTEs which are on the right node")the PTE of local pages will not be > changed in change_pte_range() for single-threaded process, so no > page_faults information will be generated in do_numa_page(). If a > single-threaded process has memory on another node, it will > unconditionally migrate all of it's local memory to that node, > even if the remote node has only one page. > > So, let's fix it. The memory of single-threaded process should follow > the cpu, not the numa faults info in order to avoid memory thrashing. Show the test results (numa stats) of the fixed kernel? > Signed-off-by: Zhongkun He > --- > kernel/sched/fair.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 24dda708b699..d7cbbda568fb 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -2898,6 +2898,12 @@ static void task_numa_placement(struct task_struct *p) > numa_group_count_active_nodes(ng); > spin_unlock_irq(group_lock); > max_nid = preferred_group_nid(p, max_nid); > + } else if (atomic_read(&p->mm->mm_users) == 1) { > + /* > + * The memory of a single-threaded process should > + * follow the CPU in order to avoid memory thrashing. > + */ > + max_nid = numa_node_id(); > } > > if (max_faults) { The change looks reasonable for me, Thanks! Acked-by: "Huang, Ying" -- Best Regards, Huang, Ying