From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37B64C3DA49 for ; Tue, 23 Jul 2024 05:07:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E8F56B007B; Tue, 23 Jul 2024 01:07:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 598A66B0083; Tue, 23 Jul 2024 01:07:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4619F6B0085; Tue, 23 Jul 2024 01:07:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 27F796B007B for ; Tue, 23 Jul 2024 01:07:47 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A457C121C03 for ; Tue, 23 Jul 2024 05:07:46 +0000 (UTC) X-FDA: 82369834932.01.AB8BFCE Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by imf29.hostedemail.com (Postfix) with ESMTP id E95D2120007 for ; Tue, 23 Jul 2024 05:07:43 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rd0qi5z8; spf=pass (imf29.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721711218; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jsnSC07zfIXon19YhC/42Be/Z1ql1w+JDYlQwMaAxqs=; b=eeaKGX2d8qPn9/ATlI7638do1UOIRfIec3oAP+Uw7wxyMP7L8iYHHqyV2ChbHshvwIRLFs EHVsUO5gUCmQWJd8EohUdCpT8wMgrxvFsn919i5ehWLUvejUNxHV+S+0L7u6Xp19fKbHbb Q/WccI7AGRTLi2HfxCDVVS7azt/1vHI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721711218; a=rsa-sha256; cv=none; b=C9XMA1nCAxmDj0ykm523FuCr/9DXBrRgoCkkdLzkZ1x0xHbN/rzgqGKklZL7PG5wwu203x 0UXEpkqQl0UCGWpG8PnHnAFRsLeZTabbYlPClchvnss8HFSVjp4yt1JUhvWH47P6WsYWUh +e/dED0ZYkpA4SZjjKvw2MfpUp+pVRo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rd0qi5z8; spf=pass (imf29.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-52f00ad303aso2758365e87.2 for ; Mon, 22 Jul 2024 22:07:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1721711262; x=1722316062; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=jsnSC07zfIXon19YhC/42Be/Z1ql1w+JDYlQwMaAxqs=; b=Rd0qi5z8ZLa4QeH4wDNx5MNi9hFKS2AtM4H2cQYoQ0h4fp7jYJH7ZxIB+I5+R5Wc99 0lQzaU4Jx/Hfw37nX7IWvBJ70VPVoxTepqdM5SYl6KIioUijTBAlygIK6h70x3Y260IN D35GByZWHUXCF3nDFeWBS99NvGA7khdwq0jSe/QXEYaUWGDfZos6FExm1UMk7lyokTMV QGLfYkbsPrTH3+c7w3omcnIR2v6Tx9pu1tSvMNAl9qFNssSyMkZXamavnSE6JsBsPcR0 jNaGHUuLVx2MfRGkaXMF2wInpfHnw8X/IHclQ8mxi9FAcdk8PRrTZFnH42sLtas52IKK NdSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721711262; x=1722316062; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jsnSC07zfIXon19YhC/42Be/Z1ql1w+JDYlQwMaAxqs=; b=hTP+7JjEVVod0KzhK0TTvvz2Kvcc9QNxCgBMsw5mh86QfL3ARRz3DGc2oN8AKcMX8S 6mY6UhqH4ZQ1be6VVt5JXWrqVTbn8krVfDIshBCHFXDiX0UkGwdUilOz5VV8ckUAZvnd hZUfGyIen4X3dz6uGVPnnkEgE3AxTKoFFsV6Te4oiJMyj+Ox+Q4Neb6Pfyrd4N10pyZT ZpQkwcIWzBp09tU1Sk+0qiefp46dUc7afOFB+c/3vw/cpm/w/xJFkcWGAcGuZhvj86nP MFlIz8yHOyLkBOCVPZwl3Rhj9EYa7yh7mbcLCbelVaVwG7qWmawNB80IaRqpFNC9B+SE yGzA== X-Forwarded-Encrypted: i=1; AJvYcCXQs5Q1nvxM8yri/3bEuMDBwpI/8ZkoM4Vtkz5nmu42YH3xFliJmGh+s8ff13ApHEMD0sdC6rBgOvS5be670AfsqGs= X-Gm-Message-State: AOJu0YzluAGmJjR24O/vuzjOfIbzy46bzDEOInzPlOvimIn2mgLrtldb kfiQyAKsj9BM6C+VU2PcfyddYBDuVYTMXhJNdie2J4+uIaT/m9Sk4igZaIhHgFmsGap49nqLZSt 0e4kDM85IAHx80XvIOp4eA7NO4qvHMQieo1pWxg== X-Google-Smtp-Source: AGHT+IEDLM/hvVYGjh4KirEmCVgBGo17a2GPijRWstrrLWkCq0JOfdCb7BsajWqphfo72SH4chhZDajYtLUxYCsCVqg= X-Received: by 2002:a05:6512:3e1e:b0:52c:c9bb:2ba4 with SMTP id 2adb3069b0e04-52fc406f0b4mr1249711e87.46.1721711261978; Mon, 22 Jul 2024 22:07:41 -0700 (PDT) MIME-Version: 1.0 References: <20240722123320.2382992-1-hezhongkun.hzk@bytedance.com> <871q3kg4r1.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <871q3kg4r1.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Zhongkun He Date: Tue, 23 Jul 2024 13:07:31 +0800 Message-ID: Subject: Re: [External] Re: [PATCH] mm/numa_balancing: Fix the memory thrashing problem in the single-threaded process To: "Huang, Ying" Cc: peterz@infradead.org, mgorman@suse.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andi Kleen Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: sr91is8uxi3bq1crtd9ywcyh74r43ez8 X-Rspamd-Queue-Id: E95D2120007 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721711263-61334 X-HE-Meta: U2FsdGVkX18Eap7QcN5uXUEhmiBd2TQEpBcCPRNjQ+vOZONyqxF7KlnHeBGnWJiWRWQ7yVMlF2aUA8thsKKioLTLE6jwsJwyngFK9Y8vXsJQ/4TuztRH8kbDWFCxyXdY0CkmlGfMTRwZE9d4wJDI842HgyDvc+r40cNXvCVP+8rrYqU/+lcr9iaEaf3zRQS5zO4uG9/qzNgfD5a6+QCYkqFEEKAk6tnMssZ+afLfEyLoHt16H11vp7CEjLjHuCBKc1CKngzfIx9cEAPYm7SIa6W0iAOIxWsF9oeXAPLdMXjKAYc9djUzpaCcT4PRJNyEDy3pcRmihJtmr6n++qj81Iua4Dx+XWB4+IBxXojoOGOljqcHNjZYmYHXg5ONVaXW6Bm+NQYkk2n/2/Re6/EedYG85VintHkwUH19Md05JmhZaQuNdZU70OZgn8QWANUuPtFlOnLkAKi18IOfdBUQvYGCZfR2zfN7DuWt+UkTxTLyVR8sp27IVWqOHlycD6s/+Bkt9iU4+MzSd64wkRAXxQVkt4SSOBOFalzAsiUCDOQem6+Q8YpL3rKnLDRJB/Kl/AKRs8hPllJJSw3++He49kTp5ktSk1p9eBdX+PDoOhZiyaqHdIHQvRHyBHgiQ6l7kaHtwCu176G7ua8kB0R1f6GjF3Emm9s7njE9N5sBLzi7Vxk5FZo0b/JuZxNMCED3slt4ltwpQENZo900bdSju/Dz8TzIsrsVl7M2//kGfiu6JuOgptiFdJOmWE7wUaAWXXm7zXkIF7u7SAGkIs70vVxB0AwOHSZzL7D6+n1euk59Dcgjmx8I/87sTuQNp9xS2ioOzAK5MGMBOyITts5ws7XK0ZYPnzvywQGLIIcLjAWPtgpIgKkj72t/mP7rHmFriVFpnwIoA5y/2MpTt/D44nropGoso26Q3YtyFxCOPJdrguyC+R9+wlgcuaxKoeHc7XrMxNzdkLarzJJhig4 mZIN12/h 2NpiaCzpcddK253zZEmdaBvuGTMvDDHGmxbt/SbbAEzj9tSYpMHQOwH3NCq8bBd1krZbyalAPz/d9+yUVrWRGT4AyuIcGa4ctoBA3wYdSO/NdIUiHQTAjDdPlwZae1X+qT5DQn8qa8BlV42ldE5kSonoIgD9WqStvOOp3kCJ7aUxOcPkjZPneP4h/BzjEQ4wVyNCsQu8G3u1eU/wDsKQFmyCOubge+fkfcgkkamKA9PPBrLzI5sa9HYkfYIEnWv2UsTaxokTcgBfX+X6mJUM6UxHMyfJ0JKYr1bfmeYUSuvM8PMbBOvwROUkczi3atljowOriNKAK7rLctv/iCgz9bAdEEw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 23, 2024 at 11:39=E2=80=AFAM Huang, Ying = wrote: > > Zhongkun He writes: > > > I found a problem in my test machine that the memory of a process is > > repeatedly migrated between two nodes and does not stop. > > > > 1.Test step and the machines. > > ------------ > > VM machine: 4 numa nodes and 10GB per node. > > > > stress --vm 1 --vm-bytes 12g --vm-keep > > > > The info of numa stat: > > while :;do cat memory.numa_stat | grep -w anon;sleep 5;done > > anon N0=3D98304 N1=3D0 N2=3D10250747904 N3=3D2634334208 > > anon N0=3D98304 N1=3D0 N2=3D10250747904 N3=3D2634334208 > > anon N0=3D98304 N1=3D0 N2=3D9937256448 N3=3D2947825664 > > anon N0=3D98304 N1=3D0 N2=3D8863514624 N3=3D4021567488 > > anon N0=3D98304 N1=3D0 N2=3D7789772800 N3=3D5095309312 > > anon N0=3D98304 N1=3D0 N2=3D6716030976 N3=3D6169051136 > > anon N0=3D98304 N1=3D0 N2=3D5642289152 N3=3D7242792960 > > anon N0=3D98304 N1=3D0 N2=3D5105442816 N3=3D7779639296 > > anon N0=3D98304 N1=3D0 N2=3D5105442816 N3=3D7779639296 > > anon N0=3D98304 N1=3D0 N2=3D4837007360 N3=3D8048074752 > > anon N0=3D98304 N1=3D0 N2=3D3763265536 N3=3D9121816576 > > anon N0=3D98304 N1=3D0 N2=3D2689523712 N3=3D10195558400 > > anon N0=3D98304 N1=3D0 N2=3D2515148800 N3=3D10369933312 > > anon N0=3D98304 N1=3D0 N2=3D2515148800 N3=3D10369933312 > > anon N0=3D98304 N1=3D0 N2=3D2515148800 N3=3D10369933312 > > anon N0=3D98304 N1=3D0 N2=3D3320455168 N3=3D9564626944 > > anon N0=3D98304 N1=3D0 N2=3D4394196992 N3=3D8490885120 > > anon N0=3D98304 N1=3D0 N2=3D5105442816 N3=3D7779639296 > > anon N0=3D98304 N1=3D0 N2=3D6174195712 N3=3D6710886400 > > anon N0=3D98304 N1=3D0 N2=3D7247937536 N3=3D5637144576 > > anon N0=3D98304 N1=3D0 N2=3D8321679360 N3=3D4563402752 > > anon N0=3D98304 N1=3D0 N2=3D9395421184 N3=3D3489660928 > > anon N0=3D98304 N1=3D0 N2=3D10247872512 N3=3D2637209600 > > anon N0=3D98304 N1=3D0 N2=3D10247872512 N3=3D2637209600 > > > > 2. Root cause: > > Since commit 3e32158767b0 ("mm/mprotect.c: don't touch single threaded > > PTEs which are on the right node")the PTE of local pages will not be > > changed in change_pte_range() for single-threaded process, so no > > page_faults information will be generated in do_numa_page(). If a > > single-threaded process has memory on another node, it will > > unconditionally migrate all of it's local memory to that node, > > even if the remote node has only one page. > > > > So, let's fix it. The memory of single-threaded process should follow > > the cpu, not the numa faults info in order to avoid memory thrashing. > > Show the test results (numa stats) of the fixed kernel? > After a long time of testing, there is no memory thrashing from the beginning. while :;do cat memory.numa_stat | grep -w anon;sleep 5;done anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 anon N0=3D2548117504 N1=3D10336903168 N2=3D139264 N3=3D0 I will add it to the commit in the next version. > > Signed-off-by: Zhongkun He > > --- > > kernel/sched/fair.c | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index 24dda708b699..d7cbbda568fb 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -2898,6 +2898,12 @@ static void task_numa_placement(struct task_stru= ct *p) > > numa_group_count_active_nodes(ng); > > spin_unlock_irq(group_lock); > > max_nid =3D preferred_group_nid(p, max_nid); > > + } else if (atomic_read(&p->mm->mm_users) =3D=3D 1) { > > + /* > > + * The memory of a single-threaded process should > > + * follow the CPU in order to avoid memory thrashing. > > + */ > > + max_nid =3D numa_node_id(); > > } > > > > if (max_faults) { > > The change looks reasonable for me, Thanks! > > Acked-by: "Huang, Ying" > Thanks. > -- > Best Regards, > Huang, Ying