From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3116CE7AA5 for ; Fri, 6 Sep 2024 01:16:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 081766B0082; Thu, 5 Sep 2024 21:16:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0310F6B0085; Thu, 5 Sep 2024 21:16:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3B1C6B0088; Thu, 5 Sep 2024 21:16:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C6AEB6B0082 for ; Thu, 5 Sep 2024 21:16:46 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id AB31840DFF for ; Fri, 6 Sep 2024 01:16:45 +0000 (UTC) X-FDA: 82532548770.26.9E9261B Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf01.hostedemail.com (Postfix) with ESMTP id 567AC4000F for ; Fri, 6 Sep 2024 01:16:41 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of liuye@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=liuye@kylinos.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725585305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WllLrPhDVtrPJ07fLT3HvfxMUmuZDLbCIjLePnU0c0k=; b=nEnL6zt89C1pXtC2ayVc6uz7ctYoSLsX2cgPHeZUDbdywDczaAnbl6xr/6KmwJUgCfm8ww 6wgt5HOTls12XkpY/8nDpX2DuEb1t3MeOUhSrGKfoxZXiCyCiCGI+c5fRwiPYuLO+B97/n EDvViaHkdUL7MMkv2J+vUNmqBws0sxI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725585305; a=rsa-sha256; cv=none; b=lxNOzWLq+fE4a1K8w8eZvDejSoOMO2bZtXiRzI/qC70vXyHfCki20x+Vh/kPERzELVHE+W LIxq9o5J20zCxGs9scZnuZEK0v4H8Ljy+QcMFEvGE/CIej5sn6sN/fatLuSwY3XCdXiBuj OqehkGYzxvjL5MWlBBuKw0rfSHZ2+O4= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of liuye@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=liuye@kylinos.cn; dmarc=none X-UUID: a953f5ea6bed11efa216b1d71e6e1362-20240906 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.38,REQID:5764e28d-80f7-4399-91fa-6c4f28fc151b,IP:0,U RL:0,TC:0,Content:0,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:0 X-CID-META: VersionHash:82c5f88,CLOUDID:51e4160c96bb2ff62655c5d7a70dc609,BulkI D:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0|-5,EDM:-3,IP:nil,UR L:0,File:nil,RT:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,S PR:NO,DKR:0,DKP:0,BRR:0,BRE:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR X-UUID: a953f5ea6bed11efa216b1d71e6e1362-20240906 Received: from node2.com.cn [(10.44.16.197)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 1203611180; Fri, 06 Sep 2024 09:16:36 +0800 Received: from node2.com.cn (localhost [127.0.0.1]) by node2.com.cn (NSMail) with SMTP id 434B5B80758A; Fri, 6 Sep 2024 09:16:36 +0800 (CST) X-ns-mid: postfix-66DA57F4-161909118 Received: from [172.30.70.72] (unknown [172.30.70.72]) by node2.com.cn (NSMail) with ESMTPA id 2ACF5B80758A; Fri, 6 Sep 2024 01:16:32 +0000 (UTC) Subject: Re: [PATCH] mm/vmscan: Fix hard LOCKUP in function isolate_lru_folios To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240815025226.8973-1-liuye@kylinos.cn> <20240823020443.7379-1-liuye@kylinos.cn> From: liuye Message-ID: Date: Fri, 6 Sep 2024 09:16:31 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <20240823020443.7379-1-liuye@kylinos.cn> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Stat-Signature: n8nnpnm6t6pu9zroeph4uuki4qe37tac X-Rspamd-Queue-Id: 567AC4000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725585401-252951 X-HE-Meta: U2FsdGVkX1+SOAp2yR75y3C1+rYUHlVCSgkNsbs5nZMvmVtDlC8tnBCPzGdCoORP7Q8xovM+FABJg45/XEjQLL57gfe+BVrWl9gB5yZ7w5+d7bSpmzz23viAZahQUTz3xrBGPvBC/V9Lnp9EoN5GXB/eNllrHTB+47apcP/LjSthZCdoXmuwOi1wd7DIP2FMDMJFQDlBPeGjPPbQ5ejoy2cE6l+De5BNTHCDYM6y0empS1lbDA2CxgQHn04s4Zz8I2W0vrpcMqJwXLzriZqcFYwuEKC4CdqikLLw5+Yaqc5eyA0Om2PQamBboyJCiMM9YWznoYyx+8A0qzYsiIA8vu/MfgOZmEu1uioTMA10AhX3QMCuMpUnR0+J0pRWNQGWkoTKLi0qMzSv2XAfR4E44jpsAdLLCvmHSfeFNfRjRtDeUz3M+oGRBp4/BdFWAJ1NDig6IjQnROZ1rhlB01CI+iVBbp0DpUPHFQ2LyILLUaSFVTBmFX7fXXFWMtLv3uvGB64otQvS9zFadHJoP3Gw33NnttOJeNCr0Wr5m18ytZNP55hgI86yQWagvYl2YOHfxm3yNBFOB5df2ItajlSd7SpxFiUEAassZ87DVfC13Fnr/d68TFeI2u22vkbW4ffJ6mIDlHBBwX6zeamyC5fo85y+X/YKLpvotq2jwx8UhZVItNPlkRRNTBrr7rp/GiLNgP8cF+O7m0aXXTvUJFTtx5JkLMB0Yr5cTgO//FocXoPUmYdUN5JVTQQB5kW0mW8RLqPk06acZfTLQB5sz4KKyN7E4dFbJe5kqgxqMV4XNlKK8qJNaarT59gZhMB8oT2u2EtyVa9YWkSWcyhKOwYOAHVaapVZBPeSl81J2TZQNmNB4UeP4qzWSJfFQSWvtmCdivFSfC36wFl9Yne+evb4TqeWZ77RpT+K6tWsV9DQeE7817qGFIEmpQP+oTFqmXszVNz83PTCTSROhgyJGlG W6Go/4c+ buZGvquutZX6WBADHKCmbw1+IaPz8gQsQex7E4CqhH3OQtvfjfok8Lr7yDdFuvoIcDcHQX2MU0zgLS2FcbfKb2KhDYN+fBmYyE5opgDV5bTa5vRv17ev2JdeQiwUC9FDiKVju23is4HY/W7c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/8/23 =E4=B8=8A=E5=8D=8810:04, liuye wrote: > I'm sorry to bother you about that, but it looks like the following ema= il send 7 days ago,=20 > did not receive a response from you. Do you mind having a look at this=20 > when you have a bit of free time please? >=20 >>>> Fixes: b2e18757f2c9 ("mm, vmscan: begin reclaiming pages on a per-no= de basis") >>> >>> Merged in 2016. >>> >>> Under what circumstances does it occur? =20 >> >> User processe are requesting a large amount of memory and keep page ac= tive. >> Then a module continuously requests memory from ZONE_DMA32 area. >> Memory reclaim will be triggered due to ZONE_DMA32 watermark alarm rea= ched. >> However pages in the LRU(active_anon) list are mostly from=20 >> the ZONE_NORMAL area. >> >>> Can you please describe how to reproduce this? =20 >> >> Terminal 1: Construct to continuously increase pages active(anon).=20 >> mkdir /tmp/memory >> mount -t tmpfs -o size=3D1024000M tmpfs /tmp/memory >> dd if=3D/dev/zero of=3D/tmp/memory/block bs=3D4M >> tail /tmp/memory/block >> >> Terminal 2: >> vmstat -a 1 >> active will increase. >> procs -----------memory---------- ---swap-- -----io---- -system-- ----= ---cpu------- >> r b swpd free inact active si so bi bo in cs us s= y id wa st gu >> 1 0 0 1445623076 45898836 83646008 0 0 0 0 1807 1= 682 0 0 100 0 0 0 >> 1 0 0 1445623076 43450228 86094616 0 0 0 0 1677 1= 468 0 0 100 0 0 0 >> 1 0 0 1445623076 41003480 88541364 0 0 0 0 1985 2= 022 0 0 100 0 0 0 >> 1 0 0 1445623076 38557088 90987756 0 0 0 4 1731 1= 544 0 0 100 0 0 0 >> 1 0 0 1445623076 36109688 93435156 0 0 0 0 1755 1= 501 0 0 100 0 0 0 >> 1 0 0 1445619552 33663256 95881632 0 0 0 0 2015 1= 678 0 0 100 0 0 0 >> 1 0 0 1445619804 31217140 98327792 0 0 0 0 2058 2= 212 0 0 100 0 0 0 >> 1 0 0 1445619804 28769988 100774944 0 0 0 0 1729 = 1585 0 0 100 0 0 0 >> 1 0 0 1445619804 26322348 103222584 0 0 0 0 1774 = 1575 0 0 100 0 0 0 >> 1 0 0 1445619804 23875592 105669340 0 0 0 4 1738 = 1604 0 0 100 0 0 0 >> >> cat /proc/meminfo | head >> Active(anon) increase. >> MemTotal: 1579941036 kB >> MemFree: 1445618500 kB >> MemAvailable: 1453013224 kB >> Buffers: 6516 kB >> Cached: 128653956 kB >> SwapCached: 0 kB >> Active: 118110812 kB >> Inactive: 11436620 kB >> Active(anon): 115345744 kB =20 >> Inactive(anon): 945292 kB >> >> When the Active(anon) is 115345744 kB, insmod module triggers the ZONE= _DMA32 watermark. >> >> perf show nr_scanned=3D28835844.=20 >> 28835844 * 4k =3D 115343376KB approximately equal to 115345744 kB. >> >> perf record -e vmscan:mm_vmscan_lru_isolate -aR >> perf script >> isolate_mode=3D0 classzone=3D1 order=3D1 nr_requested=3D32 nr_scanned=3D= 2 nr_skipped=3D2 nr_taken=3D0 lru=3Dactive_anon >> isolate_mode=3D0 classzone=3D1 order=3D1 nr_requested=3D32 nr_scanned=3D= 0 nr_skipped=3D0 nr_taken=3D0 lru=3Dactive_anon >> isolate_mode=3D0 classzone=3D1 order=3D0 nr_requested=3D32 nr_scanned=3D= 28835844 nr_skipped=3D28835844 nr_taken=3D0 lru=3Dactive_anon >> isolate_mode=3D0 classzone=3D1 order=3D1 nr_requested=3D32 nr_scanned=3D= 28835844 nr_skipped=3D28835844 nr_taken=3D0 lru=3Dactive_anon >> isolate_mode=3D0 classzone=3D1 order=3D0 nr_requested=3D32 nr_scanned=3D= 29 nr_skipped=3D29 nr_taken=3D0 lru=3Dactive_anon >> isolate_mode=3D0 classzone=3D1 order=3D0 nr_requested=3D32 nr_scanned=3D= 0 nr_skipped=3D0 nr_taken=3D0 lru=3Dactive_anon >> >> If increase Active(anon) to 1000G then insmod module triggers the ZONE= _DMA32 watermark. hard lockup will occur. >> >> In my device nr_scanned =3D 0000000003e3e937 when hard lockup. Convert= to memory size 0x0000000003e3e937 * 4KB =3D 261072092 KB. >> >> #5 [ffffc90006fb7c28] isolate_lru_folios at ffffffffa597df53 >> ffffc90006fb7c30: 0000000000000020 0000000000000000=20 >> ffffc90006fb7c40: ffffc90006fb7d40 ffff88812cbd3000=20 >> ffffc90006fb7c50: ffffc90006fb7d30 0000000106fb7de8=20 >> ffffc90006fb7c60: ffffea04a2197008 ffffea0006ed4a48=20 >> ffffc90006fb7c70: 0000000000000000 0000000000000000=20 >> ffffc90006fb7c80: 0000000000000000 0000000000000000=20 >> ffffc90006fb7c90: 0000000000000000 0000000000000000=20 >> ffffc90006fb7ca0: 0000000000000000 0000000003e3e937=20 >> ffffc90006fb7cb0: 0000000000000000 0000000000000000=20 >> ffffc90006fb7cc0: 8d7c0b56b7874b00 ffff88812cbd3000=20 >> >>> Why do you think it took eight years to be discovered? >> >> The problem requires the following conditions to occur: >> 1. The device memory should be large enough. >> 2. Pages in the LRU(active_anon) list are mostly from the ZONE_NORMAL = area. >> 3. The memory in ZONE_DMA32 needs to reach the watermark. >> >> If the memory is not large enough, or if the usage design of ZONE_DMA3= 2 area memory is reasonable, this problem is difficult to detect. >> >> notes: >> The problem is most likely to occur in ZONE_DMA32 and ZONE_NORMAL, but= other suitable scenarios may also trigger the problem. >> >>> It looks like that will fix, but perhaps something more fundamental >>> needs to be done - we're doing a tremendous amount of pretty pointles= s >>> work here. Answers to my above questions will help us resolve this. >>> >>> Thanks. >> >> Please refer to the above explanation for details. >> >> Thanks. >=20 > Thanks. >=20 Friendly ping.=20