From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52565C27C6E for ; Fri, 14 Jun 2024 03:31:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCB3D6B00AD; Thu, 13 Jun 2024 23:31:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7AF26B00B0; Thu, 13 Jun 2024 23:31:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C430F6B00B1; Thu, 13 Jun 2024 23:31:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A22986B00AD for ; Thu, 13 Jun 2024 23:31:29 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3C7D516041F for ; Fri, 14 Jun 2024 03:31:29 +0000 (UTC) X-FDA: 82228069098.11.788D568 Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by imf14.hostedemail.com (Postfix) with ESMTP id 59F96100006 for ; Fri, 14 Jun 2024 03:31:27 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MfVXtAlB; spf=pass (imf14.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718335885; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XZ6udnD39dCm4AVi83klDXXN0/OUvxCDSDYNMIZLgYo=; b=OCBdGtP3m7wR89JRDqnFk6yzIHdDehMysLud3H5HB4Sy+TE4mqrOgRIvFZLQYstysGJEBp uTVBk0nOTmFART0knMRBjPGYrztUugW4kdpx2y5C/zIQKb4YkPC8vdJhNRjP0QOllhM6Ln 1vpDqZAEe9zaN4iz00GmxEbhItr8M1o= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MfVXtAlB; spf=pass (imf14.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718335885; a=rsa-sha256; cv=none; b=jG+atUZxnuCWCA30gPfqIpF4B+Z5QofnquLtTCwsyvF8CadAUTNbLYYj+NiH2yRlwt6UQZ 6iXZv44ibNKnoMUzJI5l/JPdK9v43zzcXyOJjBiMzG7Mpl8Ii0p4CkJc2lHTSmx0OY7pTN v7QtdQsXLnrc1G2uLfrxgP9RacjCsEM= Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-2e78fe9fc2bso20205431fa.3 for ; Thu, 13 Jun 2024 20:31:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718335885; x=1718940685; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=XZ6udnD39dCm4AVi83klDXXN0/OUvxCDSDYNMIZLgYo=; b=MfVXtAlBGbH9IB/dGr17x9icgVs7BXNm78KL84U+1yTvfNm9OY5slyVlP9WlUozrmR jQ47buJ5wCHk4IpZqsOHOIdOBK/+7mmWZGSYka3pPOVcHRIM2bS2R/BcIJuDPxS9klcm pFQg1zCciYAYNyqDBNhKJd/2IMCpD5F7sOBSoQynv2t+YyGFvqlB3B2b/MNjirjJvpfq WUj/TA+wCZoIj4LzegDIpOCRD8590kb3e4QXTGelgQBI6CC8k+oPTbbjcMTlj9Y6uYV6 nwQES2nHpUwwKPIwYeDGEyV/2lwL0Dhhy0veTkGnMmaprz95x//PVVfZiuOqeqDTJeTa b3CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718335885; x=1718940685; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XZ6udnD39dCm4AVi83klDXXN0/OUvxCDSDYNMIZLgYo=; b=TI8E1sj3pG+GlW6Vo8sww7+kx8X7T3+2azrW1iIiZ1H/LHmt59lcSqivYDXOk+TGKh 0WPyV0xT9u8RMXQkZ7nAAbdPkGuqgB2Qtbf/KOwI5ZuUOHBUcgFsvfGPcGri5/MWTi9a BDY0K3hAUwProc4Uo2K+6mgst2eg1WOh3Q4VbGVym0kMXMRGjVqE2XBSLcrMqK310Qgi s0Dbk0iYNZJ/7Pl6r2811+R3MZw4fOBvuMErYjhkp40YWVQNRfJwIEKABmfup/t4wYzm RaHYwYseNLRLHf3ECdt522+IKo2T/XLmyS5q5H5gEZ9CSzK5ElC4Kwpu/3n0ldaQ4Um5 8ioQ== X-Forwarded-Encrypted: i=1; AJvYcCWmNsi1OJqny0s8J8xx4H82+RvegRgHkzUNa5Z5K4d5jA1Lzh54S7y3pNi3Ny8jUpDRigafA4PsWeOd2Tmm0pYeuAQ= X-Gm-Message-State: AOJu0YymcJSvC9BKJJSKDR7oI1aZ9FifN2/9vXTcQG2IEfHHaQi6IJqI fm+QG3mWDIEeLK1XBzZxi588AXFQ07EASWLOBWaV/WhZMK1Fgv26+nyzB/yLmJSig+BoIP8mGnV cEL81xKHN8H4avEr+zePVMOaIUNc= X-Google-Smtp-Source: AGHT+IHXdtc9K5SUE8A5VPLgvrL9c6Lb8Si5GZnrAkoQ9bdtRJrgRvkcG9b9CD5cWW0o1zvpiJ2+gFbGUVVpEbx985I= X-Received: by 2002:a2e:9e95:0:b0:2ec:1a6:71ad with SMTP id 38308e7fff4ca-2ec0e5c661fmr9185191fa.19.1718335884522; Thu, 13 Jun 2024 20:31:24 -0700 (PDT) MIME-Version: 1.0 References: <20240412064353.133497-1-zhaoyang.huang@unisoc.com> <20240412143457.5c6c0ae8f6df0f647d7cf0be@linux-foundation.org> <2652f0c1-acc9-4288-8bca-c95ee49aa562@marcinwanat.pl> <5f989315-e380-46aa-80d1-ce8608889e5f@marcinwanat.pl> In-Reply-To: <5f989315-e380-46aa-80d1-ce8608889e5f@marcinwanat.pl> From: Zhaoyang Huang Date: Fri, 14 Jun 2024 11:31:13 +0800 Message-ID: Subject: Re: [PATCH 1/1] mm: protect xa split stuff under lruvec->lru_lock during migration To: Marcin Wanat Cc: Dave Chinner , Andrew Morton , "zhaoyang.huang" , Alex Shi , "Kirill A . Shutemov" , Hugh Dickins , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, steve.kang@unisoc.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 59F96100006 X-Stat-Signature: hhrbyu96mwk7f7orqhyezs9irm9ghun6 X-Rspam-User: X-HE-Tag: 1718335887-363053 X-HE-Meta: U2FsdGVkX1/iAxQy9Qt1KwieusNfhQZiLNI7F93h4Z1/xdHL9Rd8UxIwbGcVPa02IWGRn+YdmUC08z1hMHgG6MQFccgwdwcrGakb8wenGaeDoEtz97S9aKYF9Szu0H1k1nPS7VqZqkXF/YQUj00zWjl966+urea++D7U5F9ybflCtkFJnidUV+iUnAjV76v4iWJH0Ld+d326oIRbkzDX9ThUimW2IuSkpDFrCALcRbPM4beEYET1KuunVm0dgsdscpCw4ijAQF9byYZxcRyWq7elAvEZ/64IpNQhTrd8DKbCQdA9sO3TVnXZZevQtnKtH95dFl3IqpPf8TcxjXIYWxq32BHNznsaalXDdnqmNMTcMdH8+M9JGBmxg3jIn2Nv5hZTDeIPL2nt5fMmnRbnLPcBPZbsCohatxnaixtrN3FLEOSGsuv3njIYxM4JMzm8S6ThMIgQadYoY305GB4l2HgXNVzsl4Xqj2h2PkgqjqbEMthWy9iJa94qSoamkm/UVsyTwZP6lrCA0lXCY3ZUGhhZC8nDiuQe5p/Crh09sVdk6pZmEYIgoGjPeTsQEoZP8FrECgeNqKhw6djW7PBBG+XS0HlHR/ERxg/Cl2NiIWPqlQhgA8RypNSAHhOd6Atff2Lqo/Iy335orjL5txQ01ml75itdq/MbQoM+jOEn7Rtq/sl80RIcFX/uUqrHsxCVI9xfIJwhcLza2YkYPMEAPPG//PUY4NPD4K/ful0Ix9OMpSgtAROiGj4itglpwTtxYKAUXcOZSt5zOUWQDIFobaHXTgT1Jn0ToKXCKQ/w4Pqgl4TFzcqf8/dtKATXlSG38ctc8sUhIM3RZ1iqA6UATs3sseD4A4L+3vLdXHIJODF6ZkNN67s/XDS4+ZPD62oP5VG+dwS6K7BYfDA6h6zF19bFNBAROBaXQNi+9nudatMk7/sRqNN00V9EivpteTmaOw1+4nzHgthFBKcMosb +oeeWwRs 7BJDgJY3uNggp3el81ymaf9qOZNPqsiSnuzDoqgtE5VkwvZZhy/DNC/p5jBMARm50wsCoreqT02ZWX8ChW8VaegeUAIc/U2Cc8onFmpZMQDO9LoSmUGlkR+oQtJYjDgfGp+KMvHZtDa3lp7ww4nK0v7cOODFXlx7j936NpyCPJJhMJy6Fn7MATIIbPtrOza8rhsGv3NAw+p95nVMj2lNz+HzF0qcIuKRSftw474AFWTE2deKPqoQGEPvIOm3SjE8FtHIF0Jbt5jjYX62kKc5/DbHIdq6GSkwYYSxIX7+VyhuuBSFn71fO/3kueKP0QYSHNPniFCWDORMFEVRgOKbs/tklj/nhBypL8DmdN1d7rCQuBCpdbPXFr4rm9Roqn1m2IVFBB8/kCzSnnLWJkumsCtQ1r95IiRFTm+v9H/jNortrHnk1/mHmFgth4dFdwTcNyybwF0H6lTw2ZcoPNQp4GtQM4p11QhV+y0K9lHao1BMI2Gwf+/oDf0akhO3OGehIYgOkC9qukeyO24nWpH4yRbUIcl0kjEzAlbrI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, May 27, 2024 at 4:22=E2=80=AFPM Marcin Wanat wrote: > > On 22.05.2024 12:13, Marcin Wanat wrote: > > On 22.05.2024 07:37, Zhaoyang Huang wrote: > >> On Tue, May 21, 2024 at 11:47=E2=80=AFPM Marcin Wanat > >> wrote: > >>> > >>> On 21.05.2024 03:00, Zhaoyang Huang wrote: > >>>> On Tue, May 21, 2024 at 8:58=E2=80=AFAM Zhaoyang Huang > >>>> wrote: > >>>>> > >>>>> On Tue, May 21, 2024 at 3:42=E2=80=AFAM Marcin Wanat > >>>>> wrote: > >>>>>> > >>>>>> On 15.04.2024 03:50, Zhaoyang Huang wrote: > >>>>>> I have around 50 hosts handling high I/O (each with 20Gbps+ uplink= s > >>>>>> and multiple NVMe drives), running RockyLinux 8/9. The stock RHEL > >>>>>> kernel 8/9 is NOT affected, and the long-term kernel 5.15.X is NOT > >>>>>> affected. > >>>>>> However, with long-term kernels 6.1.XX and 6.6.XX, > >>>>>> (tested at least 10 different versions), this lockup always appear= s > >>>>>> after 2-30 days, similar to the report in the original thread. > >>>>>> The more load (for example, copying a lot of local files while > >>>>>> serving 20Gbps traffic), the higher the chance that the bug will > >>>>>> appear. > >>>>>> > >>>>>> I haven't been able to reproduce this during synthetic tests, > >>>>>> but it always occurs in production on 6.1.X and 6.6.X within 2-30 > >>>>>> days. > >>>>>> If anyone can provide a patch, I can test it on multiple machines > >>>>>> over the next few days. > >>>>> Could you please try this one which could be applied on 6.6 > >>>>> directly. Thank you! > >>>> URL: https://lore.kernel.org/linux-mm/20240412064353.133497-1- > >>>> zhaoyang.huang@unisoc.com/ > >>>> > >>> > >>> Unfortunately, I am unable to cleanly apply this patch against the > >>> latest 6.6.31 > >> Please try below one which works on my v6.6 based android. Thank you > >> for your test in advance :D > >> > >> mm/huge_memory.c | 22 ++++++++++++++-------- > >> 1 file changed, 14 insertions(+), 8 deletions(-) > >> > >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c > > > > I have compiled 6.6.31 with this patch and will test it on multiple > > machines over the next 30 days. I will provide an update after 30 days > > if everything is fine or sooner if any of the hosts experience the same > > soft lockup again. > > > > First server with 6.6.31 and this patch hang today. Soft lockup changed > to hard lockup: > > [26887.389623] watchdog: Watchdog detected hard LOCKUP on cpu 21 > [26887.389626] Modules linked in: nft_limit xt_limit xt_hashlimit > ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_connlimit > nf_conncount tls xt_set ip_set_hash_net ip_set xt_CT xt_conntrack > nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables > nfnetlink rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency > intel_uncore_frequency_common isst_if_common skx_edac nfit libnvdimm > x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass > rapl intel_cstate ipmi_ssif irdma ext4 mbcache ice iTCO_wdt jbd2 mgag200 > intel_pmc_bxt iTCO_vendor_support ib_uverbs i2c_algo_bit acpi_ipmi > intel_uncore mei_me drm_shmem_helper pcspkr ib_core i2c_i801 ipmi_si > drm_kms_helper mei lpc_ich i2c_smbus ioatdma intel_pch_thermal > ipmi_devintf ipmi_msghandler acpi_pad acpi_power_meter joydev tcp_bbr > drm fuse xfs libcrc32c sd_mod t10_pi sg crct10dif_pclmul crc32_pclmul > crc32c_intel ixgbe polyval_clmulni ahci polyval_generic libahci mdio > i40e libata megaraid_sas dca ghash_clmulni_intel wmi > [26887.389682] CPU: 21 PID: 264 Comm: kswapd0 Kdump: loaded Tainted: G > W 6.6.31.el9 #3 > [26887.389685] Hardware name: FUJITSU PRIMERGY RX2540 M4/D3384-A1, BIOS > V5.0.0.12 R1.22.0 for D3384-A1x 06/04/2018 > [26887.389687] RIP: 0010:native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389696] Code: 08 0f 92 c2 8b 45 00 0f b6 d2 c1 e2 08 30 e4 09 d0 > a9 00 01 ff ff 0f 85 ea 01 00 00 85 c0 74 12 0f b6 45 00 84 c0 74 0a f3 > 90 <0f> b6 45 00 84 c0 75 f6 b8 01 00 00 00 66 89 45 00 5b 5d 41 5c 41 > [26887.389698] RSP: 0018:ffffb3e587a87a20 EFLAGS: 00000002 > [26887.389700] RAX: 0000000000000001 RBX: ffff9ad6c6f67050 RCX: > 0000000000000000 > [26887.389701] RDX: 0000000000000000 RSI: 0000000000000000 RDI: > ffff9ad6c6f67050 > [26887.389703] RBP: ffff9ad6c6f67050 R08: 0000000000000000 R09: > 0000000000000067 > [26887.389704] R10: 0000000000000000 R11: 0000000000000000 R12: > 0000000000000046 > [26887.389705] R13: 0000000000000200 R14: 0000000000000000 R15: > ffffe1138aa98000 > [26887.389707] FS: 0000000000000000(0000) GS:ffff9ade20340000(0000) > knlGS:0000000000000000 > [26887.389708] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [26887.389710] CR2: 000000002912809b CR3: 000000064401e003 CR4: > 00000000007706e0 > [26887.389711] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > [26887.389712] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: > 0000000000000400 > [26887.389713] PKRU: 55555554 > [26887.389714] Call Trace: > [26887.389717] > [26887.389720] ? watchdog_hardlockup_check+0xac/0x150 > [26887.389725] ? __perf_event_overflow+0x102/0x1d0 > [26887.389729] ? handle_pmi_common+0x189/0x3e0 > [26887.389735] ? set_pte_vaddr_p4d+0x4a/0x60 > [26887.389738] ? flush_tlb_one_kernel+0xa/0x20 > [26887.389742] ? native_set_fixmap+0x65/0x80 > [26887.389745] ? ghes_copy_tofrom_phys+0x75/0x110 > [26887.389751] ? __ghes_peek_estatus.isra.0+0x49/0xb0 > [26887.389755] ? intel_pmu_handle_irq+0x10b/0x230 > [26887.389756] ? perf_event_nmi_handler+0x28/0x50 > [26887.389759] ? nmi_handle+0x58/0x150 > [26887.389764] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389768] ? default_do_nmi+0x6b/0x170 > [26887.389770] ? exc_nmi+0x12c/0x1a0 > [26887.389772] ? end_repeat_nmi+0x16/0x1f > [26887.389777] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389780] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389784] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389787] > [26887.389788] > [26887.389789] __raw_spin_lock_irqsave+0x3d/0x50 > [26887.389793] folio_lruvec_lock_irqsave+0x5e/0x90 > [26887.389798] __page_cache_release+0x68/0x230 > [26887.389801] ? remove_migration_ptes+0x5c/0x80 > [26887.389807] __folio_put+0x24/0x60 > [26887.389808] __split_huge_page+0x368/0x520 > [26887.389812] split_huge_page_to_list+0x4b3/0x570 > [26887.389816] deferred_split_scan+0x1c8/0x290 > [26887.389819] do_shrink_slab+0x12f/0x2d0 > [26887.389824] shrink_slab_memcg+0x133/0x1d0 > [26887.389829] shrink_node_memcgs+0x18e/0x1d0 > [26887.389832] shrink_node+0xa7/0x370 > [26887.389836] balance_pgdat+0x332/0x6f0 > [26887.389842] kswapd+0xf0/0x190 > [26887.389845] ? balance_pgdat+0x6f0/0x6f0 > [26887.389848] kthread+0xee/0x120 > [26887.389851] ? kthread_complete_and_exit+0x20/0x20 > [26887.389853] ret_from_fork+0x2d/0x50 > [26887.389857] ? kthread_complete_and_exit+0x20/0x20 > [26887.389859] ret_from_fork_asm+0x11/0x20 > [26887.389864] > [26887.389865] Kernel panic - not syncing: Hard LOCKUP > [26887.389867] CPU: 21 PID: 264 Comm: kswapd0 Kdump: loaded Tainted: G > W 6.6.31.el9 #3 > [26887.389869] Hardware name: FUJITSU PRIMERGY RX2540 M4/D3384-A1, BIOS > V5.0.0.12 R1.22.0 for D3384-A1x 06/04/2018 > [26887.389870] Call Trace: > [26887.389871] > [26887.389872] dump_stack_lvl+0x44/0x60 > [26887.389877] panic+0x241/0x330 > [26887.389881] nmi_panic+0x2f/0x40 > [26887.389883] watchdog_hardlockup_check+0x119/0x150 > [26887.389886] __perf_event_overflow+0x102/0x1d0 > [26887.389889] handle_pmi_common+0x189/0x3e0 > [26887.389893] ? set_pte_vaddr_p4d+0x4a/0x60 > [26887.389896] ? flush_tlb_one_kernel+0xa/0x20 > [26887.389899] ? native_set_fixmap+0x65/0x80 > [26887.389902] ? ghes_copy_tofrom_phys+0x75/0x110 > [26887.389906] ? __ghes_peek_estatus.isra.0+0x49/0xb0 > [26887.389909] intel_pmu_handle_irq+0x10b/0x230 > [26887.389911] perf_event_nmi_handler+0x28/0x50 > [26887.389913] nmi_handle+0x58/0x150 > [26887.389916] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389920] default_do_nmi+0x6b/0x170 > [26887.389922] exc_nmi+0x12c/0x1a0 > [26887.389923] end_repeat_nmi+0x16/0x1f > [26887.389926] RIP: 0010:native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389930] Code: 08 0f 92 c2 8b 45 00 0f b6 d2 c1 e2 08 30 e4 09 d0 > a9 00 01 ff ff 0f 85 ea 01 00 00 85 c0 74 12 0f b6 45 00 84 c0 74 0a f3 > 90 <0f> b6 45 00 84 c0 75 f6 b8 01 00 00 00 66 89 45 00 5b 5d 41 5c 41 > [26887.389931] RSP: 0018:ffffb3e587a87a20 EFLAGS: 00000002 > [26887.389933] RAX: 0000000000000001 RBX: ffff9ad6c6f67050 RCX: > 0000000000000000 > [26887.389934] RDX: 0000000000000000 RSI: 0000000000000000 RDI: > ffff9ad6c6f67050 > [26887.389935] RBP: ffff9ad6c6f67050 R08: 0000000000000000 R09: > 0000000000000067 > [26887.389936] R10: 0000000000000000 R11: 0000000000000000 R12: > 0000000000000046 > [26887.389937] R13: 0000000000000200 R14: 0000000000000000 R15: > ffffe1138aa98000 > [26887.389940] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389943] ? native_queued_spin_lock_slowpath+0x6e/0x2c0 > [26887.389946] > [26887.389947] > [26887.389947] __raw_spin_lock_irqsave+0x3d/0x50 > [26887.389950] folio_lruvec_lock_irqsave+0x5e/0x90 > [26887.389953] __page_cache_release+0x68/0x230 > [26887.389955] ? remove_migration_ptes+0x5c/0x80 > [26887.389958] __folio_put+0x24/0x60 > [26887.389960] __split_huge_page+0x368/0x520 > [26887.389963] split_huge_page_to_list+0x4b3/0x570 > [26887.389967] deferred_split_scan+0x1c8/0x290 > [26887.389971] do_shrink_slab+0x12f/0x2d0 > [26887.389974] shrink_slab_memcg+0x133/0x1d0 > [26887.389978] shrink_node_memcgs+0x18e/0x1d0 > [26887.389982] shrink_node+0xa7/0x370 > [26887.389985] balance_pgdat+0x332/0x6f0 > [26887.389991] kswapd+0xf0/0x190 > [26887.389994] ? balance_pgdat+0x6f0/0x6f0 > [26887.389997] kthread+0xee/0x120 > [26887.389998] ? kthread_complete_and_exit+0x20/0x20 > [26887.390000] ret_from_fork+0x2d/0x50 > [26887.390003] ? kthread_complete_and_exit+0x20/0x20 > [26887.390004] ret_from_fork_asm+0x11/0x20 > [26887.390009] > Hi Marcin. Sorry for this late reply. I think the above hard lockup is caused by a recursive deadlock as [1] and has been fixed by [2] which is on v6.8+. I would like to know if your regression test is still going on? Thanks very much. [1] static void __split_huge_page(struct page *page, struct list_head *list, pgoff_t end, unsigned int new_order) { /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ lruvec =3D folio_lruvec_lock(folio); //take lruvec_lock here 1st for (i =3D nr - new_nr; i >=3D new_nr; i -=3D new_nr) { __split_huge_page_tail(folio, i, lruvec, list, new_order); /* Some pages can be beyond EOF: drop them from page cache = */ if (head[i].index >=3D end) { folio_put(tail); __page_cache_release folio_lruvec_lock_irqsave //hanged by 2nd try [2] commit f1ee018baee9f4e724e08859c2559323be768be3 Author: Matthew Wilcox (Oracle) Date: Tue Feb 27 17:42:42 2024 +0000 mm: use __page_cache_release() in folios_put() Pass a pointer to the lruvec so we can take advantage of the folio_lruvec_relock_irqsave(). Adjust the calling convention of folio_lruvec_relock_irqsave() to suit and add a page_cache_release() wrapper.