From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05C17C282C1 for ; Fri, 28 Feb 2025 01:52:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56EF36B0085; Thu, 27 Feb 2025 20:52:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 51EAA280001; Thu, 27 Feb 2025 20:52:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E63E6B008A; Thu, 27 Feb 2025 20:52:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 21F296B0085 for ; Thu, 27 Feb 2025 20:52:12 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C998AA4ACD for ; Fri, 28 Feb 2025 01:52:11 +0000 (UTC) X-FDA: 83167678062.22.2860AC2 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf17.hostedemail.com (Postfix) with ESMTP id 655A640010 for ; Fri, 28 Feb 2025 01:52:08 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of liushixin2@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=liushixin2@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740707529; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WI2R4q2Ukq7qcToN0Yzl/aHlIWq5cow75k8NEU1Kbtk=; b=CkBWf4VIjZHweRLmOD6RoSBc3k0O5K4iWsxX15uMlHa7HIw4rF6VVA6GmNhTPopjzCZO8W kdOQMl40POAwiBRMyQTrBKBvW1C1NM/QZ2V8YFyrF1pKQfpYmjMXpYDnTUOlZzeWgiZ7wV KhKPGBPvlWEuuzbLQAK3P9qFQmcCrgM= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of liushixin2@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=liushixin2@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740707529; a=rsa-sha256; cv=none; b=jNeVISZoM5jvG+YYiF1FG4NUU0zzeFr4fuznEZUf/FOAmL6l+Z0ftIMprQYDuN8dUIo/QM 1N7GmF2h3CJ3hk4lSqQ/J0Hu+1WTzGxyaJJIT6IQPM/+QYBrbZm7YrlNefq4yVAoHq41pU vmnyISOU0NPmwqO/e/VfngemI7/s1bo= Received: from mail.maildlp.com (unknown [172.19.163.48]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Z3rhJ6fbczvWqh; Fri, 28 Feb 2025 09:48:16 +0800 (CST) Received: from kwepemg200013.china.huawei.com (unknown [7.202.181.64]) by mail.maildlp.com (Postfix) with ESMTPS id 0D395180087; Fri, 28 Feb 2025 09:52:02 +0800 (CST) Received: from [10.174.179.24] (10.174.179.24) by kwepemg200013.china.huawei.com (7.202.181.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 28 Feb 2025 09:52:00 +0800 Subject: Re: Softlockup when test shmem swapout-swapin and compaction To: Zi Yan References: <28546fb4-5210-bf75-16d6-43e1f8646080@huawei.com> <696E1819-D7E3-42BD-B3F0-8B3AC67A8ADB@nvidia.com> CC: Baolin Wang , , Linux Kernel Mailing List , Barry Song , David Hildenbrand , Hugh Dickins , Kefeng Wang , Lance Yang , Matthew Wilcox , Ryan Roberts , Andrew Morton From: Liu Shixin Message-ID: Date: Fri, 28 Feb 2025 09:52:00 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <696E1819-D7E3-42BD-B3F0-8B3AC67A8ADB@nvidia.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.179.24] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemg200013.china.huawei.com (7.202.181.64) X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 655A640010 X-Stat-Signature: gg9m1h6zmf3kjqp1dfbdxqxfb69agsh7 X-HE-Tag: 1740707527-709090 X-HE-Meta: U2FsdGVkX1/HvWbI9xWMDGLxycOjThWv/WqrSJzrLe6BR0/ZKaRM1pSwI/L5e85xW86gZd/KAYWFpAAxf4jP/UeOu2ZzpQ0tVXp5EAY92CbE8eC75g+m6D5x7V/vTW5lQq8m0rSVYFr+EO1s+CcTXrDEy8IXQEUn35Rro4m31D8tGfGTvxZUfuykVu7GQ1n370oAaKF5hztP1ik8pkFnWDfwnR30CHnTbYRpndM6Ha+tKSHbCk+yGqV08K6zGmlBjPPqLF8dLzRxhF2rVee2/QEUIUQIholE/gPdcYC9Nutb4EZ1VnU0QhVBjt4CrIUe4Tatoy0oLQQ2AmInqaOzgE3tOUUiKYDoSj1IqjvcF/z0Vfpq6JuB+0m8G0ZMhBxUSGRkQOhl+7zgbO3aCp6aF300unbInODeenhGw6u8XA1CbnSav6IPW5tFADk7I0+DBN8xDpASe8WIaYH2eCZvvHn1VgBOUhY3/JFRSfP8F6o03Fl3R90WjAcTdxwbBft4MO4j8cTsevDnQ3T9AEOKzzk3AaqUm42rZL7anLTKl65/KXWvDP2fsVJ5rgShebxPOcsI1UIw2Bj0jlNewIYBsEQe35a1CpWa15KmCbOemybB7L7PZ/2ozDAMWTBHTbjenbbAGk7QP+0Etgy6nYQ2zgLQ3bPb0xvuDPGjfGQcoYJXLcb1PR7ZhLPOSzELKyIRWoiNp3sC8kI7llRNS3AWKn+YgcwvRA1f8R5JTANW219Vda+qHhu1MrSCrA7cg6c+PULjCTt2/hsXk85aWCT0fOenlUsh+dQBwYdFu5iRsHmjVSomskByB1rnHDnqa+FDvEZSdsgi3VNlSCtAML2Zn9w00YAfQ9PM8cFCX5ujiR9kyv/MyRU8GfGu9gwVf6WCX6If0xa0epm6UztJCNitK1KhFXmFkZEC40aIXhPPzhkBmNl8hoo94N6g9Is0V0T8sBvCo3mXPbv8VsAidKL GhJ6ChH4 fREH+sY3PPZ5bx8S5zIg0Sc+pjmY+X0tyXdNCHWpQaRvRDPaNq2PPw5aX1Qfo3iDOieZcrggCtWMYMTXIydRtaMSfx7wyw3XDqSlT9tTxlSTvhdQydVUJfOIjDl77JTbI6qZLOxlQqxA3O4lZVBXwX61X9UXBCadzMRXvEhFXv3L5/XE5bnN1TAoGcsSDbsWK3/t7JtsRUYeoxic= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025/2/28 7:43, Zi Yan wrote: > On 27 Feb 2025, at 2:04, Liu Shixin wrote: > >> On 2025/2/26 15:22, Baolin Wang wrote: >>> Add Zi. >>> >>> On 2025/2/26 15:03, Liu Shixin wrote: >>>> Hi all, >>>> >>>> I found a softlockup when testing shmem large folio swapout-swapin and compaction: >>>> >>>> watchdog: BUG: soft lockup - CPU#30 stuck for 179s! [folio_swap:4714] >>>> Modules linked in: zram xt_MASQUERADE nf_conntrack_netlink nfnetlink iptable_nat xt_addrtype iptable_filter ip_tantel_rapl_msr intel_rapl_common intel_uncore_frequency_common skx_edac_common nfit libnvdimm kvm_intel kvm rapl cixt4 mbcache jbd2 sr_mod cdrom ata_generic ata_piix virtio_net net_failover ghash_clmulni_intel libata sha512_ssse3 >>>> CPU: 30 UID: 0 PID: 4714 Comm: folio_swap Kdump: loaded Tainted: G L 6.14.0-rc4-next-20250225+ #2 >>>> Tainted: [L]=SOFTLOCKUP >>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014 >>>> RIP: 0010:xas_load+0x5d/0xc0 >>>> Code: 08 48 d3 ea 83 e2 3f 89 d0 48 83 c0 04 48 8b 44 c6 08 48 89 73 18 48 89 c1 83 e1 03 48 83 f9 02 75 08 48 3d >>>> RSP: 0000:ffffadf142f1ba60 EFLAGS: 00000293 >>>> RAX: ffffe524cc4f6700 RBX: ffffadf142f1ba90 RCX: 0000000000000000 >>>> RDX: 0000000000000011 RSI: ffff9a3e058acb68 RDI: ffffadf142f1ba90 >>>> RBP: fffffffffffffffe R08: ffffadf142f1bb50 R09: 0000000000000392 >>>> R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000011 >>>> R13: ffffadf142f1bb48 R14: ffff9a3e04e9c588 R15: 0000000000000000 >>>> FS: 00007fd957666740(0000) GS:ffff9a41ac0e5000(0000) knlGS:0000000000000000 >>>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>>> CR2: 00007fd922860000 CR3: 000000025c360001 CR4: 0000000000772ef0 >>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 >>>> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 >>>> PKRU: 55555554 >>>> Call Trace: >>>> >>>> ? watchdog_timer_fn+0x1c9/0x250 >>>> ? __pfx_watchdog_timer_fn+0x10/0x10 >>>> ? __hrtimer_run_queues+0x10e/0x250 >>>> ? hrtimer_interrupt+0xfb/0x240 >>>> ? __sysvec_apic_timer_interrupt+0x4e/0xe0 >>>> ? sysvec_apic_timer_interrupt+0x68/0x90 >>>> >>>> >>>> ? asm_sysvec_apic_timer_interrupt+0x16/0x20 >>>> ? xas_load+0x5d/0xc0 >>>> xas_find+0x153/0x1a0 >>>> find_get_entries+0x73/0x280 >>>> shmem_undo_range+0x1fc/0x640 >>>> shmem_evict_inode+0x109/0x270 >>>> evict+0x107/0x240 >>>> ? fsnotify_destroy_marks+0x25/0x180 >>>> ? _atomic_dec_and_lock+0x35/0x50 >>>> __dentry_kill+0x71/0x190 >>>> dput+0xd1/0x190 >>>> __fput+0x128/0x2a0 >>>> task_work_run+0x57/0x90 >>>> syscall_exit_to_user_mode+0x1cb/0x1e0 >>>> do_syscall_64+0x67/0x170 >>>> entry_SYSCALL_64_after_hwframe+0x76/0x7e >>>> RIP: 0033:0x7fd95776eb8b >>>> >>>> If CONFIG_DEBUG_VM is enabled, we will meet VM_BUG_ON_FOLIO(!folio_test_locked(folio)) in >>>> shmem_add_to_page_cache() too. It seems that the problem is related to memory migration or >>>> compaction which is necessary for reproduction, although without a clear why. >>>> >>>> To reproduce the problem, we need firstly a zram device as swap backend, and then run the >>>> reproduction program. The reproduction program consists of three parts: >>>> 1. A process constantly changes the status of shmem large folio by these interfaces: >>>> /sys/kernel/mm/transparent_hugepage/hugepages-/shmem_enabled >>>> 2. A process constantly echo 1 > /proc/sys/vm/compact_memory >>>> 3. A process constantly alloc/free/swapout/swapin shmem large folios. >>>> >>>> I'm not sure whether the first process is necessary but the second and third are. In addition, >>>> I tried hacking to modify compaction_alloc to return NULL, and the problem disappeared, >>>> so I guess the problem is in migration. >>>> >>>> The problem is different with https://lore.kernel.org/all/1738717785.im3r5g2vxc.none@localhost/ >>>> since I have confirmed this porblem still existed after merge the fixed patch. >>> Could you check if your version includes Zi's fix[1]? Not sure if it's related to the shmem large folio split. >>> >>> [1] https://lore.kernel.org/all/AF487A7A-F685-485D-8D74-756C843D6F0A@nvidia.com/ >>> . >>> >> Already include this patch when test. > Hi Shixin, > > Can you try the diff below? It fixed my local repro. > > The issue is that after Baolin’s patch, shmem folios now use high-order > entry, so the migration code should not update multiple xarray slots. Looks reasonable. I'll try it, thanks. > > Hi Baolin, > > Is your patch affecting anonymous swapping out? If yes, we can remove > the for loop of updating xarray in __folio_migrate_mapping(). > > diff --git a/mm/migrate.c b/mm/migrate.c > index 365c6daa8d1b..be77932596b3 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -44,6 +44,7 @@ > #include > #include > #include > +#include > > #include > > @@ -524,7 +525,11 @@ static int __folio_migrate_mapping(struct address_space *mapping, > folio_set_swapcache(newfolio); > newfolio->private = folio_get_private(folio); > } > - entries = nr; > + /* shmem now uses high-order entry */ > + if (folio->mapping && shmem_mapping(folio->mapping)) > + entries = 1; > + else > + entries = nr; > } else { > VM_BUG_ON_FOLIO(folio_test_swapcache(folio), folio); > entries = 1; > > > Best Regards, > Yan, Zi > . >