From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C4A2C5AD49 for ; Sun, 8 Jun 2025 19:27:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D5186B007B; Sun, 8 Jun 2025 15:27:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 586386B0088; Sun, 8 Jun 2025 15:27:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 475D06B0089; Sun, 8 Jun 2025 15:27:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 29E276B007B for ; Sun, 8 Jun 2025 15:27:25 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 98537B5BA3 for ; Sun, 8 Jun 2025 19:27:24 +0000 (UTC) X-FDA: 83533217208.24.B50389F Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf06.hostedemail.com (Postfix) with ESMTP id ACE47180002 for ; Sun, 8 Jun 2025 19:27:22 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HYxsx7h4; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749410842; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Umc3XJ8UA6r3DJfGKEnxONQj6IfWcR7nebw8X04cSMk=; b=4O70O4naSWT0erDcpp6QjJ73NV+8zxiDAyCmtmN6H+/z4szt6uYNUpAFuZ4Z2v8qqec98O Ztc/4a/KBmmCR+ZUr9WRQwrf2K17dYikQ8iCic+rSEeN0H1Y+4tdzyT/eIhTLQi/C0GFHg SOLx87Ay2Wl49ohnXaZUsSDxV6eki7o= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HYxsx7h4; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749410842; a=rsa-sha256; cv=none; b=L2mMPOpzA+h5extY1v8HmpSZYs52lQsf2KVMVjgRR5I1Z0AefNizTuzXE6Q05Qfyy9WUCV L9GyrKTsz/HPpXyL9RzWNyCiU9sUYx1Xkn2iYrACYIdB8iWfHhSsyrw2PtRVDQlHldGvde rCwyLAa23wN17jOu8Fbm3+/P5FJYyxE= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-747fba9f962so3045159b3a.0 for ; Sun, 08 Jun 2025 12:27:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749410841; x=1750015641; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=Umc3XJ8UA6r3DJfGKEnxONQj6IfWcR7nebw8X04cSMk=; b=HYxsx7h4K/R8TQZ8bZVqcQ3J25zeZ1uIIyYRt4Id/bRUZKMUvVgDbdXypogG+pCHHT hgzLgHN8TyC7qr8iNXNiIZ/ANlFAy0fyhy2sD8yFN5W1HAnm5PEYMjZ75w6rIGa1vKmw FEHIkRbKs7/hvc7joUqTK+JDhgfuVORrVSN+o16ljI3xEX+1jabMqJ/+GH9FiJKXAyVd rQKfwJ66VUfQD2yPFM6/tzOPjpz+61AspE6RmI2mXe8imAnIwg4yXf9bFWDQgupfTUtj Yerd9IYjUW7lpBOZcjVY0PNDFf8dNXq1afVRBkfg9kpmz4ZtLW/u3NB+QIvvhQQCBGeI WSNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749410841; x=1750015641; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Umc3XJ8UA6r3DJfGKEnxONQj6IfWcR7nebw8X04cSMk=; b=O95me3iLpldKRwWFoyyP/BI0lgSUV0f3ORPfJSFBXQrbaQ+njpmeC4itLQk58EaBWm lW/hjazTUVGPdUKMmqpG/umlzJnFNCueNEtqn4ydaN0C4LGTC9/Bsf0L4vHTMZFBxtZX Pv9MNq81qWZ8gVZX+fyRECEKAdX1/EyNRy1jEq8CpLZQO6fEuHP2DEx8Y52n4JuxPOZN 6JO4W36RZx8GUVrb5V6SMej+oN7TjgiiXTMmJj927DGR0Gv4AShWcW3jitI/B7hZQ7Yj piW9zYzgmEK4FdoxjuZEnLMzki8/P1NN/bwfiYJ/lMN7MTx1A5Opiu6NC96f9s6uYtgP nfOA== X-Gm-Message-State: AOJu0YzH5/aOHOo1MgG/oG3XgW9h8Svf3qpMBVMtwUSnNG1kLEV94Ye5 zbtstFyJS7TWpcVXIiSgoDkZQYoBMsQzCCIDztgV353LVk4XtEAS30J9Qgj2Daqn X-Gm-Gg: ASbGncu0L7gdIZM1BO7JhT+4VJSFD7eJTUqiHPwtJqf15wldX26kDmJykSkmmYi4Wjl F0zm1rIRXCNHVyTl67fTKHEKYTv0mRUrs7rQy2hEWoNwvc1bKEezD2jwk8XMWFLq24KSV+Rrqfc fLTDoCgX68b/ZBouigUgkrSDfH0mT4hmAFwMOEY1sS2EPTV9FZepnkow/vJeWNHeOTugS5AtbDH 0CmPTrr0fMfbiGakomHoqKAl+UxntLmEzn9Edwf3ra72ArYpmJ1To7s+C9QhdjBJuWkUuKx1zdH Dojpc4fUdv4TCLQ+0B2Kwqhhutmd3QBhI5nLs128vI0RF115KythTIrgR7gbXMeqzLUr6C2KMc1 KAa0/ X-Google-Smtp-Source: AGHT+IFrR29IeMa9z8t6h/YKdzb/OpTgDOF5+AG5muZffHW8pGxLe5Jf+1ISSr+3bItufYcqY48ARw== X-Received: by 2002:a05:6a00:b42:b0:736:4c3d:2cba with SMTP id d2e1a72fcca58-74839ddbc39mr7007399b3a.9.1749410840780; Sun, 08 Jun 2025 12:27:20 -0700 (PDT) Received: from KASONG-MC4.tencent.com ([1.203.117.72]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7482af7ad57sm4437091b3a.65.2025.06.08.12.27.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Jun 2025 12:27:20 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Kemeng Shi , Chris Li , Nhat Pham , Baoquan He , Barry Song , Usama Arif , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH] mm/shmem, swap: fix softlockup with mTHP swapin Date: Mon, 9 Jun 2025 03:27:13 +0800 Message-ID: <20250608192713.95875-1-ryncsn@gmail.com> X-Mailer: git-send-email 2.49.0 Reply-To: Kairui Song MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Stat-Signature: 94ju4owsts51o5qt6u7b3dmh8e1tjeed X-Rspamd-Queue-Id: ACE47180002 X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1749410842-684061 X-HE-Meta: U2FsdGVkX1/V72wb1lbPjyJ5DsbOjZqvkAfEpMaHSqqCy/hkSilc/8CDx+PtgU7trqr9hXRz3qF+PDRfWlwagIkQ5Ezieca6/RmewMhc0CSQGr+jAHOXSiwdCqb6fuAq1SIWqEFqFeEHD8d+7sBcwC7dCIl2T0BO9l6hhZ0lkBQU5EpPAiv6z/Cvc752mmGjvy/HGoRoZIsZmIZg8ttqwT8dudd/4v1DoiW08VhWqRGlZfYpreerpLMuLolUodUxUhPzL36sth4Wtvk0Fhn//MG1RBfsKEcpGWvnzSkcwLx39McKqnvgdwPetcgjn5/8UswBnMsuVlenzCaE53zDdqf9uoXFKbvSVcdUAINofVi5OpEfCq/neiIca2aSyAYk4Y6zVQqMi30+nTtLs1I5ufsHLomUAhY+HwfB/eLVhUqP57Ol2DzR5j5TQMC25/0z3U4biRerH8EE3XHhG/OcaRkgZtdfGC8j0lZIr9lmfxeCpGpHkPTkmXSUDJRrlKimeQIv3KHnd0l6ppxDacSr3kZDOApk52l4FbTP3C3C9WxTZaLqrYErOvDV75k56qgiixYa2poG/fRPeEWYzEFI3FvC3vodAC92WWUYWoxfDMicKmmhIBna8oRu8QnE5W+cbEIuvFCWfmnn1bxq9EGXZJVgBq0ftBkp2KMHytb1GUm0NisW4CSleZBKJPaAxolHZU0IwMj4+jqhdcTu4M6p6sBtcXnZFWOU41jOouW4I91yxJ2Sh7KejoUITCvAkpcsBK5TnLwKBCzJgm1qtJBPYGj56Jq7hJt1zJtGPEsZUfV8a5jE/oPLCe4mYMgYeJ9MqEYVwePr2jAEcCRdZpbUHLQA689te8T9mpt7P6HrjlzdGJnY+Wsx8U92GBqp16MLDTRruB/WFxKAoynIk34j215LemvHYB7WSnF4XADqG6kmlzJDvvAeM2/GhH9zp1JYyHuvJpVhYNt+6EGpd28 zStJh55U ZXGzInsLMe/63yt3JDlo5NuA2+VJj7lRuyOsp4yPEs/J5mya0VCfRJQkM9z8nAtu7ItKmhvE3b+V98KW/lH7jf2bgAmUsKs/uBr7YOZCkHGgHrK28x8xr/Yqp7Kw8CITtDHbBfq6eWicUmTb71ENdLsNmTdpfr0eazyjmyJGTC0/1WrpvXXgYV1apsOPyTUvJaZlxjqHrsYQbrLs6cZpd8oQV87aKUSXAhzkP/8UZPKuMq+U9ZFjaq5pBu0wSZvQU0Tzav92jtdQpIWWtf6daZNjXYjp7vKdjMSRZS5h/+KNgqHct4fBTpIjIe4I1Yx5np1jpqmSIbx5UgKShwf0qcY/Scrs6dfwKjsfJgfXjE645vwFB1dcOVOOc0V3ycVRIGan6e0TlKc3dyQwlOhUEmchNL3OrLusvs63uIod3reldnoPM3Sk90H+L9Zdb7oy6uRjRqVAuGWow6F/GBbVDndgoiVevQtr7B3SuKg/u7ZErqSet8KWEOpZbvXbd0fq4CAUQ2zSLgctVWV3jXUxchY+ohh84BukKx5u3yQLZX3vP5iw6nPS3lvJmRChEnW4gYrxe98V8x6iBBmSzJVCjVO4ZKKhnePLnOhJssY5fdKD0e8RnoPcTbEL6TB5gIuzvKholvLMfR1pbLvnmTr2BtxAxgw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Following softlockup can be easily reproduced on my test machine with: echo always > /sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled swapon /dev/zram0 # zram0 is a 48G swap device mkdir -p /sys/fs/cgroup/memory/test echo 1G > /sys/fs/cgroup/test/memory.max echo $BASHPID > /sys/fs/cgroup/test/cgroup.procs while true; do dd if=/dev/zero of=/tmp/test.img bs=1M count=5120 cat /tmp/test.img > /dev/null rm /tmp/test.img done Then after a while: watchdog: BUG: soft lockup - CPU#0 stuck for 763s! [cat:5787] Modules linked in: zram virtiofs CPU: 0 UID: 0 PID: 5787 Comm: cat Kdump: loaded Tainted: G L 6.15.0.orig-gf3021d9246bc-dirty #118 PREEMPT(voluntary)ยท Tainted: [L]=SOFTLOCKUP Hardware name: Red Hat KVM/RHEL-AV, BIOS 0.0.0 02/06/2015 RIP: 0010:mpol_shared_policy_lookup+0xd/0x70 Code: e9 b8 b4 ff ff 31 c0 c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f 44 00 00 41 54 55 53 <48> 8b 1f 48 85 db 74 41 4c 8d 67 08 48 89 fb 48 89 f5 4c 89 e7 e8 RSP: 0018:ffffc90002b1fc28 EFLAGS: 00000202 RAX: 00000000001c20ca RBX: 0000000000724e1e RCX: 0000000000000001 RDX: ffff888118e214c8 RSI: 0000000000057d42 RDI: ffff888118e21518 RBP: 000000000002bec8 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000bf4 R11: 0000000000000000 R12: 0000000000000001 R13: 00000000001c20ca R14: 00000000001c20ca R15: 0000000000000000 FS: 00007f03f995c740(0000) GS:ffff88a07ad9a000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f03f98f1000 CR3: 0000000144626004 CR4: 0000000000770eb0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: shmem_alloc_folio+0x31/0xc0 shmem_swapin_folio+0x309/0xcf0 ? filemap_get_entry+0x117/0x1e0 ? xas_load+0xd/0xb0 ? filemap_get_entry+0x101/0x1e0 shmem_get_folio_gfp+0x2ed/0x5b0 shmem_file_read_iter+0x7f/0x2e0 vfs_read+0x252/0x330 ksys_read+0x68/0xf0 do_syscall_64+0x4c/0x1c0 entry_SYSCALL_64_after_hwframe+0x76/0x7e RIP: 0033:0x7f03f9a46991 Code: 00 48 8b 15 81 14 10 00 f7 d8 64 89 02 b8 ff ff ff ff eb bd e8 20 ad 01 00 f3 0f 1e fa 80 3d 35 97 10 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 4f c3 66 0f 1f 44 00 00 55 48 89 e5 48 83 ec RSP: 002b:00007fff3c52bd28 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: ffffffffffffffda RBX: 0000000000040000 RCX: 00007f03f9a46991 RDX: 0000000000040000 RSI: 00007f03f98ba000 RDI: 0000000000000003 RBP: 00007fff3c52bd50 R08: 0000000000000000 R09: 00007f03f9b9a380 R10: 0000000000000022 R11: 0000000000000246 R12: 0000000000040000 R13: 00007f03f98ba000 R14: 0000000000000003 R15: 0000000000000000 The reason is simple, readahead brought some order 0 folio in swap cache, and the swapin mTHP folio being allocated is in confict with it, so swapcache_prepare fails and causes shmem_swap_alloc_folio to return -EEXIST, and shmem simply retries again and again causing this loop. Fix it by applying a similar fix for anon mTHP swapin. The performance change is very slight, time of swapin 10g zero folios (test for 12 times): Before: 2.49s After: 2.52s Fixes: 1dd44c0af4fa1 ("mm: shmem: skip swapcache for swapin of synchronous swap device") Signed-off-by: Kairui Song --- I found this issue while doing a performance comparing of mm-new with swap table series [1] on top of mm-new. This issue no longer exists if the swap table series is applied, because it elimated both SWAP_HAS_CACHE and SWP_SYNCHRONOUS_IO swapin completely while improving the performance and simplify the code, and the race swapin is solved differently by then. (The zero map fix might still need to stay for a while, but could be optimized too later with swap table). It will be good if the swap table series could get reviewed and merged to avoid more fixes like this. SWAP_HAS_CACHE and SWP_SYNCHRONOUS_IO has a history of causing many issues. I'll do a swap table rebase on top of this fix, if this one is accepted. And for a comparision, swap in 10G into shmem: Before this patch: 2.49s After this patch: 2.52s After swap table: 2.37s (Removing SWAP_HAS_CACHE and SWP_SYNCHRONOUS_IO, still not in the best shape but looking good) Link: https://lore.kernel.org/linux-mm/20250514201729.48420-1-ryncsn@gmail.com/ [1] mm/memory.c | 20 -------------------- mm/shmem.c | 12 +++++++++++- mm/swap.h | 19 +++++++++++++++++++ 3 files changed, 30 insertions(+), 21 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 9ead7ab07e8e..3845ed068d74 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4313,26 +4313,6 @@ static struct folio *__alloc_swap_folio(struct vm_fault *vmf) } #ifdef CONFIG_TRANSPARENT_HUGEPAGE -static inline int non_swapcache_batch(swp_entry_t entry, int max_nr) -{ - struct swap_info_struct *si = swp_swap_info(entry); - pgoff_t offset = swp_offset(entry); - int i; - - /* - * While allocating a large folio and doing swap_read_folio, which is - * the case the being faulted pte doesn't have swapcache. We need to - * ensure all PTEs have no cache as well, otherwise, we might go to - * swap devices while the content is in swapcache. - */ - for (i = 0; i < max_nr; i++) { - if ((si->swap_map[offset + i] & SWAP_HAS_CACHE)) - return i; - } - - return i; -} - /* * Check if the PTEs within a range are contiguous swap entries * and have consistent swapcache, zeromap. diff --git a/mm/shmem.c b/mm/shmem.c index 73182e904f9c..484cd3043a78 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1995,6 +1995,14 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode, */ if (swapcache_prepare(entry, nr_pages)) { folio_put(new); + + /* + * A smaller folio is in the swap cache, mTHP swapin will always fail + * until it's gone. Return -EINVAL to fallback to order 0. + */ + if (non_swapcache_batch(entry, nr_pages) != nr_pages) + return ERR_PTR(-EINVAL); + return ERR_PTR(-EEXIST); } @@ -2256,6 +2264,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, folio = swap_cache_get_folio(swap, NULL, 0); order = xa_get_order(&mapping->i_pages, index); if (!folio) { + int nr_pages = 1 << order; bool fallback_order0 = false; /* Or update major stats only when swapin succeeds?? */ @@ -2271,7 +2280,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, * to swapin order-0 folio, as well as for zswap case. */ if (order > 0 && ((vma && unlikely(userfaultfd_armed(vma))) || - !zswap_never_enabled())) + !zswap_never_enabled() || + nr_pages != swap_zeromap_batch(swap, nr_pages, NULL))) fallback_order0 = true; /* Skip swapcache for synchronous device. */ diff --git a/mm/swap.h b/mm/swap.h index e87a0f19a0ee..2d8ce1102153 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -108,6 +108,25 @@ static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, return find_next_bit(sis->zeromap, end, start) - start; } +static inline int non_swapcache_batch(swp_entry_t entry, int max_nr) +{ + struct swap_info_struct *si = swp_swap_info(entry); + pgoff_t offset = swp_offset(entry); + int i; + + /* + * While allocating a large folio and doing mTHP swapin, we need to + * ensure all entries are not cached, otherwise, the mTHP folio will + * be in conflict with the folio in swap cache. + */ + for (i = 0; i < max_nr; i++) { + if ((si->swap_map[offset + i] & SWAP_HAS_CACHE)) + return i; + } + + return i; +} + #else /* CONFIG_SWAP */ struct swap_iocb; static inline void swap_read_folio(struct folio *folio, struct swap_iocb **plug) -- 2.49.0