Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

[ffhgfv]

Hello, I found a bug titled " BUG: spinlock bad magic in z3fold_zpool_free   " with modified syzkaller in the Linux6.14-rc5.
If you fix this issue, please add the following tag to the commit:  Reported-by: Jianzhou Zhao <xnxc22xnxc22@qq.com>,    xingwei lee <xrivendell7@gmail.com>, Zhizhuo Tang <strforexctzzchange@foxmail.com>

I use the same kernel as syzbot instance upstream: 7eb172143d5508b4da468ed59ee857c6e5e01da6
kernel config: https://syzkaller.appspot.com/text?tag=KernelConfig&amp;x=da4b04ae798b7ef6
compiler: gcc version 11.4.0
------------[ cut here ]-----------------------------------------
 TITLE:     BUG: spinlock bad magic in z3fold_zpool_free
==================================================================
BUG: spinlock bad magic on CPU#0, syz-executor/16907
 lock: 0xffff88805a9de010, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
CPU: 0 UID: 0 PID: 16907 Comm: syz-executor Not tainted 6.14.0-rc5-dirty #17
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Call Trace:
 <task>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x180/0x1b0 lib/dump_stack.c:120
 spin_bug kernel/locking/spinlock_debug.c:78 [inline]
 debug_spin_unlock kernel/locking/spinlock_debug.c:100 [inline]
 do_raw_spin_unlock+0x201/0x270 kernel/locking/spinlock_debug.c:141
 __raw_spin_unlock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_unlock+0x1e/0x50 kernel/locking/spinlock.c:186
 spin_unlock include/linux/spinlock.h:391 [inline]
 z3fold_page_unlock mm/z3fold.c:235 [inline]
 get_z3fold_header mm/z3fold.c:260 [inline]
 get_z3fold_header mm/z3fold.c:239 [inline]
 z3fold_free mm/z3fold.c:1100 [inline]
 z3fold_zpool_free+0x6f/0xe40 mm/z3fold.c:1392
 zswap_entry_free+0x235/0xa80 mm/zswap.c:806
 zswap_invalidate+0x11f/0x190 mm/zswap.c:1682
 swap_range_free mm/swapfile.c:1133 [inline]
 swap_entry_range_free+0x30a/0x830 mm/swapfile.c:1512
 __swap_entry_free mm/swapfile.c:1440 [inline]
 __swap_entries_free mm/swapfile.c:1478 [inline]
 free_swap_and_cache_nr+0x5b9/0xba0 mm/swapfile.c:1797
 zap_nonpresent_ptes mm/memory.c:1636 [inline]
 do_zap_pte_range mm/memory.c:1702 [inline]
 zap_pte_range mm/memory.c:1742 [inline]
 zap_pmd_range mm/memory.c:1834 [inline]
 zap_pud_range mm/memory.c:1863 [inline]
 zap_p4d_range mm/memory.c:1884 [inline]
 unmap_page_range+0x120c/0x4af0 mm/memory.c:1905
 unmap_single_vma+0x19a/0x2b0 mm/memory.c:1951
 unmap_vmas+0x1fe/0x450 mm/memory.c:1995
 exit_mmap+0x1b4/0xbf0 mm/mmap.c:1284
 __mmput kernel/fork.c:1356 [inline]
 mmput+0x178/0x450 kernel/fork.c:1378
 exit_mm kernel/exit.c:570 [inline]
 do_exit+0x94b/0x3080 kernel/exit.c:925
 do_group_exit+0xd3/0x2a0 kernel/exit.c:1087
 get_signal+0x261f/0x2790 kernel/signal.c:3036
 arch_do_signal_or_restart+0x81/0x8b0 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0x228/0x2a0 kernel/entry/common.c:218
 do_syscall_64+0xdc/0x250 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f5bf7b9f86a
Code: Unable to access opcode bytes at 0x7f5bf7b9f840.
RSP: 002b:00007ffcec08fd08 EFLAGS: 00000246 ORIG_RAX: 000000000000003d
RAX: fffffffffffffe00 RBX: 000000000000423a RCX: 00007f5bf7b9f86a
RDX: 0000000040000000 RSI: 00007ffcec08fd14 RDI: 00000000ffffffff
RBP: 00007ffcec08fd14 R08: 000000000000423a R09: 0000000080000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000c
R13: 0000000000000003 R14: 00007f5bf7c4e881 R15: 0000000000000002
 </task>
------------[ cut here ]------------
pvqspinlock: lock 0xffff88805a9de010 has corrupted value 0x0!
WARNING: CPU: 0 PID: 16907 at kernel/locking/qspinlock_paravirt.h:504 __pv_queued_spin_unlock_slowpath+0x238/0x340 kernel/locking/qspinlock_paravirt.h:504
Modules linked in:
CPU: 0 UID: 0 PID: 16907 Comm: syz-executor Not tainted 6.14.0-rc5-dirty #17
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
RIP: 0010:__pv_queued_spin_unlock_slowpath+0x238/0x340 kernel/locking/qspinlock_paravirt.h:504
Code: 02 4c 89 e0 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85 8a 00 00 00 41 8b 14 24 4c 89 e6 48 c7 c7 00 52 6d 8b e8 89 07 1e f6 90 &lt;0f&gt; 0b 90 90 e9 64 ff ff ff 90 0f 0b e8 f7 fb c1 f6 e9 1e ff ff ff
RSP: 0018:ffffc900065cf260 EFLAGS: 00010286
RAX: 0000000000000000 RBX: ffff88802afa7188 RCX: ffffffff8179ec7a
RDX: 0000000000000000 RSI: ffff888020558000 RDI: 0000000000000002
RBP: ffff88805a9de010 R08: fffffbfff1c0b800 R09: ffffed1005705182
R10: ffffed1005705181 R11: ffff88802b828c0b R12: ffff88805a9de010
R13: ffff88805a9de020 R14: ffff88805a9de010 R15: ffffea00016a77a8
FS:  0000000000000000(0000) GS:ffff88802b800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555f7caa4fe0 CR3: 000000004f5b4000 CR4: 0000000000752ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 80000000
Call Trace:
 <task>
 __raw_callee_save___pv_queued_spin_unlock_slowpath+0x15/0x30
 .slowpath+0x9/0x18
 pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:582 [inline]
 queued_spin_unlock arch/x86/include/asm/qspinlock.h:57 [inline]
 do_raw_spin_unlock+0x174/0x270 kernel/locking/spinlock_debug.c:142
 __raw_spin_unlock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_unlock+0x1e/0x50 kernel/locking/spinlock.c:186
 spin_unlock include/linux/spinlock.h:391 [inline]
 z3fold_page_unlock mm/z3fold.c:235 [inline]
 get_z3fold_header mm/z3fold.c:260 [inline]
 get_z3fold_header mm/z3fold.c:239 [inline]
 z3fold_free mm/z3fold.c:1100 [inline]
 z3fold_zpool_free+0x6f/0xe40 mm/z3fold.c:1392
 zswap_entry_free+0x235/0xa80 mm/zswap.c:806
 zswap_invalidate+0x11f/0x190 mm/zswap.c:1682
 swap_range_free mm/swapfile.c:1133 [inline]
 swap_entry_range_free+0x30a/0x830 mm/swapfile.c:1512
 __swap_entry_free mm/swapfile.c:1440 [inline]
 __swap_entries_free mm/swapfile.c:1478 [inline]
 free_swap_and_cache_nr+0x5b9/0xba0 mm/swapfile.c:1797
 zap_nonpresent_ptes mm/memory.c:1636 [inline]
 do_zap_pte_range mm/memory.c:1702 [inline]
 zap_pte_range mm/memory.c:1742 [inline]
 zap_pmd_range mm/memory.c:1834 [inline]
 zap_pud_range mm/memory.c:1863 [inline]
 zap_p4d_range mm/memory.c:1884 [inline]
 unmap_page_range+0x120c/0x4af0 mm/memory.c:1905
 unmap_single_vma+0x19a/0x2b0 mm/memory.c:1951
 unmap_vmas+0x1fe/0x450 mm/memory.c:1995
 exit_mmap+0x1b4/0xbf0 mm/mmap.c:1284
 __mmput kernel/fork.c:1356 [inline]
 mmput+0x178/0x450 kernel/fork.c:1378
 exit_mm kernel/exit.c:570 [inline]
 do_exit+0x94b/0x3080 kernel/exit.c:925
 do_group_exit+0xd3/0x2a0 kernel/exit.c:1087
 get_signal+0x261f/0x2790 kernel/signal.c:3036
 arch_do_signal_or_restart+0x81/0x8b0 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0x228/0x2a0 kernel/entry/common.c:218
 do_syscall_64+0xdc/0x250 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f5bf7b9f86a
Code: Unable to access opcode bytes at 0x7f5bf7b9f840.
RSP: 002b:00007ffcec08fd08 EFLAGS: 00000246 ORIG_RAX: 000000000000003d
RAX: fffffffffffffe00 RBX: 000000000000423a RCX: 00007f5bf7b9f86a
RDX: 0000000040000000 RSI: 00007ffcec08fd14 RDI: 00000000ffffffff
RBP: 00007ffcec08fd14 R08: 000000000000423a R09: 0000000080000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000c
R13: 0000000000000003 R14: 00007f5bf7c4e881 R15: 0000000000000002
 </task>

==================================================================




I hope it helps.
Best regards
Jianzhou Zhao</none></strforexctzzchange@foxmail.com></xrivendell7@gmail.com></xnxc22xnxc22@qq.com>

Re: Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

[Nhat Pham]

On Wed, Mar 26, 2025 at 10:11 AM ffhgfv <xnxc22xnxc22@qq.com> wrote:
>
> Hello, I found a bug titled " BUG: spinlock bad magic in z3fold_zpool_free   " with modified syzkaller in the Linux6.14-rc5.
> If you fix this issue, please add the following tag to the commit:  Reported-by: Jianzhou Zhao <xnxc22xnxc22@qq.com>,    xingwei lee <xrivendell7@gmail.com>, Zhizhuo Tang <strforexctzzchange@foxmail.com>

Please stop using z3fold :) We already removed it upstream.

Re: Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

[Nhat Pham]

On Wed, Mar 26, 2025 at 10:32 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Wed, Mar 26, 2025 at 10:11 AM ffhgfv <xnxc22xnxc22@qq.com> wrote:
> >
> > Hello, I found a bug titled " BUG: spinlock bad magic in z3fold_zpool_free   " with modified syzkaller in the Linux6.14-rc5.
> > If you fix this issue, please add the following tag to the commit:  Reported-by: Jianzhou Zhao <xnxc22xnxc22@qq.com>,    xingwei lee <xrivendell7@gmail.com>, Zhizhuo Tang <strforexctzzchange@foxmail.com>
>
> Please stop using z3fold :) We already removed it upstream.

To clarify a little bit - we've found that z3fold is buggy (for a very
long time), and does not outperform zsmalloc in many of the workloads
we test on (both microbenchmark and real production workloads). We've
deprecated it since 6.12:

https://github.com/torvalds/linux/commit/7a2369b74abf76cd3e54c45b30f6addb497f831b

and will remove it altogether:

https://lore.kernel.org/all/20250129180633.3501650-1-yosry.ahmed@linux.dev/

Perhaps Vitaly can fix the issue for stability's sake (or in case
there is a reason why you MUST use z3fold)? But I strongly recommend
you experiment with zsmalloc :)

Re: Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

[Matthew Wilcox]

On Wed, Mar 26, 2025 at 03:43:28PM -0400, Nhat Pham wrote:
> On Wed, Mar 26, 2025 at 10:32 AM Nhat Pham <nphamcs@gmail.com> wrote:
> > On Wed, Mar 26, 2025 at 10:11 AM ffhgfv <xnxc22xnxc22@qq.com> wrote:
> > >
> > > Hello, I found a bug titled " BUG: spinlock bad magic in z3fold_zpool_free   " with modified syzkaller in the Linux6.14-rc5.
> > > If you fix this issue, please add the following tag to the commit:  Reported-by: Jianzhou Zhao <xnxc22xnxc22@qq.com>,    xingwei lee <xrivendell7@gmail.com>, Zhizhuo Tang <strforexctzzchange@foxmail.com>
> >
> > Please stop using z3fold :) We already removed it upstream.
> 
> To clarify a little bit - we've found that z3fold is buggy (for a very
> long time), and does not outperform zsmalloc in many of the workloads
> we test on (both microbenchmark and real production workloads). We've
> deprecated it since 6.12:
> 
> https://github.com/torvalds/linux/commit/7a2369b74abf76cd3e54c45b30f6addb497f831b
> 
> and will remove it altogether:
> 
> https://lore.kernel.org/all/20250129180633.3501650-1-yosry.ahmed@linux.dev/
> 
> Perhaps Vitaly can fix the issue for stability's sake (or in case
> there is a reason why you MUST use z3fold)? But I strongly recommend
> you experiment with zsmalloc :)

This group are syzkaller kiddies.  They have no understanding of what
they're testing; they're just running their fuzzer and sending emails.
They don't care what's useful, so there's a lot of noise from unmaintained
filesystems and so on.

Re: Linux6.14-rc5 BUG: spinlock bad magic in z3fold_zpool_free

[Nhat Pham]

On Wed, Mar 26, 2025 at 4:42 PM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Wed, Mar 26, 2025 at 03:43:28PM -0400, Nhat Pham wrote:
> > On Wed, Mar 26, 2025 at 10:32 AM Nhat Pham <nphamcs@gmail.com> wrote:
> > > On Wed, Mar 26, 2025 at 10:11 AM ffhgfv <xnxc22xnxc22@qq.com> wrote:
> > > >
> > > > Hello, I found a bug titled " BUG: spinlock bad magic in z3fold_zpool_free   " with modified syzkaller in the Linux6.14-rc5.
> > > > If you fix this issue, please add the following tag to the commit:  Reported-by: Jianzhou Zhao <xnxc22xnxc22@qq.com>,    xingwei lee <xrivendell7@gmail.com>, Zhizhuo Tang <strforexctzzchange@foxmail.com>
> > >
> > > Please stop using z3fold :) We already removed it upstream.
> >
> > To clarify a little bit - we've found that z3fold is buggy (for a very
> > long time), and does not outperform zsmalloc in many of the workloads
> > we test on (both microbenchmark and real production workloads). We've
> > deprecated it since 6.12:
> >
> > https://github.com/torvalds/linux/commit/7a2369b74abf76cd3e54c45b30f6addb497f831b
> >
> > and will remove it altogether:
> >
> > https://lore.kernel.org/all/20250129180633.3501650-1-yosry.ahmed@linux.dev/
> >
> > Perhaps Vitaly can fix the issue for stability's sake (or in case
> > there is a reason why you MUST use z3fold)? But I strongly recommend
> > you experiment with zsmalloc :)
>
> This group are syzkaller kiddies.  They have no understanding of what
> they're testing; they're just running their fuzzer and sending emails.
> They don't care what's useful, so there's a lot of noise from unmaintained
> filesystems and so on.

You're right - I just realized they did it once already for zswap:

https://lore.kernel.org/all/tencent_49DA3E780998A9B96ADC9FF658CC84641808@qq.com/

It was also due to the (soon-to-be-removed) z3fold backend. To save
time, I'll stop engaging from now on, unless it's a proper issue.