Re: [BUG] Double-free in blk_mq_free_sched_tags() after commit f5a6604f7a44

[Nilay Shroff <nilay@linux.ibm.com>]

On 9/28/25 5:38 PM, Niklas Fischer wrote:
> Hello,
> 
> I'm reporting a kernel crash that occurs during boot on systems with multiple storage devices. The issue manifests as a double-free bug in the SLUB allocator, triggered by block layer elevator switching code.
> 
> === Problem Summary ===
> 
> The system crashes during early boot when udev configures I/O schedulers on multiple storage devices. The crash occurs in mm/slub.c with a double-free detection, traced back to blk_mq_free_sched_tags().
> 
> === Crash Details ===
> 
> Multiple crashes occur during boot, showing a severe race condition. Seven separate kernel oops/panics are observed:
> 
> * Oops #1 (CPU 13, PID 928): General protection fault in kfree+0x69/0x3b0 - corrupted address 0x14b9d856a995288
> * Oops #2-4, #6-7 (multiple CPUs/PIDs): kernel BUG at mm/slub.c:546 in __slab_free+0x111/0x2a0 - SLUB double-free detection
> * Oops #5 (CPU 1, PID 952): General protection fault in kfree+0x69/0x3b0    - corrupted address 0x2480af562995288
> 
> All crashes share the same call stack pattern:
> 
> elv_iosched_store+0x149/0x180
> elevator_change+0xdb/0x180
> elevator_change_done+0x4a/0x1f0
> blk_mq_free_sched_tags+0x34/0x70
> blk_mq_free_tags+0x4b/0x60
> kfree+0x334/0x3b0  <-- crash here
> 
> === Bisection Results ===
> 
> I bisected the issue to this commit:
> 
> commit f5a6604f7a4405450e4a1f54e5430f47290c500f
> Author: Nilay Shroff nilay@linux.ibm.com
> Date: Wed Jul 30 13:16:08 2025 +0530
> "block: fix lockdep warning caused by lock dependency in elv_iosched_store"
> 
> This commit moved sched_tags allocation/deallocation outside of elevator locks to fix a lockdep warning, but appears to have introduced a use-after-free or double-free bug in the process.
> 
> Reverting commit f5a6604f7a44 against the v6.16.7 tag results in merge conflicts due to subsequent block subsystem changes, making a clean revert test difficult without significant manual conflict resolution. I have therefore not tried this.
> 
[...]
> P.S.: This is my first kernel bug report. I've tried to follow the proper conventions, but please let me know if I should format or present anything differently.

Thanks for the report!

It looks like you were running a tainted kernel, and the kernel version
differs from the stock upstream version. This likely means some modules
were modified, built out-of-tree, or unsigned.

I recommend trying to reproduce this issue on a pristine upstream/stock kernel.
That will help ensure the bug isn’t caused by external modifications. Otherwise,
you may consider reporting the issue to your distribution. Bug reports get the
most attention when reproduced on an untainted, upstream kernel.

From your log, it seems the tagset (request queues) being freed had already
been freed, or memory may have been corrupted in between. If you can reproduce
the issue on a stock kernel, I suggest enabling KASAN (CONFIG_KASAN) and 
reproducing the bug. The KASAN logs will be very helpful in identifying the
root cause of any memory corruption or double-free issues.

Thanks,
--Nilay