[BUG] WARNING in unlink_anon_vmas()

[BUG] WARNING in unlink_anon_vmas()

[Jiakai Xu]

Hi all,

While fuzzing the KVM subsystem on RISC-V, I stumbled upon a kernel WARNING
that triggers in unlink_anon_vmas().

 WARNING: mm/rmap.c:528 at unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
 unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
 free_pgtables+0x2a0/0x860 mm/memory.c:427
 exit_mmap+0x406/0xd14 mm/mmap.c:1314
 __mmput+0x114/0x3d4 kernel/fork.c:1174
 mmput+0x74/0x88 kernel/fork.c:1197
 exit_mm kernel/exit.c:581 [inline]
 do_exit+0x7de/0x2adc kernel/exit.c:959
 do_group_exit+0xd4/0x26c kernel/exit.c:1112
 __do_sys_exit_group kernel/exit.c:1123 [inline]
 __se_sys_exit_group kernel/exit.c:1121 [inline]
 __riscv_sys_exit_group+0x4a/0x54 kernel/exit.c:1121
 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112
 do_trap_ecall_u+0x39e/0x62e arch/riscv/kernel/traps.c:344
 handle_exception+0x15e/0x16a arch/riscv/kernel/entry.S:232

I am not an expert in this area and have not done a deep manual analysis.

The full crash log, a reproducer, the kernel .config, and the relevant
source/commit info are available in my GitHub repository:
    https://github.com/j1akai/temp/tree/main/20260318

If this turns out to be a real bug and there is anything I can do to help
with fixing or testing, I am happy to do so. I hope this report is useful
and sorry for any noise if it has already been addressed.

Thanks,
Jiakai

Re: [BUG] WARNING in unlink_anon_vmas()

[David Hildenbrand (Arm)]

On 3/18/26 11:42, Jiakai Xu wrote:
> Hi all,

Hi,

> 
> While fuzzing the KVM subsystem on RISC-V, I stumbled upon a kernel WARNING
> that triggers in unlink_anon_vmas().
> 

Looking at the log, this is against 7.0.0-rc2-00014-gc61ec3e8cc5d

>  WARNING: mm/rmap.c:528 at unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
>  unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
>  free_pgtables+0x2a0/0x860 mm/memory.c:427
>  exit_mmap+0x406/0xd14 mm/mmap.c:1314
>  __mmput+0x114/0x3d4 kernel/fork.c:1174
>  mmput+0x74/0x88 kernel/fork.c:1197
>  exit_mm kernel/exit.c:581 [inline]
>  do_exit+0x7de/0x2adc kernel/exit.c:959
>  do_group_exit+0xd4/0x26c kernel/exit.c:1112
>  __do_sys_exit_group kernel/exit.c:1123 [inline]
>  __se_sys_exit_group kernel/exit.c:1121 [inline]
>  __riscv_sys_exit_group+0x4a/0x54 kernel/exit.c:1121
>  syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112
>  do_trap_ecall_u+0x39e/0x62e arch/riscv/kernel/traps.c:344
>  handle_exception+0x15e/0x16a arch/riscv/kernel/entry.S:232
> 
> I am not an expert in this area and have not done a deep manual analysis.
> 
> The full crash log, a reproducer, the kernel .config, and the relevant
> source/commit info are available in my GitHub repository:
>     https://github.com/j1akai/temp/tree/main/20260318
> 
> If this turns out to be a real bug and there is anything I can do to help
> with fixing or testing, I am happy to do so. I hope this report is useful
> and sorry for any noise if it has already been addressed.


We trigger a VM_WARN_ON, so that's certainly something to resolve.

Seems to be:

	VM_WARN_ON(anon_vma->num_active_vmas);

Which was added by

commit 2555283eb40df89945557273121e9393ef9b542b
Author: Jann Horn <jannh@google.com>
Date:   Wed Aug 31 19:06:00 2022 +0200

    mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse
    
    anon_vma->degree tracks the combined number of child anon_vmas and VMAs
    that use the anon_vma as their ->anon_vma.



-- 
Cheers,

David

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

On Wed, Mar 18, 2026 at 11:57:24AM +0100, David Hildenbrand (Arm) wrote:
> On 3/18/26 11:42, Jiakai Xu wrote:
> > Hi all,
>
> Hi,
>
> >
> > While fuzzing the KVM subsystem on RISC-V, I stumbled upon a kernel WARNING
> > that triggers in unlink_anon_vmas().
> >
>
> Looking at the log, this is against 7.0.0-rc2-00014-gc61ec3e8cc5d
>
> >  WARNING: mm/rmap.c:528 at unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
> >  unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
> >  free_pgtables+0x2a0/0x860 mm/memory.c:427
> >  exit_mmap+0x406/0xd14 mm/mmap.c:1314
> >  __mmput+0x114/0x3d4 kernel/fork.c:1174
> >  mmput+0x74/0x88 kernel/fork.c:1197
> >  exit_mm kernel/exit.c:581 [inline]
> >  do_exit+0x7de/0x2adc kernel/exit.c:959
> >  do_group_exit+0xd4/0x26c kernel/exit.c:1112
> >  __do_sys_exit_group kernel/exit.c:1123 [inline]
> >  __se_sys_exit_group kernel/exit.c:1121 [inline]
> >  __riscv_sys_exit_group+0x4a/0x54 kernel/exit.c:1121
> >  syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112
> >  do_trap_ecall_u+0x39e/0x62e arch/riscv/kernel/traps.c:344
> >  handle_exception+0x15e/0x16a arch/riscv/kernel/entry.S:232
> >
> > I am not an expert in this area and have not done a deep manual analysis.
> >
> > The full crash log, a reproducer, the kernel .config, and the relevant
> > source/commit info are available in my GitHub repository:
> >     https://github.com/j1akai/temp/tree/main/20260318
> >
> > If this turns out to be a real bug and there is anything I can do to help
> > with fixing or testing, I am happy to do so. I hope this report is useful
> > and sorry for any noise if it has already been addressed.
>
>
> We trigger a VM_WARN_ON, so that's certainly something to resolve.
>
> Seems to be:
>
> 	VM_WARN_ON(anon_vma->num_active_vmas);
>
> Which was added by
>
> commit 2555283eb40df89945557273121e9393ef9b542b
> Author: Jann Horn <jannh@google.com>
> Date:   Wed Aug 31 19:06:00 2022 +0200
>
>     mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse
>
>     anon_vma->degree tracks the combined number of child anon_vmas and VMAs
>     that use the anon_vma as their ->anon_vma.

I suspect my recent series + some error path is a cause here, investigating! :)

>
>
>
> --
> Cheers,
>
> David

Cheers, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

(-cc old email)

On Wed, Mar 18, 2026 at 06:42:49PM +0800, Jiakai Xu wrote:
> Hi all,
>
> While fuzzing the KVM subsystem on RISC-V, I stumbled upon a kernel WARNING
> that triggers in unlink_anon_vmas().

Thanks!

Will have a look at this.

>
>  WARNING: mm/rmap.c:528 at unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
>  unlink_anon_vmas+0x562/0x768 mm/rmap.c:528

Assuming there's not some big mismatch with kernel versions this is:

VM_WARN_ON(anon_vma->num_active_vmas);

I seem to remember a possible case where the resource cleanup was wrong on
certain error paths which I was sure I fixed, but I can't find the patch
right now :/ so let me check that also.

>  free_pgtables+0x2a0/0x860 mm/memory.c:427
>  exit_mmap+0x406/0xd14 mm/mmap.c:1314

Fact this is on exit path suggests either misaccounting of
anon_vma->num_active_vmas, or a VMA is somehow not being cleaned up
properly, most likely the former I would say.


>  __mmput+0x114/0x3d4 kernel/fork.c:1174
>  mmput+0x74/0x88 kernel/fork.c:1197
>  exit_mm kernel/exit.c:581 [inline]
>  do_exit+0x7de/0x2adc kernel/exit.c:959
>  do_group_exit+0xd4/0x26c kernel/exit.c:1112
>  __do_sys_exit_group kernel/exit.c:1123 [inline]
>  __se_sys_exit_group kernel/exit.c:1121 [inline]
>  __riscv_sys_exit_group+0x4a/0x54 kernel/exit.c:1121
>  syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112
>  do_trap_ecall_u+0x39e/0x62e arch/riscv/kernel/traps.c:344
>  handle_exception+0x15e/0x16a arch/riscv/kernel/entry.S:232
>
> I am not an expert in this area and have not done a deep manual analysis.
>
> The full crash log, a reproducer, the kernel .config, and the relevant
> source/commit info are available in my GitHub repository:
>     https://github.com/j1akai/temp/tree/main/20260318

Thanks, much appreciated, will try to repro locally.

>
> If this turns out to be a real bug and there is anything I can do to help
> with fixing or testing, I am happy to do so. I hope this report is useful
> and sorry for any noise if it has already been addressed.

Thanks, and no it's not been addressed AFAIK but let me dig into this.

>
> Thanks,
> Jiakai

Cheers, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

> On Wed, Mar 18, 2026 at 06:42:49PM +0800, Jiakai Xu wrote:
> > If this turns out to be a real bug and there is anything I can do to help
> > with fixing or testing, I am happy to do so. I hope this report is useful
> > and sorry for any noise if it has already been addressed.
>

Do you happen to have a copy of the full dmesg output? Particularly looking
for where the fault injections were made.

Does this reliably repro for you consistently or is it now and again?

If reliable repro, could you try it with the upstream tree also?
e.g. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ at
master?

Much appreciated!

Thanks, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

On Wed, Mar 18, 2026 at 11:23:20AM +0000, Lorenzo Stoakes (Oracle) wrote:
> > On Wed, Mar 18, 2026 at 06:42:49PM +0800, Jiakai Xu wrote:
> > > If this turns out to be a real bug and there is anything I can do to help
> > > with fixing or testing, I am happy to do so. I hope this report is useful
> > > and sorry for any noise if it has already been addressed.
> >
>
> Do you happen to have a copy of the full dmesg output? Particularly looking
> for where the fault injections were made.
>
> Does this reliably repro for you consistently or is it now and again?
>
> If reliable repro, could you try it with the upstream tree also?
> e.g. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ at
> master?

Actually no need figured it out :)

Cheers, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

+cc Sasha

On Wed, Mar 18, 2026 at 10:59:33AM +0000, Lorenzo Stoakes (Oracle) wrote:
> (-cc old email)
>
> On Wed, Mar 18, 2026 at 06:42:49PM +0800, Jiakai Xu wrote:
> > Hi all,
> >
> > While fuzzing the KVM subsystem on RISC-V, I stumbled upon a kernel WARNING
> > that triggers in unlink_anon_vmas().
>
> Thanks!
>
> Will have a look at this.
>
> >
> >  WARNING: mm/rmap.c:528 at unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
> >  unlink_anon_vmas+0x562/0x768 mm/rmap.c:528
>
> Assuming there's not some big mismatch with kernel versions this is:
>
> VM_WARN_ON(anon_vma->num_active_vmas);

OK so this _was_ reported by Sasha (via an AI assessment), but it completely
dropped off my radar sorry about that!

https://lore.kernel.org/linux-mm/20260302151547.2389070-1-sashal@kernel.org/

I want to fix this a slightly different way though.

SO what's happening is in dup_anon_vma() we do:

static int dup_anon_vma(struct vm_area_struct *dst,
			struct vm_area_struct *src, struct vm_area_struct **dup)
{
	...

		dst->anon_vma = src->anon_vma;
		ret = anon_vma_clone(dst, src, VMA_OP_MERGE_UNFAULTED);
		if (ret)
			return ret; <-- fault injection error here

		*dup = dst;         <-- NOT set

	...
}

Then:

int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src,
		   enum vma_operation operation)
{
	...

	list_for_each_entry(pavc, &src->anon_vma_chain, same_vma) {
		avc = anon_vma_chain_alloc(GFP_KERNEL);  <-- FAULT INJECTED HERE
		if (!avc)
			goto enomem_failure;             <-- so we do this

		anon_vma_chain_assign(dst, avc, pavc->anon_vma);
	}

	...

	if (operation != VMA_OP_FORK)
		dst->anon_vma->num_active_vmas++;        <-- does NOT get run

	...

 enomem_failure:
	cleanup_partial_anon_vmas(dst);
	return -ENOMEM;
}

We only clear down the partially allocated anon_vma_chain objects:

static void cleanup_partial_anon_vmas(struct vm_area_struct *vma)
{
	struct anon_vma_chain *avc, *next;

	list_for_each_entry_safe(avc, next, &vma->anon_vma_chain, same_vma) {
		list_del(&avc->same_vma);
		anon_vma_chain_free(avc);
	}
}

But, crucially, dst->anon_vma is LEFT IN PLACE.

So on process exit, we call into unlink_anon_vmas() for that VMA:

void unlink_anon_vmas(struct vm_area_struct *vma)
{
	...
	struct anon_vma *active_anon_vma = vma->anon_vma; <-- is SET

	...

	/* Unfaulted is a no-op. */
	if (!active_anon_vma) {                           <-- is not called
		VM_WARN_ON_ONCE(!list_empty(&vma->anon_vma_chain));
		return;
	}

	...

	active_anon_vma->num_active_vmas--;               <-- Incorrect (*)

	...

	list_for_each_entry_safe(avc, next, &vma->anon_vma_chain, same_vma) {
		...

		VM_WARN_ON(anon_vma->num_active_vmas);    <-- triggers
		put_anon_vma(anon_vma);

		...
	}
}

* If anon_vma->num_active_vmas was 0, it underflows, but otherwise it'll get
  decrement one time too many, and so will _eventually underflow_ guaranteed and
  trigger the bug for for VMAs associated with this anon_vma.

The fix is to set vma->anon_vma = NULL in this situation, which I think is best
done in the cleanup code as I said at
https://lore.kernel.org/linux-mm/a709c736-fd76-4bc9-a1d2-e1351742b321@lucifer.local/
but then... didn't do as it dropped off my radar (oops!)

Will send a fix + cc, attribute Reported-by etc., thanks very much for reporting
this Jiakai, was very useful!

Cheers, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

Fix at
https://lore.kernel.org/linux-mm/20260318122632.63404-1-ljs@kernel.org/

We like to turn things around quick in mm ;)

Could you check to make sure that fixes the issue? If so could you reply to
that email with:

Tested-by: Jiakai Xu <jiakaipeanut@gmail.com>

Thanks, Lorenzo

Re: [BUG] WARNING in unlink_anon_vmas()

[Jiakai Xu]

> Fix at
> https://lore.kernel.org/linux-mm/20260318122632.63404-1-ljs@kernel.org/

Thank you for the quick investigation and fix.

> 
> We like to turn things around quick in mm ;)
> 
> Could you check to make sure that fixes the issue? If so could you reply to
> that email with:
> 
> Tested-by: Jiakai Xu <jiakaipeanut@gmail.com>

I tested the patch on my setup with the reproducer that originally
triggered the WARNING in unlink_anon_vmas(), and the issue no longer
reproduces.

Best regards,
Jiakai

Re: [BUG] WARNING in unlink_anon_vmas()

[Lorenzo Stoakes (Oracle)]

On Thu, Mar 19, 2026 at 09:16:33AM +0800, Jiakai Xu wrote:
> > Fix at
> > https://lore.kernel.org/linux-mm/20260318122632.63404-1-ljs@kernel.org/
>
> Thank you for the quick investigation and fix.
>
> >
> > We like to turn things around quick in mm ;)
> >
> > Could you check to make sure that fixes the issue? If so could you reply to
> > that email with:
> >
> > Tested-by: Jiakai Xu <jiakaipeanut@gmail.com>
>
> I tested the patch on my setup with the reproducer that originally
> triggered the WARNING in unlink_anon_vmas(), and the issue no longer
> reproduces.

Perfect, much appreciated! :)

Thanks again for submitting the bug, it's been very helpful in ensuring this
issue is addressed in an -rc rather than in a released kernel.

>
> Best regards,
> Jiakai

Thanks, Lorenzo