[PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[Deepanshu Kartikey]

When anon_vma_fork() encounters a memory allocation failure after
anon_vma_clone() has succeeded, unlink_anon_vmas() is called with
vma->anon_vma being NULL but the anon_vma_chain populated with entries
that are present in the anon_vma interval trees.

This happens in the following sequence:
1. anon_vma_clone() succeeds, populating vma->anon_vma_chain and
   inserting entries into interval trees
2. maybe_reuse_anon_vma() does not set vma->anon_vma because reuse
   conditions are not met (common case for active processes)
3. anon_vma_alloc() or anon_vma_chain_alloc() fails due to memory
   pressure
4. Error path invokes unlink_anon_vmas() with vma->anon_vma == NULL

The existing code triggered VM_WARN_ON_ONCE and returned without
performing cleanup, leaving entries in interval trees and causing
memory leaks.

Fix this by detecting the condition and properly cleaning up:
- Iterate through the populated chain
- Lock each anon_vma
- Remove entries from interval trees
- Unlock and free chain entries

This prevents both the warning and the resource leaks.

Reported-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=c27fa543e10a45d4e149
Tested-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>
---
 mm/rmap.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index f13480cb9f2e..acc8df6ad4a7 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -477,7 +477,31 @@ void unlink_anon_vmas(struct vm_area_struct *vma)
 
 	/* Unfaulted is a no-op. */
 	if (!active_anon_vma) {
-		VM_WARN_ON_ONCE(!list_empty(&vma->anon_vma_chain));
+		/*
+		 * Handle anon_vma_fork() error path where anon_vma_clone()
+		 * succeeded and populated the chain (with entries in interval
+		 * trees), but maybe_reuse_anon_vma() didn't set vma->anon_vma
+		 * because reuse conditions weren't met, and a later allocation
+		 * failed before we could allocate and assign a new anon_vma.
+		 *
+		 * We must properly remove entries from interval trees before
+		 * freeing to avoid leaving dangling pointers.
+		 */
+		if (!list_empty(&vma->anon_vma_chain)) {
+			struct anon_vma_chain *avc, *next;
+
+			list_for_each_entry_safe(avc, next, &vma->anon_vma_chain,
+						same_vma) {
+				struct anon_vma *anon_vma = avc->anon_vma;
+
+				anon_vma_lock_write(anon_vma);
+				anon_vma_interval_tree_remove(avc, &anon_vma->rb_root);
+				anon_vma_unlock_write(anon_vma);
+				list_del(&avc->same_vma);
+				anon_vma_chain_free(avc);
+			}
+		}
+
 		return;
 	}
 
-- 
2.43.0

Re: [PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[Lorenzo Stoakes]

Andrew - please ignore, I'll update my series to address the issue.

Deepanshu - thanks very much for the patch, I think perhaps you are new to
mm so you may not be aware of how the life cycle of stuff works here, but
in this case as not upstream a patch isn't appropriate.

See below for details.

On Sun, Jan 18, 2026 at 04:28:17PM +0530, Deepanshu Kartikey wrote:
> When anon_vma_fork() encounters a memory allocation failure after
> anon_vma_clone() has succeeded, unlink_anon_vmas() is called with
> vma->anon_vma being NULL but the anon_vma_chain populated with entries
> that are present in the anon_vma interval trees.
>
> This happens in the following sequence:
> 1. anon_vma_clone() succeeds, populating vma->anon_vma_chain and
>    inserting entries into interval trees
> 2. maybe_reuse_anon_vma() does not set vma->anon_vma because reuse
>    conditions are not met (common case for active processes)
> 3. anon_vma_alloc() or anon_vma_chain_alloc() fails due to memory
>    pressure
> 4. Error path invokes unlink_anon_vmas() with vma->anon_vma == NULL
>
> The existing code triggered VM_WARN_ON_ONCE and returned without
> performing cleanup, leaving entries in interval trees and causing
> memory leaks.
>
> Fix this by detecting the condition and properly cleaning up:
> - Iterate through the populated chain
> - Lock each anon_vma
> - Remove entries from interval trees
> - Unlock and free chain entries
>
> This prevents both the warning and the resource leaks.
>
> Reported-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=c27fa543e10a45d4e149
> Tested-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>

Thanks for writing a patch for this, but this is not upstream yet, so the
right approach is to comment on the series thread, where you could suggest
the patch as an idea, not to submit a separate patch.

The series will be rebased to account for any issues found by -next bots
before going upstream.

> ---
>  mm/rmap.c | 25 ++++++++++++++++++++++++-
>  1 file changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/mm/rmap.c b/mm/rmap.c
> index f13480cb9f2e..acc8df6ad4a7 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -477,7 +477,31 @@ void unlink_anon_vmas(struct vm_area_struct *vma)
>
>  	/* Unfaulted is a no-op. */
>  	if (!active_anon_vma) {
> -		VM_WARN_ON_ONCE(!list_empty(&vma->anon_vma_chain));
> +		/*
> +		 * Handle anon_vma_fork() error path where anon_vma_clone()
> +		 * succeeded and populated the chain (with entries in interval
> +		 * trees), but maybe_reuse_anon_vma() didn't set vma->anon_vma
> +		 * because reuse conditions weren't met, and a later allocation
> +		 * failed before we could allocate and assign a new anon_vma.
> +		 *
> +		 * We must properly remove entries from interval trees before
> +		 * freeing to avoid leaving dangling pointers.
> +		 */
> +		if (!list_empty(&vma->anon_vma_chain)) {
> +			struct anon_vma_chain *avc, *next;
> +
> +			list_for_each_entry_safe(avc, next, &vma->anon_vma_chain,
> +						same_vma) {
> +				struct anon_vma *anon_vma = avc->anon_vma;
> +
> +				anon_vma_lock_write(anon_vma);
> +				anon_vma_interval_tree_remove(avc, &anon_vma->rb_root);
> +				anon_vma_unlock_write(anon_vma);
> +				list_del(&avc->same_vma);
> +				anon_vma_chain_free(avc);
> +			}
> +		}
> +

This is very duplicative and works against the intent of the series.

Also this is a 'impossible in practice' fault that requires fault injection
to trigger (a typical syzbot scenario) and adding a bunch of code to handle
such scenarios makes little practical sense.

I will update my series to address this scenario in a way that fits there.

>  		return;
>  	}
>
> --
> 2.43.0
>

Thanks, Lorenzo

Re: [PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[Lorenzo Stoakes]

On Sun, Jan 18, 2026 at 04:28:17PM +0530, Deepanshu Kartikey wrote:
> When anon_vma_fork() encounters a memory allocation failure after
> anon_vma_clone() has succeeded, unlink_anon_vmas() is called with
> vma->anon_vma being NULL but the anon_vma_chain populated with entries
> that are present in the anon_vma interval trees.
>
> This happens in the following sequence:
> 1. anon_vma_clone() succeeds, populating vma->anon_vma_chain and
>    inserting entries into interval trees
> 2. maybe_reuse_anon_vma() does not set vma->anon_vma because reuse
>    conditions are not met (common case for active processes)
> 3. anon_vma_alloc() or anon_vma_chain_alloc() fails due to memory
>    pressure
> 4. Error path invokes unlink_anon_vmas() with vma->anon_vma == NULL
>
> The existing code triggered VM_WARN_ON_ONCE and returned without
> performing cleanup, leaving entries in interval trees and causing
> memory leaks.
>
> Fix this by detecting the condition and properly cleaning up:
> - Iterate through the populated chain
> - Lock each anon_vma
> - Remove entries from interval trees
> - Unlock and free chain entries
>
> This prevents both the warning and the resource leaks.

BTW this reads rather like AI generated it, can you indicate whether that
was the case or not? :) Thanks.

We generally require acknowledgment of substantial AI-assistance in
submission.

Cheers, Lorenzo

>
> Reported-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=c27fa543e10a45d4e149
> Tested-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>
> ---
>  mm/rmap.c | 25 ++++++++++++++++++++++++-
>  1 file changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/mm/rmap.c b/mm/rmap.c
> index f13480cb9f2e..acc8df6ad4a7 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -477,7 +477,31 @@ void unlink_anon_vmas(struct vm_area_struct *vma)
>
>  	/* Unfaulted is a no-op. */
>  	if (!active_anon_vma) {
> -		VM_WARN_ON_ONCE(!list_empty(&vma->anon_vma_chain));
> +		/*
> +		 * Handle anon_vma_fork() error path where anon_vma_clone()
> +		 * succeeded and populated the chain (with entries in interval
> +		 * trees), but maybe_reuse_anon_vma() didn't set vma->anon_vma
> +		 * because reuse conditions weren't met, and a later allocation
> +		 * failed before we could allocate and assign a new anon_vma.
> +		 *
> +		 * We must properly remove entries from interval trees before
> +		 * freeing to avoid leaving dangling pointers.
> +		 */
> +		if (!list_empty(&vma->anon_vma_chain)) {
> +			struct anon_vma_chain *avc, *next;
> +
> +			list_for_each_entry_safe(avc, next, &vma->anon_vma_chain,
> +						same_vma) {
> +				struct anon_vma *anon_vma = avc->anon_vma;
> +
> +				anon_vma_lock_write(anon_vma);
> +				anon_vma_interval_tree_remove(avc, &anon_vma->rb_root);
> +				anon_vma_unlock_write(anon_vma);
> +				list_del(&avc->same_vma);
> +				anon_vma_chain_free(avc);
> +			}
> +		}
> +
>  		return;
>  	}
>
> --
> 2.43.0
>

Re: [PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[Deepanshu Kartikey]

On Sun, Jan 18, 2026 at 5:36 PM Lorenzo Stoakes
<lorenzo.stoakes@oracle.com> wrote:
>
> On Sun, Jan 18, 2026 at 04:28:17PM +0530, Deepanshu Kartikey wrote:
> > When anon_vma_fork() encounters a memory allocation failure after
> > anon_vma_clone() has succeeded, unlink_anon_vmas() is called with
> > vma->anon_vma being NULL but the anon_vma_chain populated with entries
> > that are present in the anon_vma interval trees.
> >
> > This happens in the following sequence:
> > 1. anon_vma_clone() succeeds, populating vma->anon_vma_chain and
> >    inserting entries into interval trees
> > 2. maybe_reuse_anon_vma() does not set vma->anon_vma because reuse
> >    conditions are not met (common case for active processes)
> > 3. anon_vma_alloc() or anon_vma_chain_alloc() fails due to memory
> >    pressure
> > 4. Error path invokes unlink_anon_vmas() with vma->anon_vma == NULL
> >
> > The existing code triggered VM_WARN_ON_ONCE and returned without
> > performing cleanup, leaving entries in interval trees and causing
> > memory leaks.
> >
> > Fix this by detecting the condition and properly cleaning up:
> > - Iterate through the populated chain
> > - Lock each anon_vma
> > - Remove entries from interval trees
> > - Unlock and free chain entries
> >
> > This prevents both the warning and the resource leaks.
>
> BTW this reads rather like AI generated it, can you indicate whether that
> was the case or not? :) Thanks.
>
> We generally require acknowledgment of substantial AI-assistance in
> submission.
>
> Cheers, Lorenzo
>
> >
> > Reported-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> > Closes: https://syzkaller.appspot.com/bug?extid=c27fa543e10a45d4e149
> > Tested-by: syzbot+c27fa543e10a45d4e149@syzkaller.appspotmail.com
> > Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>
> > ---
> >  mm/rmap.c | 25 ++++++++++++++++++++++++-
> >  1 file changed, 24 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/rmap.c b/mm/rmap.c
> > index f13480cb9f2e..acc8df6ad4a7 100644
> > --- a/mm/rmap.c
> > +++ b/mm/rmap.c
> > @@ -477,7 +477,31 @@ void unlink_anon_vmas(struct vm_area_struct *vma)
> >
> >       /* Unfaulted is a no-op. */
> >       if (!active_anon_vma) {
> > -             VM_WARN_ON_ONCE(!list_empty(&vma->anon_vma_chain));
> > +             /*
> > +              * Handle anon_vma_fork() error path where anon_vma_clone()
> > +              * succeeded and populated the chain (with entries in interval
> > +              * trees), but maybe_reuse_anon_vma() didn't set vma->anon_vma
> > +              * because reuse conditions weren't met, and a later allocation
> > +              * failed before we could allocate and assign a new anon_vma.
> > +              *
> > +              * We must properly remove entries from interval trees before
> > +              * freeing to avoid leaving dangling pointers.
> > +              */
> > +             if (!list_empty(&vma->anon_vma_chain)) {
> > +                     struct anon_vma_chain *avc, *next;
> > +
> > +                     list_for_each_entry_safe(avc, next, &vma->anon_vma_chain,
> > +                                             same_vma) {
> > +                             struct anon_vma *anon_vma = avc->anon_vma;
> > +
> > +                             anon_vma_lock_write(anon_vma);
> > +                             anon_vma_interval_tree_remove(avc, &anon_vma->rb_root);
> > +                             anon_vma_unlock_write(anon_vma);
> > +                             list_del(&avc->same_vma);
> > +                             anon_vma_chain_free(avc);
> > +                     }
> > +             }
> > +
> >               return;
> >       }
> >
> > --
> > 2.43.0
> >

Hi Lorenzo,

Yes, I used  AI for commit message drafting and to help
articulate the technical explanation. The bug analysis, code fix,
and testing are mine.

I'll send v2 with proper acknowledgment. Thanks for catching this.

Deepanshu

Re: [PATCH] mm/rmap: fix unlink_anon_vmas() handling of error case from anon_vma_fork

[Lorenzo Stoakes]

On Sun, Jan 18, 2026 at 05:54:35PM +0530, Deepanshu Kartikey wrote:
> Hi Lorenzo,
>
> Yes, I used  AI for commit message drafting and to help
> articulate the technical explanation. The bug analysis, code fix,
> and testing are mine.
>
> I'll send v2 with proper acknowledgment. Thanks for catching this.

No, please don't - As I told you in the other reply, this patch is not
acceptable.

When a patch is not upstream the appropriate thing to do is to comment on
the series.

We rebase development patches before they go upstream, so it just makes no
sense.

In addition this patch is not upstreamble as-is. You are getting at the
issue, but you're effectively undermining what my series does with this
proposal.

So while I appreciate the report, unfortunately this is not a patch we can
do anything with.

>
> Deepanshu

Thanks, Lorenzo