[PATCH] mm: fix anon_vma->degree underflow in anon_vma endless growing prevention

[Leon Yu <chianglungyu@gmail.com>]

I have constantly stumbled upon "kernel BUG at mm/rmap.c:399!" after upgrading
to 3.19 and had no luck with 4.0-rc1 neither.

So, after looking into new logic introduced by commit 7a3ef208e662, ("mm:
prevent endless growth of anon_vma hierarchy"), I found chances are that
unlink_anon_vmas() is called without incrementing dst->anon_vma->degree in
anon_vma_clone() due to allocation failure. If dst->anon_vma is not NULL in
error path, its degree will be incorrectly decremented in unlink_anon_vmas()
and eventually underflow when exiting as a result of another call to
unlink_anon_vmas(). That's how "kernel BUG at mm/rmap.c:399!" is triggered
for me.

This patch fixes the underflow by dropping dst->anon_vma when allocation
fails. It's safe to do so regardless of original value of dst->anon_vma
because dst->anon_vma doesn't have valid meaning if anon_vma_clone() fails.
Besides, callers don't care dst->anon_vma in such case neither.

Signed-off-by: Leon Yu <chianglungyu@gmail.com>
Fixes: 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
Cc: stable@vger.kernel.org # v3.19
---
 mm/rmap.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/rmap.c b/mm/rmap.c
index 5e3e090..bed3cf2 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -287,6 +287,13 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
 	return 0;
 
  enomem_failure:
+	/*
+	 * dst->anon_vma is dropped here otherwise its degree can be incorrectly
+	 * decremented in unlink_anon_vmas().
+	 * We can safely do this because calllers of anon_vma_clone() wouldn't
+	 * care dst->anon_vma if anon_vma_clone() failed.
+	 */
+	dst->anon_vma = NULL;
 	unlink_anon_vmas(dst);
 	return -ENOMEM;
 }
-- 
2.3.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>