[PATCH 5.10/5.15] mm/hugetlb: fix assertion splat in hugetlb_split()

[Fedor Pchelkin <pchelkin@ispras.ru>]

No upstream commit exists for this patch.

The following assertion is firing on 5.10 to 6.1 stable kernels after
backport of upstream commit 081056dc00a2 ("mm/hugetlb: unshare page tables
during VMA split, not before"):

  WARNING: CPU: 0 PID: 11489 at include/linux/fs.h:503 i_mmap_assert_write_locked include/linux/fs.h:503 [inline]
  WARNING: CPU: 0 PID: 11489 at include/linux/fs.h:503 hugetlb_split+0x267/0x300 mm/hugetlb.c:4917
  Modules linked in:
  CPU: 0 PID: 11489 Comm: syz-executor.4 Not tainted 6.1.142-syzkaller-00296-gfd0df5221577 #0
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
  RIP: 0010:i_mmap_assert_write_locked include/linux/fs.h:503 [inline]
  RIP: 0010:hugetlb_split+0x267/0x300 mm/hugetlb.c:4917
  Call Trace:
   <TASK>
   __vma_adjust+0xd73/0x1c10 mm/mmap.c:736
   vma_adjust include/linux/mm.h:2745 [inline]
   __split_vma+0x459/0x540 mm/mmap.c:2385
   do_mas_align_munmap+0x5f2/0xf10 mm/mmap.c:2497
   do_mas_munmap+0x26c/0x2c0 mm/mmap.c:2646
   __mmap_region mm/mmap.c:2694 [inline]
   mmap_region+0x19f/0x1770 mm/mmap.c:2912
   do_mmap+0x84b/0xf20 mm/mmap.c:1432
   vm_mmap_pgoff+0x1af/0x280 mm/util.c:520
   ksys_mmap_pgoff+0x41f/0x5a0 mm/mmap.c:1478
   do_syscall_x64 arch/x86/entry/common.c:51 [inline]
   do_syscall_64+0x35/0x80 arch/x86/entry/common.c:81
   entry_SYSCALL_64_after_hwframe+0x6e/0xd8
  RIP: 0033:0x46a269
   </TASK>

Those branches lack commit ccf1d78d8b86 ("mm/mmap: move vma_prepare before
vma_adjust_trans_huge") so the needed locks are taken just after the newly
added hugetlb_split().

Adjust the position of vma_adjust_trans_huge() blocks with the lock-taking
code.

Found by Linux Verification Center (linuxtesting.org) with Syzkaller.

Fixes: 081056dc00a2 ("mm/hugetlb: unshare page tables during VMA split, not before")
Signed-off-by: Fedor Pchelkin <pchelkin@ispras.ru>
---

Tested with testcases/kernel/mem/hugetlb/hugemmap suite provided by LTP.

For the report see:
https://lore.kernel.org/stable/CAG48ez3LqUwXxhRY56tqQCpfGJsJ-GeXFG9FCcTZEBo2bWOK8Q@mail.gmail.com/T/#u

 mm/mmap.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/mm/mmap.c b/mm/mmap.c
index 8c188ed3738a..13669a33a515 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -832,16 +832,6 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start,
 		}
 	}
 again:
-	/*
-	 * Get rid of huge pages and shared page tables straddling the split
-	 * boundary.
-	 */
-	vma_adjust_trans_huge(orig_vma, start, end, adjust_next);
-	if (is_vm_hugetlb_page(orig_vma)) {
-		hugetlb_split(orig_vma, start);
-		hugetlb_split(orig_vma, end);
-	}
-
 	if (file) {
 		mapping = file->f_mapping;
 		root = &mapping->i_mmap;
@@ -881,6 +871,16 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start,
 			vma_interval_tree_remove(next, root);
 	}
 
+	/*
+	 * Get rid of huge pages and shared page tables straddling the split
+	 * boundary.
+	 */
+	vma_adjust_trans_huge(orig_vma, start, end, adjust_next);
+	if (is_vm_hugetlb_page(orig_vma)) {
+		hugetlb_split(orig_vma, start);
+		hugetlb_split(orig_vma, end);
+	}
+
 	if (start != vma->vm_start) {
 		vma->vm_start = start;
 		start_changed = true;
-- 
2.50.0