[PATCH 2/3] mm: Move the tlb flushing into free_pgtables

[Haggai Eran <haggaie@mellanox.com>]

From: Sagi Grimberg <sagig@mellanox.com>

The conversion of the locks taken for reverse map scanning would
require taking sleeping locks in free_pgtables() and we cannot sleep
while gathering pages for a tlb flush.

Move the tlb_gather/tlb_finish call to free_pgtables() to be done
for each vma. This may add a number of tlb flushes depending on the
number of vmas that cannot be coalesced into one.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Haggai Eran <haggaie@mellanox.com>
---
 mm/memory.c | 3 +++
 mm/mmap.c   | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index b657a2e..e721432 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -553,6 +553,7 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma,
 		unlink_file_vma(vma);
 
 		if (is_vm_hugetlb_page(vma)) {
+			tlb_gather_mmu(tlb, vma->vm_mm, 0);
 			hugetlb_free_pgd_range(tlb, addr, vma->vm_end,
 				floor, next? next->vm_start: ceiling);
 		} else {
@@ -566,9 +567,11 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma,
 				unlink_anon_vmas(vma);
 				unlink_file_vma(vma);
 			}
+			tlb_gather_mmu(tlb, vma->vm_mm, 0);
 			free_pgd_range(tlb, addr, vma->vm_end,
 				floor, next? next->vm_start: ceiling);
 		}
+		tlb_finish_mmu(tlb, addr, vma->vm_end);
 		vma = next;
 	}
 }
diff --git a/mm/mmap.c b/mm/mmap.c
index e3e8691..731da04 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1912,9 +1912,9 @@ static void unmap_region(struct mm_struct *mm,
 	tlb_gather_mmu(&tlb, mm, 0);
 	update_hiwater_rss(mm);
 	unmap_vmas(&tlb, vma, start, end);
+	tlb_finish_mmu(&tlb, start, end);
 	free_pgtables(&tlb, vma, prev ? prev->vm_end : FIRST_USER_ADDRESS,
 				 next ? next->vm_start : 0);
-	tlb_finish_mmu(&tlb, start, end);
 }
 
 /*
@@ -2295,8 +2295,8 @@ void exit_mmap(struct mm_struct *mm)
 	/* Use -1 here to ensure all VMAs in the mm are unmapped */
 	unmap_vmas(&tlb, vma, 0, -1);
 
-	free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, 0);
 	tlb_finish_mmu(&tlb, 0, -1);
+	free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, 0);
 
 	/*
 	 * Walk the list again, actually closing and freeing it,
-- 
1.7.11.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>