[PATCH] mm/khugepaged: Convert VM_BUG_ON() to collapse fail

[PATCH] mm/khugepaged: Convert VM_BUG_ON() to collapse fail

[Kirill A. Shutemov]

khugepaged is not yet able to convert PTE-mapped huge pages back to PMD
mapped. We do not collapse such pages. See check khugepaged_scan_pmd().

But if between khugepaged_scan_pmd() and __collapse_huge_page_isolate()
somebody managed to instantiate THP in the range and then split the PMD
back to PTEs we would have a problem -- VM_BUG_ON_PAGE(PageCompound(page))
will get triggered.

It's possible since we drop mmap_sem during collapse to re-take for
write.

Replace the VM_BUG_ON() with graceful collapse fail.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Fixes: b1caa957ae6d ("khugepaged: ignore pmd tables with THP mapped with ptes")
---
 mm/khugepaged.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index b7e2268dfc9a..c15da1ea7e63 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -530,7 +530,12 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma,
 			goto out;
 		}
 
-		VM_BUG_ON_PAGE(PageCompound(page), page);
+		/* TODO: teach khugepaged to collapse THP mapped with pte */
+		if (PageCompound(page)) {
+			result = SCAN_PAGE_COMPOUND;
+			goto out;
+		}
+
 		VM_BUG_ON_PAGE(!PageAnon(page), page);
 
 		/*
-- 
2.16.1

Re: [PATCH] mm/khugepaged: Convert VM_BUG_ON() to collapse fail

[Andrew Morton]

On Thu, 15 Mar 2018 18:23:53 +0300 "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> wrote:

> khugepaged is not yet able to convert PTE-mapped huge pages back to PMD
> mapped. We do not collapse such pages. See check khugepaged_scan_pmd().
> 
> But if between khugepaged_scan_pmd() and __collapse_huge_page_isolate()
> somebody managed to instantiate THP in the range and then split the PMD
> back to PTEs we would have a problem -- VM_BUG_ON_PAGE(PageCompound(page))
> will get triggered.
> 
> It's possible since we drop mmap_sem during collapse to re-take for
> write.
> 
> Replace the VM_BUG_ON() with graceful collapse fail.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: b1caa957ae6d ("khugepaged: ignore pmd tables with THP mapped with ptes")

Jan 2016.  Do we need a cc:stable?

Re: [PATCH] mm/khugepaged: Convert VM_BUG_ON() to collapse fail

[Kirill A. Shutemov]

On Thu, Mar 15, 2018 at 11:04:53PM +0000, Andrew Morton wrote:
> On Thu, 15 Mar 2018 18:23:53 +0300 "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> wrote:
> 
> > khugepaged is not yet able to convert PTE-mapped huge pages back to PMD
> > mapped. We do not collapse such pages. See check khugepaged_scan_pmd().
> > 
> > But if between khugepaged_scan_pmd() and __collapse_huge_page_isolate()
> > somebody managed to instantiate THP in the range and then split the PMD
> > back to PTEs we would have a problem -- VM_BUG_ON_PAGE(PageCompound(page))
> > will get triggered.
> > 
> > It's possible since we drop mmap_sem during collapse to re-take for
> > write.
> > 
> > Replace the VM_BUG_ON() with graceful collapse fail.
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Fixes: b1caa957ae6d ("khugepaged: ignore pmd tables with THP mapped with ptes")
> 
> Jan 2016.  Do we need a cc:stable?

Yes, please. I forgot to put it.

-- 
 Kirill A. Shutemov