[PATCH] more ZERO_PAGE handling in follow_page()

[PATCH] more ZERO_PAGE handling in follow_page()

[KAMEZAWA Hiroyuki]

Rewrote the description of patch. (no changes in the logic.)

Thank you for all help.
-Kame
==
follow_page() is called from get_user_pages(), which returns specified user page.
follow_page() can return 1) a page or 2) NULL or 3)ZERO_PAGE.
If NULL, handle_mm_fault() is called.

Now, follow_page() to unused pte returns NULL if page table exists. As a result
get_user_pages() calls handle_mm_fault() and allocate new memory.
This behavior increases memory consumption at coredump, which does
read-once-but-never-written page fault.
By returning ZERO_PAGE() against READ/ANON request, we can avoid it.

(Because exec's arguments copy needs to call handle_mm_fault at WRITE/ANON
 request, we just handle READ/ANON case here.)

Change log:
  - Rewrote patch description and Added comments.
  - fixed to check pte_present()/pte_none() in proper way.


Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Tested-by: Tony Battersby <tonyb@cybernetics.com>
Acked-by: Nick Piggin <npiggin@suse.de>

---
 mm/memory.c |   16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

Index: linux-2.6.25/mm/memory.c
===================================================================
--- linux-2.6.25.orig/mm/memory.c
+++ linux-2.6.25/mm/memory.c
@@ -926,15 +926,15 @@ struct page *follow_page(struct vm_area_
 	page = NULL;
 	pgd = pgd_offset(mm, address);
 	if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd)))
-		goto no_page_table;
+		goto null_or_zeropage;
 
 	pud = pud_offset(pgd, address);
 	if (pud_none(*pud) || unlikely(pud_bad(*pud)))
-		goto no_page_table;
+		goto null_or_zeropage;
 	
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))
-		goto no_page_table;
+		goto null_or_zeropage;
 
 	if (pmd_huge(*pmd)) {
 		BUG_ON(flags & FOLL_GET);
@@ -947,8 +947,14 @@ struct page *follow_page(struct vm_area_
 		goto out;
 
 	pte = *ptep;
-	if (!pte_present(pte))
+	if (!pte_present(pte)) {
+		/* Read fault to empty pte can return ZERO_PAGE */
+		if (!(flags & FOLL_WRITE) && pte_none(pte)) {
+			pte_unmap_unlock(ptep, ptl);
+			goto null_or_zeropage;
+		}
 		goto unlock;
+	}
 	if ((flags & FOLL_WRITE) && !pte_write(pte))
 		goto unlock;
 	page = vm_normal_page(vma, address, pte);
@@ -968,7 +974,7 @@ unlock:
 out:
 	return page;
 
-no_page_table:
+null_or_zeropage:
 	/*
 	 * When core dumping an enormous anonymous area that nobody
 	 * has touched so far, we don't want to allocate page tables.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] more ZERO_PAGE handling in follow_page()

[Andrew Morton]

On Wed, 7 May 2008 16:36:43 +0900
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:

> Rewrote the description of patch. (no changes in the logic.)
> 
> Thank you for all help.
> -Kame
> ==
> follow_page() is called from get_user_pages(), which returns specified user page.
> follow_page() can return 1) a page or 2) NULL or 3)ZERO_PAGE.
> If NULL, handle_mm_fault() is called.
> 
> Now, follow_page() to unused pte returns NULL if page table exists. As a result
> get_user_pages() calls handle_mm_fault() and allocate new memory.
> This behavior increases memory consumption at coredump, which does
> read-once-but-never-written page fault.
> By returning ZERO_PAGE() against READ/ANON request, we can avoid it.
> 
> (Because exec's arguments copy needs to call handle_mm_fault at WRITE/ANON
>  request, we just handle READ/ANON case here.)
> 
> Change log:
>   - Rewrote patch description and Added comments.
>   - fixed to check pte_present()/pte_none() in proper way.

So... how serious is the problem which we're fixing here?

I can see that if one is core-dumping large sparse address spaces this
could improve things a lot, but please help us understand the implications
so we can decide whether we need this in 2.6.26, thanks.


> Index: linux-2.6.25/mm/memory.c
> ===================================================================
> --- linux-2.6.25.orig/mm/memory.c
> +++ linux-2.6.25/mm/memory.c
> @@ -926,15 +926,15 @@ struct page *follow_page(struct vm_area_
>  	page = NULL;
>  	pgd = pgd_offset(mm, address);
>  	if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd)))
> -		goto no_page_table;
> +		goto null_or_zeropage;
>  
>  	pud = pud_offset(pgd, address);
>  	if (pud_none(*pud) || unlikely(pud_bad(*pud)))
> -		goto no_page_table;
> +		goto null_or_zeropage;
>  	
>  	pmd = pmd_offset(pud, address);
>  	if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))

The mainline kernel does not have " || unlikely(pmd_bad(*pmd))" here. 
That got changed yesterday by

commit aeed5fce37196e09b4dac3a1c00d8b7122e040ce
Author: Hugh Dickins <hugh@veritas.com>
Date:   Tue May 6 20:49:23 2008 +0100

    x86: fix PAE pmd_bad bootup warning

So please confirm that the patch which I merged is still OK (I'd be
surprised if it isn't...)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] more ZERO_PAGE handling in follow_page()

[KAMEZAWA Hiroyuki]

On Wed, 7 May 2008 14:25:39 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Wed, 7 May 2008 16:36:43 +0900
> KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:
> 
> > Rewrote the description of patch. (no changes in the logic.)
> > 
> > Thank you for all help.
> > -Kame
> > ==
> > follow_page() is called from get_user_pages(), which returns specified user page.
> > follow_page() can return 1) a page or 2) NULL or 3)ZERO_PAGE.
> > If NULL, handle_mm_fault() is called.
> > 
> > Now, follow_page() to unused pte returns NULL if page table exists. As a result
> > get_user_pages() calls handle_mm_fault() and allocate new memory.
> > This behavior increases memory consumption at coredump, which does
> > read-once-but-never-written page fault.
> > By returning ZERO_PAGE() against READ/ANON request, we can avoid it.
> > 
> > (Because exec's arguments copy needs to call handle_mm_fault at WRITE/ANON
> >  request, we just handle READ/ANON case here.)
> > 
> > Change log:
> >   - Rewrote patch description and Added comments.
> >   - fixed to check pte_present()/pte_none() in proper way.
> 
> So... how serious is the problem which we're fixing here?
> 
> I can see that if one is core-dumping large sparse address spaces this
> could improve things a lot, but please help us understand the implications
> so we can decide whether we need this in 2.6.26, thanks.
> 
I don't think this is a fix for serious trouble just a improvement.
But not sure on small systems....

a consideration.
== at coredump before patch
  killed by something
     -> generate core dump
           -> allocate "a" page before starting I/O even if a page is empty
                 -> do I/O
A page which is not mapped but there is page tables will be written out.
Here, newly allocated page is mapped_and_used after I/O. So, when we
reclaim this page, we need swap. This means terrible slow down or we cannot
go ahead when we exhaust swap.

A user can avoid this kind ot situation by setting rlimit. (and RLIMIT_CORE
is 0 at default.) or set overcommit memory or set dirty_ratio to very small.
But one terrible thing which I can think of is  that a process in coredump
cannot be killed. So once this happens, a user have to be patient or reboot
system.

It seems this patch can help coredump in following system 
  - swapless or An application which can generate core has some amount of
    ANON memory and it is multi-threaded. (pthread's stack is typical case
    for this memory usage.)
  - RLIMIT_CORE is RLIMIT_INIFINITY
  - core_pattern is file.
  - Don't have enough memory to do buffer I/O at coredump.
  - dirty_ratio is default.

But an application on this kind of system tends to be well controlled.

> 
> > Index: linux-2.6.25/mm/memory.c
> > ===================================================================
> > --- linux-2.6.25.orig/mm/memory.c
> > +++ linux-2.6.25/mm/memory.c
> > @@ -926,15 +926,15 @@ struct page *follow_page(struct vm_area_
> >  	page = NULL;
> >  	pgd = pgd_offset(mm, address);
> >  	if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd)))
> > -		goto no_page_table;
> > +		goto null_or_zeropage;
> >  
> >  	pud = pud_offset(pgd, address);
> >  	if (pud_none(*pud) || unlikely(pud_bad(*pud)))
> > -		goto no_page_table;
> > +		goto null_or_zeropage;
> >  	
> >  	pmd = pmd_offset(pud, address);
> >  	if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))
> 
> The mainline kernel does not have " || unlikely(pmd_bad(*pmd))" here. 
> That got changed yesterday by
> 
> commit aeed5fce37196e09b4dac3a1c00d8b7122e040ce
> Author: Hugh Dickins <hugh@veritas.com>
> Date:   Tue May 6 20:49:23 2008 +0100
> 
>     x86: fix PAE pmd_bad bootup warning
> 
> So please confirm that the patch which I merged is still OK (I'd be
> surprised if it isn't...)
> 
Ok, I'll check and update this against the newest git tree.
(But may took some hours.)

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH] more ZERO_PAGE handling in follow_page() v2

[KAMEZAWA Hiroyuki]

updated against 2.6.26-rc1-git6.
Tested on a x86-64 server. No troubles in generated coredump file.
It seems to work well.

Thanks,
-Kame
==
follow_page() is called from get_user_pages(), which returns specified user page
in the kernel context. follow_page() can return 1) a page or 2) NULL or
3)ZERO_PAGE.

Now, follow_page() to unused pte returns NULL if page table exists. As a result
get_user_pages() calls handle_mm_fault() and allocate new memory.
This behavior increases memory consumption at coredump, which does
read-once-but-never-written page fault.
By returning ZERO_PAGE() against READ/ANON request, we can avoid it.

(Because exec's arguments copy needs to call handle_mm_fault at WRITE/ANON
 request, we just handle READ/ANON case here.)

Change log:
  - Rebased agasinst 2.6.26-rc1-git6
  - Rewrote patch description and Added comments.
  - fixed to check pte_present()/pte_none() in proper way.


Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

---
 mm/memory.c |   18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

Index: linux-2.6.26-rc1/mm/memory.c
===================================================================
--- linux-2.6.26-rc1.orig/mm/memory.c
+++ linux-2.6.26-rc1/mm/memory.c
@@ -962,15 +962,15 @@ struct page *follow_page(struct vm_area_
 	page = NULL;
 	pgd = pgd_offset(mm, address);
 	if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd)))
-		goto no_page_table;
+		goto null_or_zeropage;
 
 	pud = pud_offset(pgd, address);
 	if (pud_none(*pud) || unlikely(pud_bad(*pud)))
-		goto no_page_table;
+		goto null_or_zeropage;
 	
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
-		goto no_page_table;
+		goto null_or_zeropage;
 
 	if (pmd_huge(*pmd)) {
 		BUG_ON(flags & FOLL_GET);
@@ -979,15 +979,21 @@ struct page *follow_page(struct vm_area_
 	}
 
 	if (unlikely(pmd_bad(*pmd)))
-		goto no_page_table;
+		goto null_or_zeropage;
 
 	ptep = pte_offset_map_lock(mm, pmd, address, &ptl);
 	if (!ptep)
 		goto out;
 
 	pte = *ptep;
-	if (!pte_present(pte))
+	if (!pte_present(pte)) {
+		/* Read fault to empty pte can return ZERO_PAGE */
+		if (!(flags & FOLL_WRITE) && pte_none(pte)) {
+			pte_unmap_unlock(ptep, ptl);
+			goto null_or_zeropage;
+		}
 		goto unlock;
+	}
 	if ((flags & FOLL_WRITE) && !pte_write(pte))
 		goto unlock;
 	page = vm_normal_page(vma, address, pte);
@@ -1007,7 +1013,7 @@ unlock:
 out:
 	return page;
 
-no_page_table:
+null_or_zeropage:
 	/*
 	 * When core dumping an enormous anonymous area that nobody
 	 * has touched so far, we don't want to allocate page tables.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>