[rfc] mm: more likely reclaim MADV_SEQUENTIAL mappings

[rfc] mm: more likely reclaim MADV_SEQUENTIAL mappings

[Johannes Weiner]

File pages mapped only in sequentially read mappings are perfect
reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like weak references, its
pages will be reclaimed unless they have a strong reference from a
normal mapping as well.

The patch changes the reclaim and the unmap path where they check if
the page has been referenced.  In both cases, accesses through
sequentially read mappings will be ignored.

Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
---

I'm afraid this is now quite a bit more aggressive than the earlier
version.  When the fault path did a mark_page_access(), we wouldn't
reclaim a page when it has been faulted into several MADV_SEQUENTIAL
mappings but now we ignore *every* activity through such a mapping.

What do you think?

Perhaps we should note a reference if there are two or more accesses
through sequentially read mappings?

 mm/memory.c |    3 ++-
 mm/rmap.c   |   13 +++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)

--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
 		goto out_unmap;
 	}
 
-	if (ptep_clear_flush_young_notify(vma, address, pte))
-		referenced++;
+	if (ptep_clear_flush_young_notify(vma, address, pte)) {
+		/*
+		 * Don't treat a reference through a sequentially read
+		 * mapping as such.  If the page has been used in
+		 * another mapping, we will catch it; if this other
+		 * mapping is already gone, the unmap path will have
+		 * set PG_referenced or activated the page.
+		 */
+		if (!VM_SequentialReadHint(vma))
+			referenced++;
+	}
 
 	/* Pretend the page is referenced if the task has the
 	   swap token and is in the middle of a page fault. */
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
 			else {
 				if (pte_dirty(ptent))
 					set_page_dirty(page);
-				if (pte_young(ptent))
+				if (pte_young(ptent) &&
+						!VM_SequentialReadHint(vma))
 					mark_page_accessed(page);
 				file_rss--;
 			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc] mm: more likely reclaim MADV_SEQUENTIAL mappings

[Nick Piggin]

On Tue, Oct 21, 2008 at 12:32:25PM +0200, Johannes Weiner wrote:
> File pages mapped only in sequentially read mappings are perfect
> reclaim canditates.
> 
> This makes MADV_SEQUENTIAL mappings behave like weak references, its
> pages will be reclaimed unless they have a strong reference from a
> normal mapping as well.
> 
> The patch changes the reclaim and the unmap path where they check if
> the page has been referenced.  In both cases, accesses through
> sequentially read mappings will be ignored.
> 
> Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
> ---
> 
> I'm afraid this is now quite a bit more aggressive than the earlier
> version.  When the fault path did a mark_page_access(), we wouldn't
> reclaim a page when it has been faulted into several MADV_SEQUENTIAL
> mappings but now we ignore *every* activity through such a mapping.
> 
> What do you think?

I think it's OK. MADV_SEQUENTIAL man page explicitly states they can
soon be freed, and we won't DoS anybody else's working set because we
are only ignoring referenced from MADV_SEQUENTIAL ptes.

It's annoying to put in extra banches especially in the unmap path.
Oh well... (at least if you can mark them as likely()).

Otherwise, this patch looks much better to me now.

Thanks,
Nick

> 
> Perhaps we should note a reference if there are two or more accesses
> through sequentially read mappings?
> 
>  mm/memory.c |    3 ++-
>  mm/rmap.c   |   13 +++++++++++--
>  2 files changed, 13 insertions(+), 3 deletions(-)
> 
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
>  		goto out_unmap;
>  	}
>  
> -	if (ptep_clear_flush_young_notify(vma, address, pte))
> -		referenced++;
> +	if (ptep_clear_flush_young_notify(vma, address, pte)) {
> +		/*
> +		 * Don't treat a reference through a sequentially read
> +		 * mapping as such.  If the page has been used in
> +		 * another mapping, we will catch it; if this other
> +		 * mapping is already gone, the unmap path will have
> +		 * set PG_referenced or activated the page.
> +		 */
> +		if (!VM_SequentialReadHint(vma))
> +			referenced++;
> +	}
>  
>  	/* Pretend the page is referenced if the task has the
>  	   swap token and is in the middle of a page fault. */
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
>  			else {
>  				if (pte_dirty(ptent))
>  					set_page_dirty(page);
> -				if (pte_young(ptent))
> +				if (pte_young(ptent) &&
> +						!VM_SequentialReadHint(vma))
>  					mark_page_accessed(page);
>  				file_rss--;
>  			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Nick Piggin <npiggin@suse.de> writes:

>> I'm afraid this is now quite a bit more aggressive than the earlier
>> version.  When the fault path did a mark_page_access(), we wouldn't
>> reclaim a page when it has been faulted into several MADV_SEQUENTIAL
>> mappings but now we ignore *every* activity through such a mapping.
>> 
>> What do you think?
>
> I think it's OK. MADV_SEQUENTIAL man page explicitly states they can
> soon be freed, and we won't DoS anybody else's working set because we
> are only ignoring referenced from MADV_SEQUENTIAL ptes.
>
> It's annoying to put in extra banches especially in the unmap path.
> Oh well... (at least if you can mark them as likely()).

Okay, added those.  Second round:

---
File pages mapped only in sequentially read mappings are perfect
reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like a weak references,
their pages will be reclaimed unless they have a strong reference from
a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if
the page has been referenced.  In both cases, accesses through
sequentially read mappings will be ignored.

Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
---
II: add likely()s to mitigate the extra branches a bit as to Nick's
    suggestion

 mm/memory.c |    3 ++-
 mm/rmap.c   |   13 +++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)

--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
 		goto out_unmap;
 	}
 
-	if (ptep_clear_flush_young_notify(vma, address, pte))
-		referenced++;
+	if (ptep_clear_flush_young_notify(vma, address, pte)) {
+		/*
+		 * Don't treat a reference through a sequentially read
+		 * mapping as such.  If the page has been used in
+		 * another mapping, we will catch it; if this other
+		 * mapping is already gone, the unmap path will have
+		 * set PG_referenced or activated the page.
+		 */
+		if (likely(!VM_SequentialReadHint(vma)))
+			referenced++;
+	}
 
 	/* Pretend the page is referenced if the task has the
 	   swap token and is in the middle of a page fault. */
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
 			else {
 				if (pte_dirty(ptent))
 					set_page_dirty(page);
-				if (pte_young(ptent))
+				if (pte_young(ptent) &&
+				    likely(!VM_SequentialReadHint(vma)))
 					mark_page_accessed(page);
 				file_rss--;
 			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Andrew Morton]

On Tue, 21 Oct 2008 13:33:45 +0200
Johannes Weiner <hannes@saeurebad.de> wrote:

> File pages mapped only in sequentially read mappings are perfect
> reclaim canditates.
> 
> This makes MADV_SEQUENTIAL mappings behave like a weak references,
> their pages will be reclaimed unless they have a strong reference from
> a normal mapping as well.
> 
> The patch changes the reclaim and the unmap path where they check if
> the page has been referenced.  In both cases, accesses through
> sequentially read mappings will be ignored.
> 
> Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
> ---
> II: add likely()s to mitigate the extra branches a bit as to Nick's
>     suggestion

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Andrew Morton <akpm@linux-foundation.org> writes:

> On Tue, 21 Oct 2008 13:33:45 +0200
> Johannes Weiner <hannes@saeurebad.de> wrote:
>
>> File pages mapped only in sequentially read mappings are perfect
>> reclaim canditates.
>> 
>> This makes MADV_SEQUENTIAL mappings behave like a weak references,
>> their pages will be reclaimed unless they have a strong reference from
>> a normal mapping as well.
>> 
>> The patch changes the reclaim and the unmap path where they check if
>> the page has been referenced.  In both cases, accesses through
>> sequentially read mappings will be ignored.
>> 
>> Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
>> ---
>> II: add likely()s to mitigate the extra branches a bit as to Nick's
>>     suggestion
>
> Is http://hannes.saeurebad.de/madvseq/ still true with this version?

No, sorry, still running benchmarks on this version.  Coming up soon...

	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Johannes Weiner <hannes@saeurebad.de> writes:

> Andrew Morton <akpm@linux-foundation.org> writes:
>
>> On Tue, 21 Oct 2008 13:33:45 +0200
>> Johannes Weiner <hannes@saeurebad.de> wrote:
>>
>>> File pages mapped only in sequentially read mappings are perfect
>>> reclaim canditates.
>>> 
>>> This makes MADV_SEQUENTIAL mappings behave like a weak references,
>>> their pages will be reclaimed unless they have a strong reference from
>>> a normal mapping as well.
>>> 
>>> The patch changes the reclaim and the unmap path where they check if
>>> the page has been referenced.  In both cases, accesses through
>>> sequentially read mappings will be ignored.
>>> 
>>> Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
>>> ---
>>> II: add likely()s to mitigate the extra branches a bit as to Nick's
>>>     suggestion
>>
>> Is http://hannes.saeurebad.de/madvseq/ still true with this version?
>
> No, sorry, still running benchmarks on this version.  Coming up
> soon...

Ok, reran the tests I used for the data on this website and updated it.
Take a look.  I am quite overwhelmed by the results, hehe.

Kosaki-san, could you perhaps run the tests you did for the previous
patch on this one, too?  I am not getting any stable results for
throughput measuring...

> 	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[KOSAKI Motohiro]

> >> Is http://hannes.saeurebad.de/madvseq/ still true with this version?
> >
> > No, sorry, still running benchmarks on this version.  Coming up
> > soon...
> 
> Ok, reran the tests I used for the data on this website and updated it.
> Take a look.  I am quite overwhelmed by the results, hehe.
> 
> Kosaki-san, could you perhaps run the tests you did for the previous
> patch on this one, too?  I am not getting any stable results for
> throughput measuring...

Usually, any reclaim throughput mesurement isn't stable.
Then I used an average of five times mesurement.


Unfortunately, I can't understand I should mesure which patch combination
because you and Nick post many patches of this issue related yesterday.
Please let me know it?



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> writes:

>> >> Is http://hannes.saeurebad.de/madvseq/ still true with this version?
>> >
>> > No, sorry, still running benchmarks on this version.  Coming up
>> > soon...
>> 
>> Ok, reran the tests I used for the data on this website and updated it.
>> Take a look.  I am quite overwhelmed by the results, hehe.
>> 
>> Kosaki-san, could you perhaps run the tests you did for the previous
>> patch on this one, too?  I am not getting any stable results for
>> throughput measuring...
>
> Usually, any reclaim throughput mesurement isn't stable.
> Then I used an average of five times mesurement.

Ah, okay.  I will give it another spin, too.

> Unfortunately, I can't understand I should mesure which patch combination
> because you and Nick post many patches of this issue related yesterday.
> Please let me know it?

mmotm
- the old mm-more-likely-reclaim-madv_sequential-mappings
+ Nick's mm: dont mark_page_accessed in fault path (from yesterday)
+ Apply this patch

Andrew's tree (not yet released) already has the first two changes, so
if he releases a new mmotm in the meantime, you only need this patch on
top of it.

        Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Andrew Morton]

On Wed, 22 Oct 2008 09:15:14 +0200 Johannes Weiner <hannes@saeurebad.de> wrote:

> Andrew's tree (not yet released) already has the first two changes, so
> if he releases a new mmotm in the meantime, you only need this patch on
> top of it.
> 

OK, there's one there now.

It gets multiple definitions of elfcorehdr_addr if you pick an
unfortunate config but I didn't bother fixing that because it'll
probably go away next time I merge everything.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Finally got some half-way stable time numbers for this patch.

My approach was to generate background activity by dd'ing my hard-drive
to /dev/null and meanwhile copy a 1G file with memcpy() between two
mmaps and then the same file again with MADV_SEQUENTIAL mmaps.  The
numbers below are averages/standard deviations from 8 copy iterations.

I dropped the caches before each copy and waited for them to become
repopulated by the background dd.

The numbers in brackets are the std dev.

mmotm:
    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]
   madvise  user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Well, time-wise not sooo much of an improvement.  But given the
massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]
I'm still looking forward to Kosaki-san's throughput measurements :)

	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[KOSAKI Motohiro]

> mmotm:
>     normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
>    madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
> mmotm+patch:
>     normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]
>    madvise  user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]
> 
> Well, time-wise not sooo much of an improvement.  But given the
> massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

My first impression, this result mean the patch is not so useful.
But anyway, I mesured it again because I think Nick's opinion is very
reasonable and I don't know your mesurement condition so detail.



> I'm still looking forward to Kosaki-san's throughput measurements :)

I'm sorry for late responce.
but I'd like to you know this mesurement need spent long time in my time.


1. copybench (http://code.google.com/p/copybench/)

   my machine mem:   8GB
   target file size: 10GB (filesize > system mem)


                         mmotm-1022  + the patch
   ==============================================================
   rw_cp                 6:32        6:34
   rw_fadv_cp            6:34        6:35
   mm_sync_cp            6:15        6:16
   mm_sync_madv_cp       6:19        6:14
   mw_cp                 6:11        6:12
   mw_madv_cp            6:13        6:12


MADV_SEQUENTIAL decrease performance a bit on mmotm.
but the patch fix it.



2. MADV_SEQUENTIAL vs dbench

                         mmotm1022   + the patch
   ==============================================================
   mm_sync_madv_cp       6:29        6:19           (min:sec)
   dbench throughput     11.633      14.4045        (MB/s)
   dbench latency        65628       18565          (ms)


mmotm's copy decrease performance largely. but the patch decrease it a bit. 
dbench throuput improve about 25%, latency improve about 3.5 times.


So, I think the patch better than v1 and we should appreciate for Nick's
good suggestion.

Hanns, Actually I recomend to spent a bit more time for proper benchmark design and settings.


	Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> writes:

> I'm sorry for late responce.
> but I'd like to you know this mesurement need spent long time in my
> time.

No problem, I really appreciate this.

> Hanns, Actually I recomend to spent a bit more time for proper
> benchmark design and settings.

I shall dig into it.

> 	Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Thanks a lot,

        Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Rik van Riel]

KOSAKI Motohiro wrote:
>> mmotm:
>>     normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
>>    madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
>> mmotm+patch:
>>     normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]
>>    madvise  user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]
>>
>> Well, time-wise not sooo much of an improvement.  But given the
>> massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]
> 
> My first impression, this result mean the patch is not so useful.
> But anyway, I mesured it again because I think Nick's opinion is very
> reasonable and I don't know your mesurement condition so detail.

It may not make much of a difference if the MADV_SEQUENTIAL
program is the only thing running on the system.

However, the goal of MADV_SEQUENTIAL is to make sure that a
streaming mapping does not kick the data from other programs
out of memory.  The patch should take care of that very well.

-- 
All rights reversed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Rik van Riel <riel@redhat.com> writes:

> KOSAKI Motohiro wrote:
>>> mmotm:
>>>     normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
>>>    madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
>>> mmotm+patch:
>>>     normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]
>>>    madvise  user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]
>>>
>>> Well, time-wise not sooo much of an improvement.  But given the
>>> massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]
>>
>> My first impression, this result mean the patch is not so useful.
>> But anyway, I mesured it again because I think Nick's opinion is very
>> reasonable and I don't know your mesurement condition so detail.
>
> It may not make much of a difference if the MADV_SEQUENTIAL
> program is the only thing running on the system.

As said, I had a big dd running in the background.  The box has only
768mb RAM, so there really was VM activity going on.

And given this small standard deviations, the numbers seem pretty
stable.  Even if I had taken more samples, I highly doubt that they
would have looked much different.

Perhaps still not enough VM pressure...

        Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[Johannes Weiner]

Johannes Weiner <hannes@saeurebad.de> writes:

> Rik van Riel <riel@redhat.com> writes:
>
>> KOSAKI Motohiro wrote:
>>>> mmotm:
>>>>     normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
>>>>    madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
>>>> mmotm+patch:
>>>>     normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]
>>>>    madvise  user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]
>>>>
>>>> Well, time-wise not sooo much of an improvement.  But given the
>>>> massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]
>>>
>>> My first impression, this result mean the patch is not so useful.
>>> But anyway, I mesured it again because I think Nick's opinion is very
>>> reasonable and I don't know your mesurement condition so detail.
>>
>> It may not make much of a difference if the MADV_SEQUENTIAL
>> program is the only thing running on the system.
>
> As said, I had a big dd running in the background.  The box has only
> 768mb RAM, so there really was VM activity going on.
>
> And given this small standard deviations, the numbers seem pretty
> stable.  Even if I had taken more samples, I highly doubt that they
> would have looked much different.
>
> Perhaps still not enough VM pressure...

Okay, Rik was kind enough to explain me what was wrong about my approach
on IRC.

Yes, the benchmark pretty much missed the point.  Please ignore.

        Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

[KOSAKI Motohiro]

>>> Well, time-wise not sooo much of an improvement.  But given the
>>> massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]
>>
>> My first impression, this result mean the patch is not so useful.
>> But anyway, I mesured it again because I think Nick's opinion is very
>> reasonable and I don't know your mesurement condition so detail.
>
> It may not make much of a difference if the MADV_SEQUENTIAL
> program is the only thing running on the system.
>
> However, the goal of MADV_SEQUENTIAL is to make sure that a
> streaming mapping does not kick the data from other programs
> out of memory.  The patch should take care of that very well.


Well, my second test(following) indicate it IMO.

> 2. MADV_SEQUENTIAL vs dbench
>
>                        mmotm1022   + the patch
>  ==============================================================
>  mm_sync_madv_cp       6:29        6:19           (min:sec)
>  dbench throughput     11.633      14.4045        (MB/s)
>  dbench latency        65628       18565          (ms)


So, I think the code of this patch is good, but I guess his mesurement
way isn't so good.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc] mm: more likely reclaim MADV_SEQUENTIAL mappings

[Rik van Riel]

Johannes Weiner wrote:

> I'm afraid this is now quite a bit more aggressive than the earlier
> version.  When the fault path did a mark_page_access(), we wouldn't
> reclaim a page when it has been faulted into several MADV_SEQUENTIAL
> mappings but now we ignore *every* activity through such a mapping.
> 
> What do you think?
> 
> Perhaps we should note a reference if there are two or more accesses
> through sequentially read mappings?

That can be easily accomplished by dropping the memory.c
part of your patch.

I do not know whether that would work any better than
the patch you just posted, though :)

-- 
All rights reversed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc] mm: more likely reclaim MADV_SEQUENTIAL mappings

[Johannes Weiner]

Rik van Riel <riel@redhat.com> writes:

> Johannes Weiner wrote:
>
>> I'm afraid this is now quite a bit more aggressive than the earlier
>> version.  When the fault path did a mark_page_access(), we wouldn't
>> reclaim a page when it has been faulted into several MADV_SEQUENTIAL
>> mappings but now we ignore *every* activity through such a mapping.
>>
>> What do you think?
>>
>> Perhaps we should note a reference if there are two or more accesses
>> through sequentially read mappings?
>
> That can be easily accomplished by dropping the memory.c
> part of your patch.

I thought about that, but wouldn't we count a reference in the chain

        fault -> unmap -> page_referenced()

opposed to counting _no_ reference in

        fault -> page_referenced() -> ... -> unmap

?

	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>