Odd swap behavior

Odd swap behavior

[Rajagopal Ananthanarayanan]

I'm running fairly stressful tests like
dbench with lots of clients. Since the new
VM changes (now in test9), I haven't noticed _any_ swap activity,
in spite of the enormous memory pressures. I have lots
of processes in the system, like 8 httpd's, 4 getty's, etc.
most of which should be "idle" ... Why aren't the
pages (eg. mapped stacks) from these processes being swapped out?


--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rik van Riel]

On Tue, 3 Oct 2000, Rajagopal Ananthanarayanan wrote:

> I'm running fairly stressful tests like dbench with lots of
> clients. Since the new VM changes (now in test9), I haven't
> noticed _any_ swap activity, in spite of the enormous memory
> pressures. I have lots of processes in the system, like 8
> httpd's, 4 getty's, etc. most of which should be "idle" ... Why
> aren't the pages (eg. mapped stacks) from these processes being
> swapped out?

That's an interesting one. Most "complaints" I've had about
test9 is that it swaps more than previous versions ;)

But let me give you the answer...

Small code changes in deactivate_page() have caused the
drop_behind() code to actually WORK AS ADVERTISED right
now, and because of that streaming IO doesn't put any
memory pressure on the system.

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rajagopal Ananthanarayanan]

Rik van Riel wrote:
> 
> On Tue, 3 Oct 2000, Rajagopal Ananthanarayanan wrote:
> 
> > I'm running fairly stressful tests like dbench with lots of
> > clients. Since the new VM changes (now in test9), I haven't
> > noticed _any_ swap activity, in spite of the enormous memory
> > pressures. I have lots of processes in the system, like 8
> > httpd's, 4 getty's, etc. most of which should be "idle" ... Why
> > aren't the pages (eg. mapped stacks) from these processes being
> > swapped out?
> 
> That's an interesting one. Most "complaints" I've had about
> test9 is that it swaps more than previous versions ;)
> 
> But let me give you the answer...
> 
> Small code changes in deactivate_page() have caused the
> drop_behind() code to actually WORK AS ADVERTISED right
> now, and because of that streaming IO doesn't put any
> memory pressure on the system.


Agreed. And since the introduction of drop_behind &
the deactivate_page() in generic_file_write, streaming I/O
performance has become pretty good.

However, in the above I was particularly talking about
swap behaviour on running dbench. Dbench is write intensive,
and also has fair amount of re-writes. So, the I'm not
sure why we still do not swap out _really_ old processes.

If old pages are not swapped out, then dbench itself
will get less than optimal amount of the page-cache during
its run. I believe this is one of the reasons for
dbench's poor showing with the new VM.


--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rik van Riel]

On Wed, 4 Oct 2000, Rajagopal Ananthanarayanan wrote:
> Rik van Riel wrote:
> > On Tue, 3 Oct 2000, Rajagopal Ananthanarayanan wrote:
> > 
> > > I'm running fairly stressful tests like dbench with lots of
> > > clients. Since the new VM changes (now in test9), I haven't
> > > noticed _any_ swap activity, in spite of the enormous memory

> > Small code changes in deactivate_page() have caused the
> > drop_behind() code to actually WORK AS ADVERTISED right
> > now, and because of that streaming IO doesn't put any
> > memory pressure on the system.
> 
> Agreed. And since the introduction of drop_behind &
> the deactivate_page() in generic_file_write, streaming I/O
> performance has become pretty good.
> 
> However, in the above I was particularly talking about
> swap behaviour on running dbench. Dbench is write intensive,
> and also has fair amount of re-writes. So, the I'm not
> sure why we still do not swap out _really_ old processes.

Please take a look at vmscan.c::refill_inactive()

Furthermore, we don't do background scanning on all
active pages, only on the unmapped ones.

This is one of the things we'll be able to fix in
2.5...

> If old pages are not swapped out, then dbench itself
> will get less than optimal amount of the page-cache during
> its run. I believe this is one of the reasons for
> dbench's poor showing with the new VM.

Agreed, but I don't see an "easy" solution for 2.4.

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rajagopal Ananthanarayanan]

Rik van Riel wrote:
	[ ... ]
> 
> Please take a look at vmscan.c::refill_inactive()
> 
> Furthermore, we don't do background scanning on all
> active pages, only on the unmapped ones.

Does that mean stack pages of processes are not included?
Non-aggressive swap can hurt performance.	

> 
> Agreed, but I don't see an "easy" solution for 2.4.
> 

Ok, I have another suggestion. Suppose you had a situation
where a page is read from disk. It has buffers. Initially
the page is active, and then aged. Where does the page go at age = 0?
In reading the current code it seems that it would go to
inactive_dirty. See how deactivate_page() chooses the dirty list
to add a page which has buffers. Of course, later page_launder()
would do try_to_free_buffers() which discards (clean) buffer heads,
and at that point the page is put on free_list/reclaimed.
Would it not be more efficient to bung clean (read) pages directly
to inactive_clean on age = 0?


--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rik van Riel]

On Wed, 4 Oct 2000, Rajagopal Ananthanarayanan wrote:
> Rik van Riel wrote:
> 	[ ... ]
> > 
> > Please take a look at vmscan.c::refill_inactive()
> > 
> > Furthermore, we don't do background scanning on all
> > active pages, only on the unmapped ones.
> 
> Does that mean stack pages of processes are not included?
> Non-aggressive swap can hurt performance.	

I don't really see a clean way to do that in 2.4 ...

> > Agreed, but I don't see an "easy" solution for 2.4.
> 
> Ok, I have another suggestion. Suppose you had a situation
> where a page is read from disk. It has buffers. Initially
> the page is active, and then aged. Where does the page go at age = 0?
> In reading the current code it seems that it would go to
> inactive_dirty. See how deactivate_page() chooses the dirty list
> to add a page which has buffers. Of course, later page_launder()
> would do try_to_free_buffers() which discards (clean) buffer heads,
> and at that point the page is put on free_list/reclaimed.
> Would it not be more efficient to bung clean (read) pages directly
> to inactive_clean on age = 0?

I don't know if this would make any difference...

And in fact, I'm contemplating adding /all/ pages
that are deactivated to the inactive_dirty list,
since that way we'll reclaim all inactive pages
in FIFO order.

Currently we may "skip" some pages that were put
on the inactive_dirty list but were cleaned up
subsequently because we can find enough active
pages that can be moved to the inactive_clean
list immediately ...

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rajagopal Ananthanarayanan]

Rik van Riel wrote:

> > Does that mean stack pages of processes are not included?
> > Non-aggressive swap can hurt performance.
> 
> I don't really see a clean way to do that in 2.4 ...

We can perhaps talk about this at the Storage Workshop ...

	[ ... ]

> > Would it not be more efficient to bung clean (read) pages directly
> > to inactive_clean on age = 0?
> 
> I don't know if this would make any difference...

I think it would. Consider steady state where pages
are all "in use". Any new allocation has to start with
pushing a page from active -> inactive, and then
inactive -> inactive_clean, if necessary, and then reclaim.
Now,  if we had pages which _are_ clean, then the path taken
is simply active -> inactive_clean -> reclaim.


> 
> And in fact, I'm contemplating adding /all/ pages
> that are deactivated to the inactive_dirty list,
> since that way we'll reclaim all inactive pages
> in FIFO order.
> 
> Currently we may "skip" some pages that were put
> on the inactive_dirty list but were cleaned up
> subsequently because we can find enough active
> pages that can be moved to the inactive_clean
> list immediately ...
> 

This is an interesting idea, although it seems
antithetical to what I said above. I think pure
FIFO has its merits in accomodating longer locality of
reference; it can help dbench.

If you have a patch (to always deactivate to inactive_dirty),
I can help you gauge it with the benchmarks ...

 
--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rik van Riel]

On Wed, 4 Oct 2000, Rajagopal Ananthanarayanan wrote:
> Rik van Riel wrote:
> 
> > > Does that mean stack pages of processes are not included?
> > > Non-aggressive swap can hurt performance.
> > 
> > I don't really see a clean way to do that in 2.4 ...
> 
> We can perhaps talk about this at the Storage Workshop ...

That would be a great idea. Maybe over dinner or lunch,
with some of the other people present as well? ;)

[I'll also bring guarana, to facilitate all-night hacking
sessions]

> > > Would it not be more efficient to bung clean (read) pages directly
> > > to inactive_clean on age = 0?
> > 
> > I don't know if this would make any difference...
> 
> I think it would. Consider steady state where pages
> are all "in use". Any new allocation has to start with
> pushing a page from active -> inactive, and then
> inactive -> inactive_clean, if necessary, and then reclaim.
> Now,  if we had pages which _are_ clean, then the path taken
> is simply active -> inactive_clean -> reclaim.

Indeed. Then again, we probably want to clear the unused
buffer heads off of active pages anyway, and in that case
these pages will be cleaned for us.

This is something I'm looking into at the moment...

[But it has lower priority than out of memory and the
high memory lockup problem ... if that still exists]

> > And in fact, I'm contemplating adding /all/ pages
> > that are deactivated to the inactive_dirty list,
> > since that way we'll reclaim all inactive pages
> > in FIFO order.
> > 
> > Currently we may "skip" some pages that were put
> > on the inactive_dirty list but were cleaned up
> > subsequently because we can find enough active
> > pages that can be moved to the inactive_clean
> > list immediately ...
> > 
> 
> This is an interesting idea, although it seems
> antithetical to what I said above. I think pure
> FIFO has its merits in accomodating longer locality of
> reference; it can help dbench.

I have no idea if it would help or not; only one
way to find out I guess...

> If you have a patch (to always deactivate to inactive_dirty),
> I can help you gauge it with the benchmarks ...

Quick and dirty patch below ;)

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/



--- swap.c.orig	Tue Oct  3 10:20:41 2000
+++ swap.c	Wed Oct  4 21:06:06 2000
@@ -201,7 +201,7 @@
 		} else if (page->mapping && !PageDirty(page) &&
 							!PageLocked(page)) {
 			del_page_from_active_list(page);
-			add_page_to_inactive_clean_list(page);
+			add_page_to_inactive_dirty_list(page);
 		}
 		/*
 		 * OK, we cannot free the page. Leave it alone.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Odd swap behavior

[Rajagopal Ananthanarayanan]

Rik van Riel wrote:
	
	[ ... ]
> 
> > If you have a patch (to always deactivate to inactive_dirty),
> > I can help you gauge it with the benchmarks ...
> 
> Quick and dirty patch below ;)
> 

	[ ... ]

Now, I'm not sure whether it helps either. It seemed
to help on 16 & 32 clients, but has worse effects w/ 48
clients. In reality, i guess having a single inactive
(dirty + clean) might help;  but seeing good end results
probably requires other things (such as the swap thing)
to be in place.

Back to looking at code.

cheers,

--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/