[uPATCH] small kswapd improvement ???

[uPATCH] small kswapd improvement ???

[Rik van Riel]

Hi,

I remember the 1.1 or 1.2 days when Stephen reworked the
swap code and I played around with a small piece of
vmscan.c. Back then a simple bug was encountered and 'fixed'
by always starting the memory scan at adress 0, which gives
a highly unfair and inefficient aging process.

I think I have 'corrected' the code. Not so much made a
large performance increase, but merely a 'correction' for
the sake of correctness and a small improvement.

Patch attached.

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

--- linux/mm/vmscan.c.orig	Tue Mar  3 14:57:53 1998
+++ linux/mm/vmscan.c	Tue Mar  3 17:51:22 1998
@@ -333,14 +333,15 @@
 	 * Go through process' page directory.
 	 */
 	address = p->swap_address;
-	p->swap_address = 0;
 
 	/*
 	 * Find the proper vm-area
 	 */
 	vma = find_vma(p->mm, address);
-	if (!vma)
+	if (!vma) {
+		p->swap_address = 0;
 		return 0;
+	}
 	if (address < vma->vm_start)
 		address = vma->vm_start;
 

Re: [uPATCH] small kswapd improvement ???

[Rik van Riel]

On Tue, 3 Mar 1998, Rik van Riel wrote:

> I think I have 'corrected' the code. Not so much made a
> large performance increase, but merely a 'correction' for
> the sake of correctness and a small improvement.

I'm currenly running a way-too-large gnuchess next
to my other applications, and I hardly notice it's
running... The disk noise is the major givaway :-)

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: [uPATCH] small kswapd improvement ???

[Stephen C. Tweedie]

Hi,

On Tue, 3 Mar 1998 18:05:18 +0100 (MET), Rik van Riel
<H.H.vanRiel@fys.ruu.nl> said:

> Hi,
> I remember the 1.1 or 1.2 days when Stephen reworked the
> swap code and I played around with a small piece of
> vmscan.c. Back then a simple bug was encountered and 'fixed'
> by always starting the memory scan at adress 0, which gives
> a highly unfair and inefficient aging process.

Ouch --- I wonder how much this is hurting 2.0.33.  I think I'll have
to try that, and perhaps look at this for 2.0.34/LMP... 

Thanks,
 Stephen.

Re: [uPATCH] small kswapd improvement ???

[Rik van Riel]

On Tue, 3 Mar 1998, Stephen C. Tweedie wrote:

> > I remember the 1.1 or 1.2 days when Stephen reworked the
> > swap code and I played around with a small piece of
> > vmscan.c. Back then a simple bug was encountered and 'fixed'
> > by always starting the memory scan at adress 0, which gives
> > a highly unfair and inefficient aging process.
> 
> Ouch --- I wonder how much this is hurting 2.0.33.  I think I'll have
> to try that, and perhaps look at this for 2.0.34/LMP... 

It doesn't hurt _that_ much... Otherwise we wouldn't
have left it in the swapper code for three years :-)

It gives somewhat more than a smallish improvement
though...

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: [uPATCH] small kswapd improvement ???

[Benjamin C.R. LaHaise]

...trim...
> > Ouch --- I wonder how much this is hurting 2.0.33.  I think I'll have
> > to try that, and perhaps look at this for 2.0.34/LMP... 
> 
> It doesn't hurt _that_ much... Otherwise we wouldn't
> have left it in the swapper code for three years :-)

AHhhh!!!  That could explain the odd behaviour of my 386 (5 megs of RAM,
masquerading/ppp and sick shell scripts) - it never quite made any sense
that after the first time it swapped, forevermore until forced to clean
out memory it would continuously touch the disk (the bdflush parameters
we set to sync every 15 minutes, so only swapping could explain it).

		-ben

Re: [uPATCH] small kswapd improvement ???

[Rik van Riel]

On Tue, 3 Mar 1998, Benjamin C.R. LaHaise wrote:

> AHhhh!!!  That could explain the odd behaviour of my 386 (5 megs of RAM,
> masquerading/ppp and sick shell scripts) - it never quite made any sense
> that after the first time it swapped, forevermore until forced to clean
> out memory it would continuously touch the disk (the bdflush parameters
> we set to sync every 15 minutes, so only swapping could explain it).

My disk is a _lot_ more quiet than it used to be... This
patch didn't have the you-can-push-linux-further effect
I had hoped for, but it makes Linux a lot quieter under
the same load.

Actually, I was planning on a complete revamp of the
scan-by-VMA algorithm in mm/vmscan.c, beginning with
the removal of AGE_CLUSTER_FRACT, but maybe it works
now :-)

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: [uPATCH] small kswapd improvement ???

[Stephen C. Tweedie]

Hi,

Regarding the tsk->swap_address stuff: swap_out_pmd() includes the line

		tsk->swap_address = address + PAGE_SIZE;

so the setting of tsk->swap_address to zero is only temporary.  Indeed,
I can't see how we can block between setting address to 0 and setting it
forward in swap_out_pmd, so we should only allow anybody to see the 0
value if there is truly nothing left to swap in the process's vm space.

Has anybody verified that this change alone, without any other of Rik's
changes, improves performance?

One other observation while looking at this code: we currently walk over
every process when swapping, and this is wrong.  We should only walk
over each mm_struct once.  Another one for the list for 2.2.

Cheers,
 Stephen.

Re: [uPATCH] small kswapd improvement ???

[Rik van Riel]

On Wed, 4 Mar 1998, Stephen C. Tweedie wrote:

> Regarding the tsk->swap_address stuff: swap_out_pmd() includes the line
> 
> 		tsk->swap_address = address + PAGE_SIZE;

Indeed, the tsk->swap_address = 0 might fit better in
swap_out(), and added to the switch statement at the
bottem, in the 0 case to be exact.

> One other observation while looking at this code: we currently walk over
> every process when swapping, and this is wrong.  We should only walk
> over each mm_struct once.  Another one for the list for 2.2.

Maybe we could give a run a serial number, and add that
number to the struct vma_struct. The serial number should
be updated each time we do the AGE_CLUSTER_SIZE thingy
because we ran out of processes with p->swap_cnt > 0.

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: [uPATCH] small kswapd improvement ???

[Dr. Werner Fink]

> > > vmscan.c. Back then a simple bug was encountered and 'fixed'
> > > by always starting the memory scan at adress 0, which gives
> > > a highly unfair and inefficient aging process.
> > 
> > Ouch --- I wonder how much this is hurting 2.0.33.  I think I'll have
> > to try that, and perhaps look at this for 2.0.34/LMP... 
> 
> It doesn't hurt _that_ much... Otherwise we wouldn't
> have left it in the swapper code for three years :-)

Maybe that's the reason why the bigger initial age for swapped in pages gives
an improvement in 2.0.33 ... it's a ``better protection'' for often needed
pages.


          Werner

Re: [uPATCH] small kswapd improvement ???

[Rik van Riel]

On Wed, 4 Mar 1998, Dr. Werner Fink wrote:

> Maybe that's the reason why the bigger initial age for swapped in pages gives
> an improvement in 2.0.33 ... it's a ``better protection'' for often needed
> pages.

I think I'm going to try a more LRU like aging algorithm
now. (I've worked with some SGI boxes at school, and it
looks like they're less phrone to trashing)

Algorithm:

age_page(page) {
	page->age >>= 1;
}
touch_page(page) {
	page->age >>= 1;
	page->age |= 0x80;
}

and in vmscan.c/filemap.c:

-	if (page->age)
+	if (page->age < 1 << (MIN_SWAP_AGE + (nr_free_pages < free_pages_low 
? 1 : 0))

Then we could use the MIN_SWAP_AGE sysctl variable
to control the CPU usage / swap 'precision' of
kswapd. Even better would be a dynamic adjusment,
so kswapd would:
- never use more than 5% CPU (over a 10 sec interval?)
- use the maximum precision possible
- scan a maximum of MIN_SWAP_AGE + 2 times the number of
  pages it frees

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+