[PATCH] latency improvements, one reschedule moved

[PATCH] latency improvements, one reschedule moved

[Roger Larsson]

[-- Attachment #1: Type: text/plain, Size: 590 bytes --]

Hi Linus, Paul, Benno, ...,

[patch against  linux-2.4.0-test3-pre2]

I cleaned up kswapd and moved its reschedule point.
Disk performance is close to the same.
Latencies have improved a lot (tested with Bennos latencytest)

* sync is still problematic
* mmap002 (Quintinela) still gives a 212 ms latency 
  (compared to 423 ms for the unpatched...)
* other disk related latencies are down under 30 ms.
  (streaming read, copy, write)
* the number of overruns has dropped considerably!
  (running 4 buffers with a deadline of 23 ms)

/RogerL

--
Home page:
  http://www.norran.net/nra02596/

[-- Attachment #2: patch-2.4.0-test3-pre2-vmscan.latency.2 --]
[-- Type: text/plain, Size: 2841 bytes --]

--- linux/mm/vmscan.c.orig	Wed May 31 20:13:37 2000
+++ linux/mm/vmscan.c	Sat Jul  1 03:29:00 2000
@@ -419,6 +419,48 @@
 }
 
 /*
+ * Check if there is any memory pressure (free_pages < pages_low)
+ */
+static inline int memory_pressure(void)
+{
+	pg_data_t *pgdat = pgdat_list;
+
+	do {
+		int i;
+		for(i = 0; i < MAX_NR_ZONES; i++) {
+			zone_t *zone = pgdat->node_zones+ i;
+			if (!zone->size &&
+			    zone->free_pages < zone->pages_low)
+				return 1;
+		}
+		pgdat = pgdat->node_next;
+	} while (pgdat);
+
+	return 0;
+}
+
+/*
+ * Check if there is any memory pressure (free_pages < pages_low)
+ */
+static inline int keep_kswapd_awake(void)
+{
+	pg_data_t *pgdat = pgdat_list;
+
+	do {
+		int i;
+		for(i = 0; i < MAX_NR_ZONES; i++) {
+			zone_t *zone = pgdat->node_zones+ i;
+			if (!zone->size &&
+			    zone->zone_wake_kswapd)
+				return 1;
+		}
+		pgdat = pgdat->node_next;
+	} while (pgdat);
+
+	return 0;
+}
+
+/*
  * We need to make the locks finer granularity, but right
  * now we need this so that we can do page allocations
  * without holding the kernel lock etc.
@@ -442,7 +484,20 @@
 
 	priority = 64;
 	do {
+		/* should __GFP_WAIT be checked? 
+		 * assume not - not WAITING for a free page
+		 * let more important task execute before
+		 * continuing (Note: kswapd does not use it).
+		 */
+	        if (current->need_resched) {
+		  schedule();
+		  /* time has passed - pressure too? */
+		  if (!memory_pressure())
+		      goto done;
+		}
+
 		while (shrink_mmap(priority, gfp_mask)) {
+		        /* check __GFP_WAIT ? see below */
 			if (!--count)
 				goto done;
 		}
@@ -477,16 +532,21 @@
 			if (--swap_count < 0)
 				break;
 
-	} while (--priority >= 0);
+		priority--;
+	} while (priority >= 0);
 
 	/* Always end on a shrink_mmap.. */
 	while (shrink_mmap(0, gfp_mask)) {
+		if (current->need_resched)
+			schedule();
+		if (!memory_pressure())
+			return 1;
 		if (!--count)
 			goto done;
 	}
 	/* We return 1 if we are freed some page */
 	return (count != FREE_COUNT);
-
+ 
 done:
 	return 1;
 }
@@ -530,29 +590,12 @@
 	tsk->flags |= PF_MEMALLOC;
 
 	for (;;) {
-		pg_data_t *pgdat;
-		int something_to_do = 0;
-
-		pgdat = pgdat_list;
-		do {
-			int i;
-			for(i = 0; i < MAX_NR_ZONES; i++) {
-				zone_t *zone = pgdat->node_zones+ i;
-				if (tsk->need_resched)
-					schedule();
-				if (!zone->size || !zone->zone_wake_kswapd)
-					continue;
-				if (zone->free_pages < zone->pages_low)
-					something_to_do = 1;
-				do_try_to_free_pages(GFP_KSWAPD);
-			}
-			pgdat = pgdat->node_next;
-		} while (pgdat);
-
-		if (!something_to_do) {
-			tsk->state = TASK_INTERRUPTIBLE;
-			interruptible_sleep_on(&kswapd_wait);
+	        if (!keep_kswapd_awake()) {
+		  tsk->state = TASK_INTERRUPTIBLE;
+		  interruptible_sleep_on(&kswapd_wait);
 		}
+
+		do_try_to_free_pages(GFP_KSWAPD);
 	}
 }
 

Re: [PATCH] latency improvements, one reschedule moved

[Roger Larsson]

Hi Linus,

Cleaned up and corrected some bugs...
(memory_pressure... !
 unintended reschedule removed)

Sadly the performance went down - slightly.
Latency looks even nicer. Still some spikes.
[sync and mmap002 behaviour not corrected]

/RogerL



Roger Larsson wrote:
> 
> Hi Linus,
> 
> [patch against  linux-2.4.0-test3-pre2]
> 
> I cleaned up kswapd and moved its reschedule point.
> Disk performance is close to the same.
> Latencies have improved a lot (tested with Bennos latencytest)
> 
> * sync is still problematic
> * mmap002 (Quintinela) still gives a 212 ms latency
>   (compared to 423 ms for the unpatched...)
> * other disk related latencies are down under 30 ms.
>   (streaming read, copy, write)
> * the number of overruns has dropped considerably!
>   (running 4 buffers with a deadline of 23 ms)
> 
> /RogerL
> 
> --
> Home page:
>   http://www.norran.net/nra02596/
> 
>   ------------------------------------------------------------------------
>                                               Name: patch-2.4.0-test3-pre2-vmscan.latency.2
>    patch-2.4.0-test3-pre2-vmscan.latency.2    Type: Plain Text (text/plain)
>                                           Encoding: 7bit

--
Home page:
  http://www.norran.net/nra02596/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

[PATCH really] latency improvements, one reschedule moved

[Roger Larsson]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

Again... :-(

Patch included this time...

/RogerL


Roger Larsson wrote:
> 
> Hi Linus,
> 
> Cleaned up and corrected some bugs...
> (memory_pressure... !
>  unintended reschedule removed)
> 
> Sadly the performance went down - slightly.
> Latency looks even nicer. Still some spikes.
> [sync and mmap002 behaviour not corrected]
> 
> /RogerL
> 
> Roger Larsson wrote:
> >
> > Hi Linus,
> >
> > [patch against  linux-2.4.0-test3-pre2]
> >
> > I cleaned up kswapd and moved its reschedule point.
> > Disk performance is close to the same.
> > Latencies have improved a lot (tested with Bennos latencytest)
> >
> > * sync is still problematic
> > * mmap002 (Quintinela) still gives a 212 ms latency
> >   (compared to 423 ms for the unpatched...)
> > * other disk related latencies are down under 30 ms.
> >   (streaming read, copy, write)
> > * the number of overruns has dropped considerably!
> >   (running 4 buffers with a deadline of 23 ms)
> >
> > /RogerL
> >
> > --
> > Home page:
> >   http://www.norran.net/nra02596/
> >
> >   ------------------------------------------------------------------------
> >                                               Name: patch-2.4.0-test3-pre2-vmscan.latency.2
> >    patch-2.4.0-test3-pre2-vmscan.latency.2    Type: Plain Text (text/plain)
> >                                           Encoding: 7bit
> 
> --
> Home page:
>   http://www.norran.net/nra02596/
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux.eu.org/Linux-MM/

--
Home page:
  http://www.norran.net/nra02596/

[-- Attachment #2: patch-2.4.0-test3-pre2-vmscan.latency.5 --]
[-- Type: text/plain, Size: 4214 bytes --]

diff -aurp linux-2.4.0-test3-pre2/mm/vmscan.c linux/mm/vmscan.c
--- linux-2.4.0-test3-pre2/mm/vmscan.c	Wed May 31 20:13:37 2000
+++ linux/mm/vmscan.c	Wed Jul  5 00:52:59 2000
@@ -419,6 +419,48 @@ out:
 }
 
 /*
+ * Check if there is any memory pressure (free_pages < pages_low)
+ */
+static inline int memory_pressure(void)
+{
+	pg_data_t *pgdat = pgdat_list;
+
+	do {
+		int i;
+		for(i = 0; i < MAX_NR_ZONES; i++) {
+			zone_t *zone = pgdat->node_zones+ i;
+			if (zone->size &&
+			    zone->free_pages < zone->pages_low)
+				return 1;
+		}
+		pgdat = pgdat->node_next;
+	} while (pgdat);
+
+	return 0;
+}
+
+/*
+ * Check if there is any memory pressure (free_pages < pages_low)
+ */
+static inline int keep_kswapd_awake(void)
+{
+	pg_data_t *pgdat = pgdat_list;
+
+	do {
+		int i;
+		for(i = 0; i < MAX_NR_ZONES; i++) {
+			zone_t *zone = pgdat->node_zones+ i;
+			if (zone->size &&
+			    zone->zone_wake_kswapd)
+				return 1;
+		}
+		pgdat = pgdat->node_next;
+	} while (pgdat);
+
+	return 0;
+}
+
+/*
  * We need to make the locks finer granularity, but right
  * now we need this so that we can do page allocations
  * without holding the kernel lock etc.
@@ -442,11 +484,28 @@ static int do_try_to_free_pages(unsigned
 
 	priority = 64;
 	do {
+		/* should __GFP_WAIT be checked? 
+		 * assume not - not WAITING for a free page
+		 * let more important task execute before
+		 * continuing (Note: kswapd does not use it).
+		 */
+	        if (current->need_resched) {
+		  schedule();
+		  /* time has passed - pressure too? */
+		  if (!memory_pressure())
+		      goto done;
+		}
+
 		while (shrink_mmap(priority, gfp_mask)) {
 			if (!--count)
 				goto done;
 		}
 
+		/* not (been) low on memory - it is
+		 * pointless to try to swap out.
+		 */
+		if (!keep_kswapd_awake())
+		      goto done;
 
 		/* Try to get rid of some shared memory pages.. */
 		if (gfp_mask & __GFP_IO) {
@@ -457,8 +516,18 @@ static int do_try_to_free_pages(unsigned
 			 */
 			count -= shrink_dcache_memory(priority, gfp_mask);
 			count -= shrink_icache_memory(priority, gfp_mask);
-			if (count <= 0)
-				goto done;
+			/*
+			 * Not currently working, see fixme in shrink_?cache_memory
+			 * In the inner funtions there is a comment:
+			 * "To help debugging, a zero exit status indicates
+			 *  all slabs were released." (-arca?)
+			 * lets handle it in a primitive but working way...
+			 *	if (count <= 0)
+			 *		goto done;
+			 */
+			if (!keep_kswapd_awake())
+			  goto done;
+
 			while (shm_swap(priority, gfp_mask)) {
 				if (!--count)
 					goto done;
@@ -477,7 +546,8 @@ static int do_try_to_free_pages(unsigned
 			if (--swap_count < 0)
 				break;
 
-	} while (--priority >= 0);
+		priority--;
+	} while (priority >= 0);
 
 	/* Always end on a shrink_mmap.. */
 	while (shrink_mmap(0, gfp_mask)) {
@@ -486,7 +556,7 @@ static int do_try_to_free_pages(unsigned
 	}
 	/* We return 1 if we are freed some page */
 	return (count != FREE_COUNT);
-
+ 
 done:
 	return 1;
 }
@@ -530,29 +600,14 @@ int kswapd(void *unused)
 	tsk->flags |= PF_MEMALLOC;
 
 	for (;;) {
-		pg_data_t *pgdat;
-		int something_to_do = 0;
-
-		pgdat = pgdat_list;
-		do {
-			int i;
-			for(i = 0; i < MAX_NR_ZONES; i++) {
-				zone_t *zone = pgdat->node_zones+ i;
-				if (tsk->need_resched)
-					schedule();
-				if (!zone->size || !zone->zone_wake_kswapd)
-					continue;
-				if (zone->free_pages < zone->pages_low)
-					something_to_do = 1;
-				do_try_to_free_pages(GFP_KSWAPD);
-			}
-			pgdat = pgdat->node_next;
-		} while (pgdat);
-
-		if (!something_to_do) {
-			tsk->state = TASK_INTERRUPTIBLE;
-			interruptible_sleep_on(&kswapd_wait);
+	        if (!keep_kswapd_awake()) {
+		  /* wake up regulary to do an early attempt too free
+		   * pages - pages will not actually be freed.
+		   */
+		  interruptible_sleep_on_timeout(&kswapd_wait, HZ);
 		}
+
+		do_try_to_free_pages(GFP_KSWAPD);
 	}
 }
 
@@ -580,6 +635,12 @@ int try_to_free_pages(unsigned int gfp_m
 		retval = do_try_to_free_pages(gfp_mask);
 		current->flags &= ~PF_MEMALLOC;
 	}
+	else {
+	        /* make sure kswapd runs */
+	        if (waitqueue_active(&kswapd_wait))
+		        wake_up_interruptible(&kswapd_wait);
+	}
+
 	return retval;
 }
 

Re: [PATCH really] latency improvements, one reschedule moved

[Zlatko Calusic]

Roger Larsson <roger.larsson@norran.net> writes:

> Again... :-(
> 
> Patch included this time...
> 

Hi, Roger, Linus, others!

2.4.0-test3-pre4 (which includes this patch) is really a pleasant
surprise. The I/O bandwidth has greatly improved and I'm still trying
to understand how can patch this simple be so effective. :)

Great work Roger!

I see this as the first (and most critical) step of returning my faith
in good performing 2.4.0-final.

Keep up the good work!
-- 
Zlatko
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [linux-audio-dev] Re: [PATCH really] latency improvements, one reschedule moved

[Roger Larsson]

Zlatko Calusic wrote:
> 
> Roger Larsson <roger.larsson@norran.net> writes:
> 
> > Again... :-(
> >
> > Patch included this time...
> >
> 
> Hi, Roger, Linus, others!
> 
> 2.4.0-test3-pre4 (which includes this patch) is really a pleasant
> surprise. The I/O bandwidth has greatly improved and I'm still trying
> to understand how can patch this simple be so effective. :)
> 
> Great work Roger!
> 
> I see this as the first (and most critical) step of returning my faith
> in good performing 2.4.0-final.
> 
> Keep up the good work!
> --
> Zlatko

It was not intended to give better performance...
(something masks the expected latency improvements - floppy is
disturbing
 me, recal_interrupt. And kmem stuff - but that is more understandable
 we will issue additional 'kmem_cache_reap']


I examined the patches again and the fact that it runs
do_try_to_free_pages
periodically may improve performance due to its page cleaning effect -
all pages won't be dirty at the same time...

But it has a downside too - it will destroy the LRU order of pages...
PG_referenced loses some of its meaning...

Streaming writes are likely to gain the most.
Non uniform random accesses may loose :-(


I have an idea...

/RogerL

--
Home page:
  http://www.norran.net/nra02596/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [linux-audio-dev] Re: [PATCH really] latency improvements, one reschedule moved

[Linus Torvalds]

On Sat, 8 Jul 2000, Roger Larsson wrote:
> 
> I examined the patches again and the fact that it runs
> do_try_to_free_pages
> periodically may improve performance due to its page cleaning effect -
> all pages won't be dirty at the same time...

One potential problem with this, though, is that it keeps shrinking the
dcache etc: even though various parts of the system refuse to touch pages
once the machine has "enough" memory, the periodic wake-up will cause
those caches that don't have the "I don't need to do this" logic to
potentially be starved.

The keep_kswapd_awake() logic protects from the worst case (it won't go on
forever), but I wonder if it might still not cause bad behaviour when one
zone is getting balanced and the other zones shouldn't be touched, but the
continual trickle will cause the dcache etc stuff to be free'd with very
little good reason.

The "one zone gets rebalanced" is normal behaviour, so this is why I worry
about the dcache.

> But it has a downside too - it will destroy the LRU order of pages...
> PG_referenced loses some of its meaning...

That part doesn't bother me in the least - it can be seen as a simple
aging of the referenced bit. Especially if we're going to re-introduce the
multi-bit page "age" code, aging the reference bits actually only improves
stuff. 

The fact that shrink_mmap() gets called regularly also makes bdflush
potentially less important, because shrink_mmap() actually does a better
job of flushing dirty data anyway these days, and in many ways it makes
sense to have this kind of background LRU list activity.

		Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/