[uPatch] Re: Graceful failure?

[uPatch] Re: Graceful failure?

[Rik van Riel]

On Mon, 5 Jun 2000, Billy Harvey wrote:

> A "make -j" slowly over the course of 5 minutes drives the load
> to about 30.  At first the degradation is controlled, with
> sendmail refusing service, but at about 160 process visible in
> top, top quits updating (set a 8 second updates), showing about
> 2 MB swap used.  At this point it sounds like the system is
> thrashing.

That probably means you're a lot more in swap now and top
has stopped displaying before you really hit the swap...

> Is this failure process acceptable?  I'd think the system should
> react differently to the thrashing, killing off the load
> demanding user process(es), rather than degrading to a point of
> freeze.

Please take into account that the system is quite a bit beyond
where you could take previous kernels ... oh, and the attached
patch should fix the problem somewhat ;)

regards,

Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.

Wanna talk about the kernel?  irc.openprojects.net / #kernelnewbies
http://www.conectiva.com/		http://www.surriel.com/



--- linux-2.4.0-t1-ac8/include/linux/mm.h.orig	Wed May 31 21:00:14 2000
+++ linux-2.4.0-t1-ac8/include/linux/mm.h	Sun Jun  4 16:21:31 2000
@@ -202,6 +202,7 @@
 #define ClearPageError(page)	clear_bit(PG_error, &(page)->flags)
 #define PageReferenced(page)	test_bit(PG_referenced, &(page)->flags)
 #define SetPageReferenced(page)	set_bit(PG_referenced, &(page)->flags)
+#define ClearPageReferenced(page)	clear_bit(PG_referenced, &(page)->flags)
 #define PageTestandClearReferenced(page)	test_and_clear_bit(PG_referenced, &(page)->flags)
 #define PageDecrAfter(page)	test_bit(PG_decr_after, &(page)->flags)
 #define SetPageDecrAfter(page)	set_bit(PG_decr_after, &(page)->flags)
--- linux-2.4.0-t1-ac8/include/linux/swap.h.orig	Wed May 31 21:00:06 2000
+++ linux-2.4.0-t1-ac8/include/linux/swap.h	Sun Jun  4 16:22:31 2000
@@ -179,6 +179,7 @@
 	list_add(&(page)->lru, &lru_cache);	\
 	nr_lru_pages++;				\
 	page->age = PG_AGE_START;		\
+	ClearPageReferenced(page);		\
 	SetPageActive(page);			\
 	spin_unlock(&pagemap_lru_lock);		\
 } while (0)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [uPatch] Re: Graceful failure?

[John Fremlin]

Rik van Riel <riel@conectiva.com.br> writes:

> On Mon, 5 Jun 2000, Billy Harvey wrote:
> 
> > A "make -j" slowly over the course of 5 minutes drives the load
> > to about 30.  At first the degradation is controlled, with
> > sendmail refusing service, but at about 160 process visible in
> > top, top quits updating (set a 8 second updates), showing about
> > 2 MB swap used.  At this point it sounds like the system is
> > thrashing.
> 
> That probably means you're a lot more in swap now and top
> has stopped displaying before you really hit the swap...

Allow me to hype my patch again. Could someone please test it?

It improves performance markedly (no horrible pauses in
vmscan.c:swap_out under heavy load).

> 
> > Is this failure process acceptable?  I'd think the system should
> > react differently to the thrashing, killing off the load
> > demanding user process(es), rather than degrading to a point of
> > freeze.

My patch fixes this for me, please test.

[...]

--- linux-2.4.0t1a7m3/mm/vmscan.c	Sat Jun  3 17:10:15 2000
+++ kernel-hacking/mm/vmscan.c	Sun Jun  4 16:35:31 2000
@@ -361,23 +361,24 @@
 	/* 
 	 * We make one or two passes through the task list, indexed by 
 	 * assign = {0, 1}:
-	 *   Pass 1: select the swappable task with maximal RSS that has
-	 *         not yet been swapped out. 
+	 *
+	 *   Pass 1: select the first swappable task that has not yet
+	 *   been swapped out.
+	 *
 	 *   Pass 2: re-assign rss swap_cnt values, then select as above.
 	 *
 	 * With this approach, there's no need to remember the last task
 	 * swapped out.  If the swap-out fails, we clear swap_cnt so the 
 	 * task won't be selected again until all others have been tried.
 	 *
-	 * Think of swap_cnt as a "shadow rss" - it tells us which process
-	 * we want to page out (always try largest first).
-	 */
+	 * Think of swap_cnt as a "shadow rss" - it tells us which
+	 * process we want to page out (always try largest first).  */
+	
 	counter = (nr_threads << 2) >> (priority >> 2);
 	if (counter < 1)
 		counter = 1;
 
 	for (; counter >= 0; counter--) {
-		unsigned long max_cnt = 0;
 		struct mm_struct *best = NULL;
 		int pid = 0;
 		int assign = 0;
@@ -391,13 +392,14 @@
 	 		if (mm->rss <= 0)
 				continue;
 			/* Refresh swap_cnt? */
-			if (assign == 1)
+			best = mm;
+			pid = p->pid;
+
+			if (assign == 1){
 				mm->swap_cnt = mm->rss;
-			if (mm->swap_cnt > max_cnt) {
-				max_cnt = mm->swap_cnt;
-				best = mm;
-				pid = p->pid;
 			}
+			else
+				break;
 		}
 		read_unlock(&tasklist_lock);
 		if (!best) {


-- 

	http://altern.org/vii
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [uPatch] Re: Graceful failure?

[Billy Harvey]

John Fremlin writes:

 > Rik van Riel <riel@conectiva.com.br> writes:

 > > On Mon, 5 Jun 2000, Billy Harvey wrote:

 > > > A "make -j" slowly over the course of 5 minutes drives the load
 > > > to about 30.  At first the degradation is controlled, with
 > > > sendmail refusing service, but at about 160 process visible in
 > > > top, top quits updating (set a 8 second updates), showing about
 > > > 2 MB swap used.  At this point it sounds like the system is
 > > > thrashing.

 > > That probably means you're a lot more in swap now and top
 > > has stopped displaying before you really hit the swap...

 > Allow me to hype my patch again. Could someone please test it?

 > It improves performance markedly (no horrible pauses in
 > vmscan.c:swap_out under heavy load).

I first tried Rik's two-line patch, and saw no noticeable difference
in the kernel's eventual failure.  I then tried John's vmscan.c patch
and observed the system slow to a crawl under a "make -j", but never
actually stop.  There were plenty of SIGKILLs sent to gcc during the
compilation by something.  After about an hour, several ^C's sent
killed the compilation and all was well.

Noticeable in this was that the swap seemed to be barely touched at
all (still showing 28K used out of 68M available).  FWIW, I am using
shm (I think it was smbmount that required it), which Rik alluded to
earlier as being suspect.

I see ac9 is out now - did either of these patches make it into that?

Regards,
Billy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/