Swap progress accounting

Swap progress accounting

[Arjan van de Ven]

Hi,

Currently, calling swap_out() on a zone doesn't count progress, and the
result can be that you swap_out() a lot of pages, and still return "no
progress possible" to try_to_free_pages(), which in turn makes a GFP_KERNEL
allocation fail (and that can kill init).

The patch below makes it count as progress, so at least the page allocator
will keep trying harder in this case (and more importantly, will start the
io on the swappage if needed etc) 

Comments? 

Greetings,
   Arjan van de Ven



--- linux/mm/vmscan.c.org	Sun Jul 22 21:06:21 2001
+++ linux/mm/vmscan.c	Sun Jul 22 21:16:41 2001
@@ -288,7 +288,7 @@
 	return nr;
 }
 
-static void swap_out(zone_t *zone, unsigned int priority, int gfp_mask)
+static int swap_out(zone_t *zone, unsigned int priority, int gfp_mask)
 {
 	int counter;
 	int retval = 0;
@@ -321,10 +321,11 @@
 		retval |= swap_out_mm(zone, mm, swap_amount(mm));
 		mmput(mm);
 	} while (--counter >= 0);
-	return;
+	return retval;
 
 empty:
 	spin_unlock(&mmlist_lock);
+	return retval;
 }
 
 
@@ -949,7 +950,8 @@
 		}
 
 		/* Walk the VM space for a bit.. */
-		swap_out(NULL, DEF_PRIORITY, gfp_mask);
+		if (swap_out(NULL, DEF_PRIORITY, gfp_mask))
+			count--; /* count swap progress as progress */
 
 		count -= refill_inactive_scan(NULL, DEF_PRIORITY, count);
 		if (count <= 0)
@@ -973,7 +975,8 @@
 	maxtry = (1 << DEF_PRIORITY);
 
 	do {
-		swap_out(zone, DEF_PRIORITY, gfp_mask);
+		if (swap_out(zone, DEF_PRIORITY, gfp_mask))
+			count--; /* count swap progress as progress */
 
 		count -= refill_inactive_scan(zone, DEF_PRIORITY, count);
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swap progress accounting

[Rik van Riel]

On Mon, 23 Jul 2001, Arjan van de Ven wrote:

> Currently, calling swap_out() on a zone doesn't count progress, and the
> result can be that you swap_out() a lot of pages, and still return "no
> progress possible" to try_to_free_pages(), which in turn makes a GFP_KERNEL
> allocation fail (and that can kill init).

"makes GFP_KERNEL allocation fail" ?!?!?!

Who the fuck broke __alloc_pages() while I wasn't looking ?

Why don't you fix __alloc_pages() instead ?


Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swap progress accounting

[Rik van Riel]

On Mon, 23 Jul 2001, Rik van Riel wrote:
> On Mon, 23 Jul 2001, Arjan van de Ven wrote:
>
> > Currently, calling swap_out() on a zone doesn't count progress, and the
> > result can be that you swap_out() a lot of pages, and still return "no
> > progress possible" to try_to_free_pages(), which in turn makes a GFP_KERNEL
> > allocation fail (and that can kill init).
>
> "makes GFP_KERNEL allocation fail" ?!?!?!

OK, after talking on IRC it turns out that recursive allocations
are failing.

This isn't influenced by either changing __alloc_pages() or
by changing swap_out(). What we need to do is limit the amount
of recursive allocations going on at the same time, probably
by making the system sleep on IO completion instead of looping
like crazy in __alloc_pages()/page_launder() until we run out
of all our memory ...

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swap progress accounting

[Stephen C. Tweedie]

Hi,

On Mon, Jul 23, 2001 at 03:35:28PM -0300, Rik van Riel wrote:

> OK, after talking on IRC it turns out that recursive allocations
> are failing.
> 
> This isn't influenced by either changing __alloc_pages() or
> by changing swap_out(). What we need to do is limit the amount
> of recursive allocations going on at the same time, probably
> by making the system sleep on IO completion instead of looping
> like crazy in __alloc_pages()/page_launder() until we run out
> of all our memory ...

That's very much the sort of thing that the reservation proposal from
a few weeks back was designed to address --- serialising access to the
last few free pages to allow the VM to proceed OK.

One gotcha is that there may be specific processes, such as loopd or
nbd servers, that need to be allowed to proceed because otherwise you
risk blocking IO requests in other independent tasks.  

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swap progress accounting

[Rik van Riel]

On Mon, 23 Jul 2001, Stephen C. Tweedie wrote:

> That's very much the sort of thing that the reservation proposal from
> a few weeks back was designed to address --- serialising access to the
> last few free pages to allow the VM to proceed OK.

Fully agreed.  We need reservations to properly fix this issue.

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/