reduce shrink_mmap rate of failure (initial attempt)

reduce shrink_mmap rate of failure (initial attempt)

[Roger Larsson]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

Hi all,

This is a trivial first attempt to reduce shrink_mmap failures
(leading to swap)

It is against 2.4.0-test1-ac7-riel3 but that is almost what
we have currently - and it is trivial to apply with an editor.

It might be possible to improve this further - but it is a start.
(Time for bed...)

/RogerL

--
Home page:
  http://www.norran.net/nra02596/

[-- Attachment #2: patch-filemap --]
[-- Type: text/plain, Size: 870 bytes --]

--- /usr/src/linux/mm/filemap.c.orig	Sat Jun  3 19:09:16 2000
+++ /usr/src/linux/mm/filemap.c	Wed Jun  7 01:21:19 2000
@@ -332,6 +332,14 @@
 		if (page->age)
 			goto dispose_continue;
 
+		/*
+		 * Page is from a zone we don't care about.
+		 * Don't drop page cache entries in vain.
+		 * Must be done before count - or do a count++
+		 */
+		if (page->zone->free_pages > page->zone->pages_high)
+			goto dispose_continue;
+
 		count--;
 		/*
 		 * Avoid unscalable SMP locking for pages we can
@@ -367,13 +375,6 @@
 				goto made_buffer_progress;
 			}
 		}
-
-		/*
-		 * Page is from a zone we don't care about.
-		 * Don't drop page cache entries in vain.
-		 */
-		if (page->zone->free_pages > page->zone->pages_high)
-			goto unlock_continue;
 
 		/* Take the pagecache_lock spinlock held to avoid
 		   other tasks to notice the page while we are looking at its

Re: reduce shrink_mmap rate of failure (initial attempt)

[Quintela Carreira Juan J.]

>>>>> "roger" == Roger Larsson <roger.larsson@norran.net> writes:

Hi

roger> This is a trivial first attempt to reduce shrink_mmap failures
roger> (leading to swap)

roger> It is against 2.4.0-test1-ac7-riel3 but that is almost what
roger> we have currently - and it is trivial to apply with an editor.

roger> It might be possible to improve this further - but it is a start.

That patch hangs my machine here when I run mmap002.  The machine is
in shrink_mmap.  It hangs trying to get the pagmap_lru_lock.

I think that the idea is good, but it doesn't work here :(.

Later, Juan.

-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: reduce shrink_mmap rate of failure (initial attempt)

[Zlatko Calusic]

Roger Larsson <roger.larsson@norran.net> writes:

> Hi all,
> 
> This is a trivial first attempt to reduce shrink_mmap failures
> (leading to swap)
> 
> It is against 2.4.0-test1-ac7-riel3 but that is almost what
> we have currently - and it is trivial to apply with an editor.
> 
> It might be possible to improve this further - but it is a start.
> (Time for bed...)
> 

Very nice idea, but...

I tried exactly that a week or two ago and it was freezing the system
like Juan noted (under heavy I/O loads). That's very unfortunate
because it is my opinion too that shrink_mmap() fails too easily and
falls back to swapping when it really shouldn't do that.

Regards,
-- 
Zlatko
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: reduce shrink_mmap rate of failure (initial attempt)

[Roger Larsson]

>That patch hangs my machine here when I run mmap002.  The machine is
>in shrink_mmap.  It hangs trying to get the pagmap_lru_lock.
>
>I think that the idea is good, but it doesn't work here :(.
>
>Later, Juan.


Ouch...

The only possible explaination is that we are searching for pages on a zone.
But no such pages are possible to free from LRU...
And we LOOP the list, holding the lru lock...
Note: without this patch you may end up in another bad situation where
shrink_mmap always fails and swapping will start until it swaps out a page
of that specific zone.
And without the test? We would free all other LRU pages without finding one
that we want :-(

This will be interesting to fix...

May the allocation of pages play a part? Filling zone after zone will give no
mix between the zones.

/RogerL
(from work)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: reduce shrink_mmap rate of failure (initial attempt)

[Rik van Riel]

On Wed, 7 Jun 2000, Roger Larsson wrote:

> >That patch hangs my machine here when I run mmap002.  The machine is
> >in shrink_mmap.  It hangs trying to get the pagmap_lru_lock.
> 
> The only possible explaination is that we are searching for
> pages on a zone. But no such pages are possible to free from
> LRU...  And we LOOP the list, holding the lru lock...

Indeed. I'll make sure to have this fixed in the multi-list
setup I'm working on. A quick fix is to have a per-zone counter
of the number of lru pages, so you can just stop when
zone->lru_pages == 0.

> May the allocation of pages play a part? Filling zone after zone
> will give no mix between the zones.

Ohh, but it does. On most systems, after memory fills up, we'll
be allocating a few pages from one zone, then a few pages from
the next zone, then kswapd kicks in and we start all over again.

The fact that you and quintela had to go through trouble to hit
the bug makes me believe that it's not a very big issue in the
normal case ... just something we want fixed and luckily the
fix is easy.

regards,

Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.

Wanna talk about the kernel?  irc.openprojects.net / #kernelnewbies
http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: reduce shrink_mmap rate of failure (initial attempt)

[Roger Larsson]

Hi again,

There is an even more likely scenario...

* shrink_mmap is called when no zone is under pressure.
  then it will search the list over and over again...

Who could do such a nasty thing?

do_try_to_free_pages !

  Its agenda is not to stop when there is no pressure - it will go on
  until FREE_COUNT (8) pages are freed...

Even in the normal ac10 this should be a problem.
Several turns in do_try_to_free_pages with shrink_mmap failing will be
done - each swapping out some pages...

It will take time and swap...

/RogerL


Roger Larsson wrote:
> 
> >That patch hangs my machine here when I run mmap002.  The machine is
> >in shrink_mmap.  It hangs trying to get the pagmap_lru_lock.
> >
> >I think that the idea is good, but it doesn't work here :(.
> >
> >Later, Juan.
> 
> Ouch...
> 
> The only possible explaination is that we are searching for pages on a zone.
> But no such pages are possible to free from LRU...
> And we LOOP the list, holding the lru lock...
> Note: without this patch you may end up in another bad situation where
> shrink_mmap always fails and swapping will start until it swaps out a page
> of that specific zone.
> And without the test? We would free all other LRU pages without finding one
> that we want :-(
> 
> This will be interesting to fix...
> 
> May the allocation of pages play a part? Filling zone after zone will give no
> mix between the zones.
> 
> /RogerL
> (from work)
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux.eu.org/Linux-MM/

--
Home page:
  http://www.norran.net/nra02596/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/