Re: 2.1.89 broken?

Re: 2.1.89 broken?

[Stephen C. Tweedie]

Hi,

On Tue, 10 Mar 1998 09:38:41 +0100 (MET), Rik van Riel
<H.H.vanRiel@fys.ruu.nl> said:

> On 10 Mar 1998, Trond Eivind =?ISO-8859-1?Q?Glomsr=F8d?= wrote:
>> That is my experience as well... I've got 80 MB. It was happy with
>> about 56 MB for caches, 4 MB free and a little less than 20 MB
>> used. Oh - and 75 MB used swap.

> You both seem to be ignoring the fact that sticking
> unused stuff in swap is better than freeing disk
> cache pages. In 2.1.89 we age disk cache pages in
> much the same way we age private (in-swap) pages.
> Because the aging is the same, you can be quite sure
> that Linux is doing the right thing...

Rik, 

No, it's not necessarily doing the Right Thing.  The trouble is that
there is no balancing between swapping and emptying the page cache.
The current balancing heuristic just about works with the old page
cache aging, but if we change that aging, then we just force the
kernel to keep trying to free pages from one source for as long as
there are freeable pages on that source.  Note that this doesn't mean
that it is _cheap_ to free these pages; just that the kernel can,
within the bounds of a single pass through try_to_free_page(), find at
least one swappable page.

Now, once we've got a single pass which can scavenge BOTH page cache
and swap pages, then we're really going to be cooking on gas. :)  For
now, however, all we're doing is tweaking what is a very very delicate
balance, and as we proved in the 1.2.4 and 1.2.5 swapping disasters,
getting such a change done in a way which doesn't make at least
somebody's performance very much worse is really quite hard to do in
the current way of managing memory.  When I was doing the first round
of work on kswap, it was this balance between cache and swap which was
the biggest problem, not the aging of individual pages from either
source.

Cheers,
 Stephen.

Re: 2.1.89 broken?

[Rik van Riel]

On Wed, 11 Mar 1998, Stephen C. Tweedie wrote:

> No, it's not necessarily doing the Right Thing.  The trouble is that
> there is no balancing between swapping and emptying the page cache.

I've still got some Digital Unix-like balancing code
lying around...
Basically, you can set 3 values for the buffer/page
cache, a minimum value, a maximum value and a steal
value. When the buffer/page memory is above steal
level and the system needs memory, it'll steal memory
from the page cache first. A good default would be
25% of main memory. Of course, these values will be
sysctl controllable (we still got 8 unused variables
in swap_control ;-).

> Now, once we've got a single pass which can scavenge BOTH page cache
> and swap pages, then we're really going to be cooking on gas. :)  For

I think we should just copy DU's scheme:
- when buffer/page cache is above steal level, we steal that memory
- otherwise, we steal in a round-robin fashion from both

> now, however, all we're doing is tweaking what is a very very delicate
> balance, and as we proved in the 1.2.4 and 1.2.5 swapping disasters,
> getting such a change done in a way which doesn't make at least
> somebody's performance very much worse is really quite hard to do in
> the current way of managing memory.  When I was doing the first round
> of work on kswap, it was this balance between cache and swap which was
> the biggest problem, not the aging of individual pages from either
> source.

That's why we have sysctl controllable swapping. And now
we're talking about it, the sysctl really needs updating
too...

You can expect these patches RSN (maybe even tomorrow).

grtz,

Rik.
+-------------------------------------------+--------------------------+
| Linux: - LinuxHQ MM-patches page          | Scouting       webmaster |
|        - kswapd ask-him & complain-to guy | Vries    cubscout leader |
|     http://www.fys.ruu.nl/~riel/          | <H.H.vanRiel@fys.ruu.nl> |
+-------------------------------------------+--------------------------+

Re: 2.1.89 broken?

[Dr. Werner Fink]

> I've still got some Digital Unix-like balancing code
> lying around...
> Basically, you can set 3 values for the buffer/page
> cache, a minimum value, a maximum value and a steal
> value. When the buffer/page memory is above steal
> level and the system needs memory, it'll steal memory
> from the page cache first. A good default would be
> 25% of main memory. Of course, these values will be
> sysctl controllable (we still got 8 unused variables
> in swap_control ;-).

Does this mean that mm/filemap.c:shrink_mmap() would call
for it's self in mm/vmscan.c:kswapd() if the level is above
the limit?  ... without using mm/vmscan.c:(do_)try_to_free_page()
to become the buffer down without trashing the tasks?

This would need also a wrapper as it does for
do_try_to_free_page() to set/unset the kernel locks.

And this upper limit should be calculated dynamically because
there is a `small' difference between a 8Mb and a 512Mb
system ... the first systems should have a smaller amount
of buffer/cache to keep the system running :-)

           Werner