Re: mmap-age patch, comments wanted

Re: mmap-age patch, comments wanted

[Rik van Riel]

On Mon, 22 Dec 1997, Noel Maddy wrote:

> That was definitely the case with your older vhand patches -- with 
> them, I could get about 20M more into virtual memory before 
> performance started degrading.  What I'm seeing now is a change in 
> performance at the same load level.  I'm not sure whether the overall 
> performance is hurt, because the vanilla kernel thrashes a lot in the 
> same situation, but the system remains responsive.  It could be that 
> the load takes longer in the vanilla kernel (I'll try to check that 
> today), but the lack of responsiveness with the mmap-age patch makes 
> it *seem* slower.  

With vhand, the kernel didn't properly age user-pages, so
swap usage was overly high compared to vanilla or mmap-age,
so comparing swap usage is no good indication of system load.

Also, between vanilla and mmap-age, the mmap-age patched kernel
uses swap more than the vanilla one. But the difference should
be very small, so swap usage should still be usable as an
indication for VM load.

I think that what's really making things slower, is that kswapd
now has to scan more pages before it can swap one out. This
makes the swapout slower (as MAX_SWAP_FAIL still is set at 3)
so there are less free pages left to swap things in again.
I'm going to try to resolve this issue right now...

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: mmap-age patch, comments wanted

[Rik van Riel]

On Sat, 20 Dec 1997, Noel Maddy wrote:

> I haven't had the time to do comprehensive testing with your mmap-age 
> patch, but I have been alternately running 2.1.71 with and without the 
> patch.  It seems like the patch really helps for steady-state 
> situations, but has problems when I start large programs when my system 
> is already overloaded.

Hmm, yes. This is a problem, it can probably be solved by
tuning some parameters in /proc/sys/vm (I have the problem too,
but not by far as severe as you have them). <see below>

> The problem comes when I run another large application, like Debian's 
> dpkg, or netscape, or gimp.  With the mmap-age patched kernel, the 
> system freezes for 5 to 10 seconds while loading one of these large 
> apps -- no mouse movement, no X updating, nothing.  When I run vmstat 
> while this is happening, I see a huge amount of page-out with no 
> page-in followed by a huge amount of page-in.
> 
> When I do the same thing without the mmap-age patch, the system slows 
> down, but does not become completely unresponsive.

At my place it doesn't become unresponsive, not even when I'm
already swapping and then I load Netscape...

> I'm not that familiar with the Linux mm system, but I'm guessing that 
> what's happening is that since all of the new pages for the starting 
> process are referenced while it loads, they all have a high priority, 
> and won't be replaced by more pages from the same new process.  Once 
> the new process starts, the  pages from the older processes that have 
> been paged out have to be paged in again.
> 
> If that's the case, might it help to start the new pages out pre-aged?  
> That way, there'd be a chance that a new page for the new process would 
> replace older pages for the new process rather than pages from older 
> processes?

They are loaded preaged. And since Linux does demand-loading,
only the pages actually required are loaded into core...

I think it's the anti-fragmentation part that's biting you,
ie. the kernel has some free pages in large chunks --> memory
gets allocated --> kswapd has to free loads of random pages until
there are some large chunks of free pages again.

But if we don't have the anti-fragmentation stuff, the kernel
wouldn't run as stable as it does now, and I'm not willing to
sacrifice stability for performance...
(and frankly, I don't think anyone is:)

> Of course, this is just speculation rather than from studying the code 
> -- if I'm all wet, just tell me...

Well, you did have a clue. Not neccesary the right clue, but
it did make me think about what to do...
I might actually have some solution^H^H^H^H^H^H^H^Hworkaround
for this by tomorrow...

Merry Cristmas,

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

mmap-age patch, comments wanted

[Rik van Riel]

Hi,

as most of you read, I released my mmap-age patch a few
weeks ago. Since then I've had a few reports of people
(all success stories), but not enough to submit the patch
to Linus.

Although there's no really new code in it, and everything
is well-tested, I'd like a few more reports from people
before sending it off to Linus.

And as to the code-freeze, this patch makes the system
more stable (thanks to Zlatko's part) and there are no
really new features in it, just an old feature aplied
to another part of the system.

As always, go to LinuxHQ, then to the 2.1 unoff patches,
and then my patches-page...

If I get about 10 more success stories (and no bug reports)
I'll send it off to Linus, for a better performing 2.2...

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+