On Wed, May 20, 2009 at 09:59:05AM +0800, KOSAKI Motohiro wrote:
> > On Tue, May 19, 2009 at 08:28:28PM +0800, KOSAKI Motohiro wrote:
> > > Hi
> > > 
> > > 2009/5/19 Wu Fengguang <fengguang.wu@intel.com>:
> > > > On Tue, May 19, 2009 at 04:06:35PM +0800, KOSAKI Motohiro wrote:
> > > >> > > > Like the console mode, the absolute nr_mapped drops considerably - to 1/13 of
> > > >> > > > the original size - during the streaming IO.
> > > >> > > >
> > > >> > > > The delta of pgmajfault is 3 vs 107 during IO, or 236 vs 393 during the whole
> > > >> > > > process.
> > > >> > >
> > > >> > > hmmm.
> > > >> > >
> > > >> > > about 100 page fault don't match Elladan's problem, I think.
> > > >> > > perhaps We missed any addional reproduce condition?
> > > >> >
> > > >> > Elladan's case is not the point of this test.
> > > >> > Elladan's IO is use-once, so probably not a caching problem at all.
> > > >> >
> > > >> > This test case is specifically devised to confirm whether this patch
> > > >> > works as expected. Conclusion: it is.
> > > >>
> > > >> Dejection ;-)
> > > >>
> > > >> The number should address the patch is useful or not. confirming as expected
> > > >> is not so great.
> > > >
> > > > OK, let's make the conclusion in this way:
> > > >
> > > > The changelog analyzed the possible beneficial situation, and this
> > > > test backs that theory with real numbers, ie: it successfully stops
> > > > major faults when the active file list is slowly scanned when there
> > > > are partially cache hot streaming IO.
> > > >
> > > > Another (amazing) finding of the test is, only around 1/10 mapped pages
> > > > are actively referenced in the absence of user activities.
> > > >
> > > > Shall we protect the remaining 9/10 inactive ones? This is a question ;-)
> > > 
> > > Unfortunately, I don't reproduce again.
> > > I don't apply your patch yet. but mapped ratio is reduced only very little.
> > 
> > mapped ratio or absolute numbers? The ratio wont change much because
> > nr_mapped is already small.
> 
> My box is running Fedora 10 initlevel 5 (GNOME desktop).
> 
> many GNOME component is mapped very many process (likes >50).
> Thus, these page aren't dropped by typical any workload.

Yeah, that's possible (but sounds bloated in regard of active
working set size).

> > > I think smem can show which library evicted.  Can you try it?
> > > 
> > > download:  http://www.selenic.com/smem/
> > > usage:   ./smem -m -r --abbreviate
> > 
> > Sure, but I don't see much change in its output (see attachments).
> > 
> > smem-console-0 is collected after fresh boot,
> > smem-console-1 is collected after the big IO.
> 
> hmmmm, your result has following characatistics.
> 
> - no graphics component
> - very few mapped library
>   (it is almost only zsh library)
> 
> Can you try test on X environment?

Sure, see the attached smem-x-0/1. This time we see sufficient differences.

> > > We can't decide 9/10 is important or not. we need know actual evicted file list.
> > 
> > Right. But what I measured is the activeness. Almost zero major page
> > faults means the evicted 90% mapped pages are inactive during the
> > long 300 seconds of IO.
> 
> Agreed.
> IOW, I don't think your test environment is typical desktop...

Kind of :)  It's fluxbox + terminal + firefox, a bare desktop for
testing things out.