vm scenario tool / mincore(2) functionality for regular pages?

vm scenario tool / mincore(2) functionality for regular pages?

[bert hubert]

I'm building a tool to subject the VM to different scenarios and I'd like to
be able to determine if a page is swapped out or not. For a file I can
easily determine if a page is in memory (in the page cache) or not using the
mincore(2) system call.

I want to expand my tool so it can investigate which of its pages are
swapped out under cache pressure or real memory pressure.

However, to do this, I need a way to determine if a page is there or if it
is swapped out. My two questions are:

	1) is there an existing way to do this
	   (the kernel obviously knows)

	2) would it be correct to expand mincore to also work on
           non-filebacked memory so it works for 'swap-backed' memory too?

Thanks.

Some current output of the scenario tool:

vmloader> alloc 25
Arena now 25 megabytes, 6250 pages

vmloader> sweep
Sweeping from mbyte 0 to 25, 6250 pages. Done

vmloader> rusage
minor: 6250, major: 2, swaps: 0

vmloader> sweep 0 12
Sweeping from mbyte 0 to 12, 1440 pages. Done

vmloader> rusage
minor: 0, major: 0, swaps: 0

vmloader> touch
Touching from mbyte 0 to 25, 6250 pages. Done

vmloader> rusage
minor: 6249, major: 0, swaps: 0

vmloader> rsweep
Random sweeping from mbyte 0 to 25, 6250 pages. Done

vmloader> rusage
minor: 0, major: 0, swaps: 0


-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: vm scenario tool / mincore(2) functionality for regular pages?

[Andrew Morton]

bert hubert wrote:
> 
> I'm building a tool to subject the VM to different scenarios and I'd like to
> be able to determine if a page is swapped out or not. For a file I can
> easily determine if a page is in memory (in the page cache) or not using the
> mincore(2) system call.
> 
> I want to expand my tool so it can investigate which of its pages are
> swapped out under cache pressure or real memory pressure.
> 
> However, to do this, I need a way to determine if a page is there or if it
> is swapped out. My two questions are:
> 
>         1) is there an existing way to do this
>            (the kernel obviously knows)
> 
>         2) would it be correct to expand mincore to also work on
>            non-filebacked memory so it works for 'swap-backed' memory too?
> 

mincore needs to be taught to walk pagetables and to look up
stuff in swapcache.

Also it currently assumes that vma->vm_file is mapped linearly,
so it will return incorrect results with Ingo's nonlinear mapping
extensions.

But if we were to use Ingo's "file pte's" for all mmappings, mincore
only needs to do the pte->pagecache lookup, so it can lose the
"vma is linear" arithmetic.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: vm scenario tool / mincore(2) functionality for regular pages?

[bert hubert]

On Tue, Oct 22, 2002 at 01:58:36PM -0700, Andrew Morton wrote:

> mincore needs to be taught to walk pagetables and to look up
> stuff in swapcache.

As mincore appears to be entirely unstandardized, we can get away with
extending its functionality.

> Also it currently assumes that vma->vm_file is mapped linearly,
> so it will return incorrect results with Ingo's nonlinear mapping
> extensions.

It also appears to fail if the memory range it is offered lives in multiple
vmas. I'm unsure if this is possible, but I recall reading about mozilla
needing 'vma merging', which seems to imply that a process can have more of
them.

> But if we were to use Ingo's "file pte's" for all mmappings, mincore
> only needs to do the pte->pagecache lookup, so it can lose the
> "vma is linear" arithmetic.

The pagetable walking and swapcache lookup is orthogonal to this? 

By the way, version 0.1 which is mildly functional is on
http://ds9a.nl/vmloader-0.1.tar.gz , it currently does mincore only for
mmapped files. Use 'mkfile name 100' to create a 100mb file, 'map name' to
map it.

It is interesting to note that 2.4.20-pre9 allows me to allocate 250
megabytes and touch it sequentially without dire behaviour on a 186 megabyte
(or so) machine. RSS is a reasonable 153MB afterwards.

Reading that 250 megabytes from the start again however causes massive
swapping and takes way longer than the initial touching. Probably some kind
of 'use once' heuristic that is suddenly disabled.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/