vmstats patch against 2.4.8pre7 and new userlevel hack

vmstats patch against 2.4.8pre7 and new userlevel hack

[Marcelo Tosatti]

Hi, 

I've updated the vmstats patch to use Andrew Morton's statcount facilities
(which is in initial development state). I've also removed/added some
statistics due to VM changes.

On the userlevel side, I got zab's cpustat nice tool and transformed it
into an ugly hack which allows me to easily add/remove statistic
counters.

Adding a new statistic counter just needs (on the kernel side): 

VMSTAT(stat_name); (for global stats)

or 

VMSTAT_ZONE(zone, stat_name); (for perzone stats) 

On userlevel side, one line with corresponding stat_name, plus the field
name to be reported in the userlevel tool output for the given stat. 

Easy. 

The patch including the statcounts facilities
http://bazar.conectiva.com.br/~marcelo/patches/v2.4/2.4.8pre7/vmstats.patch

The userlevel tool
http://bazar.conectiva.com.br/~marcelo/nvmstat-0.1/nvmstat-0.1.tar.gz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: vmstats patch against 2.4.8pre7 and new userlevel hack

[Daniel Phillips]

On Thursday 09 August 2001 08:45, Marcelo Tosatti wrote:
> I've updated the vmstats patch to use Andrew Morton's statcount facilities
> (which is in initial development state). I've also removed/added some
> statistics due to VM changes.

I applied it and added some of my own statistics.  Very nice, much nicer than 
the traditional compile-reboot-measure-the-time cycle.

For one thing, it means you can watch the system in operation under a test 
load and see what it's really doing.  Chances are, you know right then 
whether it's running well or not and don't have to wait till the end of a 
long test run.

Problem: none of the statistics show up in proc until the first time the 
kernel hits them.  The /proc/stats entry isn't even there until the kernel 
hits the first statistic.  This isn't user-friendly.

I can see that this patch is going to break a lot between kernel updates, 
because it touches precisely the places we work on all the time - that's why 
the stats are there, right?  I'd suggest breaking it into two patchs, one 
with all the support and a few basic statistics in stable places, and another 
that adds in the rest of your current favorite vm stats.  It would also be 
nice if the stats were broken up into sets that can be catted out of proc 
onto the screen, in other words, sets of 23 or less.  This would mean that 
that something like watch cat /proc/stats/vm is already an effective 
interface.

I already learned a lot more about the what's actually happening inside the 
vm using this.  One thing that surprised me is how few locked pages there 
actually are on the inactive_dirty list.  I suppose I'd need a heavy mmap 
load to see more activity there.  Maybe a heavy write load would show up more 
there, but for now it looks like there are so few of those locked pages it 
won't interfere with scanning performance at all.

> On the userlevel side, I got zab's cpustat nice tool and transformed it
> into an ugly hack which allows me to easily add/remove statistic
> counters.

I didn't get that to work.  It seemed to be looking at the wrong /proc file.
I didn't look into it further.

--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: vmstats patch against 2.4.8pre7 and new userlevel hack

[Marcelo Tosatti]

On Fri, 10 Aug 2001, Daniel Phillips wrote:

> On Thursday 09 August 2001 08:45, Marcelo Tosatti wrote:
> > I've updated the vmstats patch to use Andrew Morton's statcount facilities
> > (which is in initial development state). I've also removed/added some
> > statistics due to VM changes.
> 
> I applied it and added some of my own statistics.  Very nice, much nicer than 
> the traditional compile-reboot-measure-the-time cycle.
> 
> For one thing, it means you can watch the system in operation under a test 
> load and see what it's really doing.  Chances are, you know right then 
> whether it's running well or not and don't have to wait till the end of a 
> long test run.
> 
> Problem: none of the statistics show up in proc until the first time the 
> kernel hits them.  The /proc/stats entry isn't even there until the kernel 
> hits the first statistic.  This isn't user-friendly.

Right. This has to be fixed.
 
> I can see that this patch is going to break a lot between kernel updates, 
> because it touches precisely the places we work on all the time - that's why 
> the stats are there, right? 

Exactly. Thats why I've thought about doing the thing in an easy way to
remove/add new statistics.

Kudos goes to Andrew for the statcount code. Thanks a lot! 

> I'd suggest breaking it into two patchs, one with all the support and
> a few basic statistics in stable places, and another that adds in the
> rest of your current favorite vm stats.  It would also be nice if the
> stats were broken up into sets that can be catted out of proc onto the
> screen, in other words, sets of 23 or less.  This would mean that that
> something like watch cat /proc/stats/vm is already an effective
> interface.
> 
> I already learned a lot more about the what's actually happening inside the 
> vm using this.  One thing that surprised me is how few locked pages there 
> actually are on the inactive_dirty list.  I suppose I'd need a heavy mmap 
> load to see more activity there.  Maybe a heavy write load would show up more 
> there, but for now it looks like there are so few of those locked pages it 
> won't interfere with scanning performance at all.
> 
> > On the userlevel side, I got zab's cpustat nice tool and transformed it
> > into an ugly hack which allows me to easily add/remove statistic
> > counters.
> 
> I didn't get that to work.  It seemed to be looking at the wrong /proc
> file. I didn't look into it further.

The default /proc file it tries to open is "/proc/vm_stat", while the
stats are at "/proc/stats/vm". Use the -p option to select the file to
read the stats from. (nvmstat  -p /proc/stats/vm)

The userlevel tool will average the stats, so you can actually see whats
happening over time, and not just see "what already happened". Its really
really useful. 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: vmstats patch against 2.4.8pre7 and new userlevel hack

[Andrew Morton]

Marcelo Tosatti wrote:
> 
> > Problem: none of the statistics show up in proc until the first time the
> > kernel hits them.  The /proc/stats entry isn't even there until the kernel
> > hits the first statistic.  This isn't user-friendly.
> 
> Right. This has to be fixed.
> 

Does it?  The userspace tool can just assume the value is zero
if it isn't available.

If we want unencountered counters to appear in the summary
we'd have to declare them, which means two lines of code
rather than one :)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/