Automated performance testing system was Re: Text form for STP tests

Automated performance testing system was Re: Text form for STP tests

[Marcelo Tosatti]

Linux-MM fellows,

I've been talking to Cliff about the need for a set of benchmarks,
covering as many different workloads as possible, for developers to have a 
better notion of impact on performance changes. 

Usually when one does a change which affects performance, he/she runs one 
or two benchmarks with a limited amount of hardware configurations.
This is a very painful, boring and time consuming process, which can 
result in misinterpretation and/or limited understading of the results 
of such changes.

It is important to automate such process, with a set of benchmarks 
covering as wide as possible range of workloads, running on common 
and most used hardware variations.

OSDL's STP provides the base framework for this.

Cliff mentioned an internal tool they are developing for this purpose, 
based on XML-like configuration files. 

I have suggested him a set of benchmarks (available on STP right now, 
we want to add other benchmarks there whenever necessary) and a set of 
CPU/memory variations.

Quoting myself:

"here is a list of a wide enough 
group of performance tests, for a start:

bonnie++
reaim (default, new_fserver, shared)
dbench_long
kernbench
tiobench

Each of these running one the following combinations:

1CPU, 2CPU, 4CPU, 8CPU (4 variants).

total memory, half memory, a quarter of total memory (3 variants).

Thats 12 results for each benchmark."

Obviously this set of benchmarks is limited (an example set), they are 
relatively similar, but it is a start. 
In the future we ough to have several sets of benchmarks.

We also need a way to easily visualize such results, which is the next
step of the project. x,y graphics are the best way for most tests 
I believe.

Construction of such automated testing infrastructure will improve our 
capabilities giving better notion of impact while decreasing 
wasted time and wasted efforts.

Of course this needs to be a collaborative effort, I expect others
interested to help, comment and get involved.

Follows my private discussion with Cliff up till now, for reference.

On Sun, Nov 28, 2004 at 12:17:02PM -0800, Cliff White wrote:
> > 
> > Hi Cliff,
> > 
> > On Wed, Nov 24, 2004 at 10:22:57AM -0800, Cliff White wrote:
> > > 
> > > Marcelo, we are working on a better version of 
> > > this as I mentioned, plus a command line tool.
> > 
> > Great :)
> > 
> > > The current file is almost XML, and some
> > > of the data is internal to our database. If you
> > > can indicate what tests you want to run on specific
> > > sizes of machines,  we can supply all the numbers. 
> > > 
> > > It the XML is too much, just send me a list of what you
> > > want, and we can go from there.
> > 
> > That looks fine to me - here is a list of a wide enough 
> > group of performance tests, for a start:
> > 
> > bonnie++
> > reaim (default, new_fserver, shared)
> > dbench_long
> > kernbench
> > tiobench
> > 
> > Each of these running one the following combinations:
> > 
> > 1CPU, 2CPU, 4CPU, 8CPU (4 variants).
> > 
> > total memory, half memory, a quarter of total memory (3 variants).
> > 
> > Thats 12 results for each benchmark. 
> 
> I can get this setup monday, ( long weekend ) 
> > 
> > We need to make those results visually easy to read/interpret, 
> > but thats another story which can be done independantly.
> > A x,y graph is the easier way to visualize - thats the next step.
> 
> Great, appreciate any input on this. 
> 
> > 
> > > We have an automated tool that does a string match against
> > > kernel.org patch names, if you have a specific regexp, we
> > > can tie an automatic test submit to that. 
> > > 
> > > The text file just specifies test details, kernel patch info
> > > is supplied in a separate step, so one text file can
> > > be re-used.
> > 
> > OK - great. We want to run the testgroup on baseline kernel 
> > and on baseline+modification, of course.
> > 
> > Can we move this discussion to linux-mm? I'm sure others are interested
> > as well.
> 
> Sure, would you start a thread? I'm on that list already.
> cliffw
> 
> > 
> > Many thanks, this will certainly improve out performance testing capabilities.
> > 
> > > ------------Current-------------------
> > > <config >
> > >     <stp2-000 host="77">
> > >         <aio-stress testID="82">
> > >             <distro>4</distro>
> > >             <run>
> > >                 <lilo>profile=2</lilo>
> > >                 <opt pID="-F" val="1" />
> > >                 <opt pID="-s" val="4g" />
> > >             </run>
> > >         </aio-stress>
> > >     </stp2-000>
> > > </config>
> > > ----------------------------------
> > > 
> > > We've changed the format some, but the tools to use
> > > it are still being fixed up. New format (so far)
> > > will look like this: ( the 'test name' and 'host name' bits allow
> > > for multiple runs of the same test/host )
> > > 
> > > --------------------------------
> > > 
> > > <config>
> > >   <host name="cpu1" id="14">
> > >     <test name="dummy_test" id="82">
> > >       <distro id="4" />
> > >       <run>
> > >         <lilo>profile=2</lilo>
> > >         <param name="-o">25</param>
> > >         <param name="-z">foobar</param>
> > >       </run>
> > >       <run>
> > >         <lilo>profile=2</lilo>
> > >         <param name="-o">25</param>
> > >         <param name="-z">foobar</param>
> > >       </run>
> > >     </test>
> > >   </host>
> > >  <host name="cpu2" id="15">
> > >     <test name="dummy_test" id="82">
> > >     <distro id="4" />
> > >       <run>
> > >         <lilo>profile=2</lilo>
> > >         <param name="-o">25</param>
> > >         <param name="-z">foobar</param>
> > >       </run>
> > >     </test>
> > >   </host>
> > > </config>
> > > ----------------------------------
> > > 
> > > cliffw
> > 
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Martin J. Bligh]

> I've been talking to Cliff about the need for a set of benchmarks,
> covering as many different workloads as possible, for developers to have a 
> better notion of impact on performance changes. 
> 
> Usually when one does a change which affects performance, he/she runs one 
> or two benchmarks with a limited amount of hardware configurations.
> This is a very painful, boring and time consuming process, which can 
> result in misinterpretation and/or limited understading of the results 
> of such changes.
> 
> It is important to automate such process, with a set of benchmarks 
> covering as wide as possible range of workloads, running on common 
> and most used hardware variations.
> 
> OSDL's STP provides the base framework for this.
> 
> Cliff mentioned an internal tool they are developing for this purpose, 
> based on XML-like configuration files. 
> 
> I have suggested him a set of benchmarks (available on STP right now, 
> we want to add other benchmarks there whenever necessary) and a set of 
> CPU/memory variations.

Sounds like a good plan in general, by why on earth would you want to do
it in XML? Personally I'm not that much into masochism. A simple text
control file is perfectly sufficient (and yes, we do this internally).

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[cliff white]

On Tue, 30 Nov 2004 09:00:59 -0800
"Martin J. Bligh" <mbligh@aracnet.com> wrote:

> > I've been talking to Cliff about the need for a set of benchmarks,
> > covering as many different workloads as possible, for developers to have a 
> > better notion of impact on performance changes. 
> > 
> > Usually when one does a change which affects performance, he/she runs one 
> > or two benchmarks with a limited amount of hardware configurations.
> > This is a very painful, boring and time consuming process, which can 
> > result in misinterpretation and/or limited understading of the results 
> > of such changes.
> > 
> > It is important to automate such process, with a set of benchmarks 
> > covering as wide as possible range of workloads, running on common 
> > and most used hardware variations.
> > 
> > OSDL's STP provides the base framework for this.
> > 
> > Cliff mentioned an internal tool they are developing for this purpose, 
> > based on XML-like configuration files. 
> > 
> > I have suggested him a set of benchmarks (available on STP right now, 
> > we want to add other benchmarks there whenever necessary) and a set of 
> > CPU/memory variations.
> 
> Sounds like a good plan in general, by why on earth would you want to do
> it in XML? Personally I'm not that much into masochism. A simple text
> control file is perfectly sufficient (and yes, we do this internally).

True, very true. What i was showing Marcelo was what we do internally, which
was not desiged for humans. What we're working on currently _is designed for humans,
and not XML.

I'm making up this AM a control file for STP for the tests marcelo requested, and
we won't be asking humans to do XML, no. 

Martin, if you could share any of your internal goop ( and save me work ) it would be 
great. If you all have a text format that large numbers of people find sensible, i'd love it.
So far, it's just been me and the robots. 

( btw, it's XML because XML::Simple just seemed so....simple. I was young then. ) 
cliffw

> 
> M.
> 


-- 
The church is near, but the road is icy.
The bar is far, but i will walk carefully. - Russian proverb
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Cliff White]

> Linux-MM fellows,
> 
> I've been talking to Cliff about the need for a set of benchmarks,
> covering as many different workloads as possible, for developers to have a 
> better notion of impact on performance changes. 
> 
> Usually when one does a change which affects performance, he/she runs one 
> or two benchmarks with a limited amount of hardware configurations.
> This is a very painful, boring and time consuming process, which can 
> result in misinterpretation and/or limited understading of the results 
> of such changes.
> 
> It is important to automate such process, with a set of benchmarks 
> covering as wide as possible range of workloads, running on common 
> and most used hardware variations.
> 
> OSDL's STP provides the base framework for this.
> 
[ snip ]
> bonnie++
> reaim (default, new_fserver, shared)
> dbench_long
> kernbench
> tiobench
> 
> Each of these running one the following combinations:
> 
> 1CPU, 2CPU, 4CPU, 8CPU (4 variants).
> 
> total memory, half memory, a quarter of total memory (3 variants).
> 
> Thats 12 results for each benchmark."
> 
The configuration files to do these tests are now written, and the humble
robots are running this test series against linux-2.6.7 ( for history data )
There will need to be some adjustments - some of these tests will no doubt
fail for reasons of script error or configuration ( i see already kernbench will 
have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )

And, the second part of the automation is already done, but needs input.
I can aim this test battery at any kernel patch, where 'any kernel patch'
is identified by a regexp. What kernels do you want this against? 
I've heard mention of 'baseline' - we call this baseline:

/^(patch|linux)-\d+\.\d+\.\d+$/

( starts with 'patch' or 'linux', then '-' followed by three decimals ) 

cliffw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Marcelo Tosatti]

On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > Linux-MM fellows,
> > 
> > I've been talking to Cliff about the need for a set of benchmarks,
> > covering as many different workloads as possible, for developers to have a 
> > better notion of impact on performance changes. 
> > 
> > Usually when one does a change which affects performance, he/she runs one 
> > or two benchmarks with a limited amount of hardware configurations.
> > This is a very painful, boring and time consuming process, which can 
> > result in misinterpretation and/or limited understading of the results 
> > of such changes.
> > 
> > It is important to automate such process, with a set of benchmarks 
> > covering as wide as possible range of workloads, running on common 
> > and most used hardware variations.
> > 
> > OSDL's STP provides the base framework for this.
> > 
> [ snip ]
> > bonnie++
> > reaim (default, new_fserver, shared)
> > dbench_long
> > kernbench
> > tiobench
> > 
> > Each of these running one the following combinations:
> > 
> > 1CPU, 2CPU, 4CPU, 8CPU (4 variants).
> > 
> > total memory, half memory, a quarter of total memory (3 variants).
> > 
> > Thats 12 results for each benchmark."
> > 
> The configuration files to do these tests are now written, and the humble
> robots are running this test series against linux-2.6.7 ( for history data )
> There will need to be some adjustments - some of these tests will no doubt
> fail for reasons of script error or configuration ( i see already kernbench will 
> have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )
> 
> And, the second part of the automation is already done, but needs input.
> I can aim this test battery at any kernel patch, where 'any kernel patch'
> is identified by a regexp. What kernels do you want this against? 

The most recent 2.6.10-rc2 and 2.6.10-rc2-mm in STP.

Will this be available through the web interface? 

Thanks again Cliff

> I've heard mention of 'baseline' - we call this baseline:
> 
> /^(patch|linux)-\d+\.\d+\.\d+$/
> 
> ( starts with 'patch' or 'linux', then '-' followed by three decimals ) 
> 
> cliffw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Cliff White]

> On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > > Linux-MM fellows,
> > > 
> > > I've been talking to Cliff about the need for a set of benchmarks,
> > > covering as many different workloads as possible, for developers to have a 
> > > better notion of impact on performance changes. 
> > > 
[snip]
> > robots are running this test series against linux-2.6.7 ( for history data )
> > There will need to be some adjustments - some of these tests will no doubt
> > fail for reasons of script error or configuration ( i see already kernbench will 
> > have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )
> > 
> > And, the second part of the automation is already done, but needs input.
> > I can aim this test battery at any kernel patch, where 'any kernel patch'
> > is identified by a regexp. What kernels do you want this against? 
> 
> The most recent 2.6.10-rc2 and 2.6.10-rc2-mm in STP.
> 
> Will this be available through the web interface? 

Yes, the results should be visible. If something looks wrongs, email.
the 'advanced search' bit needs some test-specific fixes, and may not work
for all tests - some of the kits still needs some patching..
cliffw
> 
> Thanks again Cliff
> 
> > I've heard mention of 'baseline' - we call this baseline:
> > 
> > /^(patch|linux)-\d+\.\d+\.\d+$/
> > 
> > ( starts with 'patch' or 'linux', then '-' followed by three decimals ) 
> > 
> > cliffw
> 
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Marcelo Tosatti]

On Wed, Dec 01, 2004 at 12:04:09PM -0800, Cliff White wrote:
> > On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > > > Linux-MM fellows,
> > > > 
> > > > I've been talking to Cliff about the need for a set of benchmarks,
> > > > covering as many different workloads as possible, for developers to have a 
> > > > better notion of impact on performance changes. 
> > > > 
> [snip]
> > > robots are running this test series against linux-2.6.7 ( for history data )
> > > There will need to be some adjustments - some of these tests will no doubt
> > > fail for reasons of script error or configuration ( i see already kernbench will 
> > > have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )
> > > 
> > > And, the second part of the automation is already done, but needs input.
> > > I can aim this test battery at any kernel patch, where 'any kernel patch'
> > > is identified by a regexp. What kernels do you want this against? 
> > 
> > The most recent 2.6.10-rc2 and 2.6.10-rc2-mm in STP.
> > 
> > Will this be available through the web interface? 
> 
> Yes, the results should be visible. If something looks wrongs, email.
> the 'advanced search' bit needs some test-specific fixes, and may not work
> for all tests - some of the kits still needs some patching..
> cliffw

Any news on the automatic test series scripts Cliff ? 

Haven't seen any results yet.

Thanks 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[cliff white]

On Mon, 13 Dec 2004 09:42:23 -0200
Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:

> On Wed, Dec 01, 2004 at 12:04:09PM -0800, Cliff White wrote:
> > > On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > > > > Linux-MM fellows,
> > > > > 
> > > > > I've been talking to Cliff about the need for a set of benchmarks,
> > > > > covering as many different workloads as possible, for developers to have a 
> > > > > better notion of impact on performance changes. 
> > > > > 
> > [snip]
> > > > robots are running this test series against linux-2.6.7 ( for history data )
> > > > There will need to be some adjustments - some of these tests will no doubt
> > > > fail for reasons of script error or configuration ( i see already kernbench will 
> > > > have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )
> > > > 
> > > > And, the second part of the automation is already done, but needs input.
> > > > I can aim this test battery at any kernel patch, where 'any kernel patch'
> > > > is identified by a regexp. What kernels do you want this against? 
> > > 
> > > The most recent 2.6.10-rc2 and 2.6.10-rc2-mm in STP.
> > > 
> > > Will this be available through the web interface? 
> > 
> > Yes, the results should be visible. If something looks wrongs, email.
> > the 'advanced search' bit needs some test-specific fixes, and may not work
> > for all tests - some of the kits still needs some patching..
> > cliffw
> 
> Any news on the automatic test series scripts Cliff ? 
> 
> Haven't seen any results yet.
> 
I ran a set for 2.6.10-rc3, PLM 3957, some results here:
 http://www.osdl.org/projects/26lnxstblztn/results/
Or by doing this:
http://www.osdl.org/lab_activities/kernel_testing/stp/display_test_requests?d_patch_id%3Astring%3Aignore_empty=3957&op=Search

Marcelo, do you want me to submit the tests under your user id?
That would make searching for results eaiser. 
cliffw

> Thanks
> 
> 


-- 
The church is near, but the road is icy.
The bar is far, but i will walk carefully. - Russian proverb
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Marcelo Tosatti]

On Mon, Dec 13, 2004 at 08:22:26AM -0800, cliff white wrote:
> On Mon, 13 Dec 2004 09:42:23 -0200
> Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> 
> > On Wed, Dec 01, 2004 at 12:04:09PM -0800, Cliff White wrote:
> > > > On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > > > > > Linux-MM fellows,
> > > > > > 
> > > > > > I've been talking to Cliff about the need for a set of benchmarks,
> > > > > > covering as many different workloads as possible, for developers to have a 
> > > > > > better notion of impact on performance changes. 
> > > > > > 
> > > [snip]
> > > > > robots are running this test series against linux-2.6.7 ( for history data )
> > > > > There will need to be some adjustments - some of these tests will no doubt
> > > > > fail for reasons of script error or configuration ( i see already kernbench will 
> > > > > have to be redunced for 1-cpu systems, as it runs > 13.5 hours :( )
> > > > > 
> > > > > And, the second part of the automation is already done, but needs input.
> > > > > I can aim this test battery at any kernel patch, where 'any kernel patch'
> > > > > is identified by a regexp. What kernels do you want this against? 
> > > > 
> > > > The most recent 2.6.10-rc2 and 2.6.10-rc2-mm in STP.
> > > > 
> > > > Will this be available through the web interface? 
> > > 
> > > Yes, the results should be visible. If something looks wrongs, email.
> > > the 'advanced search' bit needs some test-specific fixes, and may not work
> > > for all tests - some of the kits still needs some patching..
> > > cliffw
> > 
> > Any news on the automatic test series scripts Cliff ? 
> > 
> > Haven't seen any results yet.
> > 
> I ran a set for 2.6.10-rc3, PLM 3957, some results here:
>  http://www.osdl.org/projects/26lnxstblztn/results/
> Or by doing this:
> http://www.osdl.org/lab_activities/kernel_testing/stp/display_test_requests?d_patch_id%3Astring%3Aignore_empty=3957&op=Search
> 
> Marcelo, do you want me to submit the tests under your user id?
> That would make searching for results eaiser. 

Cliff, 

How have you started these tests? I dont to run LTP for example.

I would like to be able to select two different patch ID's and run them from the web
interface, on a set of benchmarks vs memory size ranges vs nrCPUs (you already do 
different number of CPUs on those series of tests I see), as we talked.
Not just me, every developer doing performance testing :)

And then generate the graphs for the results of one patchID vs another. 

On reaim for example it would be nice to have graphs of global jobs per minute vs 
memory size, with a different colors for each patch ID. 

Maybe we can even fit results for different nrCPUS on the same graph 
with line types (with symbols like triangle, square, to differentiate).
But then it might become too polluted to easily visualize, but maybe not.

Can you make the scripts which you are using for graphic generation and the
gnuplot configuration files available? So I can play around with them.
I want to help with that.

Another question: Is the source for reaim available? 

I see you're already generating the graphs for vmstat/iostat and user/system 
time.

Thats really nice.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Automated performance testing system was Re: Text form for STP tests

[Cliff White]

> On Mon, Dec 13, 2004 at 08:22:26AM -0800, cliff white wrote:
> > On Mon, 13 Dec 2004 09:42:23 -0200
> > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> > 
> > > On Wed, Dec 01, 2004 at 12:04:09PM -0800, Cliff White wrote:
> > > > > On Wed, Dec 01, 2004 at 10:28:24AM -0800, Cliff White wrote:
> > > > > > > Linux-MM fellows,
[snip]
> ork
> > > > for all tests - some of the kits still needs some patching..
> > > > cliffw
> > > 
> > > Any news on the automatic test series scripts Cliff ? 
> > > 
> > > Haven't seen any results yet.
> > > 
> > I ran a set for 2.6.10-rc3, PLM 3957, some results here:
> >  http://www.osdl.org/projects/26lnxstblztn/results/
> > Or by doing this:
> > http://www.osdl.org/lab_activities/kernel_testing/stp/display_test_requests
> ?d_patch_id%3Astring%3Aignore_empty=3957&op=Search
> > 
> > Marcelo, do you want me to submit the tests under your user id?
> > That would make searching for results eaiser. 
> 
> Cliff, 
> 
> How have you started these tests? I dont to run LTP for example.

We have a master script that checks kernel.org and bkbits.net for new
stuff ( run by cron ) If the master script sees new checkins that
match our regexp, it automagically kicks off the series of tests you 
requested. That covers the base,etc. 


> 
> I would like to be able to select two different patch ID's and run them from 
> the web
> interface, on a set of benchmarks vs memory size ranges vs nrCPUs (you alread
> y do 
> different number of CPUs on those series of tests I see), as we talked.
> Not just me, every developer doing performance testing :)

The current web requires you to set this up by hand,one test at a time.
We're working on some command line tools to replace the web, but
we're also thinking about another rev of the web interface, so the 
comments are helpful.
> 
> And then generate the graphs for the results of one patchID vs another. 
> 
> On reaim for example it would be nice to have graphs of global jobs per minut
> e vs 
> memory size, with a different colors for each patch ID. 
Okay.
> 
> Maybe we can even fit results for different nrCPUS on the same graph 
> with line types (with symbols like triangle, square, to differentiate).
> But then it might become too polluted to easily visualize, but maybe not.

I haven't had time to figure out fancy gnuplottage..that sounds neat.
> 
> Can you make the scripts which you are using for graphic generation and the
> gnuplot configuration files available? So I can play around with them.
> I want to help with that.
> 
> Another question: Is the source for reaim available? 
Of course, we're the Open Source Development Lab :)
Reaim:
bk bk://developer.osdl.org/reaim
or SF, tarballs
http://sourceforge.net/projects/re-aim-7/

any of the STP tests can be found as:
bk bk://developer.osdl.org/stp-test/<testname>
so 
bk://developer.osdl.org/stp-test/reaim

> 
> I see you're already generating the graphs for vmstat/iostat and user/system 
> time.
> 
> Thats really nice.
Those bits are in a Perl module that a few tests re-use.

> 
cliffw

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>