Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[chrubis@suse.cz]

Hi!
> > Is there a reason not to run the latest version of LTP (unless bisecting
> > LTP ;-)? The syscall API is supposed to be stable.
> 
> I think not, and we have strong reasons for wanting to run the latest
> LTP against every kernel (including stable ones), not just the version
> in the test directory, so in practise, it looks like this doesn't meet
> the changes with the kernel test for inclusion.  On the other hand,
> having the tests available is also useful.   Perhaps we just need a
> tests repo which pulls from all our other disparate tests so there's one
> location everyone knows to go for the latest?

That sounds good to me. But as allready said, creating some
scripts/repos that pulls and runs all the tests is relatively easy.
Creating configurations and figuring out who needs to run which parts is
not.

I think that the main problem here is the communication and information
sharing. Maybe we can start with a wiki page or a similar document that
summarizes maintained testsuites, their purpose and structure. Because
just now, if there is any information about kernel testing, it is
scattered around the web, forums, etc.

Also I would like to see more communication between the Kernel and QA.

It's getting a bit better as we have linux-api mailing list and (thanks
to Michaal Kerrisk) commits that change kernel API are starting to CC
it. Which I consider as a great improvement because now we at least know
what we need to write tests for. However I still think that there is
some work lost in the process, particulary because the kernel devs who
wrote the userspace API have surely implemented some kind of tests for
it and these may have been adapted and included into LTP which would be
far less work than writing testcases from scratch.

-- 
Cyril Hrubis
chrubis@suse.cz