Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Jason Cooper <jason@lakedaemon.net>]

On Sat, May 24, 2014 at 03:10:51AM +0900, Masami Hiramatsu wrote:
> (2014/05/24 1:24), Olof Johansson wrote:
> >> The boot farms that Kevin and Olof run currently tests booting to a
> >> command prompt.  We're catching a lot of regressions before they hit
> >> mainline, which is great.  But I'd like to see how we can extend that.
> >> And yes, I know those farms are saturated, and we need to bring
> >> something else on line to do more functional testing,  Perhaps break up
> >> the testing load:  boot-test linux-next, and runtime tests of the -rcX
> >> tags and stable tags.
> > 
> > I wouldn't call them saturated, but neither of us will be able to
> > scale to 10x the current size. 2-3x should be doable.
> 
> Right, the size of test should be considered. If the number of tests
> are too big and testing takes too long time, no one executes it.
> 
> >>> So, I'd like to discuss how we can standardize them for each subsystem
> >>> at this kernel summit.
> >>>
> >>> My suggestion are,
> >>> - Organizing existing in-tree kernel test frameworks (as "make test")
> > 
> > For my type of setup, I'd prefer a "make install_tests" target,
> > similar to modules/firmware that I can give a prefix to, and then
> > something in that directory to actually run them.
> 
> So it installs tests to /lib/testing/, similar to modules :) ?
> Yeah, that's also good, perhaps we can add "make testconfig" too.

Ideally, the kernel's .config could be used to determine which tests are
relevant.  I can also see the installed runtime tests being tied to the
kernel version they came from.

thx,

Jason.