Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Daniel Vetter <daniel.vetter@ffwll.ch>]

On Sat, May 24, 2014 at 2:30 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> On Fri, May 23, 2014 at 09:05:02AM -0500, Justin M. Forbes wrote:
>> All good suggestions. As nice as it would be if tests were in tree, this
>> might be unmanageable. But even out of tree tests could be automatically
>> brought in provided they are listed somewhere in tree.  Ideally you
>> would be able to "make tests" and get all in tree tests run, or "make
>> alltests" and have it grab/build/run out of tree tests with git urls as
>> well.
>
> Um.... how long do you expect "make alltests" to run?
>
> And how do you deal with tests that require specific hardware?
>
> For ext4, just doing a light smoke test takes about 30 minutes.  For
> me to run the full set of tests using multiple file system
> configurations, it takes about 12 to 16 hours.  And that's just for
> one file system.  (I do the tests using KVM, with a 90 megabyte
> compressed root file system, and 55 gigabytes worth of scratch
> partitions.)

Full drm/i915 regression testing takes about equally long, multiplied
by the need to run this on different physical hw platforms to cover
all relevant code paths. tbh I really don't see much point in having a
fully integrated testsuite for everything. At least for developers.

Otoh if distros/stable trees and other consumers of upstream want to
run this I think it would make sense to have something unified. I've
tried to haggle the drm/i915 testsuite to various people, but thus far
very little success. So I'm not sure whether it's just a lack of
awareness about the tests or whether it is a more fundamental lack of
interest (usually called "we don't have time").
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch