Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Guenter Roeck <linux@roeck-us.net>]

On 05/23/2014 09:24 AM, Olof Johansson wrote:
> On Fri, May 23, 2014 at 6:32 AM, Jason Cooper <jason@lakedaemon.net> wrote:
>> Masami,
>>
>> On Fri, May 23, 2014 at 08:47:29PM +0900, Masami Hiramatsu wrote:
>>> Issue:
>>> There are many ways to test the kernel but it's neither well documented
>>> nor standardized/organized.
>>>
>>> As you may know, testing kernel is important on each phase of kernel
>>> life-cycle. For example, even at the designing phase, actual test-case
>>> shows us what the new feature/design does, how is will work, and how
>>> to use it. This can improve the quality of the discussion.
>>>
>>> Through the previous discussion I realized there are many different methods/
>>> tools/functions for testing kernel, LTP, trinity, tools/testing/selftest,
>>> in-kernel selftest etc. Each has good points and bad points.
>>
>> * automated boot testing (embedded platforms)
>> * runtime testing
>>
>> A lot of development that we see is embedded platforms using
>> cross-compilers.  That makes a whole lot of tests impossible to run on
>> the host.  Especially when it deals with hardware interaction.  So
>> run-time testing definitely needs to be a part of the discussion.
>>
>> The boot farms that Kevin and Olof run currently tests booting to a
>> command prompt.  We're catching a lot of regressions before they hit
>> mainline, which is great.  But I'd like to see how we can extend that.
>> And yes, I know those farms are saturated, and we need to bring
>> something else on line to do more functional testing,  Perhaps break up
>> the testing load:  boot-test linux-next, and runtime tests of the -rcX
>> tags and stable tags.
>
> I wouldn't call them saturated, but neither of us will be able to
> scale to 10x the current size. 2-3x should be doable.
>

A lot can be done with qemu. My tests run to boot prompt for several architectures,
but are very basic (no networking, for example). I would love to add more tests,
but time is a problem.

Guenter