Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

["Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com>]

On 06/09/2014 04:44 PM, chrubis@suse.cz wrote:
> Hi!
>>> Is there a reason not to run the latest version of LTP (unless bisecting
>>> LTP ;-)? The syscall API is supposed to be stable.
>>
>> I think not, and we have strong reasons for wanting to run the latest
>> LTP against every kernel (including stable ones), not just the version
>> in the test directory, so in practise, it looks like this doesn't meet
>> the changes with the kernel test for inclusion.  On the other hand,
>> having the tests available is also useful.   Perhaps we just need a
>> tests repo which pulls from all our other disparate tests so there's one
>> location everyone knows to go for the latest?
> 
> That sounds good to me. But as allready said, creating some
> scripts/repos that pulls and runs all the tests is relatively easy.
> Creating configurations and figuring out who needs to run which parts is
> not.
> 
> I think that the main problem here is the communication and information
> sharing. Maybe we can start with a wiki page or a similar document that
> summarizes maintained testsuites, their purpose and structure. Because
> just now, if there is any information about kernel testing, it is
> scattered around the web, forums, etc.
> 
> Also I would like to see more communication between the Kernel and QA.
> 
> It's getting a bit better as we have linux-api mailing list and (thanks
> to Michaal Kerrisk) commits that change kernel API are starting to CC
> it. Which I consider as a great improvement because now we at least know
> what we need to write tests for. However I still think that there is
> some work lost in the process, particulary because the kernel devs who
> wrote the userspace API have surely implemented some kind of tests for
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Actually, I can point to numerous examples that show that this sadly
could not have been the case. Very many times I've written tests for
an API, or some API feature, only to discover that the most basic of
tests fails--in other words, clearly no-on--including the kernel
dev--did any testing of that particular feature before me.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/