Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Daniel Vetter <daniel.vetter@ffwll.ch>]

On Thu, Jun 5, 2014 at 1:17 PM, Masami Hiramatsu
<masami.hiramatsu.pt@hitachi.com> wrote:
> (2014/06/05 17:53), Daniel Vetter wrote:
>> On Thu, Jun 5, 2014 at 10:30 AM, Geert Uytterhoeven
>> <geert@linux-m68k.org> wrote:
>>> On Thu, Jun 5, 2014 at 8:54 AM, Mel Gorman <mgorman@suse.de> wrote:
>>>> There is a hazard that someone bisecting the tree would need to be careful
>>>> to not bisect LTP instead.
>>>
>>> That may actually be a good reason not to import LTP...
>>> I'd imagine you usually want to bisect the kernel to find when a regression
>>> was introduced in the syscall API.
>>>
>>> Is there a reason not to run the latest version of LTP (unless bisecting
>>> LTP ;-)? The syscall API is supposed to be stable.
>>
>> Same for validating backports - you want the latest testsuite to make
>> sure you don't miss important fixes. Downside is that the testsuite
>> needs to be compatible over a much wider range of kernels to be
>> useful, which is a pain for e.g. checking that garbage in reserved
>> fields (like reserved flags) are properly rejected on each kernel
>> version.
>
> Perhaps, the testsuite can recognize which patch is merged or not
> if it can access the git repository by "git log | grep <commit-id>"
> or something like that. Then, it can self-configure to reject
> non-supported test. :)

Well we have feature flags and stuff (need them anyway) and magic
macros to skip tests if something isn't there/supported. So (at least
for us) not a technical issue at all. The problem is in getting these
tests right in all cases and maintaining them while updating tests for
new features.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch