Re: [Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>]

(2014/06/05 20:58), Daniel Vetter wrote:
> On Thu, Jun 5, 2014 at 1:17 PM, Masami Hiramatsu
> <masami.hiramatsu.pt@hitachi.com> wrote:
>> (2014/06/05 17:53), Daniel Vetter wrote:
>>> On Thu, Jun 5, 2014 at 10:30 AM, Geert Uytterhoeven
>>> <geert@linux-m68k.org> wrote:
>>>> On Thu, Jun 5, 2014 at 8:54 AM, Mel Gorman <mgorman@suse.de> wrote:
>>>>> There is a hazard that someone bisecting the tree would need to be careful
>>>>> to not bisect LTP instead.
>>>>
>>>> That may actually be a good reason not to import LTP...
>>>> I'd imagine you usually want to bisect the kernel to find when a regression
>>>> was introduced in the syscall API.
>>>>
>>>> Is there a reason not to run the latest version of LTP (unless bisecting
>>>> LTP ;-)? The syscall API is supposed to be stable.
>>>
>>> Same for validating backports - you want the latest testsuite to make
>>> sure you don't miss important fixes. Downside is that the testsuite
>>> needs to be compatible over a much wider range of kernels to be
>>> useful, which is a pain for e.g. checking that garbage in reserved
>>> fields (like reserved flags) are properly rejected on each kernel
>>> version.
>>
>> Perhaps, the testsuite can recognize which patch is merged or not
>> if it can access the git repository by "git log | grep <commit-id>"
>> or something like that. Then, it can self-configure to reject
>> non-supported test. :)
> 
> Well we have feature flags and stuff (need them anyway) and magic
> macros to skip tests if something isn't there/supported. So (at least
> for us) not a technical issue at all.

Nice :) , and I think we can use git commit-id as feature flags in other
subsystems. Since the patches in stable tree have [ Upstream commit <commit-id> ]
tag in the description, we can use them for checking which fixes are already
merged.

> The problem is in getting these
> tests right in all cases and maintaining them while updating tests for
> new features.

Is that a limitation comes from feature flags? or version synchronizing
problem?

Thank you,

-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com