Re: [Ksummit-discuss] [CORE TOPIC] stable issues

[Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>]

(2014/05/07 21:45), Daniel Vetter wrote:
> On Wed, May 7, 2014 at 1:45 PM, Masami Hiramatsu
> <masami.hiramatsu.pt@hitachi.com> wrote:
>> (2014/05/07 17:39), Matt Fleming wrote:
>>> On Wed, 07 May, at 05:27:05PM, Masami Hiramatsu wrote:
>>>>
>>>> IOW, would the test cases be better to be out-of-tree or in-tree? If it is
>>>> out-of-tree(like LTP), how can we maintain both test-cases and upstream kernels?
>>>> What infrastructure should we have (e.g. bugzilla which provides a database for
>>>> relationship between bug# and test-case) ?
>>>> Those are my interests :)
>>>
>>> There's definitely huge merit in having in-tree tests like the current
>>> selftests stuff because it allows you to roll up fixes and regression
>>> tests into a single commit, see commit 123abd76edf5 ("efivars:
>>> efivarfs_valid_name() should handle pstore syntax").
>>>
>>
>> Ah, that's a good example for adding new feature/bugfix with test case! :)
>> I think this type of combined patch will be good to run tests with git-bisect.
>> At least out-of-tree test should work with git-bisect.
> 
> At least for drm/i915 I don't think merging the tests into the kernel
> would be beneficial, at least now:

Hm, it seems some other subsystems have their own testsuites, I think we'd
better clarify the testing policy for each subsystem,
using dedicated testing tools or in-kernel selftest.

> - Our tests are integrated into the regression test framework used by
> graphics people in general (piglit), and that most certainly won't
> move into the kernel.
> - We have lots of debug tools in the same repo (with shared code), and
> it tends to be less scary for bug reporters to grab
> intel-gpu-tools.git to run one of them instead of the entire kernel.
> - Documentation tooling in userspace sucks a lot less than kerneldoc.
> Which is important since we use testcases and tooling as getting
> started tasks for newcomers.
> - Also I want much stricter review requirements on kernel patches than
> testcase patches, separate git trees helps with that.
> 
> Hence why we thus far just link the kernel patch to its testcase with
> an Testcase: tag.

Ah, that's also nice to find an appropriate testcases. I think adding a
link(or git hash) to testcase allows us to automate test configuration
when git-bisecting, even if the test is out-of-tree. :)

Thank you,


-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com