[Ksummit-discuss] [CORE TOPIC] kernel testing standard

[Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>]

Hi,

As I discussed with Greg K.H. at LinuxCon Japan yesterday,
I'd like to propose kernel testing standard as a separated topic.

Issue:
There are many ways to test the kernel but it's neither well documented
nor standardized/organized.

As you may know, testing kernel is important on each phase of kernel
life-cycle. For example, even at the designing phase, actual test-case
shows us what the new feature/design does, how is will work, and how
to use it. This can improve the quality of the discussion.

Through the previous discussion I realized there are many different methods/
tools/functions for testing kernel, LTP, trinity, tools/testing/selftest,
in-kernel selftest etc. Each has good points and bad points.

So, I'd like to discuss how we can standardize them for each subsystem
at this kernel summit.

My suggestion are,
- Organizing existing in-tree kernel test frameworks (as "make test")
- Documenting the standard testing method, including how to run,
  how to add test-cases, and how to report.
- Commenting standard testing for each subsystem, maybe by adding
  UT: or TS: tags to MAINTAINERS, which describes the URL of
  out-of-tree tests or the directory of the selftest.

Note that I don't tend to change the ways to test for subsystems which
already have own tests, but organize it for who wants to get involved in
and/or to evaluate it. :-)

I think we can strongly request developers to add test-cases for new features
if we standardize the testing method.

Suggested participants: greg k.h., Li Zefan, test-tool maintainers and
                       subsystem maintainers.


Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com