Re: [Ksummit-discuss] [CORE TOPIC] kernel unit testing

[Dan Williams <dan.j.williams@intel.com>]

On Sun, Jul 10, 2016 at 9:03 PM, Trond Myklebust
<trondmy@primarydata.com> wrote:
> So, we might as well make this a formal proposal.
>
> I’d like to propose that we have a discussion around how to make it easier to implement kernel unit tests. I’ve co-opted Dan as he has expressed both an interest and hands-on experience. :-)

Yes, I think there is some benefit to talk about unit test
implementation details.  Also, there are some wider questions, beyond
what I wrote up for LWN [1], that could be productive to discuss in
person:

* Are unit tests strictly out of tree (where tools/testing/ is "out-of-tree")?
There's maintenance overhead and code readability concerns if unit
test infrastructure is included in-line.  While it may enable testing
of deep internals, there's significant coverage that can be obtained
by mocking interfaces at the level of exported-symbols.

* What does "we never regress userspace" mean when we have unit tests
that are tightly coupled to the kernel?
For example, there have been occasions where I have "regressed" a test
case to improve kernel behavior with the knowledge that no real
(non-test) application was dependent on the old behavior.

* Are tests only for developers, or should an end consumer of the
kernel expect to be able to run the tests and baseline their kernel?
The unit tests for libnvdimm [2] are meant to run against latest
mainline. There's some support for checking the kernel version, but I
don't promise that tests for new kernel functionality will be skipped
on older kernels without that specific enabling. I'd revisit that
stance if -stable tree maintainers or distros were looking to run the
tests.

[1]: https://lwn.net/Articles/654071/
[2]: https://github.com/pmem/ndctl/tree/master/test