Re: [Ksummit-discuss] Maintainer's Summit 2017 Feedback Thread

[Theodore Ts'o <tytso@mit.edu>]

On Mon, Oct 30, 2017 at 05:42:33PM +0000, Bart Van Assche wrote:
> On Sun, 2017-10-29 at 06:08 -0400, Theodore Ts'o wrote:
> > Please reply to this thread if you have any comments about how we can
> > organize the Maintainer's Summit for next year.  Given that Linus
> > seemed fairly happy with how things went, it's likely we will stick
> > with the same format for next year, but if there are any details about
> > how we could do things better, I'd greatly appreciate them.
> 
> Something Linus was very clear about is that regressions are not acceptable.
> The best way I know to reduce the number of regressions is to increase the
> efforts on automatic testing. Should the number of tests that is run by the
> zero-day testing infrastructure be increased?

The 0-day system has sent a whole series of reports in response to
Linus's v4.14-rc6 announcement.  So it sounds like Fengguang is
already on it.  :-)

At least, for those things where we have tests and where the 0-day
hardware tickles the code paths of various arch-specific and drivers.

The good news is that we've picked a lot of the low-hanging fruit.
The bad news is that a lot of what is left is hardware-specific driver
issues, where fixing a bug for the current generation of devices might
break some device from an older generation.  I recall one change to
the Intel Wireless which only regressed when talking to an Aruba
Enterprise-grade Access Point, for which the Intel wireless driver
folks did not have in their hardware test library.

I would hope that most maintainers are doing testing on their own
subsystems.  I certainly do a large amount of exhaustive testing of
ext4 before I send a pull request (including most cases doing a trial
merge before launching a gce-xfstests run).  And I would assume (for
example), that Damien does a large amount of test of his ZBC support
code against SMR HDD's (bonus points if he tests SMR drives from both
WDC and Seagate :-) before he does he submits patches for review, and
as they get merged into Linus's tree.

More test in the zero-day testing infrastructure is good, but this
can't be a substitute for maintainers testing their own subsystems.
That's the only thing that can possibly scale.

	     	    	     	   	    - Ted