Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

[Mark Brown <broonie@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]

On Thu, Jul 06, 2017 at 07:48:05AM -0700, James Bottomley wrote:
> On Thu, 2017-07-06 at 10:28 +0100, Mark Brown wrote:

> > I think before anything like that is viable we need to show a
> > concerted and visible interest in actually running the tests we
> > already have and paying attention to the results - if people can see
> > that they're just checking a checkbox that will often result in low
> > quality tests which can do more harm than good.

> it depends what you mean by "we".  I used to run a battery of tests
> over every SCSI commit.  It was time consuming and slowed down the

We as a community, I think something viable needs to be central services
like kernelci that's automated and allows multiple people to be involved
with the analysis.  Hand running tests at scale just doesn't.

> The corollary I take away from this is that the less intrusive the test
> infrastructure is (at least to my process) the happier I am.  The 0day
> quantum leap for me was going from testing my tree and telling me of
> problems after I've added the patch to testing patches posted to the
> mailing list, which tells me of problems *before* the commit gets added
> to the tree.

I think we'd get a long way just by looking at what's ending up in -next
- it's not as good as detecting things before they go in but it's
workable if people keep on top of things.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]