Re: [Ksummit-discuss] [CORE TOPIC] Testing

[Mark Brown <broonie@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

On Sun, Jul 12, 2015 at 07:15:47PM +0800, Fengguang Wu wrote:

> If 0day failed to catch the error, it might be due to

> - regressions in 0day itself -- it's still in active development,
>   in the past year I did 1k patches to 0day build/boot scripts and
>   2k patches to the follow up LKP (Linux Kernel Performance) tests.

> - the "report once" logic, it's tricky and can possibly hide issues

> - failed bisect, rare but possible

I think some of the issues have been due to bisection getting confused
by issues appearing in merge commits but ICBW.

> - machine hang, network/disk fails, etc. maintenance incidents

> "Loading" may add latency, however it's not the cause to miss errors.

Latency seems like it might be an issue here, when I say I'm not seeing
things that's issues coming up in the -next build before they're
reported by 0day so they might well get fixed before 0day catches up.

> > > Are people ignoring them?

> > They're not reliably followed through on, no, and one of the things with
> > 0day is that it just generates a one time report so if things don't get
> > followed up on then that's that.  A regular "these are all the issues"
> > mail helps chase down those issues.

> 0day has such report type. It will be sent after each git push (unless
> you push too quickly) and it looks like this. Just drop me a note and
> list the git trees/branches you wish to receive such notice emails.

For me personally it'd be more interesting to be able to get them on
demand (eg, from a web page) than e-mailed, or e-mailed by a human
(possibly with fixes!).  The kernelci.org reporting does a lot of this
but doesn't cover anything except raw compiler warnings.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]