Re: [Ksummit-discuss] [CORE TOPIC] Testing

[Guenter Roeck <linux@roeck-us.net>]

On 07/07/2015 02:24 AM, Mark Brown wrote:
> One thing we typically cover at Kernel Summit is some of the activity
> that's going on around testing upstream.  I think it'd be useful to have
> some more of those discussions, both in terms of making people aware of
> what's available and in terms of helping the people doing testing figure
> out what would be useful.  A lot of this is probably well suited to a
> workshop session between the interested people but I do think some
> element in the core day beyond just a readout will be useful.
>
> The main things I'm aware of that are happening at the minute are
> kselftest development, the 0day tester, plus kernelci.org and the other
> build and boot/test bots that are running against various trees.
>
Maybe list all known ones as a start ?

> In terms of discussion topics some of the issues I'm seeing are:
>
>   - Can we pool resources to share the workload of running things and
>     interpreting results, ideally also providing some central way for
>     people to discover what results are out there for them to look at
>     for a given kernel in the different systems?
>
That might be quite useful. However, I have seen that it doesn't really
help to just provide the test results. kissb test results have been
available for ages, and people just don't look at it. Even the regular
"Build regression" e-mails sent out by Geert seem to be widely ignored.

What I really found to help is to bisect new problems and send an e-mail
to the responsible maintainer and to the submitter of the patch which
introduced it. I'd like to automate that with my test system, but
unfortunately I just don't have the time to do it.

>   - Should we start carrying config fragments upstream designed to
>     support testing, things like the distro config fragments that keep
>     getting discussed are one example here but there's other things like
>     collections of debug options we could be looking at.  Should we be
>     more generally slimming defconfigs and moving things into fragments?
>
> and there's always the the perennial ones about what people would like
> to see testing for.
>

Sharing as many test bot configuration scripts and relevant configurations
as possible would be quite helpful. For example, I am building various
configurations for all architectures, but I don't really know if they
are relevant. Also, I would like to run more qemu configurations,
but it is really hard to find working ones.

Guenter