[Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Luis R. Rodriguez]

It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
so I'd like to nominate him. The mechanical process should probably include in
the future a scrape for top Reported-by contributors.

Fengguang's 0-day infrastructure is invaluable to day to day kernel
development, having him present would be great for any questions that may come
up. Getting a statistical overview / update of impact / any major architectural
changes of the 0-day infrastructure would also be very useful. If maintainers
are not yet using 0-day it would be great to hear why. If your contributors are
not using 0-day (I know some of you exist) I'd like to know why you don't use it,
I often run into issues on linux-next which at times I have to fix, if 0-day
would have been used a folowup fix would not have been needed.

  Luis

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Alexandre Belloni]

On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> 
> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
> so I'd like to nominate him. The mechanical process should probably include in
> the future a scrape for top Reported-by contributors.
> 

That's a good point.

> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> development, having him present would be great for any questions that may come
> up. Getting a statistical overview / update of impact / any major architectural
> changes of the 0-day infrastructure would also be very useful. If maintainers
> are not yet using 0-day it would be great to hear why. If your contributors are
> not using 0-day (I know some of you exist) I'd like to know why you don't use it,
> I often run into issues on linux-next which at times I have to fix, if 0-day
> would have been used a folowup fix would not have been needed.
> 

Well, I think it would also help to know how the patch/git tree
selection is done. Lately, I've been receiving less reports from 0-day
and some issues were found by Arnd's autobuilders after hitting
linux-next. I used to get report of compilation breakage 10-15 minutes
after the patch submission.


-- 
Alexandre Belloni, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Vinod Koul]

On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> > 
> > It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
> > so I'd like to nominate him. The mechanical process should probably include in
> > the future a scrape for top Reported-by contributors.
> > 
> 
> That's a good point.
> 
> > Fengguang's 0-day infrastructure is invaluable to day to day kernel
> > development, having him present would be great for any questions that may come
> > up. Getting a statistical overview / update of impact / any major architectural
> > changes of the 0-day infrastructure would also be very useful. If maintainers
> > are not yet using 0-day it would be great to hear why. If your contributors are
> > not using 0-day (I know some of you exist) I'd like to know why you don't use it,
> > I often run into issues on linux-next which at times I have to fix, if 0-day
> > would have been used a folowup fix would not have been needed.

I am using it a lot :)

> Well, I think it would also help to know how the patch/git tree
> selection is done. Lately, I've been receiving less reports from 0-day
> and some issues were found by Arnd's autobuilders after hitting
> linux-next. I used to get report of compilation breakage 10-15 minutes
> after the patch submission.

Pls check if the mailing list is still subscribed. I had the same issue
and found that his ID was kicked off from the list, which is quite
typical w/ Intel accounts & vger.

Since work is done by scripts, people may not have noticed.

-- 
~Vinod

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Laurent Pinchart]

Hello,

On Tuesday 26 Jul 2016 08:40:07 Vinod Koul wrote:
> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> > On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> >> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated
> >> yet, so I'd like to nominate him. The mechanical process should
> >> probably include in the future a scrape for top Reported-by
> >> contributors.
> > 
> > That's a good point.
> > 
> >> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> >> development, having him present would be great for any questions that
> >> may come up. Getting a statistical overview / update of impact / any
> >> major architectural changes of the 0-day infrastructure would also be
> >> very useful. If maintainers are not yet using 0-day it would be great
> >> to hear why. If your contributors are not using 0-day (I know some of
> >> you exist) I'd like to know why you don't use it, I often run into
> >> issues on linux-next which at times I have to fix, if 0-day would have
> >> been used a folowup fix would not have been needed.
> 
> I am using it a lot :)
> 
> > Well, I think it would also help to know how the patch/git tree
> > selection is done. Lately, I've been receiving less reports from 0-day
> > and some issues were found by Arnd's autobuilders after hitting
> > linux-next. I used to get report of compilation breakage 10-15 minutes
> > after the patch submission.

I second that. 0-day is an amazing tool, but relying on it requires developers 
to trust that the tool will perform its job as expected. A proper 
understanding of the logic behind patch/git tree selection would be very 
valuable, as well as how the build bot handles its 500+ kernel configurations.

Even though 0-day is an external tool, maybe it deserves an entry in 
Documentation/ ? A public web page hosting the list of all git registered git 
trees could be useful too.

> Pls check if the mailing list is still subscribed. I had the same issue
> and found that his ID was kicked off from the list, which is quite
> typical w/ Intel accounts & vger.
> 
> Since work is done by scripts, people may not have noticed.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Vinod Koul]

On Tue, Jul 26, 2016 at 11:16:40AM +0300, Laurent Pinchart wrote:
> Hello,
> 
> On Tuesday 26 Jul 2016 08:40:07 Vinod Koul wrote:
> > On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> > > On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> > >> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated
> > >> yet, so I'd like to nominate him. The mechanical process should
> > >> probably include in the future a scrape for top Reported-by
> > >> contributors.
> > > 
> > > That's a good point.
> > > 
> > >> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> > >> development, having him present would be great for any questions that
> > >> may come up. Getting a statistical overview / update of impact / any
> > >> major architectural changes of the 0-day infrastructure would also be
> > >> very useful. If maintainers are not yet using 0-day it would be great
> > >> to hear why. If your contributors are not using 0-day (I know some of
> > >> you exist) I'd like to know why you don't use it, I often run into
> > >> issues on linux-next which at times I have to fix, if 0-day would have
> > >> been used a folowup fix would not have been needed.
> > 
> > I am using it a lot :)
> > 
> > > Well, I think it would also help to know how the patch/git tree
> > > selection is done. Lately, I've been receiving less reports from 0-day
> > > and some issues were found by Arnd's autobuilders after hitting
> > > linux-next. I used to get report of compilation breakage 10-15 minutes
> > > after the patch submission.
> 
> I second that. 0-day is an amazing tool, but relying on it requires developers 
> to trust that the tool will perform its job as expected. A proper 
> understanding of the logic behind patch/git tree selection would be very 
> valuable, as well as how the build bot handles its 500+ kernel configurations.

Agreed, although in his report he does mention the tree used, but would be
great if the logic is Documented somewhere.

There seems some basic Documentation up at [1] and [2]

Also the reports can be put on webpage. Last test results of maintainer trees
and results of patches run on mailing list. I think that data is available
but not present...

> Even though 0-day is an external tool, maybe it deserves an entry in 
> Documentation/ ? A public web page hosting the list of all git registered git 
> trees could be useful too.
> 
> > Pls check if the mailing list is still subscribed. I had the same issue
> > and found that his ID was kicked off from the list, which is quite
> > typical w/ Intel accounts & vger.
> > 
> > Since work is done by scripts, people may not have noticed.

[1]: https://01.org/lkp/documentation/0-day-test-service
[2]: https://01.org/lkp/

-- 
~Vinod

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Tue, Jul 26, 2016 at 02:26:46PM +0530, Vinod Koul wrote:
>On Tue, Jul 26, 2016 at 11:16:40AM +0300, Laurent Pinchart wrote:
>> Hello,
>>
>> On Tuesday 26 Jul 2016 08:40:07 Vinod Koul wrote:
>> > On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
>> > > On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
>> > >> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated
>> > >> yet, so I'd like to nominate him. The mechanical process should
>> > >> probably include in the future a scrape for top Reported-by
>> > >> contributors.
>> > >
>> > > That's a good point.
>> > >
>> > >> Fengguang's 0-day infrastructure is invaluable to day to day kernel
>> > >> development, having him present would be great for any questions that
>> > >> may come up. Getting a statistical overview / update of impact / any
>> > >> major architectural changes of the 0-day infrastructure would also be
>> > >> very useful. If maintainers are not yet using 0-day it would be great
>> > >> to hear why. If your contributors are not using 0-day (I know some of
>> > >> you exist) I'd like to know why you don't use it, I often run into
>> > >> issues on linux-next which at times I have to fix, if 0-day would have
>> > >> been used a folowup fix would not have been needed.
>> >
>> > I am using it a lot :)
>> >
>> > > Well, I think it would also help to know how the patch/git tree
>> > > selection is done. Lately, I've been receiving less reports from 0-day
>> > > and some issues were found by Arnd's autobuilders after hitting
>> > > linux-next. I used to get report of compilation breakage 10-15 minutes
>> > > after the patch submission.
>>
>> I second that. 0-day is an amazing tool, but relying on it requires developers
>> to trust that the tool will perform its job as expected. A proper
>> understanding of the logic behind patch/git tree selection would be very
>> valuable, as well as how the build bot handles its 500+ kernel configurations.
>
>Agreed, although in his report he does mention the tree used, but would be
>great if the logic is Documented somewhere.

Yeah, sorry the Documentation could be improved.

The 600+ git trees 0-day monitors are tracked here:

https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/tree/repo/linux

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
>On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
>>
>> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
>> so I'd like to nominate him. The mechanical process should probably include in
>> the future a scrape for top Reported-by contributors.
>>
>
>That's a good point.
>
>> Fengguang's 0-day infrastructure is invaluable to day to day kernel
>> development, having him present would be great for any questions that may come
>> up. Getting a statistical overview / update of impact / any major architectural
>> changes of the 0-day infrastructure would also be very useful. If maintainers
>> are not yet using 0-day it would be great to hear why. If your contributors are
>> not using 0-day (I know some of you exist) I'd like to know why you don't use it,
>> I often run into issues on linux-next which at times I have to fix, if 0-day
>> would have been used a folowup fix would not have been needed.
>>
>
>Well, I think it would also help to know how the patch/git tree
>selection is done. Lately, I've been receiving less reports from 0-day
>and some issues were found by Arnd's autobuilders after hitting
>linux-next. I used to get report of compilation breakage 10-15 minutes
>after the patch submission.

Yeah sorry about that! There are many things that can impact 0-day
system's stability: regressions, hard disk replacement, proxy issues,
mailing list kicking off, vocations, etc. Which should improve over
time as we add monitoring and self-test facilities. On the other hand,
if you find such issues, please don't hesitate forwarding the error
emails to us, so that we can check and take action quickly.

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Laurent Pinchart]

Hi Fengguang,

On Wednesday 27 Jul 2016 22:50:27 Fengguang Wu wrote:
> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> >On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> >> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated
> >> yet, so I'd like to nominate him. The mechanical process should probably
> >> include in the future a scrape for top Reported-by contributors.
> >
> > That's a good point.
> >
> >> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> >> development, having him present would be great for any questions that may
> >> come up. Getting a statistical overview / update of impact / any major
> >> architectural changes of the 0-day infrastructure would also be very
> >> useful. If maintainers are not yet using 0-day it would be great to hear
> >> why. If your contributors are not using 0-day (I know some of you exist)
> >> I'd like to know why you don't use it, I often run into issues on
> >> linux-next which at times I have to fix, if 0-day would have been used a
> >> folowup fix would not have been needed.
> >
> > Well, I think it would also help to know how the patch/git tree
> > selection is done. Lately, I've been receiving less reports from 0-day
> > and some issues were found by Arnd's autobuilders after hitting
> > linux-next. I used to get report of compilation breakage 10-15 minutes
> > after the patch submission.
> 
> Yeah sorry about that! There are many things that can impact 0-day
> system's stability: regressions, hard disk replacement, proxy issues,
> mailing list kicking off, vocations, etc. Which should improve over
> time as we add monitoring and self-test facilities. On the other hand,
> if you find such issues, please don't hesitate forwarding the error
> emails to us, so that we can check and take action quickly.

There's no need to apologize, really. You've done and keep doing an amazing 
work for which we're all thankful. The biggest problem I see that would stop 
me from fully relying on 0-day is the lack of information about such 
downtimes, leading me to wonder if the absence of a report really means that 
everything is fine. A 0-day status page with information about the current 
status and the planned downtime, as well as possibly the list of trees covered 
by 0-day (I don't know whether part of that information could be considered as 
confidential), would help a lot.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

Hi Laurent,

On Thu, Jul 28, 2016 at 07:15:22PM +0300, Laurent Pinchart wrote:
>Hi Fengguang,
>
>On Wednesday 27 Jul 2016 22:50:27 Fengguang Wu wrote:
>> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
>> >On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
>> >> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated
>> >> yet, so I'd like to nominate him. The mechanical process should probably
>> >> include in the future a scrape for top Reported-by contributors.
>> >
>> > That's a good point.
>> >
>> >> Fengguang's 0-day infrastructure is invaluable to day to day kernel
>> >> development, having him present would be great for any questions that may
>> >> come up. Getting a statistical overview / update of impact / any major
>> >> architectural changes of the 0-day infrastructure would also be very
>> >> useful. If maintainers are not yet using 0-day it would be great to hear
>> >> why. If your contributors are not using 0-day (I know some of you exist)
>> >> I'd like to know why you don't use it, I often run into issues on
>> >> linux-next which at times I have to fix, if 0-day would have been used a
>> >> folowup fix would not have been needed.
>> >
>> > Well, I think it would also help to know how the patch/git tree
>> > selection is done. Lately, I've been receiving less reports from 0-day
>> > and some issues were found by Arnd's autobuilders after hitting
>> > linux-next. I used to get report of compilation breakage 10-15 minutes
>> > after the patch submission.
>>
>> Yeah sorry about that! There are many things that can impact 0-day
>> system's stability: regressions, hard disk replacement, proxy issues,
>> mailing list kicking off, vocations, etc. Which should improve over
>> time as we add monitoring and self-test facilities. On the other hand,
>> if you find such issues, please don't hesitate forwarding the error
>> emails to us, so that we can check and take action quickly.
>
>There's no need to apologize, really. You've done and keep doing an amazing
>work for which we're all thankful. The biggest problem I see that would stop
>me from fully relying on 0-day is the lack of information about such
>downtimes, leading me to wonder if the absence of a report really means that
>everything is fine. A 0-day status page with information about the current
>status and the planned downtime, as well as possibly the list of trees covered
>by 0-day (I don't know whether part of that information could be considered as
>confidential), would help a lot.

For that purpose you may subscribe to the BUILD SUCCESS notification
by sending me an "opt-in" email with your tree URL.

That notification will be sent 1-2 hours after each branch's git push,
which contains the build test progress on that branch. Based on which
(or missing of it) you'll know if everything is on the track.

This will actually turn that feature on for your tree:

https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/commit/?id=aa0480dd8d58d234fdf0ff41bcb71a930aa024b9

commit aa0480dd8d58d234fdf0ff41bcb71a930aa024b9
Author:     Fengguang Wu <fengguang.wu@intel.com>
Commit:     Fengguang Wu <fengguang.wu@intel.com>
CommitDate: Fri Jul 29 04:48:44 2016 +0800

    repo: notify build success for pinchartl-fbdev

    CC: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
    Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
---
 repo/linux/pinchartl-fbdev | 1 +
 1 file changed, 1 insertion(+)

diff --git a/repo/linux/pinchartl-fbdev b/repo/linux/pinchartl-fbdev
index bbb4dd1..cc6e19b 100644
--- a/repo/linux/pinchartl-fbdev
+++ b/repo/linux/pinchartl-fbdev
@@ -1,5 +1,6 @@
 url: git://linuxtv.org/pinchartl/fbdev.git
 owner: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
+notify_build_success_branch: .*
 subsystems:
 - drm
 - rcar-du

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Laurent Pinchart]

Hi Fengguang,

On Friday 29 Jul 2016 04:53:24 Fengguang Wu wrote:
> On Thu, Jul 28, 2016 at 07:15:22PM +0300, Laurent Pinchart wrote:
> > On Wednesday 27 Jul 2016 22:50:27 Fengguang Wu wrote:
> >> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> >>> On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> >>>> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been
> >>>> nominated yet, so I'd like to nominate him. The mechanical process
> >>>> should probably include in the future a scrape for top Reported-by
> >>>> contributors.
> >>> 
> >>> That's a good point.
> >>> 
> >>>> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> >>>> development, having him present would be great for any questions that
> >>>> may come up. Getting a statistical overview / update of impact / any
> >>>> major architectural changes of the 0-day infrastructure would also be
> >>>> very useful. If maintainers are not yet using 0-day it would be great
> >>>> to hear why. If your contributors are not using 0-day (I know some of
> >>>> you exist) I'd like to know why you don't use it, I often run into
> >>>> issues on linux-next which at times I have to fix, if 0-day would have
> >>>> been used a folowup fix would not have been needed.
> >>> 
> >>> Well, I think it would also help to know how the patch/git tree
> >>> selection is done. Lately, I've been receiving less reports from 0-day
> >>> and some issues were found by Arnd's autobuilders after hitting
> >>> linux-next. I used to get report of compilation breakage 10-15 minutes
> >>> after the patch submission.
> >> 
> >> Yeah sorry about that! There are many things that can impact 0-day
> >> system's stability: regressions, hard disk replacement, proxy issues,
> >> mailing list kicking off, vocations, etc. Which should improve over
> >> time as we add monitoring and self-test facilities. On the other hand,
> >> if you find such issues, please don't hesitate forwarding the error
> >> emails to us, so that we can check and take action quickly.
> >
> > There's no need to apologize, really. You've done and keep doing an
> > amazing work for which we're all thankful. The biggest problem I see that
> > would stop me from fully relying on 0-day is the lack of information about
> > such downtimes, leading me to wonder if the absence of a report really
> > means that everything is fine. A 0-day status page with information about
> > the current status and the planned downtime, as well as possibly the list
> > of trees covered by 0-day (I don't know whether part of that information
> > could be considered as confidential), would help a lot.
> 
> For that purpose you may subscribe to the BUILD SUCCESS notification
> by sending me an "opt-in" email with your tree URL.
> 
> That notification will be sent 1-2 hours after each branch's git push,
> which contains the build test progress on that branch. Based on which
> (or missing of it) you'll know if everything is on the track.

I know that such a feature exists, but I'm not sure all developers would 
appreciate the additional e-mail traffic. That's why I proposed a status page 
that could be polled when needed.

> This will actually turn that feature on for your tree:
> 
> https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/commit/?id=aa
> 0480dd8d58d234fdf0ff41bcb71a930aa024b9
> 
> commit aa0480dd8d58d234fdf0ff41bcb71a930aa024b9
> Author:     Fengguang Wu <fengguang.wu@intel.com>
> Commit:     Fengguang Wu <fengguang.wu@intel.com>
> CommitDate: Fri Jul 29 04:48:44 2016 +0800
> 
>     repo: notify build success for pinchartl-fbdev
> 
>     CC: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>     Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
> ---
>  repo/linux/pinchartl-fbdev | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/repo/linux/pinchartl-fbdev b/repo/linux/pinchartl-fbdev
> index bbb4dd1..cc6e19b 100644
> --- a/repo/linux/pinchartl-fbdev
> +++ b/repo/linux/pinchartl-fbdev
> @@ -1,5 +1,6 @@
>  url: git://linuxtv.org/pinchartl/fbdev.git
>  owner: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> +notify_build_success_branch: .*
>  subsystems:
>  - drm
>  - rcar-du

Thanks. I will actually deprecate that tree in favour of 
git://linuxtv.org/pinchartl/media.git :-) I'll send you a mail when that will 
be effective.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Dmitry Torokhov]

On Thu, Jul 28, 2016 at 11:59:13PM +0300, Laurent Pinchart wrote:
> Hi Fengguang,
> 
> On Friday 29 Jul 2016 04:53:24 Fengguang Wu wrote:
> > On Thu, Jul 28, 2016 at 07:15:22PM +0300, Laurent Pinchart wrote:
> > > On Wednesday 27 Jul 2016 22:50:27 Fengguang Wu wrote:
> > >> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
> > >>> On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
> > >>>> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been
> > >>>> nominated yet, so I'd like to nominate him. The mechanical process
> > >>>> should probably include in the future a scrape for top Reported-by
> > >>>> contributors.
> > >>> 
> > >>> That's a good point.
> > >>> 
> > >>>> Fengguang's 0-day infrastructure is invaluable to day to day kernel
> > >>>> development, having him present would be great for any questions that
> > >>>> may come up. Getting a statistical overview / update of impact / any
> > >>>> major architectural changes of the 0-day infrastructure would also be
> > >>>> very useful. If maintainers are not yet using 0-day it would be great
> > >>>> to hear why. If your contributors are not using 0-day (I know some of
> > >>>> you exist) I'd like to know why you don't use it, I often run into
> > >>>> issues on linux-next which at times I have to fix, if 0-day would have
> > >>>> been used a folowup fix would not have been needed.
> > >>> 
> > >>> Well, I think it would also help to know how the patch/git tree
> > >>> selection is done. Lately, I've been receiving less reports from 0-day
> > >>> and some issues were found by Arnd's autobuilders after hitting
> > >>> linux-next. I used to get report of compilation breakage 10-15 minutes
> > >>> after the patch submission.
> > >> 
> > >> Yeah sorry about that! There are many things that can impact 0-day
> > >> system's stability: regressions, hard disk replacement, proxy issues,
> > >> mailing list kicking off, vocations, etc. Which should improve over
> > >> time as we add monitoring and self-test facilities. On the other hand,
> > >> if you find such issues, please don't hesitate forwarding the error
> > >> emails to us, so that we can check and take action quickly.
> > >
> > > There's no need to apologize, really. You've done and keep doing an
> > > amazing work for which we're all thankful. The biggest problem I see that
> > > would stop me from fully relying on 0-day is the lack of information about
> > > such downtimes, leading me to wonder if the absence of a report really
> > > means that everything is fine. A 0-day status page with information about
> > > the current status and the planned downtime, as well as possibly the list
> > > of trees covered by 0-day (I don't know whether part of that information
> > > could be considered as confidential), would help a lot.
> > 
> > For that purpose you may subscribe to the BUILD SUCCESS notification
> > by sending me an "opt-in" email with your tree URL.
> > 
> > That notification will be sent 1-2 hours after each branch's git push,
> > which contains the build test progress on that branch. Based on which
> > (or missing of it) you'll know if everything is on the track.
> 
> I know that such a feature exists, but I'm not sure all developers would 
> appreciate the additional e-mail traffic. That's why I proposed a status page 
> that could be polled when needed.

I for one really appreciate that I do not need to poll anything and
instead I am notified when stuff that I pushed passed/failed some tests.
So, even if there are status pages and what's not, please do keep opt-in
email notifications.

Thanks Fengguang!

-- 
Dmitry

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Vinod Koul]

On Thu, Jul 28, 2016 at 03:38:45PM -0700, Dmitry Torokhov wrote:
> > I know that such a feature exists, but I'm not sure all developers would 
> > appreciate the additional e-mail traffic. That's why I proposed a status page 
> > that could be polled when needed.
> 
> I for one really appreciate that I do not need to poll anything and
> instead I am notified when stuff that I pushed passed/failed some tests.
> So, even if there are status pages and what's not, please do keep opt-in
> email notifications.

Same here, I do look for mail from Fengguang after I have pushed stuff up.

Said that, if we have a lookup interface, I won't mind at all, given that
sometimes latencies vary, so knowing the latest status, branch and commit
built would help.

Also on mailing list tests, I would really like a status page. Right now it
only reports errors, which is great. But I won't know if tests were run or
not. It happened on one of our lists recently

> Thanks Fengguang!

Absolutely, this bot has been a great service to kernel community.

Thanks
-- 
~Vinod

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Thu, Jul 28, 2016 at 11:59:13PM +0300, Laurent Pinchart wrote:
>Hi Fengguang,
>
>On Friday 29 Jul 2016 04:53:24 Fengguang Wu wrote:
>> On Thu, Jul 28, 2016 at 07:15:22PM +0300, Laurent Pinchart wrote:
>> > On Wednesday 27 Jul 2016 22:50:27 Fengguang Wu wrote:
>> >> On Mon, Jul 25, 2016 at 10:23:43PM +0200, Alexandre Belloni wrote:
>> >>> On 25/07/2016 at 21:01:25 +0200, Luis R. Rodriguez wrote :
>> >>>> It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been
>> >>>> nominated yet, so I'd like to nominate him. The mechanical process
>> >>>> should probably include in the future a scrape for top Reported-by
>> >>>> contributors.
>> >>>
>> >>> That's a good point.
>> >>>
>> >>>> Fengguang's 0-day infrastructure is invaluable to day to day kernel
>> >>>> development, having him present would be great for any questions that
>> >>>> may come up. Getting a statistical overview / update of impact / any
>> >>>> major architectural changes of the 0-day infrastructure would also be
>> >>>> very useful. If maintainers are not yet using 0-day it would be great
>> >>>> to hear why. If your contributors are not using 0-day (I know some of
>> >>>> you exist) I'd like to know why you don't use it, I often run into
>> >>>> issues on linux-next which at times I have to fix, if 0-day would have
>> >>>> been used a folowup fix would not have been needed.
>> >>>
>> >>> Well, I think it would also help to know how the patch/git tree
>> >>> selection is done. Lately, I've been receiving less reports from 0-day
>> >>> and some issues were found by Arnd's autobuilders after hitting
>> >>> linux-next. I used to get report of compilation breakage 10-15 minutes
>> >>> after the patch submission.
>> >>
>> >> Yeah sorry about that! There are many things that can impact 0-day
>> >> system's stability: regressions, hard disk replacement, proxy issues,
>> >> mailing list kicking off, vocations, etc. Which should improve over
>> >> time as we add monitoring and self-test facilities. On the other hand,
>> >> if you find such issues, please don't hesitate forwarding the error
>> >> emails to us, so that we can check and take action quickly.
>> >
>> > There's no need to apologize, really. You've done and keep doing an
>> > amazing work for which we're all thankful. The biggest problem I see that
>> > would stop me from fully relying on 0-day is the lack of information about
>> > such downtimes, leading me to wonder if the absence of a report really
>> > means that everything is fine. A 0-day status page with information about
>> > the current status and the planned downtime, as well as possibly the list
>> > of trees covered by 0-day (I don't know whether part of that information
>> > could be considered as confidential), would help a lot.
>>
>> For that purpose you may subscribe to the BUILD SUCCESS notification
>> by sending me an "opt-in" email with your tree URL.
>>
>> That notification will be sent 1-2 hours after each branch's git push,
>> which contains the build test progress on that branch. Based on which
>> (or missing of it) you'll know if everything is on the track.
>
>I know that such a feature exists, but I'm not sure all developers would
>appreciate the additional e-mail traffic. That's why I proposed a status page
>that could be polled when needed.

Such a status web page would be similar to the email notification
with no much extra contents, except that it's always there for your
polling. If that'd be valuable to you and some few other developers,
I'd be glad to push forward the Web UI feature.

I need your understand that there are severe limitations what useful
contents we can provide due to the nature of merge-test-bisect. For
example, there will be no build logs specific for your branches/commits
because the extensive build tests are possible only because your code
are tested together with a large number of random others. For the same
reason there will be no error/warnings available for browsing before
the noisy error/warnings are bisected down. If we expose the unclassified
noisy error messages and encourage developers to browse through them
and do a guess work "are they likely caused by my commits?", we may be
wasting the highly valued developer time.

btw, maybe some maintainers are already informed: 0-day statistics
show that ~60% errors can be reported in 2 hours, ~90% errors are
reported in 24 hours and there are 1% errors reported after 1 week.

As we extend test coverage aggressively, there are no longer clear
moment that we can claim all tests for a branch are complete. The
actual build/boot tests may well go on for over a week, covering
thousands of kconfigs. The more time consuming functional/performance
tests may go on for a month. That's made possible and cost effective
thanks to the merge-test-bisect methodology.

So the safe bet is to wait for 1 day's tests before sending out patches.
The BUILD SUCCESS notification serves as an indication 0-day robot is
not sleeping through. My personal style suggestion would be to stay
relaxed, take time and avoid polling. :)

>> This will actually turn that feature on for your tree:
>>
>> https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/commit/?id=aa
>> 0480dd8d58d234fdf0ff41bcb71a930aa024b9
>>
>> commit aa0480dd8d58d234fdf0ff41bcb71a930aa024b9
>> Author:     Fengguang Wu <fengguang.wu@intel.com>
>> Commit:     Fengguang Wu <fengguang.wu@intel.com>
>> CommitDate: Fri Jul 29 04:48:44 2016 +0800
>>
>>     repo: notify build success for pinchartl-fbdev
>>
>>     CC: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>     Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
>> ---
>>  repo/linux/pinchartl-fbdev | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/repo/linux/pinchartl-fbdev b/repo/linux/pinchartl-fbdev
>> index bbb4dd1..cc6e19b 100644
>> --- a/repo/linux/pinchartl-fbdev
>> +++ b/repo/linux/pinchartl-fbdev
>> @@ -1,5 +1,6 @@
>>  url: git://linuxtv.org/pinchartl/fbdev.git
>>  owner: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>> +notify_build_success_branch: .*
>>  subsystems:
>>  - drm
>>  - rcar-du
>
>Thanks. I will actually deprecate that tree in favour of
>git://linuxtv.org/pinchartl/media.git :-) I'll send you a mail when that will
>be effective.

OK!

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Luis R. Rodriguez]

On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
> btw, maybe some maintainers are already informed: 0-day statistics
> show that ~60% errors can be reported in 2 hours, ~90% errors are
> reported in 24 hours and there are 1% errors reported after 1 week.

If one were to take 0-day code an slap it on some internal big-iron
server, and prioritize work for a few developers (say SUSE would do
this for its developers) do you expect the turnaround time for
reports would be faster if we had bigger-meaner hardware ? These
days actually would like to get results back in a few minutes
for 90% of errors so wondering how / if others have taken on
0-day internally and made it faster by beefing up the hardware.

  Luis

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Fri, Jul 29, 2016 at 01:33:27AM +0200, Luis R. Rodriguez wrote:
>On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
>> btw, maybe some maintainers are already informed: 0-day statistics
>> show that ~60% errors can be reported in 2 hours, ~90% errors are
>> reported in 24 hours and there are 1% errors reported after 1 week.
>
>If one were to take 0-day code an slap it on some internal big-iron
>server, and prioritize work for a few developers (say SUSE would do
>this for its developers) do you expect the turnaround time for
>reports would be faster if we had bigger-meaner hardware ? These
>days actually would like to get results back in a few minutes
>for 90% of errors so wondering how / if others have taken on
>0-day internally and made it faster by beefing up the hardware.

I'd suspect roughly the same timing given powerful servers but still
with reasonable cost considerations.

Intel pretty values the 0-day service and backs it up with 12 parallel
build servers, including 4 4S Xeon machines. Since we do merged tests,
one may assume most of the servers are working parallel for his code
whenever he does a git push. Kernel hackers can feel free to push
frequently because the extra pushes are virtually cost free -- the
build workers are working cyclicly on latest merged code anyway.

To take free ride of that restless horse, it'd be good to push small
topic branches on latest RC kernels, which will have better chances
to merge and play well with others code.

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Bjorn Helgaas]

On Thu, Jul 28, 2016 at 7:09 PM, Fengguang Wu <fengguang.wu@intel.com> wrote:
> On Fri, Jul 29, 2016 at 01:33:27AM +0200, Luis R. Rodriguez wrote:
>>
>> On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
>>>
>>> btw, maybe some maintainers are already informed: 0-day statistics
>>> show that ~60% errors can be reported in 2 hours, ~90% errors are
>>> reported in 24 hours and there are 1% errors reported after 1 week.
>>
>>
>> If one were to take 0-day code an slap it on some internal big-iron
>> server, and prioritize work for a few developers (say SUSE would do
>> this for its developers) do you expect the turnaround time for
>> reports would be faster if we had bigger-meaner hardware ? These
>> days actually would like to get results back in a few minutes
>> for 90% of errors so wondering how / if others have taken on
>> 0-day internally and made it faster by beefing up the hardware.
>
>
> I'd suspect roughly the same timing given powerful servers but still
> with reasonable cost considerations.
>
> Intel pretty values the 0-day service and backs it up with 12 parallel
> build servers, including 4 4S Xeon machines. Since we do merged tests,
> one may assume most of the servers are working parallel for his code
> whenever he does a git push. Kernel hackers can feel free to push
> frequently because the extra pushes are virtually cost free -- the
> build workers are working cyclicly on latest merged code anyway.
>
> To take free ride of that restless horse, it'd be good to push small
> topic branches on latest RC kernels, which will have better chances
> to merge and play well with others code.

I'm interested in learning more about how this works.  I typically
base all my topic branches on -rc2 or so, no matter where we are in
the cycle, because it makes it easier to rebuild my "next" branch if I
discover a problem and want to amend something.  But maybe this
consumes more 0-day cycles than necessary.

I second the desire for a web page of status.  Sometimes things seem
to get lost or delayed and I hate to bug Fengguang unnecessarily.

I typically push a topic branch and wait for BUILD_SUCCESS before
merging into my "next" branch.  I do this manually by watching for the
email, but maybe it could be scripted if there were a way to query
"build status for SHA-1".

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Luis R. Rodriguez]

On Fri, Jul 29, 2016 at 08:09:12AM +0800, Fengguang Wu wrote:
> On Fri, Jul 29, 2016 at 01:33:27AM +0200, Luis R. Rodriguez wrote:
> >On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
> >>btw, maybe some maintainers are already informed: 0-day statistics
> >>show that ~60% errors can be reported in 2 hours, ~90% errors are
> >>reported in 24 hours and there are 1% errors reported after 1 week.
> >
> >If one were to take 0-day code an slap it on some internal big-iron
> >server, and prioritize work for a few developers (say SUSE would do
> >this for its developers) do you expect the turnaround time for
> >reports would be faster if we had bigger-meaner hardware ? These
> >days actually would like to get results back in a few minutes
> >for 90% of errors so wondering how / if others have taken on
> >0-day internally and made it faster by beefing up the hardware.
> 
> I'd suspect roughly the same timing given powerful servers but still
> with reasonable cost considerations.

Interesting, thanks. Let's say I still want to, where is the code?

> Intel pretty values the 0-day service and backs it up with 12 parallel
> build servers, including 4 4S Xeon machines. Since we do merged tests,
> one may assume most of the servers are working parallel for his code
> whenever he does a git push. Kernel hackers can feel free to push
> frequently because the extra pushes are virtually cost free -- the
> build workers are working cyclicly on latest merged code anyway.

Awesome thanks! I recently ran into the situation where I may have
run into what seems to be a binutils (ld) bug and I want to verify
if it is only associated to an odd-ball architecture, I have 2 kconfig
entries I know I want tested on all architectures. In the future it
may be nice to be able to suggest a given set of kconfig entires one
wants as base for all architectures. This may need something like
kconfig-sat though [0].

[0] https://kernelnewbies.org/KernelProjects/kconfig-sat

> To take free ride of that restless horse, it'd be good to push small
> topic branches on latest RC kernels, which will have better chances
> to merge and play well with others code.

How about linux-next ?

  Luis

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Guenter Roeck]

On 07/30/2016 10:05 AM, Luis R. Rodriguez wrote:
> On Fri, Jul 29, 2016 at 08:09:12AM +0800, Fengguang Wu wrote:
>> On Fri, Jul 29, 2016 at 01:33:27AM +0200, Luis R. Rodriguez wrote:
>>> On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
>>>> btw, maybe some maintainers are already informed: 0-day statistics
>>>> show that ~60% errors can be reported in 2 hours, ~90% errors are
>>>> reported in 24 hours and there are 1% errors reported after 1 week.
>>>
>>> If one were to take 0-day code an slap it on some internal big-iron
>>> server, and prioritize work for a few developers (say SUSE would do
>>> this for its developers) do you expect the turnaround time for
>>> reports would be faster if we had bigger-meaner hardware ? These
>>> days actually would like to get results back in a few minutes
>>> for 90% of errors so wondering how / if others have taken on
>>> 0-day internally and made it faster by beefing up the hardware.
>>
>> I'd suspect roughly the same timing given powerful servers but still
>> with reasonable cost considerations.
>
> Interesting, thanks. Let's say I still want to, where is the code?
>
>> Intel pretty values the 0-day service and backs it up with 12 parallel
>> build servers, including 4 4S Xeon machines. Since we do merged tests,
>> one may assume most of the servers are working parallel for his code
>> whenever he does a git push. Kernel hackers can feel free to push
>> frequently because the extra pushes are virtually cost free -- the
>> build workers are working cyclicly on latest merged code anyway.
>
> Awesome thanks! I recently ran into the situation where I may have
> run into what seems to be a binutils (ld) bug and I want to verify
> if it is only associated to an odd-ball architecture, I have 2 kconfig
> entries I know I want tested on all architectures. In the future it

If you let me know what those are, I should be able to run them through
my build system.

Guenter

> may be nice to be able to suggest a given set of kconfig entires one
> wants as base for all architectures. This may need something like
> kconfig-sat though [0].
>
> [0] https://kernelnewbies.org/KernelProjects/kconfig-sat
>
>> To take free ride of that restless horse, it'd be good to push small
>> topic branches on latest RC kernels, which will have better chances
>> to merge and play well with others code.
>
> How about linux-next ?
>
>    Luis
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Sat, Jul 30, 2016 at 07:05:56PM +0200, Luis R. Rodriguez wrote:
>On Fri, Jul 29, 2016 at 08:09:12AM +0800, Fengguang Wu wrote:
>> On Fri, Jul 29, 2016 at 01:33:27AM +0200, Luis R. Rodriguez wrote:
>> >On Fri, Jul 29, 2016 at 07:07:13AM +0800, Fengguang Wu wrote:
>> >>btw, maybe some maintainers are already informed: 0-day statistics
>> >>show that ~60% errors can be reported in 2 hours, ~90% errors are
>> >>reported in 24 hours and there are 1% errors reported after 1 week.
>> >
>> >If one were to take 0-day code an slap it on some internal big-iron
>> >server, and prioritize work for a few developers (say SUSE would do
>> >this for its developers) do you expect the turnaround time for
>> >reports would be faster if we had bigger-meaner hardware ? These
>> >days actually would like to get results back in a few minutes
>> >for 90% of errors so wondering how / if others have taken on
>> >0-day internally and made it faster by beefing up the hardware.
>>
>> I'd suspect roughly the same timing given powerful servers but still
>> with reasonable cost considerations.
>
>Interesting, thanks. Let's say I still want to, where is the code?

It's not public available. Sorry. I love kernel community and open
source, unfortunately it's not a decision can be made by me alone.

>> Intel pretty values the 0-day service and backs it up with 12 parallel
>> build servers, including 4 4S Xeon machines. Since we do merged tests,
>> one may assume most of the servers are working parallel for his code
>> whenever he does a git push. Kernel hackers can feel free to push
>> frequently because the extra pushes are virtually cost free -- the
>> build workers are working cyclicly on latest merged code anyway.
>
>Awesome thanks! I recently ran into the situation where I may have
>run into what seems to be a binutils (ld) bug and I want to verify
>if it is only associated to an odd-ball architecture, I have 2 kconfig
>entries I know I want tested on all architectures. In the future it
>may be nice to be able to suggest a given set of kconfig entires one
>wants as base for all architectures. This may need something like
>kconfig-sat though [0].
>
>[0] https://kernelnewbies.org/KernelProjects/kconfig-sat

I'll reply that in the other email.

>> To take free ride of that restless horse, it'd be good to push small
>> topic branches on latest RC kernels, which will have better chances
>> to merge and play well with others code.
>
>How about linux-next ?

linux-next could be perfectly supported, however in a different
"rebase" way than the "merges" for RC kernels.

Suppose there are 100 branches based on -rc2 and another 100 branches
on -rc3. We can typically merge most of the 200 branches onto -rc3,
since there are relatively few changes between -rc2 and -rc3.

On the other hand, linux-next is a rolling base, it'd be hard to merge
next-20160728 itself onto next-20160729, not to mention patches based
on them. So if there are 10 branches based on next-20160728 and
another 10 branches based on next-20160729, there's little chance to
merge test them together. In current situation these linux-next based
branches will end up being tested less in 0-day.

We'd like to improve that situation by "rebasing" all linux-next based
branches to latest linux-next and test them together. It will make the
error reports less faithful to the original commits, however I guess
developers working on linux-next more or less accept it as a rolling
base and won't care much about the exact linux-next version.

If developers are interested in knowing merge conflicts with others'
code, so that he can fix it in advance and get more test coverage in
0-day, we can help auto locate down and report the merge or rebase
failures, too. Or build errors due to logical merge conflicts.

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Vinod Koul]

On Sun, Jul 31, 2016 at 02:35:58PM +0800, Fengguang Wu wrote:
> >
> >How about linux-next ?
> 
> linux-next could be perfectly supported, however in a different
> "rebase" way than the "merges" for RC kernels.
> 
> Suppose there are 100 branches based on -rc2 and another 100 branches
> on -rc3. We can typically merge most of the 200 branches onto -rc3,
> since there are relatively few changes between -rc2 and -rc3.

Since linux-next already contains all maintainer branches slated for -next,
merging that won't help much and seems tedious. I think 0day should just
runs tests on top of linux-next everyday and report, if not done already.

-- 
~Vinod

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Sun, Jul 31, 2016 at 11:02:15PM +0530, Vinod Koul wrote:
>On Sun, Jul 31, 2016 at 02:35:58PM +0800, Fengguang Wu wrote:
>> >
>> >How about linux-next ?
>>
>> linux-next could be perfectly supported, however in a different
>> "rebase" way than the "merges" for RC kernels.
>>
>> Suppose there are 100 branches based on -rc2 and another 100 branches
>> on -rc3. We can typically merge most of the 200 branches onto -rc3,
>> since there are relatively few changes between -rc2 and -rc3.
>
>Since linux-next already contains all maintainer branches slated for -next,
>merging that won't help much and seems tedious. I think 0day should just
>runs tests on top of linux-next everyday and report, if not done already.

Yes we are already testing linux-next. There are some few trees based
on linux-next because many LKML patches could be applied to it for
testing -- if they cannot apply to "guessed" maintainer trees and RC
kernels. So we have to handle patches on linux-next well.

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Jiri Kosina]

On Fri, 29 Jul 2016, Fengguang Wu wrote:

> I need your understand that there are severe limitations what useful
> contents we can provide due to the nature of merge-test-bisect. For
> example, there will be no build logs specific for your branches/commits
> because the extensive build tests are possible only because your code
> are tested together with a large number of random others. 

I think this'd be an interesting thing to discuss per se, as my personal 
feeling is that the expectations of maintainers / git tree owners might be 
slightly diverging from reality.

The reason I am bringing this up is a very recent example here:

	https://lkml.org/lkml/2016/7/28/212

The 0day bot identified my patch as a build-breaker, and it took me quite 
some time to figure out that this has been actually broken for years 
already (IOW yes, the treee fails to build with some configs after my 
patch has been applied, but I'm pretty sure it breaks with the same config 
even without my patch).

Hence, if you could explain in a little bit more detailed way how 
developers should interpret the reports, I'd be very grateful for such 
session.

Special focus should probably be put on regressions -- once reported by 
your bot, I'd love to know the last-known-good base to compare against.

Oh, and thanks a *lot* for all the 0day efforts, I am pretty sure that it 
increases the speed of development while maintaining the quality, and a 
status update from you should IMO always be a welcome KS topic,

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

Hi Jiri,

On Fri, Jul 29, 2016 at 01:38:36AM +0200, Jiri Kosina wrote:
>On Fri, 29 Jul 2016, Fengguang Wu wrote:
>
>> I need your understand that there are severe limitations what useful
>> contents we can provide due to the nature of merge-test-bisect. For
>> example, there will be no build logs specific for your branches/commits
>> because the extensive build tests are possible only because your code
>> are tested together with a large number of random others.
>
>I think this'd be an interesting thing to discuss per se, as my personal
>feeling is that the expectations of maintainers / git tree owners might be
>slightly diverging from reality.
>
>The reason I am bringing this up is a very recent example here:
>
>	https://lkml.org/lkml/2016/7/28/212
>
>The 0day bot identified my patch as a build-breaker, and it took me quite
>some time to figure out that this has been actually broken for years
>already (IOW yes, the treee fails to build with some configs after my
>patch has been applied, but I'm pretty sure it breaks with the same config
>even without my patch).

Yeah we may be running into an interesting case here: the build error
is true and bisect for that build error is correct. The robot end up
happily sent report for an innocent patch which happen to trigger a
more general problem.

>Hence, if you could explain in a little bit more detailed way how
>developers should interpret the reports, I'd be very grateful for such
>session.
>
>Special focus should probably be put on regressions -- once reported by
>your bot, I'd love to know the last-known-good base to compare against.

In bisect POV, the parent commit builds fine (all the way to vmlinux)
-- so it is the last-known-good base. If follow the reproduce steps in
the report email, you should be able to confirm that.

The main problem here is, first-bad-commit may not necessarily mean
root cause. It'll require human analyzes to reach such a conclusion.

>Oh, and thanks a *lot* for all the 0day efforts, I am pretty sure that it
>increases the speed of development while maintaining the quality, and a
>status update from you should IMO always be a welcome KS topic,

Thank you! :)

Regards,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Steven Rostedt]

On Fri, 29 Jul 2016 07:07:13 +0800
Fengguang Wu <fengguang.wu@intel.com> wrote:


> Such a status web page would be similar to the email notification
> with no much extra contents, except that it's always there for your
> polling. If that'd be valuable to you and some few other developers,
> I'd be glad to push forward the Web UI feature.
> 
> I need your understand that there are severe limitations what useful
> contents we can provide due to the nature of merge-test-bisect. For
> example, there will be no build logs specific for your branches/commits
> because the extensive build tests are possible only because your code
> are tested together with a large number of random others. For the same
> reason there will be no error/warnings available for browsing before
> the noisy error/warnings are bisected down. If we expose the unclassified
> noisy error messages and encourage developers to browse through them
> and do a guess work "are they likely caused by my commits?", we may be
> wasting the highly valued developer time.
> 
> btw, maybe some maintainers are already informed: 0-day statistics
> show that ~60% errors can be reported in 2 hours, ~90% errors are
> reported in 24 hours and there are 1% errors reported after 1 week.
> 
> As we extend test coverage aggressively, there are no longer clear
> moment that we can claim all tests for a branch are complete. The
> actual build/boot tests may well go on for over a week, covering
> thousands of kconfigs. The more time consuming functional/performance
> tests may go on for a month. That's made possible and cost effective
> thanks to the merge-test-bisect methodology.
> 
> So the safe bet is to wait for 1 day's tests before sending out patches.
> The BUILD SUCCESS notification serves as an indication 0-day robot is
> not sleeping through. My personal style suggestion would be to stay
> relaxed, take time and avoid polling. :)
> 

I'm signed up to the BUILD_SUCCESS notification, but there's been times
where they never showed up, probably due to the issues you mentioned.
What would be really useful, is just a status page of what commit has
been pulled into testing, and if it finished successful or not. It
doesn't need to display anything else but where it is in the test.

For example, if there's a web page of my git repo, I could see:

 branch ftrace commit SHA1-A - finished failed
 branch ftrace commit SHA1-B - finished success
 branch ftrace commit SHA1-C - running

Something like that, where I can see that your tests have picked up my
lasted push, and if I should be waiting for an email notification or
not.

Make sense?

-- Steve

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Thu, Jul 28, 2016 at 10:00:11PM -0400, Steven Rostedt wrote:
>On Fri, 29 Jul 2016 07:07:13 +0800
>Fengguang Wu <fengguang.wu@intel.com> wrote:
>
>
>> Such a status web page would be similar to the email notification
>> with no much extra contents, except that it's always there for your
>> polling. If that'd be valuable to you and some few other developers,
>> I'd be glad to push forward the Web UI feature.
>>
>> I need your understand that there are severe limitations what useful
>> contents we can provide due to the nature of merge-test-bisect. For
>> example, there will be no build logs specific for your branches/commits
>> because the extensive build tests are possible only because your code
>> are tested together with a large number of random others. For the same
>> reason there will be no error/warnings available for browsing before
>> the noisy error/warnings are bisected down. If we expose the unclassified
>> noisy error messages and encourage developers to browse through them
>> and do a guess work "are they likely caused by my commits?", we may be
>> wasting the highly valued developer time.
>>
>> btw, maybe some maintainers are already informed: 0-day statistics
>> show that ~60% errors can be reported in 2 hours, ~90% errors are
>> reported in 24 hours and there are 1% errors reported after 1 week.
>>
>> As we extend test coverage aggressively, there are no longer clear
>> moment that we can claim all tests for a branch are complete. The
>> actual build/boot tests may well go on for over a week, covering
>> thousands of kconfigs. The more time consuming functional/performance
>> tests may go on for a month. That's made possible and cost effective
>> thanks to the merge-test-bisect methodology.
>>
>> So the safe bet is to wait for 1 day's tests before sending out patches.
>> The BUILD SUCCESS notification serves as an indication 0-day robot is
>> not sleeping through. My personal style suggestion would be to stay
>> relaxed, take time and avoid polling. :)
>>
>
>I'm signed up to the BUILD_SUCCESS notification, but there's been times
>where they never showed up, probably due to the issues you mentioned.
>What would be really useful, is just a status page of what commit has
>been pulled into testing, and if it finished successful or not. It
>doesn't need to display anything else but where it is in the test.
>
>For example, if there's a web page of my git repo, I could see:
>
> branch ftrace commit SHA1-A - finished failed
> branch ftrace commit SHA1-B - finished success
> branch ftrace commit SHA1-C - running
>
>Something like that, where I can see that your tests have picked up my
>lasted push, and if I should be waiting for an email notification or
>not.
>
>Make sense?

Yes that makes good sense. A more structured layout could be

trace
├── for-next
│   ├── 97f8827a8c7963756ae7d3ee898675b4667eca73
│   └── HEAD -> 97f8827a8c7963756ae7d3ee898675b4667eca73
└── ftrace
    ├── core
    │   ├── 501c2375253c0795048f48368e0b3e8b2f6646dc
    │   ├── 78aebca2c955c1c5aeb48e12645e13fe3c3461f2
    │   ├── 7fa8b7171a638ad896acabd9a17183b75b70aeb4
    │   ├── 8329e818f14926a6040df86b2668568bde342ebf
    │   ├── de9be5961936523e6cc3a1358b166e4efb42595b
    │   ├── fc54f5a288feb132d089404ef5748725ac669a5c
    │   └── HEAD -> 78aebca2c955c1c5aeb48e12645e13fe3c3461f2
    └── urgent
        ├── 0ded5174e976e2b2a354fe38abf1ebf4492c6dc3
        ├── 90d6c42c7766fe63c724a6ec7758a85b6231b08b
        └── HEAD -> 90d6c42c7766fe63c724a6ec7758a85b6231b08b

Each end node contains log messages showing all the progresses that
have been made.

If necessary, symlinks (like HEAD commit's patch-file-name) could be
created at top level to make it more convenient to navigate. Such
symlinks can be auto deleted after 1 day (when people are less
unlikely to look at its status).

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Fengguang Wu]

On Mon, Jul 25, 2016 at 09:01:25PM +0200, Luis R. Rodriguez wrote:
>
>It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
>so I'd like to nominate him. The mechanical process should probably include in
>the future a scrape for top Reported-by contributors.

Thanks for your appreciations, Luis!

>Fengguang's 0-day infrastructure is invaluable to day to day kernel
>development, having him present would be great for any questions that may come
>up.

I'd be glad to answer questions and more importantly, collect
feedbacks on where and how to improve the 0-day infrastructure.

There are 2 major parts in 0-day: build tests and runtime tests.
While build tests will be continuously improved, there may be a lot
more to be desired for runtime tests, which should be my main focus
in the coming year.

>Getting a statistical overview / update of impact / any major architectural
>changes of the 0-day infrastructure would also be very useful.

Sure if there are interests.

>If maintainers
>are not yet using 0-day it would be great to hear why. If your contributors are
>not using 0-day (I know some of you exist) I'd like to know why you don't use it,
>I often run into issues on linux-next which at times I have to fix, if 0-day
>would have been used a folowup fix would not have been needed.

0-day tries to monitor as many git trees as possible, so that fresh
code can be tested before they land maintainer trees and linux-next.
To achieve better early code coverage, we periodically check if there
are new git trees showing up in git.kernel.org, in mainline git log
or mentioned in LKML emails. And add the newly discovered ones
unsolicited. :)

That said, it's still possible errors hit linux-next. Sometimes it may
be due to bug in 0-day or temporarily out of service. The solution
would be to improve 0-day system's stability and add more self-tests
to the system. Quick feedbacks about build/boot errors missed by 0-day
would also be highly appreciated.

Thanks,
Fengguang

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Guenter Roeck]

On Wed, Jul 27, 2016 at 10:41:14PM +0800, Fengguang Wu wrote:
> On Mon, Jul 25, 2016 at 09:01:25PM +0200, Luis R. Rodriguez wrote:
> >
> >It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
> >so I'd like to nominate him. The mechanical process should probably include in
> >the future a scrape for top Reported-by contributors.
> 
> Thanks for your appreciations, Luis!
> 
> >Fengguang's 0-day infrastructure is invaluable to day to day kernel
> >development, having him present would be great for any questions that may come
> >up.
> 
> I'd be glad to answer questions and more importantly, collect
> feedbacks on where and how to improve the 0-day infrastructure.
> 
> There are 2 major parts in 0-day: build tests and runtime tests.
> While build tests will be continuously improved, there may be a lot
> more to be desired for runtime tests, which should be my main focus
> in the coming year.
> 
For runtime test improvements: We should discuss if and how to integrate
kasan/syzkaller testing, and/or if it would make sense to set up a separate
test bed for that purpose (to avoid overloading 0day).

Thanks,
Guenter

> >Getting a statistical overview / update of impact / any major architectural
> >changes of the 0-day infrastructure would also be very useful.
> 
> Sure if there are interests.
> 
> >If maintainers
> >are not yet using 0-day it would be great to hear why. If your contributors are
> >not using 0-day (I know some of you exist) I'd like to know why you don't use it,
> >I often run into issues on linux-next which at times I have to fix, if 0-day
> >would have been used a folowup fix would not have been needed.
> 
> 0-day tries to monitor as many git trees as possible, so that fresh
> code can be tested before they land maintainer trees and linux-next.
> To achieve better early code coverage, we periodically check if there
> are new git trees showing up in git.kernel.org, in mainline git log
> or mentioned in LKML emails. And add the newly discovered ones
> unsolicited. :)
> 
> That said, it's still possible errors hit linux-next. Sometimes it may
> be due to bug in 0-day or temporarily out of service. The solution
> would be to improve 0-day system's stability and add more self-tests
> to the system. Quick feedbacks about build/boot errors missed by 0-day
> would also be highly appreciated.
> 
> Thanks,
> Fengguang
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] Nominating Fengguang Wu - 0-day

[Dmitry Vyukov]

On Thu, Jul 28, 2016 at 7:15 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On Wed, Jul 27, 2016 at 10:41:14PM +0800, Fengguang Wu wrote:
>> On Mon, Jul 25, 2016 at 09:01:25PM +0200, Luis R. Rodriguez wrote:
>> >
>> >It surprises Fengguang Wu <fengguang.wu@intel.com> hasn't been nominated yet,
>> >so I'd like to nominate him. The mechanical process should probably include in
>> >the future a scrape for top Reported-by contributors.
>>
>> Thanks for your appreciations, Luis!
>>
>> >Fengguang's 0-day infrastructure is invaluable to day to day kernel
>> >development, having him present would be great for any questions that may come
>> >up.
>>
>> I'd be glad to answer questions and more importantly, collect
>> feedbacks on where and how to improve the 0-day infrastructure.
>>
>> There are 2 major parts in 0-day: build tests and runtime tests.
>> While build tests will be continuously improved, there may be a lot
>> more to be desired for runtime tests, which should be my main focus
>> in the coming year.
>>
> For runtime test improvements: We should discuss if and how to integrate
> kasan/syzkaller testing, and/or if it would make sense to set up a separate
> test bed for that purpose (to avoid overloading 0day).


I am all for it.
We are attending plumbers, so we will be able to discuss this in person.


>> >Getting a statistical overview / update of impact / any major architectural
>> >changes of the 0-day infrastructure would also be very useful.
>>
>> Sure if there are interests.
>>
>> >If maintainers
>> >are not yet using 0-day it would be great to hear why. If your contributors are
>> >not using 0-day (I know some of you exist) I'd like to know why you don't use it,
>> >I often run into issues on linux-next which at times I have to fix, if 0-day
>> >would have been used a folowup fix would not have been needed.
>>
>> 0-day tries to monitor as many git trees as possible, so that fresh
>> code can be tested before they land maintainer trees and linux-next.
>> To achieve better early code coverage, we periodically check if there
>> are new git trees showing up in git.kernel.org, in mainline git log
>> or mentioned in LKML emails. And add the newly discovered ones
>> unsolicited. :)
>>
>> That said, it's still possible errors hit linux-next. Sometimes it may
>> be due to bug in 0-day or temporarily out of service. The solution
>> would be to improve 0-day system's stability and add more self-tests
>> to the system. Quick feedbacks about build/boot errors missed by 0-day
>> would also be highly appreciated.
>>
>> Thanks,
>> Fengguang
>> _______________________________________________
>> Ksummit-discuss mailing list
>> Ksummit-discuss@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss