Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches

[Steven Rostedt <rostedt@goodmis.org>]

On Tue, 11 Sep 2018 16:09:32 -0400
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:

> On Tue, 2018-09-11 at 14:39 -0400, Steven Rostedt wrote:
> > On Tue, 11 Sep 2018 10:47:02 -0700
> > James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
> >    
> > > I really don't think that helps.  The 0day mailing list bot seems
> > > to be a bit overloaded and about 80% of the automation isn't run
> > > *unless*  
> > 
> > Really? I get reports on my branches about a lot of issues without
> > ever pushing it to next. Maybe I lucked out and these issues were
> > caught by the 20%.
> > 
> > When I have a branch ready to test, I push it to my local branch on
> > kernel.org and then kick off my own test suite. I like to see which
> > one will catch any bugs first. I usually get a response from 0day and
> > a couple of hours later my tests will fail with the same error. Thus
> > I found 0day to be quite efficient.  
> 
> I treat -next as the integration tree: I push to it when we're ready to
> integrate.  I don't think that's unreasonable and I do see it being
> unreasonable to impose a delay on this process.

I believe that's exactly what -next is for. Integration with
development from other sub systems.

> 
> > > your branch hits -next.  Our criterion for -next queueing is local
> > > tests pass, code inspection complete and 0day ML didn't complain. 
> > > However, we still get quite a few reports from the -next automated
> > > testing even after our local stuff.  I really don't see what
> > > delaying into -next buys you except delaying finding and fixing
> > > bugs.  
> > 
> > But you state you have your own local tests, which I think could be
> > enough of a requirement. Although running allmod and allyes should be
> > part of a local test, but I think 0day does that too (as that's
> > usually the test that fails most often that 0day catches first).  
> 
> So when local tests are complete we're ready for integration.

If your local tests catch most common bugs. (testing on non-SMP,
x86_32, and other non-common cases where things can break).

> 
> > > This applies to 0day as well, because, by agreement, it has a much
> > > deeper set of xfstest runs for branches we actually queue for -next
> > > rather than the more cursory set of tests it runs on ML patches.  
> > 
> > What about the tests on local branches? Not what you post to the ML.  
> 
> I don't really see any point having a local -pre-next pass and hoping
> 0day will find it.  It's much more valuable (and faster) to push to
> -next and have all the integrated tests work on it once we've done
> everything we can locally.

Why not do what I do and push to a -pre-next branch when you kick off
your local tests? Then perhaps 0day may catch something you introduced
before it heads off to -next. -next is about catching bugs due to
integration with other sub systems, it shouldn't be used to catch bugs
that could have been caught without the integration.

Kind of like if you give blood in the US. There's a note that tells you
to not use blood donation as a way to get tested for a particular
disease. Just get tested for it before giving blood and possibly
infecting others.

-- Steve