Re: [MAINTAINERS SUMMIT] [0/4] Common scenario for four proposals regarding regressions

[James Bottomley <James.Bottomley@HansenPartnership.com>]

[-- Attachment #1: Type: text/plain, Size: 2725 bytes --]

On Thu, 2024-06-20 at 15:42 +0100, Mark Brown wrote:
> On Thu, Jun 20, 2024 at 10:01:57AM -0400, James Bottomley wrote:
> > On Thu, 2024-06-20 at 14:55 +0100, Mark Brown wrote:
> 
> > > If your tests take more than a day to run then it gets more
> > > tricky, but that's just generally harder no matter which tree
> > > you're testing.
> 
> > The difficulty is usually that by the time you get a signal
> > something is wrong, the next tree is different.  I agree you can
> > freeze on the
> 
> That'd be the tests taking more than a day bit.

Depends ... we might be using different terms.  I think of testing as
simply finding the bug.  After that there's usually a whole load of
work to pinpoint the commit that caused it, so even if a test only
takes say 30 minutes to run, the bisection can take over a day.

> > next tree you have and hope that the identified commit (by the time
> > you find it) is still in the current version of -next, but there is
> > a non-zero chance it would get rebased which makes testing next a
> > bit more of a chore than testing main, which is why it's tested
> > less often than main
> 
> Obviously some trees do rebase, but not constantly and a lot of trees
> simply don't rebase - carrying things forward to the next day tends
> to be more of a mild annoyance IME, especially if you remember all
> the good and bad commits and don't need to restart from scratch.

I agree that -next is mostly an unstable tree built from reasonably
stable branches, yes.

> > Regardless, I don't think -next is a useful tree for the wider pool
> > who usually test stable to try because of all the difficulties.  I
> > do think it's not impossible to get some of them to move up to main
> > (after all it's the .0 of stable).
> 
> AFAICT we have a far wider pool of people testing -next than we do
> testing the stable -rcs at the minute, there's more people trying to
> *use* stables and finding issues but that's not quite the same thing
> and I suspect much of the plain testing is going to be qualification
> for release so it'd be hard to get people to substitute mainline.

Right, but the point I'm making is that even that wider pool doesn't
have the app use or hardware breadth of the pool who try out stable.  I
also agree the stable users would rather not be testers but given that
they are, it's not impossible we could sell them on the idea of testing
out .0 to find bugs they would otherwise be finding in .n.

After all, given that stable is now delaying backports in the merge
window, there should be at least a 2 week period where .0 is it
(although it's also the two week period where we're not paying
attention ...)

James



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]