On Thu, Jun 20, 2024 at 12:02:21PM -0400, James Bottomley wrote:
> On Thu, 2024-06-20 at 15:42 +0100, Mark Brown wrote:
> > On Thu, Jun 20, 2024 at 10:01:57AM -0400, James Bottomley wrote:
> > > On Thu, 2024-06-20 at 14:55 +0100, Mark Brown wrote:

> > > > If your tests take more than a day to run then it gets more
> > > > tricky, but that's just generally harder no matter which tree
> > > > you're testing.

> > > The difficulty is usually that by the time you get a signal
> > > something is wrong, the next tree is different.  I agree you can
> > > freeze on the

> > That'd be the tests taking more than a day bit.

> Depends ... we might be using different terms.  I think of testing as
> simply finding the bug.  After that there's usually a whole load of
> work to pinpoint the commit that caused it, so even if a test only
> takes say 30 minutes to run, the bisection can take over a day.

Sure, but unless the tree with the issue rebases constantly so long as
you can bisect into the tree and then some within a day that's not going
to stop progress (and a lot of the time just finishing the bisect and
then validating on today's -next is fine).  IME the effort with -next is
worth it for the turnaround time, it's a lot easier to get attention on
recently merged patches.

> > > Regardless, I don't think -next is a useful tree for the wider pool
> > > who usually test stable to try because of all the difficulties.  I
> > > do think it's not impossible to get some of them to move up to main
> > > (after all it's the .0 of stable).

> > AFAICT we have a far wider pool of people testing -next than we do
> > testing the stable -rcs at the minute, there's more people trying to
> > *use* stables and finding issues but that's not quite the same thing
> > and I suspect much of the plain testing is going to be qualification
> > for release so it'd be hard to get people to substitute mainline.

> Right, but the point I'm making is that even that wider pool doesn't
> have the app use or hardware breadth of the pool who try out stable.  I
> also agree the stable users would rather not be testers but given that
> they are, it's not impossible we could sell them on the idea of testing
> out .0 to find bugs they would otherwise be finding in .n.

I suspect you'll find that a lot of the people who have the capacity and
engagement to do that are already doing so.

> After all, given that stable is now delaying backports in the merge
> window, there should be at least a 2 week period where .0 is it
> (although it's also the two week period where we're not paying
> attention ...)

Yeah, and it also depends on people being able to easily run mainline
which if for example people are carrying out of tree patches might be a
bit of an issue.