Re: linus-next: improving functional testing for to-be-merged pull requests

["Paul E. McKenney" <paulmck@kernel.org>]

On Wed, Oct 23, 2024 at 10:24:29PM +0100, Mark Brown wrote:
> On Wed, Oct 23, 2024 at 11:06:59AM -0700, Linus Torvalds wrote:
> 
> > And yes, I know some people do functional testing on linux-next
> > already. The message at the maintainer summit was a bit mixed with
> > some people saying linux-next tends to work even for that, others
> > saying it's often too broken to be useful.
> 
> It very much depends on what you're trying to get out of the testing -
> -next does work well most of the time, but it will absolutely just blow
> up catastrophically on you from time time to time so you have to be
> prepared to cope with loosing some or all of your coverage sometimes.
> Usually anything major gets fixed fairly promptly, but sometimes you'll
> be missing coverage for extended periods especially if it's something
> like a more niche platform that's been broken or there's some problem
> getting people to actually apply the fixes.  Submaintainer trees that
> people don't want to add to -next can be an issue too.
> 
> You're also going to run into issues that are nothing to do with
> whatever you're actually working on yourself and need to consider what
> you're covering based on your tolerance for dealing with that.  The rate
> of change can also be an issue if the tests you're intersted in are
> expensive.  OTOH if you're doing things that are likely to be affected
> by changes in a broad set of trees (eg, maintaining some embedded
> platform where you care about all the various subsystems breaking
> platform specific drivers) it can be a lot easier to cover -next rather
> than all the individual subsystems.

You said it much better than I did, thank you!

For me, -next is a convenient point to test much of what will be going in.
Yes, it can be frustrating, finding problems just as someone else fixed
them, finding problems irrelevant to any of my use cases, and so on.
But sometimes I find something that would have been quite painful to
deal with later in process that others don't find.  Overall, it is well
worth the my effort.

							Thanx, Paul