Re: linus-next: improving functional testing for to-be-merged pull requests

["Paul E. McKenney" <paulmck@kernel.org>]

On Mon, Oct 21, 2024 at 12:07:13PM -0400, Sasha Levin wrote:
> Hi folks,
> 
> The linux-next tree we all know and love is widely used by the kernel
> community for integration work. It offers several advantages:
> 
> 	1. Early detection of conflicts between matinainer trees
> 
> 	2. Catching most new build errors/warnings
> 
> However, it faces significant testing challenges:
> 
> 	1. Contains a mix of "ready-to-go" code and experimental additions
> 
> 	2. A single "bad" piece of code can affect testing of everything else
> 
> 	3. Low barrier of entry, encouraging inclusion over exclusion
> 
> 	4. While linux-next offers early conflict resolution and
> 	identifies build issues, it is very difficult to actually test
> 	due to the abundance of runtime issues it tends to have
> 
> These factors combine to make linux-next a valuable tool for integration
> but problematic for comprehensive testing.
> 
> During the Maintainer's Summit, Linus Torvalds expressed concerns about
> the quality of testing that code receives before he pulls it. The
> subsequent discussion side-tracked to the testability of linux-next, but
> we didn't directly address Linus's original concern about pre-pull
> testing quality.

I have to ask...

Wouldn't more people testing -next result in more pressure to fix
linux-next problems quickly?  Or perhaps more pressure for people to
avoid linux-next?  But this later would also apply to a new linus-next.
Unless Linus were to start rejecting pull requests that had not been
in linu[sx]-next for "long enough", whatever that might be.  ;-)

							Thanx, Paul

> In an attempt to address the concerns, we're trying out a new "linus-next"
> tree is being created and maintained with the following characteristics:
> 
> 	1. Composed of pull requests sent directly to Linus
> 
> 	2. Contains branches destined for imminent inclusion by Linus
> 
> 	3. Higher code quality expectation (these are pull requests that
> 	maintainers expect Linus to pull)
> 
> 	4. Continuous tree (not daily tags like in linux-next),
> 	facilitating easier bisection
> 
> The linus-next tree aims to provide a more stable and testable
> integration point compared to linux-next, addressing the runtime issues
> that make testing linux-next challenging and focusing on code that's
> about to be pulled by Linus.
> 
> linus-next is (expected to be) particularly effective before the merge
> window opens, as maintainers tend to send their pull requests early,
> allowing for more thorough testing of to-be-merged changes.
> 
> We also want to avoid altering the existing workflow. In particular:
> 
> 	1. No increase in latency. If anything, the expectation is that
> 	the cadence of merges would be improved given that Linus will
> 	need to do less builds and tests.
> 
> 	2. Require "sign up" for the tree like linux-next does. Instead,
> 	pull requests are monitored and grabbed directly from the
> 	mailing list.
> 
> Tree location: `git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linus-next.git linus-next`
> 
> Current testing:
>   - LKFT: https://qa-reports.linaro.org/lkft/sashal-linus-next/
>   - KernelCI: https://t.ly/KEW7F
> 
> Feedback and suggestions for improving usability are welcome!
> 
> -- 
> Thanks,
> Sasha
>