Re: linus-next: improving functional testing for to-be-merged pull requests

[Benjamin Tissoires <bentiss@kernel.org>]

On Oct 21 2024, Kees Cook wrote:
> 
> 
> On October 21, 2024 9:07:13 AM PDT, Sasha Levin <sashal@kernel.org> wrote:
> >In an attempt to address the concerns, we're trying out a new "linus-next"
> >tree is being created and maintained with the following characteristics:
> >
> >	1. Composed of pull requests sent directly to Linus
> >
> >	2. Contains branches destined for imminent inclusion by Linus
> 
> But this means hours or a day or 2 at most.

Yeah :/

> 
> >	3. Higher code quality expectation (these are pull requests that
> >	maintainers expect Linus to pull)
> 
> Are people putting things in linux-next that they don't expect to send to Linus? That seems like the greater problem.
> 
> >	4. Continuous tree (not daily tags like in linux-next),
> >	facilitating easier bisection
> 
> I'm not sure how useful that is given the very small time window to find bugs.

I think there is some value for this tree, but the target is not what
Sasha explained (or what I understood at least). The linus-next (or
whatever linus-pr) is seems to me to be solely targetting Linus, for
running pre-merge checks.

And that is *very* interesting for him, so yes, I'd vote for a plus one
on this.

IMO this new tree shouldn't be advertised as a:
"please run as many possible tests on this before Linus pulls it",
but as a:
"we are gathering all PR, run a dedicated sets of tests on it, and if
they passes, Linus will look at your PR. If not, your PR will be
automatically put on hold".

Think of it as a pre-merge testing, and if a PR fails, the maintainer
has to sort it out instead of having Linus to sort it out.

> 
> >The linus-next tree aims to provide a more stable and testable
> >integration point compared to linux-next,
> 
> Why not just use linux-next? I don't understand how this is any different except that it provides very little time to do testing and will need manual conflict resolutions that have already been done in linux-next.

Agree. Gathering general tests should not come *after* the PR has been
sent, but before.

It could be interesting IMO to have a linux-current-fixes tree which
gather trees that are meant to be sent to this current cycle (fixes only
basically).

In my small subsystem we sometime gather fixes that are not urgent but
could go into the current cycle but are not send them right away. They
could be integrated in such a tree, but OTOH, they are tested in
linux-next already, so it's more of a matter for testers to decide if
they want to have a "current tree + upcoming fixes" or not.

> 
> How about this, instead: no one sends -rc1 PRs to Linus that didn't go through -next. Just have a bot that replies to all PRs with a health check, and Linus can pull it if he thinks it looks good. 
> 
> For example, for a given PR, the bot can report:
> 
> - Were the patches CCed to a mailing list?
> - A histogram of how long the patches were in next (to show bake times)
> - Are any patches associated with test failures? (0day and many other CIs are already running tests against -next; parse those reports)
> 
> We could have a real pre-submit checker! :)

As long as it only helps making an educated guess (or catch obvious
failures) I'm all for it.
In the same way stable receives emails from testers before Greg tags the
new tree.

Cheers,
Benjamin