Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] stable kernel process automation and improvement

[Greg KH <greg@kroah.com>]

On Mon, Jul 08, 2019 at 04:33:28PM +0200, Takashi Iwai wrote:
> On Mon, 08 Jul 2019 16:05:44 +0200,
> Guenter Roeck wrote:
> > 
> > On 7/8/19 5:37 AM, Mark Brown wrote:
> > > On Mon, Jul 08, 2019 at 07:02:08AM -0400, Sasha Levin wrote:
> > >> On Sat, Jul 06, 2019 at 01:32:14AM +0100, Mark Brown wrote:
> > >
> > >>> I'm not saying leave it alone, it's more a question of how
> > >>> aggressive we are about picking up things we think might be
> > >>> relevant fixes but haven't had some sort of domain specific
> > >>> analysis of.  Testing is a good way to mitigate the potential
> > >>> risks here.
> > >
> > >> I agree, and for various subsystems and drivers where the maintainers
> > >> volunteer their domain specific expertise to send backports to stable, I
> > >> have "blacklisted" it from AUTOSEL since indeed it's a much better
> > >> option.
> > >
> > > Hrm, it's definitely getting a bunch of stuff for my subsystems
> > > where I do tag things for stable...
> > >
> > >>>> This came up in the last MS, and the agreement there was that we expect
> > >>>> stable kernel users to test their workloads before throwing it into
> > >>>> production.
> > >
> > >>> That's kind of the problem - if people are doing testing and end
> > >>> up finding problems coming back in the stable kernel that's the
> > >>> sort of thing that encourages them to not just take stable en
> > >>> masse as we say they should.  Part of the deal with stable is
> > >>> that it is conservative, people can trust it to be a low risk
> > >>> update.  That's not happening now as far as I'm aware but it does
> > >>> worry me that it might happen.
> > >
> > >> Right, and the rate at which AUTOSEL commits are reverted is lower than
> > >> commits that are actually tagged for stable. If AUTOSEL commits on their
> > >> own were being reverted left and right I'd agree we need to tone it
> > >> down, but I don't see it happening now.
> > >
> > > I'm not sure how many people will actually report problems they
> > > experience upstream rather than just fixing things locally and
> > > just moving on.  The more code is the more likely it is that one
> > > of the users will report things.
> > >
> > 
> > I for my part will most definitely report any such problems, since each
> > regression in stable releases is used as argument against merging
> > stable releases (even if the regression rate is negligible), and I am
> > very interested in getting that regression rate as close to zero as
> > possible. Reporting each and every regression is an essential part
> > of that.
> 
> BTW, regarding regression: currently we have no central regression
> tracking.  This is another big missing piece, and a thing to be
> discussed in KS, IMO.

Well, I think the conversation will go just like it has in the past for
this issue:
	"We need to have someone track regressions!"
	"X said they would do it but they need to be paid, any company
	willing to sponsor this?"
	{crickets}

We know we need this, we have at least one talented and capable person
to do the work, but no company is willing to step up and fund it :(

It's like where we were 5 years ago with testing, everyone knew there
was a problem, but no one was willing to do anything about it.  That
time I convinced some LF member companies to start doing work within
their companies toward this, but that really doesn't solve this type of
problem as being "distributed" isn't the issue here...

thanks,

greg k-h