Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] stable kernel process automation and improvement

[Takashi Iwai <tiwai@suse.de>]

On Mon, 08 Jul 2019 17:10:40 +0200,
Greg KH wrote:
> 
> On Mon, Jul 08, 2019 at 04:33:28PM +0200, Takashi Iwai wrote:
> > On Mon, 08 Jul 2019 16:05:44 +0200,
> > Guenter Roeck wrote:
> > > 
> > > On 7/8/19 5:37 AM, Mark Brown wrote:
> > > > On Mon, Jul 08, 2019 at 07:02:08AM -0400, Sasha Levin wrote:
> > > >> On Sat, Jul 06, 2019 at 01:32:14AM +0100, Mark Brown wrote:
> > > >
> > > >>> I'm not saying leave it alone, it's more a question of how
> > > >>> aggressive we are about picking up things we think might be
> > > >>> relevant fixes but haven't had some sort of domain specific
> > > >>> analysis of.  Testing is a good way to mitigate the potential
> > > >>> risks here.
> > > >
> > > >> I agree, and for various subsystems and drivers where the maintainers
> > > >> volunteer their domain specific expertise to send backports to stable, I
> > > >> have "blacklisted" it from AUTOSEL since indeed it's a much better
> > > >> option.
> > > >
> > > > Hrm, it's definitely getting a bunch of stuff for my subsystems
> > > > where I do tag things for stable...
> > > >
> > > >>>> This came up in the last MS, and the agreement there was that we expect
> > > >>>> stable kernel users to test their workloads before throwing it into
> > > >>>> production.
> > > >
> > > >>> That's kind of the problem - if people are doing testing and end
> > > >>> up finding problems coming back in the stable kernel that's the
> > > >>> sort of thing that encourages them to not just take stable en
> > > >>> masse as we say they should.  Part of the deal with stable is
> > > >>> that it is conservative, people can trust it to be a low risk
> > > >>> update.  That's not happening now as far as I'm aware but it does
> > > >>> worry me that it might happen.
> > > >
> > > >> Right, and the rate at which AUTOSEL commits are reverted is lower than
> > > >> commits that are actually tagged for stable. If AUTOSEL commits on their
> > > >> own were being reverted left and right I'd agree we need to tone it
> > > >> down, but I don't see it happening now.
> > > >
> > > > I'm not sure how many people will actually report problems they
> > > > experience upstream rather than just fixing things locally and
> > > > just moving on.  The more code is the more likely it is that one
> > > > of the users will report things.
> > > >
> > > 
> > > I for my part will most definitely report any such problems, since each
> > > regression in stable releases is used as argument against merging
> > > stable releases (even if the regression rate is negligible), and I am
> > > very interested in getting that regression rate as close to zero as
> > > possible. Reporting each and every regression is an essential part
> > > of that.
> > 
> > BTW, regarding regression: currently we have no central regression
> > tracking.  This is another big missing piece, and a thing to be
> > discussed in KS, IMO.
> 
> Well, I think the conversation will go just like it has in the past for
> this issue:
> 	"We need to have someone track regressions!"
> 	"X said they would do it but they need to be paid, any company
> 	willing to sponsor this?"
> 	{crickets}
> 
> We know we need this, we have at least one talented and capable person
> to do the work, but no company is willing to step up and fund it :(

Yeah, it's a sad deja vu...

> It's like where we were 5 years ago with testing, everyone knew there
> was a problem, but no one was willing to do anything about it.  That
> time I convinced some LF member companies to start doing work within
> their companies toward this, but that really doesn't solve this type of
> problem as being "distributed" isn't the issue here...

The past attempts and their failing patterns look like a SPOF, it's
been always a load to a single person, who eventually gave up
maintaining.  A more automated and distributed work would help in this
regard, I hope sincerely.


thanks,

Takashi