Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] stable kernel process automation and improvement

[Laura Abbott <labbott@redhat.com>]

On 7/8/19 11:10 AM, Greg KH wrote:
> On Mon, Jul 08, 2019 at 04:33:28PM +0200, Takashi Iwai wrote:
>> On Mon, 08 Jul 2019 16:05:44 +0200,
>> Guenter Roeck wrote:
>>>
>>> On 7/8/19 5:37 AM, Mark Brown wrote:
>>>> On Mon, Jul 08, 2019 at 07:02:08AM -0400, Sasha Levin wrote:
>>>>> On Sat, Jul 06, 2019 at 01:32:14AM +0100, Mark Brown wrote:
>>>>
>>>>>> I'm not saying leave it alone, it's more a question of how
>>>>>> aggressive we are about picking up things we think might be
>>>>>> relevant fixes but haven't had some sort of domain specific
>>>>>> analysis of.  Testing is a good way to mitigate the potential
>>>>>> risks here.
>>>>
>>>>> I agree, and for various subsystems and drivers where the maintainers
>>>>> volunteer their domain specific expertise to send backports to stable, I
>>>>> have "blacklisted" it from AUTOSEL since indeed it's a much better
>>>>> option.
>>>>
>>>> Hrm, it's definitely getting a bunch of stuff for my subsystems
>>>> where I do tag things for stable...
>>>>
>>>>>>> This came up in the last MS, and the agreement there was that we expect
>>>>>>> stable kernel users to test their workloads before throwing it into
>>>>>>> production.
>>>>
>>>>>> That's kind of the problem - if people are doing testing and end
>>>>>> up finding problems coming back in the stable kernel that's the
>>>>>> sort of thing that encourages them to not just take stable en
>>>>>> masse as we say they should.  Part of the deal with stable is
>>>>>> that it is conservative, people can trust it to be a low risk
>>>>>> update.  That's not happening now as far as I'm aware but it does
>>>>>> worry me that it might happen.
>>>>
>>>>> Right, and the rate at which AUTOSEL commits are reverted is lower than
>>>>> commits that are actually tagged for stable. If AUTOSEL commits on their
>>>>> own were being reverted left and right I'd agree we need to tone it
>>>>> down, but I don't see it happening now.
>>>>
>>>> I'm not sure how many people will actually report problems they
>>>> experience upstream rather than just fixing things locally and
>>>> just moving on.  The more code is the more likely it is that one
>>>> of the users will report things.
>>>>
>>>
>>> I for my part will most definitely report any such problems, since each
>>> regression in stable releases is used as argument against merging
>>> stable releases (even if the regression rate is negligible), and I am
>>> very interested in getting that regression rate as close to zero as
>>> possible. Reporting each and every regression is an essential part
>>> of that.
>>
>> BTW, regarding regression: currently we have no central regression
>> tracking.  This is another big missing piece, and a thing to be
>> discussed in KS, IMO.
> 
> Well, I think the conversation will go just like it has in the past for
> this issue:
> 	"We need to have someone track regressions!"
> 	"X said they would do it but they need to be paid, any company
> 	willing to sponsor this?"
> 	{crickets}
> 
> We know we need this, we have at least one talented and capable person
> to do the work, but no company is willing to step up and fund it :(
> 
> It's like where we were 5 years ago with testing, everyone knew there
> was a problem, but no one was willing to do anything about it.  That
> time I convinced some LF member companies to start doing work within
> their companies toward this, but that really doesn't solve this type of
> problem as being "distributed" isn't the issue here...
> 
> thanks,
> 
> greg k-h

There's two parts here: a centralized place to track bugs and regressions
and person to help manage those. While having a person to manage everything
would be good, getting the central tracking going without relying on a
single person is important.

Thanks,
Laura