Re: [Ksummit-discuss] [Stable kernel] feature backporting collaboration

["Levin, Alexander" <alexander.levin@verizon.com>]

On Fri, Sep 02, 2016 at 05:54:17AM -0400, Mark Brown wrote:
> Sep 01, 2016 at 09:25:31PM -0400, Levin, Alexander via Ksummit-discuss wrote:
> > On Wed, Aug 31, 2016 at 10:01:13PM -0400, Alex Shi wrote:
> 
> > > I am a Linaro stable kernel maintainer. Our stable kernel is base on LTS
> > > plus much of upstream features backporting on them. Here is the detailed
> 
> > I really disagree with this approach. I think that backporting board support
> > like what LTSI does might make sense since it's self contained, but what LSK
> > does is just crazy.
> 
> The bulk of these features are exactly that - they're isolated driver
> specific code or new subsystems.  There are also some things with wider
> impact but it's nowhere near all of them.

It's nowhere near all of them, but all it takes is one.

Look at KASLR and KASan, it has complex interactions with pretty much the rest
of the kernel. Quite a few things not directly related to either of those had
to be fixed just because they were found to not integrate right (For example,
KASLR uncovered a bunch of bugs before it was actually merged in), who says
that there aren't any similar interactions with the older kernels that no one
looked into?
 
> > Stable kernels have very strict restrictions that are focused on not taking
> > commits that have high potential to cause unintended side effects, incorrect
> > interactions with the rest of the kernel or just introduce new bugs.
> 
> > Mixing in new features that interact with multiple subsystems is a recipe for
> > disaster. We barely pull off backporting what looks like trivial fixes, trying
> > to do the same for more than that is bound be broken.
> 
> It's what people are doing for products, they want newer features but
> they also don't want to rebase their product kernel onto mainline as
> that's an even bigger integration risk.  People aren't using this kernel
> raw, they're using it as the basis for product kernels.  What this is
> doing is getting a bunch of people using the same backports which shares
> effort and hopefully makes it more likely that some of the security
> relevant features will get deployed in products.  Ideally some of the
> saved time can be spent on upstreaming things though I fear that's a
> little optimistic.

I'm sorry but just calling a kernel "stable" doesn't mean that suddenly it
acquires the qualities of a stable kernel that follows the very strict rules
we have for those.

Given that you're backporting features into a stable kernel it really inherits
the code quality of a release candidate kernel; nowhere close to a stable
kernel.

This following is just my opinion as an LTS kernel maintainer: if you think
that the integration risk of a newer stable/LTS is bigger than using these
frankenstein kernels you are very much mistaken.

In your case it's nice if you could share backports betweek multiple users
(just like we try doing for all the stable/LTS trees), but the coverage and
testing you're going to get for that isn't anywhere close to what you'll have
for a more recent stable kernel that already has those features baked into
that.

> > As an alternative, why not use more recent stable kernels and customize the
> > config specifically for each user to enable on features that that specific
> > user wants to have.
> 
> That's just shipping a kernel - I don't think anyone is silly enough to
> ship an allmodconfig or similar in production (though I'm sure someone
> can come up with an example).

I highly doubt that most shipped kernels actually go through the process of
auditing every single config option and figuring out if they actually need it
or not (in part because the kernel's config is quite a mess). I really doubt
that the kernel is fine-tuned for majority of the released products that run
linux.

I think that time invested in improving the config code is much more important
that investing time in attempting to backport features.

> > The benefit here is that if used correctly you'll get to use all the new shiny
> > features you want on a more recent kernel, and none of the things you don't
> > want. So yes, you're upgrading to a newer kernel all the time, but if I
> > understant your use-case right it shouldn't matter too much, more so if
> > you're already taking chances on backporting major features yourself.
> 
> Like I say in this case updating to a newer kernel also means rebasing
> the out of tree patch stack and taking a bunch of test risk from that -
> in product development for the sorts of products that end up including
> the LSK the churn and risk from targeted backports is seen as much safer
> than updating to an entire new upstream kernel.

Same as I said before, the risk LSK introduces, IMO, is much greater than
rebasing and out-of-tree driver stack.

-- 

Thanks,
Sasha