Re: [Ksummit-discuss] [MAINTAINER SUMMIT] Distribution kernel bugzillas considered harmful

[Takashi Iwai <tiwai@suse.de>]

On Tue, 18 Sep 2018 15:43:08 +0200,
Martin K. Petersen wrote:
> 
> 
> Hi Jiri,
> 
> Sorry I'm a little late to the game here. Been out on vacation.
> 
> > We order patches in our trees in the same git-topological-ordering as they 
> > are upstream. It has a lot of benefits, most importantly: it doesn't 
> > introduce artificial conflicts that don't exist in reality.
> >
> > In order to achieve that, we of course need 1:1 mapping between our
> > patches and upstream commits.  Rebases destroy that mapping.
> >
> > And in some areas (scsi is one, but not the only one), we basically had no 
> > other choice than considering maintainer's tree to be already "upstream 
> > enough", without waiting for Linus' tree merge.
> 
> When I discussed this with Johannes a little while ago, I suggested you
> guys used git patch-id to track patches instead of commit ids. That's
> how we track patches applied across many different trees internally.
> Works much better than using the upstream sha.

What is git patch id?  Is there any internal patch tag?

> I would like to understand your "upstream enough" requirement. Why do
> you need a tree that's stable before Linus pulls the changes?

A few reasons:
- We'd like to sort also the multiple would-be-upstream patches in the
  git topo order, which allows us to keep the patches closer to the
  original shape.

- Once when such patches are merged to upstream, they are moved to the
  upstream section, again ordered.  For moving them, of course it'd be
  easy if the commit ids are kept.

And here the worst case is that the old commit ID disappears after the
rebase.  In that sense, at least, it'd be helpful if you can preserve
the old branches via tag or another named branch in the repo; then the
commits are not pruned, and we can still track the order of patches.

> Note that I am generally only rebasing as a last resort and typically
> only very early in the rc cycle. It usually happens when I need to drop
> a patch series that turned out to be unfixable in its current state.

... and we shouldn't take such a patch series always as
"would-be-upstream" thing, either.

So it sounds like a sort of lack of communication to me; you might be
interested in that a certain patch series are already pushed to distro
kernels, and we are definitely interested in whether the patches are
merged or dropped.  Hmm...


Takashi

> And before everyone screams because I'm not supposed to be pushing stuff
> that breaks, please realize that it is impossible to test all the
> different types of hardware I have to merge drivers for. There is no
> regression test suite or lab setup with anything resembling
> comprehensive coverage. I test changes to the SCSI core code and do some
> rudimentary testing on SAS and FC on x86_64. But that's really the best
> I can do.
> 
> Even though most patches posted to linux-scsi get picked up by 0day,
> more often than not they only get x86_64 build coverage. Whereas 0day
> build failures on arm, mips, sparc32, whatever typically only get
> reported after patches have been simmering in linux-next for a
> while. Depends how busy 0day is.
> 
> Also, actual driver failures on platforms not officially supported and
> tested by the controller vendor are only found after the fact. And most
> of the time it's not a matter of reverting a single patch but
> effectively dropping all of the patches in the series until they can be
> reworked. Sometimes a workaround takes a week or two to deliver, and
> people don't appreciate not being able to boot their systems in the
> meantime. So that's why I generally drop the series instead.
> 
> I would love for every patch sent to linux-scsi to be bug free and
> instantly build tested by 0day on every architecture.  And I would love
> for hardware vendors to be more cognizant about architectures they don't
> commercially support.  But reality is that things break frequently when
> I merge big, complex driver update patch series.
> 
> As a result, the preference has been to have the flexibility to amend or
> drop patches early in every cycle. It hasn't really been a problem
> because there have been no downstream users of SCSI at all. I only
> recently found out about your use case.
> 
> I'm pretty flexible about how to address this, there are a couple of
> ways to go about it.
> 
> 1. I could just always revert instead of dropping the patches. The
>    downside is that we end up with a pretty messy history because, as I
>    pointed out above, it's usually a matter of dropping tens of patches
>    at a time and not reverting a single offending commit. In addition,
>    having a messy history makes it harder on distro kernel people to
>    track driver updates.
> 
> 2. The other option is that I set up a scsi-staging tree where stuff can
>    simmer for a bit and hopefully get some 0day coverage before getting
>    shuffled over to scsi-queue. However, I do question how much actual
>    real hardware coverage we'll get by having a SCSI tree that people
>    would explicitly have to pull to test. As opposed to linux-next which
>    at least gets some coverage by test farms and users.
> 
> 3. The third option is that we pick a number like rc4 and I will promise
>    not to rebase after that. We can see how that works out.
> 
> I'm open to ideas. The most important thing to me is that 0day and
> linux-next are indispensable tools in my workflow given that I have no
> way to personally test most of the code I merge. So it is imperative
> that I have the ability to push code early because that's where I get
> the bulk of my (non-SCSI-core) build and test coverage.
> 
> -- 
> Martin K. Petersen	Oracle Linux Engineering
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>