[Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

A lot of the problems accepting patches into the kernel stem from trying
to get them reviewed ... some patch series even go for months without
being reviewed.  This is caused by a couple of problems.  The first one
is a submission issue, where you get a long N patch series (usually N >
10) you review and provide feedback, then you get a long N+Y series back
eventually with no indication what disposition was taken on any of the
review points ... even the most hardy reviewers get a bit demotivated at
this point. The second is an actual lack of reviewers in particular
fields, meaning that a small number of people tend to be the ones
reviewing patches in particular subsystems.

The former is probably just an addition to SubmittingPatches explaining
how to resend after addressing review comments.  The latter was supposed
to be helped by having the Reviewed-by: tag so we gave credit to
reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
sword: it is a good way of giving review credits, but I also see patches
that come in initially with it on (usually the signoff and the
reviewed-by are from people in the same company) ... it's not
necessarily a bad thing, but it doesn't add much value to the kernel
review process, because we're looking for independent reviews.  The
other thing I find problematic is that some people respond to a patch
with a Reviewed-by: tag and nothing more.  I'm really looking for
evidence of actually having read (and understood) the patch, so the best
review usually comes with a sequence of comments, questions and minor
nits and a reviewed-by at the end.

However, the fundamental problem is we still need to increase our review
pool, so we need more than the Reviewed-by: tag.  It has been suggested
that people who submit patches should also be required to review patches
from other people ... I think that's a bit drastic, but it is a
possibility. Another possibility for the kernel summit might be to have
review time instead of hack time: we each nominate a patch set needing
review and it gets reviewed by others in that time ... if that one's
successful, we could extend the concept to other conferences and
development sprints which do hack time.  Another possibility is to
publish a subsystem to review list (similar to the to do list idea).
This at least shows submitters why their patch series is languishing and
could serve as a call to arms if it gets too long.

I'm sure there are many other things people could suggest.

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]


> A lot of the problems accepting patches into the kernel stem from trying
> to get them reviewed

So true.

> I'm really looking for evidence of actually having read (and
> understood) the patch, so the best review usually comes with a
> sequence of comments, questions and minor nits and a reviewed-by at
> the end.

Yes, this is good (and as much needed as describing the tests done for a
patch by the submitter). Still, trust is an issue here. Even with
comments you described above, if it is from a person you don't know the
review needs to be reviewed. Depending on the reviewer that might take
more time than to do the review on your own. Which pays off if the
person stays. If not, well, then I still see this as something a
maintainer should do, simply to educate people. It just might not help
getting patches reviewed faster.

> review pool, so we need more than the Reviewed-by: tag.  It has been
> suggested that people who submit patches should also be required to
> review patches from other people

Eeks. Enforced reviews are likely to be sloppy IMO. And sloppy reviews
can easily cause more work, just like sloppy documentation.

> conferences and development sprints which do hack time.  Another
> possibility is to publish a subsystem to review list (similar to the
> to do list idea). This at least shows submitters why their patch
> series is languishing and could serve as a call to arms if it gets too
> long.

For those using patchwork, such lists are on the web and referenced in
MAINTAINERS. Submitters don't use it much (yet), according to my
experiences.

The thing I'd like to see way more in the Linux ecosystem:

Paid reviewers/maintainers (selected people, no hiring offers). The
number of developers increases faster than the number of quality
keepers. So, the latter should be given the chance to focus on it, if
they want to.

   Wolfram


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Williams]

On Sat, May 24, 2014 at 2:53 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> A lot of the problems accepting patches into the kernel stem from trying
> to get them reviewed ... some patch series even go for months without
> being reviewed.  This is caused by a couple of problems.  The first one
> is a submission issue, where you get a long N patch series (usually N >
> 10) you review and provide feedback, then you get a long N+Y series back
> eventually with no indication what disposition was taken on any of the
> review points ... even the most hardy reviewers get a bit demotivated at
> this point. The second is an actual lack of reviewers in particular
> fields, meaning that a small number of people tend to be the ones
> reviewing patches in particular subsystems.
>
> The former is probably just an addition to SubmittingPatches explaining
> how to resend after addressing review comments.  The latter was supposed
> to be helped by having the Reviewed-by: tag so we gave credit to
> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> sword: it is a good way of giving review credits, but I also see patches
> that come in initially with it on (usually the signoff and the
> reviewed-by are from people in the same company) ... it's not
> necessarily a bad thing, but it doesn't add much value to the kernel
> review process, because we're looking for independent reviews.  The
> other thing I find problematic is that some people respond to a patch
> with a Reviewed-by: tag and nothing more.  I'm really looking for
> evidence of actually having read (and understood) the patch, so the best
> review usually comes with a sequence of comments, questions and minor
> nits and a reviewed-by at the end.
>
> However, the fundamental problem is we still need to increase our review
> pool, so we need more than the Reviewed-by: tag.  It has been suggested
> that people who submit patches should also be required to review patches
> from other people ... I think that's a bit drastic, but it is a
> possibility. Another possibility for the kernel summit might be to have
> review time instead of hack time: we each nominate a patch set needing
> review and it gets reviewed by others in that time ... if that one's
> successful, we could extend the concept to other conferences and
> development sprints which do hack time.  Another possibility is to
> publish a subsystem to review list (similar to the to do list idea).
> This at least shows submitters why their patch series is languishing and
> could serve as a call to arms if it gets too long.

Mandatory patchwork?  If no one can perceive the patch queue depth how
do they know where to help?  Would also be nice if patchwork had
states like "needs review from another fibre channel developer" to
make it explicit the class of review that is being sought from the
maintainer.

> I'm sure there are many other things people could suggest.

Forgive the co-opt / tie-in to my own topic proposal, but I think one
way to increase review is to do it after the code is merged.  Quality
reviews tend to also show up after bug reports.  So, maybe finding a
way to get more code in people's hands, in a responsible way, also
increases the quantity and quality of reviews.

--
Dan

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Trond Myklebust]

On Sat, May 24, 2014 at 5:53 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> A lot of the problems accepting patches into the kernel stem from trying
> to get them reviewed ... some patch series even go for months without
> being reviewed.  This is caused by a couple of problems.  The first one
> is a submission issue, where you get a long N patch series (usually N >
> 10) you review and provide feedback, then you get a long N+Y series back
> eventually with no indication what disposition was taken on any of the
> review points ... even the most hardy reviewers get a bit demotivated at
> this point. The second is an actual lack of reviewers in particular
> fields, meaning that a small number of people tend to be the ones
> reviewing patches in particular subsystems.
>
> The former is probably just an addition to SubmittingPatches explaining
> how to resend after addressing review comments.  The latter was supposed
> to be helped by having the Reviewed-by: tag so we gave credit to
> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> sword: it is a good way of giving review credits, but I also see patches
> that come in initially with it on (usually the signoff and the
> reviewed-by are from people in the same company) ... it's not
> necessarily a bad thing, but it doesn't add much value to the kernel
> review process, because we're looking for independent reviews.  The
> other thing I find problematic is that some people respond to a patch
> with a Reviewed-by: tag and nothing more.  I'm really looking for
> evidence of actually having read (and understood) the patch, so the best
> review usually comes with a sequence of comments, questions and minor
> nits and a reviewed-by at the end.
>
> However, the fundamental problem is we still need to increase our review
> pool, so we need more than the Reviewed-by: tag.  It has been suggested
> that people who submit patches should also be required to review patches
> from other people ... I think that's a bit drastic, but it is a
> possibility. Another possibility for the kernel summit might be to have
> review time instead of hack time: we each nominate a patch set needing
> review and it gets reviewed by others in that time ... if that one's
> successful, we could extend the concept to other conferences and
> development sprints which do hack time.  Another possibility is to
> publish a subsystem to review list (similar to the to do list idea).
> This at least shows submitters why their patch series is languishing and
> could serve as a call to arms if it gets too long.


Hi James,

You are assuming that organising reviews of patches is the
responsibility of the maintainer.
In my opinion, it should rather be the responsibility of the submitter
to do so as part of the process of rallying support for what they are
proposing. It is fine for the maintainer to assist that process, to
set parameters around it (e.g. how many reviews are required before it
can be merged), and to act as one of the reviewers, but there is no
reason why the maintainer must be the one to drive the process.

I agree that ensuring quality of review is difficult. Part of the
problem there is that our toolchain has nothing to capture reviews,
and to preserve them for posterity other than the mailing lists. The
best you can do then is to use 'Link:' in order to tag the review
thread in the commit log. It would be nice to have a better solution.
Patchworks is one option, but it doesn't really work well for capture
the full process if the submitter pushes out a new patchset in
response to a review (you end up starting afresh).
Bugzillas are another option, but their main task isn't really to
preserve patch reviews, and furthermore, they are less well adapted to
our working habits (which has lead to pushback from a number of
maintainers).
Other options?

Cheers
  Trond
-- 
Trond Myklebust

Linux NFS client maintainer, PrimaryData

trond.myklebust@primarydata.com

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Sat, 2014-05-24 at 11:50 -0400, Trond Myklebust wrote:
> On Sat, May 24, 2014 at 5:53 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > A lot of the problems accepting patches into the kernel stem from trying
> > to get them reviewed ... some patch series even go for months without
> > being reviewed.  This is caused by a couple of problems.  The first one
> > is a submission issue, where you get a long N patch series (usually N >
> > 10) you review and provide feedback, then you get a long N+Y series back
> > eventually with no indication what disposition was taken on any of the
> > review points ... even the most hardy reviewers get a bit demotivated at
> > this point. The second is an actual lack of reviewers in particular
> > fields, meaning that a small number of people tend to be the ones
> > reviewing patches in particular subsystems.
> >
> > The former is probably just an addition to SubmittingPatches explaining
> > how to resend after addressing review comments.  The latter was supposed
> > to be helped by having the Reviewed-by: tag so we gave credit to
> > reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> > sword: it is a good way of giving review credits, but I also see patches
> > that come in initially with it on (usually the signoff and the
> > reviewed-by are from people in the same company) ... it's not
> > necessarily a bad thing, but it doesn't add much value to the kernel
> > review process, because we're looking for independent reviews.  The
> > other thing I find problematic is that some people respond to a patch
> > with a Reviewed-by: tag and nothing more.  I'm really looking for
> > evidence of actually having read (and understood) the patch, so the best
> > review usually comes with a sequence of comments, questions and minor
> > nits and a reviewed-by at the end.
> >
> > However, the fundamental problem is we still need to increase our review
> > pool, so we need more than the Reviewed-by: tag.  It has been suggested
> > that people who submit patches should also be required to review patches
> > from other people ... I think that's a bit drastic, but it is a
> > possibility. Another possibility for the kernel summit might be to have
> > review time instead of hack time: we each nominate a patch set needing
> > review and it gets reviewed by others in that time ... if that one's
> > successful, we could extend the concept to other conferences and
> > development sprints which do hack time.  Another possibility is to
> > publish a subsystem to review list (similar to the to do list idea).
> > This at least shows submitters why their patch series is languishing and
> > could serve as a call to arms if it gets too long.
> 
> 
> Hi James,
> 
> You are assuming that organising reviews of patches is the
> responsibility of the maintainer.

Actually, I don't believe I said that anywhere ... however, I do see, at
least in my subsystem, that the Maintainer is the reviewer of last
resort, so my selfish reason for wanting more reviewers is to offload
that from me.

> In my opinion, it should rather be the responsibility of the submitter
> to do so as part of the process of rallying support for what they are
> proposing. It is fine for the maintainer to assist that process, to
> set parameters around it (e.g. how many reviews are required before it
> can be merged), and to act as one of the reviewers, but there is no
> reason why the maintainer must be the one to drive the process.

Sure, this is all fine.  I think the dynamic will depend on the
subsystem anyway.

> I agree that ensuring quality of review is difficult. Part of the
> problem there is that our toolchain has nothing to capture reviews,
> and to preserve them for posterity other than the mailing lists. The
> best you can do then is to use 'Link:' in order to tag the review
> thread in the commit log.

Heh, perhaps, at last, we might have found a use for git notes.

>  It would be nice to have a better solution.
> Patchworks is one option, but it doesn't really work well for capture
> the full process if the submitter pushes out a new patchset in
> response to a review (you end up starting afresh).

I've not really tried patchwork, so can't comment

> Bugzillas are another option, but their main task isn't really to
> preserve patch reviews, and furthermore, they are less well adapted to
> our working habits (which has lead to pushback from a number of
> maintainers).
> Other options?

Well, the other obvious one is gerrit, but I'm really not keen on it ...
it provides a nice log of the reviews, but it's set up around the
process of owning the git tree as well.

However, I don't really think better tools would help us grow
reviewers ... OpenStack has exactly the same review bottleneck problem
we have and they already use a full Gerrit workflow.

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Guenter Roeck]

>
> The thing I'd like to see way more in the Linux ecosystem:
>
> Paid reviewers/maintainers (selected people, no hiring offers). The
> number of developers increases faster than the number of quality
> keepers. So, the latter should be given the chance to focus on it, if
> they want to.
>

Problem with that is that in most company hierarchies code reviewers
get little  if no credit for their work. If anything, I have seen
the opposite - code reviewers, if they take their responsibility
serious, end up getting blamed for project delays because they keep
finding problems in the code.

Imagine a project where one employee writes the code and another
reviews it. Who do you think will get the credit (and bonus) ?
I bet it will be the person who wrote the code, not the person
who made sure that it is clean and free of bugs.

Guenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Bjorn Helgaas]

On Sat, May 24, 2014 at 11:31 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Sat, 2014-05-24 at 11:50 -0400, Trond Myklebust wrote:
>> On Sat, May 24, 2014 at 5:53 AM, James Bottomley
>> <James.Bottomley@hansenpartnership.com> wrote:
>> > A lot of the problems accepting patches into the kernel stem from trying
>> > to get them reviewed ... some patch series even go for months without
>> > being reviewed.  This is caused by a couple of problems.  The first one
>> > is a submission issue, where you get a long N patch series (usually N >
>> > 10) you review and provide feedback, then you get a long N+Y series back
>> > eventually with no indication what disposition was taken on any of the
>> > review points ... even the most hardy reviewers get a bit demotivated at
>> > this point. The second is an actual lack of reviewers in particular
>> > fields, meaning that a small number of people tend to be the ones
>> > reviewing patches in particular subsystems.

Reviewing is a big problem for me, as evidenced by my long backlog of
PCI patches.

When people are adding brand-new functionality, e.g., a new host
bridge driver, sometimes several people are interested in it, and
people in that group tend to review each other's stuff.  That's good,
of course.

But the bigger problem for me is reviewing changes to the core
architecture.  Usually there aren't very many people interested in
those, and at the same time, they have a much broader impact, so I
feel like I have to vet them much more carefully.

Technical debt also makes it harder to work in the core than in a
brand-new driver.  We often have changes that seemed to solve a
problem at the time, but later turned out to be band-aids that only
covered it up.  This sort of stuff accumulates and makes future work
harder.  There's a lot of history that has to be understood before
making changes, and usually you can't test to make sure a refactoring
doesn't reintroduce a problem.

I guess I'm just agreeing that a lack of reviewers is a problem.  I
don't know how to solve it.  I do think that if core code were cleaner
and more approachable, people would be more inclined to read it and
work on it.

>> I agree that ensuring quality of review is difficult. Part of the
>> problem there is that our toolchain has nothing to capture reviews,
>> and to preserve them for posterity other than the mailing lists. The
>> best you can do then is to use 'Link:' in order to tag the review
>> thread in the commit log.

I haven't found preservation of reviews to be a big problem for me.
Maybe this is because PCI reviews are usually two-party conversations
(just the author and me), and I just keep them forever in my gmail
archives.

> Heh, perhaps, at last, we might have found a use for git notes.
>
>>  It would be nice to have a better solution.
>> Patchworks is one option, but it doesn't really work well for capture
>> the full process if the submitter pushes out a new patchset in
>> response to a review (you end up starting afresh).

To the extent that archived reviews are useful, I think it's good that
they're in the mailing list archives on the web rather than buried in
a tool like git notes or patchwork.

> I've not really tried patchwork, so can't comment
>
>> Bugzillas are another option, but their main task isn't really to
>> preserve patch reviews, and furthermore, they are less well adapted to
>> our working habits (which has lead to pushback from a number of
>> maintainers).
>> Other options?
>
> Well, the other obvious one is gerrit, but I'm really not keen on it ...
> it provides a nice log of the reviews, but it's set up around the
> process of owning the git tree as well.
>
> However, I don't really think better tools would help us grow
> reviewers ... OpenStack has exactly the same review bottleneck problem
> we have and they already use a full Gerrit workflow.

I agree.  I use gerrit internally, and I don't care for it.  It's sort
of OK for minor checkpatch-type issues, but it tends to fragment
commentary into disconnected little pieces.  I feel like I'm always
clicking something and typing into tiny little text boxes.

Most of my reviewing effort goes into researching the code, reading
specs, searching the web for similar issues, formulating alternatives,
and writing responses.  Tools like patchwork and gerrit help with
organizational stuff (and I do use patchwork as a to-do list), but in
my experience they don't help at all with the activities where I spend
the bulk of my time.

Better tools are always nice, but I agree they won't address the "lack
of reviewers" problem.  I'm a little concerned that the real problem
is that the complexity of the system makes it unapproachable except to
the few who can afford to work in one area full-time (I'm talking
about the complexity of accumulated warts, not the unavoidable
complexity of the hardware itself).

Bjorn

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]

Hi all,

On Sat, 24 May 2014 13:53:45 +0400 James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
>
> The latter was supposed
> to be helped by having the Reviewed-by: tag so we gave credit to
> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> sword: it is a good way of giving review credits, but I also see patches
> that come in initially with it on (usually the signoff and the
> reviewed-by are from people in the same company) ... it's not
> necessarily a bad thing, but it doesn't add much value to the kernel
> review process, because we're looking for independent reviews.  The
> other thing I find problematic is that some people respond to a patch
> with a Reviewed-by: tag and nothing more.  I'm really looking for
> evidence of actually having read (and understood) the patch, so the best
> review usually comes with a sequence of comments, questions and minor
> nits and a reviewed-by at the end.

Some stats (I know you all love stats :-)):

for next-20140523, no merge commits, origin/master..HEAD^ (exclude
Linus' tree and my Next files commit)

commits: 7717
commits with more than one Signed-off-by: 6291
commits with Reviewed-by: 1369
commits with Tested-by: 354

Not sure what these show ...
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

On Sat, 24 May 2014 12:18:42 -0700 Guenter Roeck <linux@roeck-us.net> wrote:

> >
> > The thing I'd like to see way more in the Linux ecosystem:
> >
> > Paid reviewers/maintainers (selected people, no hiring offers). The
> > number of developers increases faster than the number of quality
> > keepers. So, the latter should be given the chance to focus on it, if
> > they want to.
> >
> 
> Problem with that is that in most company hierarchies code reviewers
> get little  if no credit for their work. If anything, I have seen
> the opposite - code reviewers, if they take their responsibility
> serious, end up getting blamed for project delays because they keep
> finding problems in the code.
> 
> Imagine a project where one employee writes the code and another
> reviews it. Who do you think will get the credit (and bonus) ?
> I bet it will be the person who wrote the code, not the person
> who made sure that it is clean and free of bugs.
> 

Sounds like a job for the Linux Foundation (easy for me to say ....).

Get funding and/or secondment from members and use it to appoint reviewers.
Their role and remuneration would be focussed (solely)
on review-for-upstream-acceptance.

I'm sure there are people who can do the work, and probably even some who
enjoy doing the work.  So I assume that they aren't given enough time to do
that work.  And time == money.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Sat, 2014-05-24 at 12:18 -0700, Guenter Roeck wrote:
> >
> > The thing I'd like to see way more in the Linux ecosystem:
> >
> > Paid reviewers/maintainers (selected people, no hiring offers). The
> > number of developers increases faster than the number of quality
> > keepers. So, the latter should be given the chance to focus on it, if
> > they want to.
> >
> 
> Problem with that is that in most company hierarchies code reviewers
> get little  if no credit for their work.

I could see this in start up type companies.  Older companies learned
long ago that customers value quality over features so they tend to have
elaborate review processes. (As an aside, customers say they value
features, but if you deliver one with a regression, it's the regression
you'll hear about the whole time).

>  If anything, I have seen
> the opposite - code reviewers, if they take their responsibility
> serious, end up getting blamed for project delays because they keep
> finding problems in the code.

I've worked for a couple of large companies over my career (and a few
start ups).  I've got to say that's not my experience.  I've always seen
us blame the submitter for bad code, not the reviewer.

> Imagine a project where one employee writes the code and another
> reviews it. Who do you think will get the credit (and bonus) ?
> I bet it will be the person who wrote the code, not the person
> who made sure that it is clean and free of bugs.

This is certainly true that credit goes to features.  However, if your
company only incents that way, QA rapidly gets disillusioned, so only
giving credit to features wouldn't work long term which is why no
company I know does it.

To give a counter point: every product we produce has defect metrics and
I've seen QA get all the prizes in the case where the initial submit was
too buggy and they turned around the reviews and tests fast enough to
meet the shipping deadlines and reduce the defects to within the
metrics.

In all things in life, it's a balance.  I've seen cockups where QA is
solely incented on defects found and minor UI bugs get classified as
critical feature defects (because that's what gets the bonus).

But anyway, back to the problem at hand, I think you're suggesting that
paying for reviews might not work, and I think I agree because it's back
to incenting QA solely on finding defects.  However, if others thought
there was merit, we might persuade the LF to offer a small incentive.

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Geert Uytterhoeven]

Hi Stephen,

On Sun, May 25, 2014 at 6:17 AM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> On Sat, 24 May 2014 13:53:45 +0400 James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
>>
>> The latter was supposed
>> to be helped by having the Reviewed-by: tag so we gave credit to
>> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
>> sword: it is a good way of giving review credits, but I also see patches
>> that come in initially with it on (usually the signoff and the
>> reviewed-by are from people in the same company) ... it's not
>> necessarily a bad thing, but it doesn't add much value to the kernel
>> review process, because we're looking for independent reviews.  The
>> other thing I find problematic is that some people respond to a patch
>> with a Reviewed-by: tag and nothing more.  I'm really looking for
>> evidence of actually having read (and understood) the patch, so the best
>> review usually comes with a sequence of comments, questions and minor
>> nits and a reviewed-by at the end.
>
> Some stats (I know you all love stats :-)):
>
> for next-20140523, no merge commits, origin/master..HEAD^ (exclude
> Linus' tree and my Next files commit)
>
> commits: 7717
> commits with more than one Signed-off-by: 6291
> commits with Reviewed-by: 1369
> commits with Tested-by: 354
>
> Not sure what these show ...

Thanks for the numbers!

How many Acked-by? Sometimes there's only a thin line between Acked-by
and Reviewed-by.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]


> Problem with that is that in most company hierarchies code reviewers
> get little  if no credit for their work.

I was more aiming for upstream reviewers being paid for upstream review.
And not so much aiming for work on company level. Without knowing
details, GregKH being hired by the Linux Foundation looks like a good
example to me. There are so many $$$ around in the Linux market, there
should be some left to keep (and better: improve) the quality we all
appreciate.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]


> Some stats (I know you all love stats :-)):

Some more stats here ;)

http://elinux.org/images/b/b0/Sang-ELCE2013_WolframSang_WeHaveAScalingProblem.pdf

One thing that you can see there is that the number of authors increases
faster than the number of reviewers and committers (= gatekeepers/maintainers).
So, IMO the situation is likely to get worse.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

Hi Geert,

On Sun, 25 May 2014 10:53:11 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> On Sun, May 25, 2014 at 6:17 AM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> >
> > Some stats (I know you all love stats :-)):
> >
> > for next-20140523, no merge commits, origin/master..HEAD^ (exclude
> > Linus' tree and my Next files commit)
> >
> > commits: 7717
> > commits with more than one Signed-off-by: 6291
> > commits with Reviewed-by: 1369
> > commits with Tested-by: 354
> >
> > Not sure what these show ...
> 
> Thanks for the numbers!
> 
> How many Acked-by? Sometimes there's only a thin line between Acked-by
> and Reviewed-by.

commits with Acked-by: 1192

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Carpenter]

I think SCSI is almost uniquely difficult for this.  The drivers are so
big and different from each other.  Competitors aren't going to review
each other's code.

With almost every other subsystem, there is a second in command who
could take over if needed.  I don't know anyone who could do your job if
you decided to go back to Cambridge for an MBA.  I did a:

git log --after=2013-01-01 drivers/scsi | grep '\-by\:' | sort | uniq -c | sort -rn

There are some smart people who work on vendor code but everyone seems
focussed on their own code.

regards,
dan carpenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

Hi James,

On Saturday, May 24, 2014 01:53:45 PM James Bottomley wrote:
> A lot of the problems accepting patches into the kernel stem from trying
> to get them reviewed ... some patch series even go for months without
> being reviewed.  This is caused by a couple of problems.  The first one
> is a submission issue, where you get a long N patch series (usually N >
> 10) you review and provide feedback, then you get a long N+Y series back
> eventually with no indication what disposition was taken on any of the
> review points ... even the most hardy reviewers get a bit demotivated at
> this point. The second is an actual lack of reviewers in particular
> fields, meaning that a small number of people tend to be the ones
> reviewing patches in particular subsystems.

I agree that both are problems currently.

I'm seeing an interesting twist of the second one, though, for example in
the cpufreq subsystem, where there is a developer who reviews the others'
patches, but then his patches often go unreviewed by anyone (except for me
when I need to decide whether or not to apply them).

> The former is probably just an addition to SubmittingPatches explaining
> how to resend after addressing review comments.  The latter was supposed
> to be helped by having the Reviewed-by: tag so we gave credit to
> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> sword: it is a good way of giving review credits, but I also see patches
> that come in initially with it on (usually the signoff and the
> reviewed-by are from people in the same company) ... it's not
> necessarily a bad thing, but it doesn't add much value to the kernel
> review process, because we're looking for independent reviews.

Well, the fact that the review is internal doesn't necessarily imply that it
is not independent.

> The other thing I find problematic is that some people respond to a patch
> with a Reviewed-by: tag and nothing more.  I'm really looking for
> evidence of actually having read (and understood) the patch, so the best
> review usually comes with a sequence of comments, questions and minor
> nits and a reviewed-by at the end.

There is some confusion about when to use Reviewd-by and when to use Acked-by.
I prefer people to say Acked-by if the review is not in-dpeth.  Reviewed-by
should always imply "I understand what the change does and don't see any
problems with it" in my opinion, while Acked-by is something like "that's
fine by me" (that may mean quite a lot when a maintainer says so, but not
in general).

It would be good to clarify that somehow in the documentation this way or
another.

> However, the fundamental problem is we still need to increase our review
> pool, so we need more than the Reviewed-by: tag.  It has been suggested
> that people who submit patches should also be required to review patches
> from other people ... I think that's a bit drastic, but it is a
> possibility. Another possibility for the kernel summit might be to have
> review time instead of hack time: we each nominate a patch set needing
> review and it gets reviewed by others in that time ... if that one's
> successful, we could extend the concept to other conferences and
> development sprints which do hack time.

I like this idea, it's worth trying anyway in my opinion.

> Another possibility is to
> publish a subsystem to review list (similar to the to do list idea).
> This at least shows submitters why their patch series is languishing and
> could serve as a call to arms if it gets too long.
> 
> I'm sure there are many other things people could suggest.

Using Patchwork helps a bit in that area (if people know that it is used
actually), because submitters can go there and check the status and see if
anyone has commented their patches.  If the status is "New" and there are
no comments, that pretty much indicates "unreviewed yet". :-)

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Sunday, May 25, 2014 10:59:09 AM Wolfram Sang wrote:
> 
> > Problem with that is that in most company hierarchies code reviewers
> > get little  if no credit for their work.
> 
> I was more aiming for upstream reviewers being paid for upstream review.
> And not so much aiming for work on company level. Without knowing
> details, GregKH being hired by the Linux Foundation looks like a good
> example to me. There are so many $$$ around in the Linux market, there
> should be some left to keep (and better: improve) the quality we all
> appreciate.

There is one more angle to that.  Namely, it would be good if at least some
reviewers were seen as independent of any particular company.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Saturday, May 24, 2014 07:24:15 AM Dan Williams wrote:
> On Sat, May 24, 2014 at 2:53 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > A lot of the problems accepting patches into the kernel stem from trying
> > to get them reviewed ... some patch series even go for months without
> > being reviewed.  This is caused by a couple of problems.  The first one
> > is a submission issue, where you get a long N patch series (usually N >
> > 10) you review and provide feedback, then you get a long N+Y series back
> > eventually with no indication what disposition was taken on any of the
> > review points ... even the most hardy reviewers get a bit demotivated at
> > this point. The second is an actual lack of reviewers in particular
> > fields, meaning that a small number of people tend to be the ones
> > reviewing patches in particular subsystems.
> >
> > The former is probably just an addition to SubmittingPatches explaining
> > how to resend after addressing review comments.  The latter was supposed
> > to be helped by having the Reviewed-by: tag so we gave credit to
> > reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
> > sword: it is a good way of giving review credits, but I also see patches
> > that come in initially with it on (usually the signoff and the
> > reviewed-by are from people in the same company) ... it's not
> > necessarily a bad thing, but it doesn't add much value to the kernel
> > review process, because we're looking for independent reviews.  The
> > other thing I find problematic is that some people respond to a patch
> > with a Reviewed-by: tag and nothing more.  I'm really looking for
> > evidence of actually having read (and understood) the patch, so the best
> > review usually comes with a sequence of comments, questions and minor
> > nits and a reviewed-by at the end.
> >
> > However, the fundamental problem is we still need to increase our review
> > pool, so we need more than the Reviewed-by: tag.  It has been suggested
> > that people who submit patches should also be required to review patches
> > from other people ... I think that's a bit drastic, but it is a
> > possibility. Another possibility for the kernel summit might be to have
> > review time instead of hack time: we each nominate a patch set needing
> > review and it gets reviewed by others in that time ... if that one's
> > successful, we could extend the concept to other conferences and
> > development sprints which do hack time.  Another possibility is to
> > publish a subsystem to review list (similar to the to do list idea).
> > This at least shows submitters why their patch series is languishing and
> > could serve as a call to arms if it gets too long.
> 
> Mandatory patchwork?  If no one can perceive the patch queue depth how
> do they know where to help?  Would also be nice if patchwork had
> states like "needs review from another fibre channel developer" to
> make it explicit the class of review that is being sought from the
> maintainer.

I agree that adding something like this to Patchwork would be a good idea (just
another status value, like "Review Required" should be sufficient), but making
it mandatory is a bit over the top to me.

> > I'm sure there are many other things people could suggest.
> 
> Forgive the co-opt / tie-in to my own topic proposal, but I think one
> way to increase review is to do it after the code is merged.

Excuse me, but who's going to have the incentive then?  And what exactly is
the incentive going to be?

> Quality reviews tend to also show up after bug reports.

I wouldn't call that "reviews", honestly, but rather code analyses.  Yes, bug
reports often cause people to analyse the code more in depth and understand it
better, but confusing after-the-fact analysis with review is like confusing
prevention with healing a disease that's already developed.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Saturday, May 24, 2014 10:15:52 PM Bjorn Helgaas wrote:
> On Sat, May 24, 2014 at 11:31 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:

[cut]

> Better tools are always nice, but I agree they won't address the "lack
> of reviewers" problem.  I'm a little concerned that the real problem
> is that the complexity of the system makes it unapproachable except to
> the few who can afford to work in one area full-time (I'm talking
> about the complexity of accumulated warts, not the unavoidable
> complexity of the hardware itself).

I agree.

If a change is deep enough, you need to know a lot about the code that's
already there to review it, and not just about one subsystem usually, and you
also need to know *why* it is there which in many cases is undocumented and may
be specific to certain special cases.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

On Mon, May 26, 2014 at 02:23:50PM +0200, Rafael J. Wysocki wrote:
> On Sunday, May 25, 2014 10:59:09 AM Wolfram Sang wrote:
> > 
> > > Problem with that is that in most company hierarchies code reviewers
> > > get little  if no credit for their work.
> > 
> > I was more aiming for upstream reviewers being paid for upstream review.
> > And not so much aiming for work on company level. Without knowing
> > details, GregKH being hired by the Linux Foundation looks like a good
> > example to me. There are so many $$$ around in the Linux market, there
> > should be some left to keep (and better: improve) the quality we all
> > appreciate.
> 
> There is one more angle to that.  Namely, it would be good if at least some
> reviewers were seen as independent of any particular company.

Agreed. This would be easiest, without friction. However, with enough
credibility, this becomes less important IMO. I'd name akpm here, for
example.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Guenter Roeck]

On 05/24/2014 09:57 PM, James Bottomley wrote:
> On Sat, 2014-05-24 at 12:18 -0700, Guenter Roeck wrote:
>>>
>>> The thing I'd like to see way more in the Linux ecosystem:
>>>
>>> Paid reviewers/maintainers (selected people, no hiring offers). The
>>> number of developers increases faster than the number of quality
>>> keepers. So, the latter should be given the chance to focus on it, if
>>> they want to.
>>>
>>
>> Problem with that is that in most company hierarchies code reviewers
>> get little  if no credit for their work.
>
> I could see this in start up type companies.  Older companies learned
> long ago that customers value quality over features so they tend to have
> elaborate review processes. (As an aside, customers say they value
> features, but if you deliver one with a regression, it's the regression
> you'll hear about the whole time).
>

I am not sure if all those companies learned the lesson. Agreed, many
of them do, but I have seen the opposite. But that is not really
the point here.

You can actually take the Linux kernel at a case in point: Let's assume
someone wants to hire a kernel engineer and looks up kernel commits for
reference. What do you think that person will look for ? Patch authors
or  "Reviewed-by" tags ? I would argue it is going to be patch authors.

Really, again, the point (or question) here is how much credit an engineer
gets for doing code reviews (or fixing bugs, for that matter) vs. for
writing code. I would argue that there is very little incentive for
senior engineers (ie those who are best suited to do code reviews)
to actually _do_ code reviews more or less for a living, or at least
for a substantial amount of their time.

>>   If anything, I have seen
>> the opposite - code reviewers, if they take their responsibility
>> serious, end up getting blamed for project delays because they keep
>> finding problems in the code.
>
> I've worked for a couple of large companies over my career (and a few
> start ups).  I've got to say that's not my experience.  I've always seen
> us blame the submitter for bad code, not the reviewer.
>

This always depends on the definition of "bad code". If "bad code"
means buggy code, I agree. If "bad code" means bad code architecture,
bad code structure, or simply a bad implementation, it gets more murky.
Maybe I just wasn't lucky, but clean architecture and implementation
was not a focus in any of the large companies I worked for. Ok, maybe
officially it was, but not really.

>> Imagine a project where one employee writes the code and another
>> reviews it. Who do you think will get the credit (and bonus) ?
>> I bet it will be the person who wrote the code, not the person
>> who made sure that it is clean and free of bugs.
>
> This is certainly true that credit goes to features.  However, if your
> company only incents that way, QA rapidly gets disillusioned, so only
> giving credit to features wouldn't work long term which is why no
> company I know does it.
>

Lucky you. In one of the companies I worked for, QA engineers could
actually get reprimanded if they found a bug while not following a well
defined test script. The best QA engineer I ever knew ultimately got fired,
not because he didn't find bugs, but because he found lots of bugs by not
following those well defined scripts (the argument being that by not
following the scripts it was difficult to reproduce the bugs).

> To give a counter point: every product we produce has defect metrics and
> I've seen QA get all the prizes in the case where the initial submit was
> too buggy and they turned around the reviews and tests fast enough to
> meet the shipping deadlines and reduce the defects to within the
> metrics.
>
> In all things in life, it's a balance.  I've seen cockups where QA is
> solely incented on defects found and minor UI bugs get classified as
> critical feature defects (because that's what gets the bonus).
>
> But anyway, back to the problem at hand, I think you're suggesting that
> paying for reviews might not work, and I think I agree because it's back
> to incenting QA solely on finding defects.  However, if others thought
> there was merit, we might persuade the LF to offer a small incentive.
>

Reviews should actually precede QA, so I am not sure I agree.
I actually think that paying for reviews (or reviewers) could work,
but that company culture prevents that from happening in practice.

On the other side, there are by now many (or at least some) kernel
maintainers who are actually getting paid for being kernel maintainers.
Some companies out there actually _want_ their engineers to become
kernel maintainers. Kernel maintainers by definition (or at least
I think so) do spend a lot of time doing code reviews. So the
situation may be improving. Maybe all it takes would be to educate
the involved companies that one does not only need kernel maintainers,
but also (paid) code reviewers, and to give those as much credit as
the actual maintainers get. Maybe declare those reviewers to be
maintainers, to give them the credit they deserve. Worth a thought.

Guenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Mon, 2014-05-26 at 01:29 +0300, Dan Carpenter wrote:
> I think SCSI is almost uniquely difficult for this.  The drivers are so
> big and different from each other.  Competitors aren't going to review
> each other's code.

To be honest, my review standard for drivers is does it pass checkpatch
(for an ignored subset of warnings, like lines over 80 characters), does
it compile individually and when I look through the patch does anything
leap out as wrong.  That's by no means an extensive review, but it's
about all that you can do without understanding the internals of the
driver.  I figure mostly that if something goes wrong within a big
driver then in won't affect other drivers, so the manufacturer would
only have themselves to blame and thus be nicely motivated to fix it.

We do have a higher standard of review for shared components (like the
transport classes or libsas).

> With almost every other subsystem, there is a second in command who
> could take over if needed.  I don't know anyone who could do your job if
> you decided to go back to Cambridge for an MBA.

An MBA isn't really an essential qualification for a CTO, so I think
you're safe on that one.

>   I did a:
> 
> git log --after=2013-01-01 drivers/scsi | grep '\-by\:' | sort | uniq -c | sort -rn
> 
> There are some smart people who work on vendor code but everyone seems
> focussed on their own code.

Well, we are trying to encourage more reviewers and backup maintainers
in SCSI ...

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Li Zefan]

On 2014/5/25 16:53, Geert Uytterhoeven wrote:
> Hi Stephen,
> 
> On Sun, May 25, 2014 at 6:17 AM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>> On Sat, 24 May 2014 13:53:45 +0400 James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
>>>
>>> The latter was supposed
>>> to be helped by having the Reviewed-by: tag so we gave credit to
>>> reviewers.  I've found the Reviewed-by tag to be a bit of a double edged
>>> sword: it is a good way of giving review credits, but I also see patches
>>> that come in initially with it on (usually the signoff and the
>>> reviewed-by are from people in the same company) ... it's not
>>> necessarily a bad thing, but it doesn't add much value to the kernel
>>> review process, because we're looking for independent reviews.  The
>>> other thing I find problematic is that some people respond to a patch
>>> with a Reviewed-by: tag and nothing more.  I'm really looking for
>>> evidence of actually having read (and understood) the patch, so the best
>>> review usually comes with a sequence of comments, questions and minor
>>> nits and a reviewed-by at the end.
>>
>> Some stats (I know you all love stats :-)):
>>
>> for next-20140523, no merge commits, origin/master..HEAD^ (exclude
>> Linus' tree and my Next files commit)
>>
>> commits: 7717
>> commits with more than one Signed-off-by: 6291
>> commits with Reviewed-by: 1369
>> commits with Tested-by: 354
>>
>> Not sure what these show ...
> 
> Thanks for the numbers!
> 
> How many Acked-by? Sometimes there's only a thin line between Acked-by
> and Reviewed-by.
> 

Correct. Some Acked-by's are actually Reviewed-by's, and somethimes people
reviewed the code withouting providing Reviewed-by's.

For me, Tejun and I are co-maintaining cgroup. At first I said acked-by
to some of his patches and said reviewed-by to others, and that made it
hard for Tejun to add tags to his patches, so we agreed that I'll just
always stick to acked-by.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Jiri Kosina]

On Mon, 26 May 2014, James Bottomley wrote:

> > I think SCSI is almost uniquely difficult for this.  The drivers are so
> > big and different from each other.  Competitors aren't going to review
> > each other's code.
> 
> To be honest, my review standard for drivers is does it pass checkpatch
> (for an ignored subset of warnings, like lines over 80 characters), does
> it compile individually and when I look through the patch does anything
> leap out as wrong.  That's by no means an extensive review, but it's
> about all that you can do without understanding the internals of the
> driver.  I figure mostly that if something goes wrong within a big
> driver then in won't affect other drivers, so the manufacturer would
> only have themselves to blame and thus be nicely motivated to fix it.

With my distro person hat on, I'd really like to call at least for pushing 
driver maintainers much harder to be a lot more verbose in their 
changelogs (if splitting the commits into smaller chunks is not an 
option). Without that, trying to find out what change might potentially 
cause what kind of behavior turns into a nightmare.

For an example picked up in a completely in random, look at this one

	commit 1ba981fd3ad1f91b8bb205ce6aac6aad45f2fa7a
	Author: James Smart <james.smart@emulex.com>
	Date:   Thu Feb 20 09:56:45 2014 -0500

	    [SCSI] lpfc 8.3.45: Incorporated support of a low-latency io path

No changelog at all, which doesn't look really right when looking at this:

	13 files changed, 1622 insertions(+), 75 deletions(-)

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Lukáš Czerner]

On Sat, 24 May 2014, Wolfram Sang wrote:

> Date: Sat, 24 May 2014 13:19:28 +0200
> From: Wolfram Sang <wsa@the-dreams.de>
> To: James Bottomley <James.Bottomley@HansenPartnership.com>
> Cc: ksummit-discuss@lists.linuxfoundation.org
> Subject: Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers
> 
> 
> > A lot of the problems accepting patches into the kernel stem from trying
> > to get them reviewed
> 
> So true.
> 
> > I'm really looking for evidence of actually having read (and
> > understood) the patch, so the best review usually comes with a
> > sequence of comments, questions and minor nits and a reviewed-by at
> > the end.
> 
> Yes, this is good (and as much needed as describing the tests done for a
> patch by the submitter). Still, trust is an issue here. Even with
> comments you described above, if it is from a person you don't know the
> review needs to be reviewed. Depending on the reviewer that might take
> more time than to do the review on your own. Which pays off if the
> person stays. If not, well, then I still see this as something a
> maintainer should do, simply to educate people. It just might not help
> getting patches reviewed faster.
> 
> > review pool, so we need more than the Reviewed-by: tag.  It has been
> > suggested that people who submit patches should also be required to
> > review patches from other people
> 
> Eeks. Enforced reviews are likely to be sloppy IMO. And sloppy reviews
> can easily cause more work, just like sloppy documentation.
> 
> > conferences and development sprints which do hack time.  Another
> > possibility is to publish a subsystem to review list (similar to the
> > to do list idea). This at least shows submitters why their patch
> > series is languishing and could serve as a call to arms if it gets too
> > long.
> 
> For those using patchwork, such lists are on the web and referenced in
> MAINTAINERS. Submitters don't use it much (yet), according to my
> experiences.
> 
> The thing I'd like to see way more in the Linux ecosystem:
> 
> Paid reviewers/maintainers (selected people, no hiring offers). The
> number of developers increases faster than the number of quality
> keepers. So, the latter should be given the chance to focus on it, if
> they want to.

That does not make much sense to me. In order to review the code you
need to understand it and if you already understand the code, you
can write it as well. I do not think that having dedicated reviewers
is realistic in the long run.

However encouraging reviewers by treating reviewed-by tag with equal
"respect" as signed-off-by seems like the better way.

-Lukas

> 
>    Wolfram
> 
> 

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[H. Peter Anvin]

On 05/24/2014 10:31 AM, James Bottomley wrote:
>>
>> You are assuming that organising reviews of patches is the
>> responsibility of the maintainer.
> 
> Actually, I don't believe I said that anywhere ... however, I do see, at
> least in my subsystem, that the Maintainer is the reviewer of last
> resort, so my selfish reason for wanting more reviewers is to offload
> that from me.
> 

And far too frequently that is the only reviewer available.

Being a kernel maintainer is a very challenging task, combining a lot of
disciplines -- you have to review, architect, manage, educate/mentor,
and depending on your exact employment you may have to manage a "real"
career depending on just how much the maintainer job overlaps your
"real" job.

	-hpa

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Tue, 2014-05-27 at 16:39 +0200, Jiri Kosina wrote:
> On Mon, 26 May 2014, James Bottomley wrote:
> 
> > > I think SCSI is almost uniquely difficult for this.  The drivers are so
> > > big and different from each other.  Competitors aren't going to review
> > > each other's code.
> > 
> > To be honest, my review standard for drivers is does it pass checkpatch
> > (for an ignored subset of warnings, like lines over 80 characters), does
> > it compile individually and when I look through the patch does anything
> > leap out as wrong.  That's by no means an extensive review, but it's
> > about all that you can do without understanding the internals of the
> > driver.  I figure mostly that if something goes wrong within a big
> > driver then in won't affect other drivers, so the manufacturer would
> > only have themselves to blame and thus be nicely motivated to fix it.
> 
> With my distro person hat on, I'd really like to call at least for pushing 
> driver maintainers much harder to be a lot more verbose in their 
> changelogs (if splitting the commits into smaller chunks is not an 
> option). Without that, trying to find out what change might potentially 
> cause what kind of behavior turns into a nightmare.
> 
> For an example picked up in a completely in random, look at this one
> 
> 	commit 1ba981fd3ad1f91b8bb205ce6aac6aad45f2fa7a
> 	Author: James Smart <james.smart@emulex.com>
> 	Date:   Thu Feb 20 09:56:45 2014 -0500
> 
> 	    [SCSI] lpfc 8.3.45: Incorporated support of a low-latency io path

Well, I don't disagree, but getting driver writers to supply changelogs
is hard.  For the ones I understand, I've rewritten (or even composed)
quite a few change logs myself because I often don't get anything usable
back when I request a rewrite.

My intolerance for bad changelogs is high in shared code, but for single
vendor drivers it's often hard just to get the code and keep it in sync,
so I have a lot lower tolerance.

James


James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Jiri Kosina]

On Wed, 28 May 2014, James Bottomley wrote:

> > With my distro person hat on, I'd really like to call at least for pushing 
> > driver maintainers much harder to be a lot more verbose in their 
> > changelogs (if splitting the commits into smaller chunks is not an 
> > option). Without that, trying to find out what change might potentially 
> > cause what kind of behavior turns into a nightmare.
> > 
> > For an example picked up in a completely in random, look at this one
> > 
> > 	commit 1ba981fd3ad1f91b8bb205ce6aac6aad45f2fa7a
> > 	Author: James Smart <james.smart@emulex.com>
> > 	Date:   Thu Feb 20 09:56:45 2014 -0500
> > 
> > 	    [SCSI] lpfc 8.3.45: Incorporated support of a low-latency io path
> 
> Well, I don't disagree, but getting driver writers to supply changelogs
> is hard.  

I know. But there just a one single force on planet Earth that can make 
this happen, and that's maintainer saying "No, you have to do better".

> For the ones I understand, I've rewritten (or even composed) quite a few 
> change logs myself because I often don't get anything usable back when I 
> request a rewrite.

Are you implying that Linux is still not in a position to force HW vendor 
companies to rather invest 30 man-minutes in order to have a proper 
changelog and driver merged in Linus' tree compared to receiving bad 
public press when they are being rejected (especially for such negligible 
reason as changelog text)?

> My intolerance for bad changelogs is high in shared code, but for single 
> vendor drivers it's often hard just to get the code and keep it in sync, 
> so I have a lot lower tolerance.

Unfortunately this doesn't make much of a difference for distro vendors 
when chasing unknown bugs.

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Andy Lutomirski]

On Tue, May 27, 2014 at 3:57 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> On Tuesday, May 27, 2014 07:27:06 PM Lukáš Czerner wrote:
>> On Sat, 24 May 2014, Wolfram Sang wrote:
>
> [cut]
>
>> >
>> > Paid reviewers/maintainers (selected people, no hiring offers). The
>> > number of developers increases faster than the number of quality
>> > keepers. So, the latter should be given the chance to focus on it, if
>> > they want to.
>>
>> That does not make much sense to me. In order to review the code you
>> need to understand it and if you already understand the code, you
>> can write it as well. I do not think that having dedicated reviewers
>> is realistic in the long run.
>
> That's correct and dedicated reviewers who don't really write code will
> become less and less reliable over time.  So they will have to be people
> who actively work on the code *and* review patch submissions from other
> developers.
>
>> However encouraging reviewers by treating reviewed-by tag with equal
>> "respect" as signed-off-by seems like the better way.
>
> I would even argue that it should be treated more seriously than sign-offs.
> After all, there are more patches applied (and all of them are signed-off
> by at least one person) than there are commits with the Reviewed-by tag.

Does Signed-off-by mean something different from "I affirm what the
DCO says" these days?

--Andy

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Tuesday, May 27, 2014 07:27:06 PM Lukáš Czerner wrote:
> On Sat, 24 May 2014, Wolfram Sang wrote:

[cut]

> > 
> > Paid reviewers/maintainers (selected people, no hiring offers). The
> > number of developers increases faster than the number of quality
> > keepers. So, the latter should be given the chance to focus on it, if
> > they want to.
> 
> That does not make much sense to me. In order to review the code you
> need to understand it and if you already understand the code, you
> can write it as well. I do not think that having dedicated reviewers
> is realistic in the long run.

That's correct and dedicated reviewers who don't really write code will
become less and less reliable over time.  So they will have to be people
who actively work on the code *and* review patch submissions from other
developers.

> However encouraging reviewers by treating reviewed-by tag with equal
> "respect" as signed-off-by seems like the better way.

I would even argue that it should be treated more seriously than sign-offs.
After all, there are more patches applied (and all of them are signed-off
by at least one person) than there are commits with the Reviewed-by tag.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Tuesday, May 27, 2014 03:43:00 PM Andy Lutomirski wrote:
> On Tue, May 27, 2014 at 3:57 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > On Tuesday, May 27, 2014 07:27:06 PM Lukáš Czerner wrote:
> >> On Sat, 24 May 2014, Wolfram Sang wrote:
> >
> > [cut]
> >
> >> >
> >> > Paid reviewers/maintainers (selected people, no hiring offers). The
> >> > number of developers increases faster than the number of quality
> >> > keepers. So, the latter should be given the chance to focus on it, if
> >> > they want to.
> >>
> >> That does not make much sense to me. In order to review the code you
> >> need to understand it and if you already understand the code, you
> >> can write it as well. I do not think that having dedicated reviewers
> >> is realistic in the long run.
> >
> > That's correct and dedicated reviewers who don't really write code will
> > become less and less reliable over time.  So they will have to be people
> > who actively work on the code *and* review patch submissions from other
> > developers.
> >
> >> However encouraging reviewers by treating reviewed-by tag with equal
> >> "respect" as signed-off-by seems like the better way.
> >
> > I would even argue that it should be treated more seriously than sign-offs.
> > After all, there are more patches applied (and all of them are signed-off
> > by at least one person) than there are commits with the Reviewed-by tag.
> 
> Does Signed-off-by mean something different from "I affirm what the
> DCO says" these days?

Formally - no, I don't think so.  Practically, though, I'm expecting people
who send me patches with their sign-offs to at least stay around for some time
to fix issues related to their changes.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Martin K. Petersen]

>>>>> "Jiri" == Jiri Kosina <jkosina@suse.cz> writes:

Jiri> Are you implying that Linux is still not in a position to force HW
Jiri> vendor companies to rather invest 30 man-minutes in order to have
Jiri> a proper changelog and driver merged in Linus' tree compared to
Jiri> receiving bad public press when they are being rejected
Jiri> (especially for such negligible reason as changelog text)?

There are a few companies in the enterprise space that get it. But in
general it can be quite the challenge to get these vendors to see the
value of being good Linux citizens. And these companies are much less
susceptible to phoronix rants than entities producing consumer widgets.

These companies have internal development processes that are often quite
unfriendly to the way we work. More often than not, upstream drivers are
trailing internal driver trees by many, many months. Patches are only
sent upstream when there is a bit of slack in the engineering schedule.

In many cases upstream Linux is not seen as an important target because
the driver vendor will provide OEMs that ship their hardware with a
"value added" outbox driver tarball or CD. The server vendors are
partially to blame for keeping this dreadful anachronism alive. One
would think that "It Just Works with RHEL/SLES/OL" would be better story
than retrofitting tarball drivers and jeopardizing future kernel
updates. According to the server vendors, however, customers expect to
install an updated Linux driver just like they do on Windows. Otherwise
the perception is that Linux is lagging behind or poorly supported.
*sigh*

Vendor driver release cycles are often tied exclusively to new silicon
availability and internal firmware release schedules. Many vendors pay
little to no attention to Linux development cycles. Furthermore, driver
code is often shared between several operating systems so every patch
needs to undergo IP/legal review before it can be submitted upstream.
Internal commit messages often have partner references, bug numbers,
code names, etc. in them. So it's typically easier to just drop the
patch description than rewriting it and have the new text reviewed by
the legal team.

That's the sorry state of affairs. Part of the problem here is that many
of the enterprise drivers have been in the kernel for a very long time.
They predate things like staging and SubmittingPatches by many years.
And we have been poor at communicating that things have changed and
sometimes a bit too lenient at accepting patches.

We have had a several discussions about this problem the last 6 months,
including at LSF/MM. I have had meetings with several hw vendors this
spring trying to educate them about our "new" rules of engagement. It's
not easy, but I feel we're making progress. Worst case we'll send gregkh
wielding a clue bat...

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Tue, May 27, 2014 at 08:10:36PM -0400, Martin K. Petersen wrote:
> We have had a several discussions about this problem the last 6 months,
> including at LSF/MM. I have had meetings with several hw vendors this
> spring trying to educate them about our "new" rules of engagement. It's
> not easy, but I feel we're making progress. Worst case we'll send gregkh
> wielding a clue bat...

Oooh, that would be fun, sign me up!

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 3282 bytes --]

On Tue, 27 May 2014 23:22:16 +0200 (CEST) Jiri Kosina <jkosina@suse.cz> wrote:

> On Wed, 28 May 2014, James Bottomley wrote:
> 
> > > With my distro person hat on, I'd really like to call at least for pushing 
> > > driver maintainers much harder to be a lot more verbose in their 
> > > changelogs (if splitting the commits into smaller chunks is not an 
> > > option). Without that, trying to find out what change might potentially 
> > > cause what kind of behavior turns into a nightmare.
> > > 
> > > For an example picked up in a completely in random, look at this one
> > > 
> > > 	commit 1ba981fd3ad1f91b8bb205ce6aac6aad45f2fa7a
> > > 	Author: James Smart <james.smart@emulex.com>
> > > 	Date:   Thu Feb 20 09:56:45 2014 -0500
> > > 
> > > 	    [SCSI] lpfc 8.3.45: Incorporated support of a low-latency io path
> > 
> > Well, I don't disagree, but getting driver writers to supply changelogs
> > is hard.  
> 
> I know. But there just a one single force on planet Earth that can make 
> this happen, and that's maintainer saying "No, you have to do better".
> 
> > For the ones I understand, I've rewritten (or even composed) quite a few 
> > change logs myself because I often don't get anything usable back when I 
> > request a rewrite.
> 
> Are you implying that Linux is still not in a position to force HW vendor 
> companies to rather invest 30 man-minutes in order to have a proper 
> changelog and driver merged in Linus' tree compared to receiving bad 
> public press when they are being rejected (especially for such negligible 
> reason as changelog text)?

Even if they spend the 30 minutes, will the changelog be of any value?
Writing a good changelog is quite an art, and we don't even have something
like checkpatch.pl to highlight to worst offenders.

I came across a patch the other day which had quite a reasonable changelog
comment which suggested the patch was a clean-up - and quite a sensible one.
But the patch was also tagged for -stable, and had a couple of Cc:s which
made me think it was probably a bug-fix and the Cc: people had reported the
bug.  But there was no hint in the changelog what the bug might have been...

I quite often write or re-write changelogs for people who seem unable or
unwilling or unmotivated to do a better job.  I could just keep saying "no"
until they get it right, but when it is a real bugfix or a useful improvement
I want the  patch and the cost/benefit of pushing for a better changelog
rises sharply.

So I agree that I would love better changelogs and I think it is an important
part of the maintainer role to keep changelog quality high just as much as
keeping code quality high.  But every maintainer is in a different situation
and there is a limit to what we can expect of other maintainers.

It is nonetheless good to regularly remind each other that changelog quality
is really important.

Thanks,
NeilBrown


> 
> > My intolerance for bad changelogs is high in shared code, but for single 
> > vendor drivers it's often hard just to get the code and keep it in sync, 
> > so I have a lot lower tolerance.
> 
> Unfortunately this doesn't make much of a difference for distro vendors 
> when chasing unknown bugs.
> 
> Thanks,
> 


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Tue, 2014-05-27 at 23:22 +0200, Jiri Kosina wrote:
> On Wed, 28 May 2014, James Bottomley wrote:
> 
> > > With my distro person hat on, I'd really like to call at least for pushing 
> > > driver maintainers much harder to be a lot more verbose in their 
> > > changelogs (if splitting the commits into smaller chunks is not an 
> > > option). Without that, trying to find out what change might potentially 
> > > cause what kind of behavior turns into a nightmare.
> > > 
> > > For an example picked up in a completely in random, look at this one
> > > 
> > > 	commit 1ba981fd3ad1f91b8bb205ce6aac6aad45f2fa7a
> > > 	Author: James Smart <james.smart@emulex.com>
> > > 	Date:   Thu Feb 20 09:56:45 2014 -0500
> > > 
> > > 	    [SCSI] lpfc 8.3.45: Incorporated support of a low-latency io path
> > 
> > Well, I don't disagree, but getting driver writers to supply changelogs
> > is hard.  
> 
> I know. But there just a one single force on planet Earth that can make 
> this happen, and that's maintainer saying "No, you have to do better".

Look, in the immortal words of Bill Clinton: I feel your pain.  However,
having it all be the Maintainer's fault isn't scalable.

> > For the ones I understand, I've rewritten (or even composed) quite a few 
> > change logs myself because I often don't get anything usable back when I 
> > request a rewrite.
> 
> Are you implying that Linux is still not in a position to force HW vendor 
> companies to rather invest 30 man-minutes in order to have a proper 
> changelog and driver merged in Linus' tree compared to receiving bad 
> public press when they are being rejected (especially for such negligible 
> reason as changelog text)?

To paraphrase Martin: hell, yes.

> > My intolerance for bad changelogs is high in shared code, but for single 
> > vendor drivers it's often hard just to get the code and keep it in sync, 
> > so I have a lot lower tolerance.
> 
> Unfortunately this doesn't make much of a difference for distro vendors 
> when chasing unknown bugs.

So please help me.  I have quite a few distro people on my list, it
would be enormously helpful if they reviewed a few vendor patches,
perhaps suggesting workable changelogs in the review ... it doesn't have
to be every patch, which is unmanageable, but a few to act as training
wheels.

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 1615 bytes --]


> > The thing I'd like to see way more in the Linux ecosystem:
> > 
> > Paid reviewers/maintainers (selected people, no hiring offers). The
> > number of developers increases faster than the number of quality
> > keepers. So, the latter should be given the chance to focus on it, if
> > they want to.
> 
> That does not make much sense to me. In order to review the code you
> need to understand it and if you already understand the code, you
> can write it as well.

Huh, have you ever maintained something? I couldn't write most of the
drivers I get for I2C, because I simply don't have access to the
hardware. Also, I it is one thing to get the driver working (dealing
with HW bugs and documentation flaws) and another to get the driver
proper (use proper kernal interfaces, coding style...).

> I do not think that having dedicated reviewers is realistic in the
> long run.

Then, think more of dedicated maintainers: I don't know one maintainer
who is not interested in hacking as well. Which is needed, of course.
There are reviews which make flaws in the subsystem core obvious.
Usually, the original patch comitter is only interested in this reviewed
patch and does not want to deal with the core. So, this kind of work is
up to the maintainer. If maintainers are backed off, more of this work
could be done IMO.

> However encouraging reviewers by treating reviewed-by tag with equal
> "respect" as signed-off-by seems like the better way.

In general, yes. As I said before, "Reviewed-by" tags need to be trusted
what also takes some time and effort in the beginning.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Lukáš Czerner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2623 bytes --]

On Wed, 28 May 2014, Wolfram Sang wrote:

> Date: Wed, 28 May 2014 07:37:49 +0200
> From: Wolfram Sang <wsa@the-dreams.de>
> To: Lukáš Czerner <lczerner@redhat.com>
> Cc: James Bottomley <James.Bottomley@HansenPartnership.com>,
>     ksummit-discuss@lists.linuxfoundation.org
> Subject: Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers
> 
> 
> > > The thing I'd like to see way more in the Linux ecosystem:
> > > 
> > > Paid reviewers/maintainers (selected people, no hiring offers). The
> > > number of developers increases faster than the number of quality
> > > keepers. So, the latter should be given the chance to focus on it, if
> > > they want to.
> > 
> > That does not make much sense to me. In order to review the code you
> > need to understand it and if you already understand the code, you
> > can write it as well.
> 
> Huh, have you ever maintained something? I couldn't write most of the
> drivers I get for I2C, because I simply don't have access to the
> hardware.

But the lack of HW was the one thing stopping you right ? Not the
lack of skills or knowledge. I am just saying that it's quite likely
that there are even less people with the skills who are willing to be
dedicated to review than developers who are willing to review
patches from time to time.

> Also, I it is one thing to get the driver working (dealing
> with HW bugs and documentation flaws) and another to get the driver
> proper (use proper kernal interfaces, coding style...).
> 
> > I do not think that having dedicated reviewers is realistic in the
> > long run.
> 
> Then, think more of dedicated maintainers: I don't know one maintainer
> who is not interested in hacking as well. Which is needed, of course.
> There are reviews which make flaws in the subsystem core obvious.
> Usually, the original patch comitter is only interested in this reviewed
> patch and does not want to deal with the core. So, this kind of work is
> up to the maintainer. If maintainers are backed off, more of this work
> could be done IMO.

Are you suggesting that existing maintainers should either stop
hacking, or give up maintainership in favour of those who does not
want to write code ? :) Sorry for exaggerating, but I do not think
that having people dedicated solely to review is realistic.

We should rather focus on encouraging developers to review more.

> 
> > However encouraging reviewers by treating reviewed-by tag with equal
> > "respect" as signed-off-by seems like the better way.
> 
> In general, yes. As I said before, "Reviewed-by" tags need to be trusted
> what also takes some time and effort in the beginning.
> 
> 

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]


> > > That does not make much sense to me. In order to review the code you
> > > need to understand it and if you already understand the code, you
> > > can write it as well.
> > 
> > Huh, have you ever maintained something? I couldn't write most of the
> > drivers I get for I2C, because I simply don't have access to the
> > hardware.
> 
> But the lack of HW was the one thing stopping you right ? Not the
> lack of skills or knowledge.

Lack of time is another one. And lack of interest a third one. Unlike
the casual developer, I'd like to work more on the core and
infrastructure than on drivers.

> I am just saying that it's quite likely
> that there are even less people with the skills who are willing to be
> dedicated to review than developers who are willing to review
> patches from time to time.

Right. And give those rare bunch some money to focus on this kind of
work (opposed to stuff like board bringup) is what I am proposing.

> > Then, think more of dedicated maintainers: I don't know one maintainer
> > who is not interested in hacking as well. Which is needed, of course.
> > There are reviews which make flaws in the subsystem core obvious.
> > Usually, the original patch comitter is only interested in this reviewed
> > patch and does not want to deal with the core. So, this kind of work is
> > up to the maintainer. If maintainers are backed off, more of this work
> > could be done IMO.
> 
> Are you suggesting that existing maintainers should either stop
> hacking, or give up maintainership in favour of those who does not
> want to write code ? :) Sorry for exaggerating, but I do not think
> that having people dedicated solely to review is realistic.

I think we are missing each other here. Maintaining is not only
reviewing but also keeping the core in shape. There is quite some
hacking involved if done properly.

> We should rather focus on encouraging developers to review more.

That is a good, helpful thing. Yet, from my experience it won't scale
enough. I'd be happy to be wrong here, though.

Regards,

   Wolfram


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Daniel Vetter]

On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>> However encouraging reviewers by treating reviewed-by tag with equal
>> "respect" as signed-off-by seems like the better way.
>
> I would even argue that it should be treated more seriously than sign-offs.
> After all, there are more patches applied (and all of them are signed-off
> by at least one person) than there are commits with the Reviewed-by tag.

Fully agreed on given reviews more credit than sobs. Authors of
feature already get all the praise and publicity for doing something
visible, which means review is always a background chore. But if we
lack reviewers then the pipeline for merging patches gets seriously
clogged up. At least that's been my experience with drm/i915, and
pretty much all the people there work for my employer so I can _make_
them review code. Still not enough.

It's a fine line though since we absolutely don't want people to
rubber-stamp 20 patches in half an hour just because someone told them
they need to "review" them. Plain more visibility to reviewers (lwn
stats?) might help even with the risk that it will be gamed for sure.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Carpenter]

We should have a special tag for reviewers who find actual bugs because
that is the best sort of reviewer.

regards,
dan carpenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Daniel Vetter]

On Wed, May 28, 2014 at 4:32 PM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
> We should have a special tag for reviewers who find actual bugs because
> that is the best sort of reviewer.

My approach has been to insist on an in-patch revision log which gets
included in the commit. And that for any changes and bugs spotted the
reviewer/commenter must be acknowleged. See e.g.
d978ef14456a38034f6c0e for a very nice example of that. But that's
also a good example for no tag to acknowledge all the work that went
into this review/patch, since I've done the final review myself and
only put my sob onto the patch.

Something more standardized here would indeed be nice. There's still
the problem that some review takes a really long time (e.g. if there's
lots of design considerations), and other patches can be reviewed
quickly. So a binary tag will always be lacking.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Guenter Roeck]

On Wed, May 28, 2014 at 04:26:47PM +0200, Daniel Vetter wrote:
> On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >> However encouraging reviewers by treating reviewed-by tag with equal
> >> "respect" as signed-off-by seems like the better way.
> >
> > I would even argue that it should be treated more seriously than sign-offs.
> > After all, there are more patches applied (and all of them are signed-off
> > by at least one person) than there are commits with the Reviewed-by tag.
> 
> Fully agreed on given reviews more credit than sobs. Authors of
> feature already get all the praise and publicity for doing something
> visible, which means review is always a background chore. But if we
> lack reviewers then the pipeline for merging patches gets seriously
> clogged up. At least that's been my experience with drm/i915, and
> pretty much all the people there work for my employer so I can _make_
> them review code. Still not enough.
> 
> It's a fine line though since we absolutely don't want people to
> rubber-stamp 20 patches in half an hour just because someone told them
> they need to "review" them. Plain more visibility to reviewers (lwn
> stats?) might help even with the risk that it will be gamed for sure.

It gets scary if people start to ignore the "Reviewer's statement of
oversight" in SubmittingPatches. I don't treat my "Reviewed-by" tag
lightly, and sincerely hope others don't either.

Guenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mimi Zohar]

On Wed, 2014-05-28 at 16:26 +0200, Daniel Vetter wrote: 
> On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >> However encouraging reviewers by treating reviewed-by tag with equal
> >> "respect" as signed-off-by seems like the better way.
> >
> > I would even argue that it should be treated more seriously than sign-offs.
> > After all, there are more patches applied (and all of them are signed-off
> > by at least one person) than there are commits with the Reviewed-by tag.
> 
> Fully agreed on given reviews more credit than sobs. Authors of
> feature already get all the praise and publicity for doing something
> visible, which means review is always a background chore. But if we
> lack reviewers then the pipeline for merging patches gets seriously
> clogged up. At least that's been my experience with drm/i915, and
> pretty much all the people there work for my employer so I can _make_
> them review code. Still not enough.
> 
> It's a fine line though since we absolutely don't want people to
> rubber-stamp 20 patches in half an hour just because someone told them
> they need to "review" them. Plain more visibility to reviewers (lwn
> stats?) might help even with the risk that it will be gamed for sure.
> -Daniel

Like the other tags, 'Reviewed-by' isn't automatically generated and
requires agreement from the person.  Perhaps the tag name implies too
much responsibility, perhaps a 'Commented-by' tag would be less daunting
for people reviewing the code.  And if it was highlighted in the
statistics at the conference talks, then maybe there would be more
participation.

Mimi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Josh Triplett]

On Wed, May 28, 2014 at 12:20:20PM -0400, Mimi Zohar wrote:
> On Wed, 2014-05-28 at 16:26 +0200, Daniel Vetter wrote: 
> > On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > >> However encouraging reviewers by treating reviewed-by tag with equal
> > >> "respect" as signed-off-by seems like the better way.
> > >
> > > I would even argue that it should be treated more seriously than sign-offs.
> > > After all, there are more patches applied (and all of them are signed-off
> > > by at least one person) than there are commits with the Reviewed-by tag.
> > 
> > Fully agreed on given reviews more credit than sobs. Authors of
> > feature already get all the praise and publicity for doing something
> > visible, which means review is always a background chore. But if we
> > lack reviewers then the pipeline for merging patches gets seriously
> > clogged up. At least that's been my experience with drm/i915, and
> > pretty much all the people there work for my employer so I can _make_
> > them review code. Still not enough.
> > 
> > It's a fine line though since we absolutely don't want people to
> > rubber-stamp 20 patches in half an hour just because someone told them
> > they need to "review" them. Plain more visibility to reviewers (lwn
> > stats?) might help even with the risk that it will be gamed for sure.
> > -Daniel
> 
> Like the other tags, 'Reviewed-by' isn't automatically generated and
> requires agreement from the person.  Perhaps the tag name implies too
> much responsibility, perhaps a 'Commented-by' tag would be less daunting
> for people reviewing the code.  And if it was highlighted in the
> statistics at the conference talks, then maybe there would be more
> participation.

Or perhaps in LWN's statistics, which focus on Signed-off-by but don't
mention Reviewed-by at all.

- Josh Triplett

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mimi Zohar]

On Wed, 2014-05-28 at 09:05 -0700, Guenter Roeck wrote: 
> On Wed, May 28, 2014 at 04:26:47PM +0200, Daniel Vetter wrote:
> > On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > >> However encouraging reviewers by treating reviewed-by tag with equal
> > >> "respect" as signed-off-by seems like the better way.
> > >
> > > I would even argue that it should be treated more seriously than sign-offs.
> > > After all, there are more patches applied (and all of them are signed-off
> > > by at least one person) than there are commits with the Reviewed-by tag.
> > 
> > Fully agreed on given reviews more credit than sobs. Authors of
> > feature already get all the praise and publicity for doing something
> > visible, which means review is always a background chore. But if we
> > lack reviewers then the pipeline for merging patches gets seriously
> > clogged up. At least that's been my experience with drm/i915, and
> > pretty much all the people there work for my employer so I can _make_
> > them review code. Still not enough.
> > 
> > It's a fine line though since we absolutely don't want people to
> > rubber-stamp 20 patches in half an hour just because someone told them
> > they need to "review" them. Plain more visibility to reviewers (lwn
> > stats?) might help even with the risk that it will be gamed for sure.
> 
> It gets scary if people start to ignore the "Reviewer's statement of
> oversight" in SubmittingPatches. I don't treat my "Reviewed-by" tag
> lightly, and sincerely hope others don't either.

Patches can be reviewed at different levels.  There are those things
that are generic, subsystem independent, that don't require low level
knowledge, but they do take time.  Unlike a low-level review, nobody
doing this type of review, would want the responsibility of the
'Reviewed-by' tag added.

Mimi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

On Wed, May 28, 2014 at 04:39:15PM +0200, Daniel Vetter wrote:

> My approach has been to insist on an in-patch revision log which gets
> included in the commit. And that for any changes and bugs spotted the
> reviewer/commenter must be acknowleged. See e.g.
> d978ef14456a38034f6c0e for a very nice example of that. But that's
> also a good example for no tag to acknowledge all the work that went
> into this review/patch, since I've done the final review myself and
> only put my sob onto the patch.

This does mean that the final changelogs that get included in the kernel
get very large and noisy and is relying on the submitters doing a good
job paying attention to review comments in the first place, recording
exactly what changed and so on.  They are sometimes useful but normally
I'm finding very little value in the changelogs in the first place,
generally it doesn't really matter what the problems were in any
previous versions.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Guenter Roeck]

On Wed, May 28, 2014 at 12:37:38PM -0400, Mimi Zohar wrote:
> On Wed, 2014-05-28 at 09:05 -0700, Guenter Roeck wrote: 
> > On Wed, May 28, 2014 at 04:26:47PM +0200, Daniel Vetter wrote:
> > > On Wed, May 28, 2014 at 12:57 AM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > > >> However encouraging reviewers by treating reviewed-by tag with equal
> > > >> "respect" as signed-off-by seems like the better way.
> > > >
> > > > I would even argue that it should be treated more seriously than sign-offs.
> > > > After all, there are more patches applied (and all of them are signed-off
> > > > by at least one person) than there are commits with the Reviewed-by tag.
> > > 
> > > Fully agreed on given reviews more credit than sobs. Authors of
> > > feature already get all the praise and publicity for doing something
> > > visible, which means review is always a background chore. But if we
> > > lack reviewers then the pipeline for merging patches gets seriously
> > > clogged up. At least that's been my experience with drm/i915, and
> > > pretty much all the people there work for my employer so I can _make_
> > > them review code. Still not enough.
> > > 
> > > It's a fine line though since we absolutely don't want people to
> > > rubber-stamp 20 patches in half an hour just because someone told them
> > > they need to "review" them. Plain more visibility to reviewers (lwn
> > > stats?) might help even with the risk that it will be gamed for sure.
> > 
> > It gets scary if people start to ignore the "Reviewer's statement of
> > oversight" in SubmittingPatches. I don't treat my "Reviewed-by" tag
> > lightly, and sincerely hope others don't either.
> 
> Patches can be reviewed at different levels.  There are those things
> that are generic, subsystem independent, that don't require low level
> knowledge, but they do take time.  Unlike a low-level review, nobody
> doing this type of review, would want the responsibility of the
> 'Reviewed-by' tag added.
> 
There is still Acked-by: if a reviewer doesn't feel entirely comfortable
with Reviewed-by:. I tend to use it to indicate "looks good to me, but
I'll want the subsystem maintainer or someone with detailed knowledge
about the subsystem and/or the code to have another look".

Guenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mimi Zohar]

On Wed, 2014-05-28 at 17:39 +0100, Mark Brown wrote: 
> On Wed, May 28, 2014 at 04:39:15PM +0200, Daniel Vetter wrote:
> 
> > My approach has been to insist on an in-patch revision log which gets
> > included in the commit. And that for any changes and bugs spotted the
> > reviewer/commenter must be acknowleged. See e.g.
> > d978ef14456a38034f6c0e for a very nice example of that. But that's
> > also a good example for no tag to acknowledge all the work that went
> > into this review/patch, since I've done the final review myself and
> > only put my sob onto the patch.
> 
> This does mean that the final changelogs that get included in the kernel
> get very large and noisy and is relying on the submitters doing a good
> job paying attention to review comments in the first place, recording
> exactly what changed and so on.  They are sometimes useful but normally
> I'm finding very little value in the changelogs in the first place,
> generally it doesn't really matter what the problems were in any
> previous versions.

True, but when you have to squash patches there needs to be at least
some recognition of who contributed what.

Mimi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Jonathan Corbet]

On Wed, 28 May 2014 09:28:33 -0700
Josh Triplett <josh@joshtriplett.org> wrote:

> Or perhaps in LWN's statistics, which focus on Signed-off-by but don't
> mention Reviewed-by at all.

I occasionally put up a table with Reviewed-by (and Reported-by) stats,
but the absolute numbers are generally quite small, so the interest
seems to be low.  But I should revisit that, it's been a little while.

jon

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]

On Wed, May 28, 2014 at 12:51:02PM -0400, Mimi Zohar wrote:
> On Wed, 2014-05-28 at 17:39 +0100, Mark Brown wrote: 
> > On Wed, May 28, 2014 at 04:39:15PM +0200, Daniel Vetter wrote:

> > > My approach has been to insist on an in-patch revision log which gets
> > > included in the commit. And that for any changes and bugs spotted the
> > > reviewer/commenter must be acknowleged. See e.g.

> > This does mean that the final changelogs that get included in the kernel
> > get very large and noisy and is relying on the submitters doing a good
> > job paying attention to review comments in the first place, recording
> > exactly what changed and so on.  They are sometimes useful but normally
> > I'm finding very little value in the changelogs in the first place,
> > generally it doesn't really matter what the problems were in any
> > previous versions.

> True, but when you have to squash patches there needs to be at least
> some recognition of who contributed what.

There's a world of difference between thanking people for review and a
detailed account of all the changes made in every single iteration of
the review.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Luck, Tony]

> There's a world of difference between thanking people for review and a
> detailed account of all the changes made in every single iteration of
> the review.

This is already covered in Documentation/SubmittingPatches. Quoting
lines 585-592 (see last sentence):

One good use for the additional comments after the "---" marker is for
a diffstat, to show what files have changed, and the number of
inserted and deleted lines per file.  A diffstat is especially useful
on bigger patches.  Other comments relevant only to the moment or the
maintainer, not suitable for the permanent changelog, should also go
here.  A good example of such comments might be "patch changelogs"
which describe what has changed between the v1 and v2 version of the
patch.

-Tony

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:

> > There's a world of difference between thanking people for review and a
> > detailed account of all the changes made in every single iteration of
> > the review.

> This is already covered in Documentation/SubmittingPatches. Quoting
> lines 585-592 (see last sentence):

Right, but Daniel is proposing lifting that above the --- and including
it in git.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Paul Walmsley]

On Sat, 24 May 2014, James Bottomley wrote:

> I'm sure there are many other things people could suggest.

What's needed is to bring quality reviewers up to the same level of 
recognition and control as maintainers.

Ideally, maintainers would recognize quality reviewers, and list them in 
the MAINTAINERS file - perhaps with an "R:" tag?  Maintainers would be 
expected to designate at least one quality reviewer, but ideally more, for 
a given subsystem.  

Then we should require every patch to have at least one "Reviewed-by:", 
aside from the maintainer's "Signed-off-by:" before being merged.  This 
"Reviewed-by:" could come from the maintainer, but ideally would come from 
a quality reviewer.

Patch submitters would need to get their patches reviewed by at least one 
of the recognized reviewers before expecting it to be merged.

Part of the goal here would also be to convert quality reviewers into 
co-maintainers over time, so maintainership duties can be spread among a 
larger group of people.


- Paul

[Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Paul Walmsley]

Also long-overdue is a clarification on exactly what "Acked-by" means.
Right now it is being used for at least two distinct and
mutually-incompatible purposes:

1. A maintainer A for code affected by a patch, who is distinct from a
maintainer B queuing a patch, has reviewed the patch and has cleared it as
being OK for maintainer B to send upstream

2. A casual review has been done by someone who is not a maintainer for
the code in question

What I would propose is to have the first use replaced by a new tag, 
"Maintainer-acked-by:", and the second use abolished, along with 
"Acked-by:", and replaced by "Reviewed-by:".


- Paul

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Mimi Zohar]

On Wed, 2014-05-28 at 18:48 +0000, Paul Walmsley wrote: 
> Also long-overdue is a clarification on exactly what "Acked-by" means.
> Right now it is being used for at least two distinct and
> mutually-incompatible purposes:
> 
> 1. A maintainer A for code affected by a patch, who is distinct from a
> maintainer B queuing a patch, has reviewed the patch and has cleared it as
> being OK for maintainer B to send upstream
> 
> 2. A casual review has been done by someone who is not a maintainer for
> the code in question
> 
> What I would propose is to have the first use replaced by a new tag, 
> "Maintainer-acked-by:", and the second use abolished, along with 
> "Acked-by:", and replaced by "Reviewed-by:".

Agreed, "Acked-by" is ambiguous and should be dis-ambiguated.
"Reviewed-by:" is too much of a barrier for people to feel comfortable
using.  Just as the "Maintainer-acked-by:" would imply a subset of the
patch related to the subsystem, "Reviewed-by" needs something similar to
limit its scope.

Mimi

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[John W. Linville]

On Wed, May 28, 2014 at 03:11:55PM -0400, Mimi Zohar wrote:
> On Wed, 2014-05-28 at 18:48 +0000, Paul Walmsley wrote: 
> > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > Right now it is being used for at least two distinct and
> > mutually-incompatible purposes:
> > 
> > 1. A maintainer A for code affected by a patch, who is distinct from a
> > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> > being OK for maintainer B to send upstream
> > 
> > 2. A casual review has been done by someone who is not a maintainer for
> > the code in question
> > 
> > What I would propose is to have the first use replaced by a new tag, 
> > "Maintainer-acked-by:", and the second use abolished, along with 
> > "Acked-by:", and replaced by "Reviewed-by:".
> 
> Agreed, "Acked-by" is ambiguous and should be dis-ambiguated.
> "Reviewed-by:" is too much of a barrier for people to feel comfortable
> using.  Just as the "Maintainer-acked-by:" would imply a subset of the
> patch related to the subsystem, "Reviewed-by" needs something similar to
> limit its scope.

I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...

-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Guenter Roeck]

On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> 
> Also long-overdue is a clarification on exactly what "Acked-by" means.
> Right now it is being used for at least two distinct and
> mutually-incompatible purposes:
> 
> 1. A maintainer A for code affected by a patch, who is distinct from a
> maintainer B queuing a patch, has reviewed the patch and has cleared it as
> being OK for maintainer B to send upstream
> 
> 2. A casual review has been done by someone who is not a maintainer for
> the code in question
> 
> What I would propose is to have the first use replaced by a new tag, 
> "Maintainer-acked-by:", and the second use abolished, along with 
> "Acked-by:", and replaced by "Reviewed-by:".
> 

Not sure I understand the problem with "It is a record that the acker
has at least reviewed the patch and has indicated acceptance".

Guess you are saying this is a bad thing, that reviewers should not
have the means to indicate such a level of acceptance except for affected
subsystem maintainers, and that everyone else should either do a full
formal review or nothing. That seems a bit extreme to me, and I don't
entirely understand how this would improve the situation.

Guenter

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Mimi Zohar]

On Wed, 2014-05-28 at 15:15 -0400, John W. Linville wrote: 
> On Wed, May 28, 2014 at 03:11:55PM -0400, Mimi Zohar wrote:
> > On Wed, 2014-05-28 at 18:48 +0000, Paul Walmsley wrote: 
> > > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > > Right now it is being used for at least two distinct and
> > > mutually-incompatible purposes:
> > > 
> > > 1. A maintainer A for code affected by a patch, who is distinct from a
> > > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> > > being OK for maintainer B to send upstream
> > > 
> > > 2. A casual review has been done by someone who is not a maintainer for
> > > the code in question
> > > 
> > > What I would propose is to have the first use replaced by a new tag, 
> > > "Maintainer-acked-by:", and the second use abolished, along with 
> > > "Acked-by:", and replaced by "Reviewed-by:".
> > 
> > Agreed, "Acked-by" is ambiguous and should be dis-ambiguated.
> > "Reviewed-by:" is too much of a barrier for people to feel comfortable
> > using.  Just as the "Maintainer-acked-by:" would imply a subset of the
> > patch related to the subsystem, "Reviewed-by" needs something similar to
> > limit its scope.
> 
> I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...

Agreed, but I don't think its much longer than any of the others. :)

Perhaps more people would feel comfortable adding "Reviewed-by", if they
could limit the scope.  Instead of defining a new tag, perhaps all that
is needed is adding an optional field.  For example, "Reviewed-by
[option]: ", where 'option' could be style, logic, syntax, ...

Mimi

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[josh]

On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> 
> Also long-overdue is a clarification on exactly what "Acked-by" means.
> Right now it is being used for at least two distinct and
> mutually-incompatible purposes:
> 
> 1. A maintainer A for code affected by a patch, who is distinct from a
> maintainer B queuing a patch, has reviewed the patch and has cleared it as
> being OK for maintainer B to send upstream
> 
> 2. A casual review has been done by someone who is not a maintainer for
> the code in question
> 
> What I would propose is to have the first use replaced by a new tag, 
> "Maintainer-acked-by:", and the second use abolished, along with 
> "Acked-by:", and replaced by "Reviewed-by:".

In practice, (2) seems to have been mostly replaced by "Reviewed-by"; I
rarely see Acked-by used for cases other than (1): "I'm the maintainer
or an affected subsystem developer, I approve of this patch, but I don't
intend to take it through my own tree."

- Josh Triplett

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[josh]

On Wed, May 28, 2014 at 06:47:43PM +0000, Paul Walmsley wrote:
> On Sat, 24 May 2014, James Bottomley wrote:
> 
> > I'm sure there are many other things people could suggest.
> 
> What's needed is to bring quality reviewers up to the same level of 
> recognition and control as maintainers.
> 
> Ideally, maintainers would recognize quality reviewers, and list them in 
> the MAINTAINERS file - perhaps with an "R:" tag?  Maintainers would be 
> expected to designate at least one quality reviewer, but ideally more, for 
> a given subsystem.  
> 
> Then we should require every patch to have at least one "Reviewed-by:", 
> aside from the maintainer's "Signed-off-by:" before being merged.  This 
> "Reviewed-by:" could come from the maintainer, but ideally would come from 
> a quality reviewer.
> 
> Patch submitters would need to get their patches reviewed by at least one 
> of the recognized reviewers before expecting it to be merged.
> 
> Part of the goal here would also be to convert quality reviewers into 
> co-maintainers over time, so maintainership duties can be spread among a 
> larger group of people.

I would love to see this changed.  Right now, only people considered
co-maintainers get listed in MAINTAINERS.  With how get_maintainer.pl
works, it would make sense to use it as a more general list of "CC these
people on patches to this subsystem".

Many subsystems use mailing lists for this, but mailing lists suffer
from diffusion of responsibility: "someone else on the list can review
this patch".

- Josh Triplett

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Dmitry Torokhov]

On Wednesday, May 28, 2014 01:12:28 PM josh@joshtriplett.org wrote:
> On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > Right now it is being used for at least two distinct and
> > mutually-incompatible purposes:
> > 
> > 1. A maintainer A for code affected by a patch, who is distinct from a
> > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> > being OK for maintainer B to send upstream
> > 
> > 2. A casual review has been done by someone who is not a maintainer for
> > the code in question
> > 
> > What I would propose is to have the first use replaced by a new tag,
> > "Maintainer-acked-by:", and the second use abolished, along with
> > "Acked-by:", and replaced by "Reviewed-by:".
> 
> In practice, (2) seems to have been mostly replaced by "Reviewed-by"; I
> rarely see Acked-by used for cases other than (1): "I'm the maintainer
> or an affected subsystem developer, I approve of this patch, but I don't
> intend to take it through my own tree."

I believe we still need a separate mark for casual review. I can scan
a patch and find a few item that i might now like, but if I do not
do full review in the first pass and then again after the issues
that have been pointed out have been corrected I won't be stamping
"Reviewed-by" on the patch.

Nitpicked-by? :)

-- 
Dmitry

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Thomas Gleixner]

On Wed, 28 May 2014, Mark Brown wrote:

> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
> 
> > > There's a world of difference between thanking people for review and a
> > > detailed account of all the changes made in every single iteration of
> > > the review.
> 
> > This is already covered in Documentation/SubmittingPatches. Quoting
> > lines 585-592 (see last sentence):
> 
> Right, but Daniel is proposing lifting that above the --- and including
> it in git.

What you really want is:

Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH

It's way more useful than any of the v1-n writeups, which are most of
the time just a completely waste of electrons. Even if written well,
without the actual review context they are pretty pointless.

Thanks,

	tglx

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Thomas Gleixner]

On Wed, 28 May 2014, Josh Triplett wrote:
> 
> Or perhaps in LWN's statistics, which focus on Signed-off-by but don't
> mention Reviewed-by at all.

Oh no. The existing madness about the LWN stats is bad enough. We
don't want to encourage another excel sheet accountable competition.

It's bad enough that the stats already foster a metric ton of horrible
patches. The least we need is a metric ton of horrible reviews
conducted just to show up in the rankings.

You cannot fix the lack of capable reviewers or the lack of time of
those who are capable by any tags, statitics or whatever.

You need to look at the underlying problem. And that's at least
related by the commit statistics. The flood of crappy patches and the
amount of review cycles it takes to get even trivial stuff into an
acceptable shape is what makes the live of a maintainer and reviewer a
nightmare.

The goals of some organizations, to reach a top X contributor level in
201X, results in a frency of half baken "works for me" patches,
completely unreviewed inside of the organization and let lose on the
maintainers/reviewers who are burdened to educate the submitters.

That's the real issue. And this needs to be fixed first.

I really started to put breaks into this cycle of hell, where I get
spammed with a 30+ patch series in the morning and after I spent some
quality time looking at it and replying to a particular patch, I get
another spam bomb within a few hours, which is not much better than
the previous one.

Hell no, that does not work. No matter how many maintainers/reviewers
you have there is no way that they can scale to that global effort
which is obviously conducted to prove a newfangled variant of the
Infinite-Monkey-Theorem: Computers instead of typewriters and well
written kernel patches instead of Shakespeares writings.

Thanks,

	tglx

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Daniel Vetter]

On Wed, May 28, 2014 at 6:39 PM, Mark Brown <broonie@kernel.org> wrote:
> On Wed, May 28, 2014 at 04:39:15PM +0200, Daniel Vetter wrote:
>> My approach has been to insist on an in-patch revision log which gets
>> included in the commit. And that for any changes and bugs spotted the
>> reviewer/commenter must be acknowleged. See e.g.
>> d978ef14456a38034f6c0e for a very nice example of that. But that's
>> also a good example for no tag to acknowledge all the work that went
>> into this review/patch, since I've done the final review myself and
>> only put my sob onto the patch.
>
> This does mean that the final changelogs that get included in the kernel
> get very large and noisy and is relying on the submitters doing a good
> job paying attention to review comments in the first place, recording
> exactly what changed and so on.  They are sometimes useful but normally
> I'm finding very little value in the changelogs in the first place,
> generally it doesn't really matter what the problems were in any
> previous versions.

Thus far it didn't annoy me - it's at the bottom with the sob section.
And occasionally I've found it useful to follow some of the steps laid
out in there to understand why a patch looks what it looks like. I
know that a lot of other maintainers want the patch revision log below
the scissors.

And it also gives me a really good tool to scold patch authors if they
don't follow up on all review comments, which is the other useful
aspect. Including it in the commit makes it look a bit more valued imo
and hopefully makes sure people take this all serious.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Laurent Pinchart]

On Wednesday 28 May 2014 13:22:42 Dmitry Torokhov wrote:
> On Wednesday, May 28, 2014 01:12:28 PM josh@joshtriplett.org wrote:
> > On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> > > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > > Right now it is being used for at least two distinct and
> > > mutually-incompatible purposes:
> > > 
> > > 1. A maintainer A for code affected by a patch, who is distinct from a
> > > maintainer B queuing a patch, has reviewed the patch and has cleared it
> > > as being OK for maintainer B to send upstream
> > > 
> > > 2. A casual review has been done by someone who is not a maintainer for
> > > the code in question
> > > 
> > > What I would propose is to have the first use replaced by a new tag,
> > > "Maintainer-acked-by:", and the second use abolished, along with
> > > "Acked-by:", and replaced by "Reviewed-by:".
> > 
> > In practice, (2) seems to have been mostly replaced by "Reviewed-by"; I
> > rarely see Acked-by used for cases other than (1): "I'm the maintainer
> > or an affected subsystem developer, I approve of this patch, but I don't
> > intend to take it through my own tree."
> 
> I believe we still need a separate mark for casual review. I can scan
> a patch and find a few item that i might now like, but if I do not
> do full review in the first pass and then again after the issues
> that have been pointed out have been corrected I won't be stamping
> "Reviewed-by" on the patch.
> 
> Nitpicked-by? :)

Do such casual reviews need to be recorded in the git history ? I tend to just 
reply with "looks good to me" when I just skim through a patch, and I don't 
think that deserves being recorded for posterity.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Laurent Pinchart]

On Thursday 29 May 2014 00:48:31 Daniel Vetter wrote:
> On Wed, May 28, 2014 at 6:39 PM, Mark Brown <broonie@kernel.org> wrote:
> > On Wed, May 28, 2014 at 04:39:15PM +0200, Daniel Vetter wrote:
> >> My approach has been to insist on an in-patch revision log which gets
> >> included in the commit. And that for any changes and bugs spotted the
> >> reviewer/commenter must be acknowleged. See e.g.
> >> d978ef14456a38034f6c0e for a very nice example of that. But that's
> >> also a good example for no tag to acknowledge all the work that went
> >> into this review/patch, since I've done the final review myself and
> >> only put my sob onto the patch.
> > 
> > This does mean that the final changelogs that get included in the kernel
> > get very large and noisy and is relying on the submitters doing a good
> > job paying attention to review comments in the first place, recording
> > exactly what changed and so on.  They are sometimes useful but normally
> > I'm finding very little value in the changelogs in the first place,
> > generally it doesn't really matter what the problems were in any
> > previous versions.
> 
> Thus far it didn't annoy me - it's at the bottom with the sob section.
> And occasionally I've found it useful to follow some of the steps laid
> out in there to understand why a patch looks what it looks like. I
> know that a lot of other maintainers want the patch revision log below
> the scissors.
> 
> And it also gives me a really good tool to scold patch authors if they
> don't follow up on all review comments, which is the other useful
> aspect.

That's one particular point that I have to spend too much time on according to 
my tastes. When receiving version N+1 of a patch, I need to compare versions N 
and N+1, open the comments I've sent for version N, and verify that they all 
have been taken into account. I'd be interested in knowing about how other 
maintainers automate (part of) that process (if they do).

> Including it in the commit makes it look a bit more valued imo and hopefully
> makes sure people take this all serious.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Dmitry Torokhov]

On Thu, May 29, 2014 at 01:02:59AM +0200, Laurent Pinchart wrote:
> On Wednesday 28 May 2014 13:22:42 Dmitry Torokhov wrote:
> > On Wednesday, May 28, 2014 01:12:28 PM josh@joshtriplett.org wrote:
> > > On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> > > > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > > > Right now it is being used for at least two distinct and
> > > > mutually-incompatible purposes:
> > > > 
> > > > 1. A maintainer A for code affected by a patch, who is distinct from a
> > > > maintainer B queuing a patch, has reviewed the patch and has cleared it
> > > > as being OK for maintainer B to send upstream
> > > > 
> > > > 2. A casual review has been done by someone who is not a maintainer for
> > > > the code in question
> > > > 
> > > > What I would propose is to have the first use replaced by a new tag,
> > > > "Maintainer-acked-by:", and the second use abolished, along with
> > > > "Acked-by:", and replaced by "Reviewed-by:".
> > > 
> > > In practice, (2) seems to have been mostly replaced by "Reviewed-by"; I
> > > rarely see Acked-by used for cases other than (1): "I'm the maintainer
> > > or an affected subsystem developer, I approve of this patch, but I don't
> > > intend to take it through my own tree."
> > 
> > I believe we still need a separate mark for casual review. I can scan
> > a patch and find a few item that i might now like, but if I do not
> > do full review in the first pass and then again after the issues
> > that have been pointed out have been corrected I won't be stamping
> > "Reviewed-by" on the patch.
> > 
> > Nitpicked-by? :)
> 
> Do such casual reviews need to be recorded in the git history ? I tend to just 
> reply with "looks good to me" when I just skim through a patch, and I don't 
> think that deserves being recorded for posterity.

I personally do not care if it gets recorded or not as long as issue is
resolved but we are talking about giving more credit to reviewers to
make process more appealing...

Thanks.

-- 
Dmitry

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Williams]

On Tue, May 27, 2014 at 5:10 PM, Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>>>>>> "Jiri" == Jiri Kosina <jkosina@suse.cz> writes:
>
> Jiri> Are you implying that Linux is still not in a position to force HW
> Jiri> vendor companies to rather invest 30 man-minutes in order to have
> Jiri> a proper changelog and driver merged in Linus' tree compared to
> Jiri> receiving bad public press when they are being rejected
> Jiri> (especially for such negligible reason as changelog text)?
>
> There are a few companies in the enterprise space that get it. But in
> general it can be quite the challenge to get these vendors to see the
> value of being good Linux citizens. And these companies are much less
> susceptible to phoronix rants than entities producing consumer widgets.
>
> These companies have internal development processes that are often quite
> unfriendly to the way we work. More often than not, upstream drivers are
> trailing internal driver trees by many, many months. Patches are only
> sent upstream when there is a bit of slack in the engineering schedule.
>
> In many cases upstream Linux is not seen as an important target because
> the driver vendor will provide OEMs that ship their hardware with a
> "value added" outbox driver tarball or CD. The server vendors are
> partially to blame for keeping this dreadful anachronism alive. One
> would think that "It Just Works with RHEL/SLES/OL" would be better story
> than retrofitting tarball drivers and jeopardizing future kernel
> updates. According to the server vendors, however, customers expect to
> install an updated Linux driver just like they do on Windows. Otherwise
> the perception is that Linux is lagging behind or poorly supported.
> *sigh*
>
> Vendor driver release cycles are often tied exclusively to new silicon
> availability and internal firmware release schedules. Many vendors pay
> little to no attention to Linux development cycles. Furthermore, driver
> code is often shared between several operating systems so every patch
> needs to undergo IP/legal review before it can be submitted upstream.
> Internal commit messages often have partner references, bug numbers,
> code names, etc. in them. So it's typically easier to just drop the
> patch description than rewriting it and have the new text reviewed by
> the legal team.
>
> That's the sorry state of affairs. Part of the problem here is that many
> of the enterprise drivers have been in the kernel for a very long time.
> They predate things like staging and SubmittingPatches by many years.
> And we have been poor at communicating that things have changed and
> sometimes a bit too lenient at accepting patches.
>
> We have had a several discussions about this problem the last 6 months,
> including at LSF/MM. I have had meetings with several hw vendors this
> spring trying to educate them about our "new" rules of engagement. It's
> not easy, but I feel we're making progress. Worst case we'll send gregkh
> wielding a clue bat...

[ speaking for myself and glad to be employed by a hw vendor that
comes to a healthier conclusion ]

Isn't the fundamental problem:

HW Vendor: "Are you saying you may hold up driver updates for an
indefinite period of time?"

Kernel community: "Yes"

HW Vendor: "Ok, we'll keep sending our official updates direct to
customers as an out-of-tree tarball and let that 'upstream-thing'
happen in the background."

I.e. a conflict of "customer first" vs "upstream first"

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Laurent Pinchart]

On Wednesday 28 May 2014 16:18:45 Dmitry Torokhov wrote:
> On Thu, May 29, 2014 at 01:02:59AM +0200, Laurent Pinchart wrote:
> > On Wednesday 28 May 2014 13:22:42 Dmitry Torokhov wrote:
> > > On Wednesday, May 28, 2014 01:12:28 PM josh@joshtriplett.org wrote:
> > > > On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> > > > > Also long-overdue is a clarification on exactly what "Acked-by"
> > > > > means. Right now it is being used for at least two distinct and
> > > > > mutually-incompatible purposes:
> > > > > 
> > > > > 1. A maintainer A for code affected by a patch, who is distinct from
> > > > > a maintainer B queuing a patch, has reviewed the patch and has
> > > > > cleared it as being OK for maintainer B to send upstream
> > > > > 
> > > > > 2. A casual review has been done by someone who is not a maintainer
> > > > > for the code in question
> > > > > 
> > > > > What I would propose is to have the first use replaced by a new tag,
> > > > > "Maintainer-acked-by:", and the second use abolished, along with
> > > > > "Acked-by:", and replaced by "Reviewed-by:".
> > > > 
> > > > In practice, (2) seems to have been mostly replaced by "Reviewed-by";
> > > > I rarely see Acked-by used for cases other than (1): "I'm the
> > > > maintainer or an affected subsystem developer, I approve of this
> > > > patch, but I don't intend to take it through my own tree."
> > > 
> > > I believe we still need a separate mark for casual review. I can scan
> > > a patch and find a few item that i might now like, but if I do not
> > > do full review in the first pass and then again after the issues
> > > that have been pointed out have been corrected I won't be stamping
> > > "Reviewed-by" on the patch.
> > > 
> > > Nitpicked-by? :)
> > 
> > Do such casual reviews need to be recorded in the git history ? I tend to
> > just reply with "looks good to me" when I just skim through a patch, and
> > I don't think that deserves being recorded for posterity.
> 
> I personally do not care if it gets recorded or not as long as issue is
> resolved but we are talking about giving more credit to reviewers to
> make process more appealing...

We most certainly do, but if we want that credit to be an incentive, it has to 
have value. A casual review tag could even be seen as having a negative value. 
My opinion is most probably strongly biased on that subject though.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[josh]

On Wed, May 28, 2014 at 11:59:02PM +0200, Thomas Gleixner wrote:
> You need to look at the underlying problem. And that's at least
> related by the commit statistics. The flood of crappy patches and the
> amount of review cycles it takes to get even trivial stuff into an
> acceptable shape is what makes the live of a maintainer and reviewer a
> nightmare.
> 
> The goals of some organizations, to reach a top X contributor level in
> 201X, results in a frency of half baken "works for me" patches,
> completely unreviewed inside of the organization and let lose on the
> maintainers/reviewers who are burdened to educate the submitters.

While I do think that problem exists, I don't think the presence or
absence of commit statistics particularly motivate it.  Either way,
you'll have random driver/patch submissions motivated by "I need to ship
a product" or "entity X made it a requirement to get my driver
upstream".

> That's the real issue. And this needs to be fixed first.
> 
> I really started to put breaks into this cycle of hell, where I get
> spammed with a 30+ patch series in the morning and after I spent some
> quality time looking at it and replying to a particular patch, I get
> another spam bomb within a few hours, which is not much better than
> the previous one.

That's definitely a good workflow question.  We tell people to break
huge patches down into pieces, and that can turn substantial changes
into long patch series.  Which of the following is your primary concern:

- That people update the patch series before you're done submitting
  feedback on it?  That does seem like an annoying problem, and I
  haven't seen anyone explicitly talk about it; we should document it,
  and develop some standard boilerplate saying "If you get feedback on
  patch 04/25, don't send out v2 of the whole series until you get the
  remaining feedback on the patches".  And until users are reasonably
  educated on that problem, frequent reviewers may want to include that
  boilerplate in their patch reviews.

- That people send patch series with too many problems, and you're
  frustrated having to educate them on a dozen fundamental problems
  (formatting, standard API usage, etc) rather than just what's specific
  to your subsystem?  It seems completely reasonable to remind people of
  the standard tools, improve those standard tools as needed, and
  ideally set up more automation to run those tools before patches hit
  reviewers.  You shouldn't need to waste your time reviewing patches
  that don't build, for instance.

- That a 30-patch series is painful to review via email?  Agreed
  completely; I think we need better git-based workflows that don't
  always have to fall back to the least-common-denominator of patchbomb
  threads.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Jiri Kosina]

On Wed, 28 May 2014, Dan Williams wrote:

> Isn't the fundamental problem:
> 
> HW Vendor: "Are you saying you may hold up driver updates for an
> indefinite period of time?"
> 
> Kernel community: "Yes"
> 
> HW Vendor: "Ok, we'll keep sending our official updates direct to
> customers as an out-of-tree tarball and let that 'upstream-thing'
> happen in the background."

This puts customers into an unfortunate position, as their OS vendor is 
not going to support them with some "random" version of the driver being 
linked into OS vendor's kernel. Hence I would expect the customers to 
start asking HW vendor questions (especially if OS vendor is able to 
explain the situation to the customer properly).

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rafael J. Wysocki]

On Tuesday, May 27, 2014 08:10:36 PM Martin K. Petersen wrote:
> >>>>> "Jiri" == Jiri Kosina <jkosina@suse.cz> writes:
> 

[cut]

> 
> We have had a several discussions about this problem the last 6 months,
> including at LSF/MM. I have had meetings with several hw vendors this
> spring trying to educate them about our "new" rules of engagement. It's
> not easy, but I feel we're making progress. Worst case we'll send gregkh
> wielding a clue bat...

Rather, the Point-of-View Gun featured in the "Hitchhiker's Guide to the Galaxy" ...

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Williams]

On Wed, May 28, 2014 at 4:32 PM, Jiri Kosina <jkosina@suse.cz> wrote:
> On Wed, 28 May 2014, Dan Williams wrote:
>
>> Isn't the fundamental problem:
>>
>> HW Vendor: "Are you saying you may hold up driver updates for an
>> indefinite period of time?"
>>
>> Kernel community: "Yes"
>>
>> HW Vendor: "Ok, we'll keep sending our official updates direct to
>> customers as an out-of-tree tarball and let that 'upstream-thing'
>> happen in the background."
>
> This puts customers into an unfortunate position, as their OS vendor is
> not going to support them with some "random" version of the driver being
> linked into OS vendor's kernel. Hence I would expect the customers to
> start asking HW vendor questions (especially if OS vendor is able to
> explain the situation to the customer properly).
>

I've been on both sides of the producer/consumer coin.  For
environments that roll their own kernel upstream acceptance is
preferred, but not necessarily required.  Also, for the enterprise
storage driver I worked on its dependency on core libraries turned out
to be a liability.  OS vendors sometimes have a different update
cadences for common components vs standalone drivers.  Customers are
caught in the gap and exert pressure, to whomever will listen, to get
their hardware to "just work (TM)".

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Thomas Gleixner]

On Wed, 28 May 2014, josh@joshtriplett.org wrote:
> On Wed, May 28, 2014 at 11:59:02PM +0200, Thomas Gleixner wrote:
> > You need to look at the underlying problem. And that's at least
> > related by the commit statistics. The flood of crappy patches and the
> > amount of review cycles it takes to get even trivial stuff into an
> > acceptable shape is what makes the live of a maintainer and reviewer a
> > nightmare.
> > 
> > The goals of some organizations, to reach a top X contributor level in
> > 201X, results in a frency of half baken "works for me" patches,
> > completely unreviewed inside of the organization and let lose on the
> > maintainers/reviewers who are burdened to educate the submitters.
> 
> While I do think that problem exists, I don't think the presence or
> absence of commit statistics particularly motivate it.  Either way,

It does. I've heard and seen written statements, that corporate goals
are to show up in the top X rank of lwn stats. That's the sad truth.

The reason is simple. stats are an "objective" measure for managers,
while patch quality cannot be expressed in any excel sheet.

> you'll have random driver/patch submissions motivated by "I need to ship
> a product" or "entity X made it a requirement to get my driver
> upstream".

That does not excuse that exactly those people try to offload the
education of their engineers to the maintainers/reviewers. That simply
does not work and it does not make sense either.
 
> > That's the real issue. And this needs to be fixed first.
> > 
> > I really started to put breaks into this cycle of hell, where I get
> > spammed with a 30+ patch series in the morning and after I spent some
> > quality time looking at it and replying to a particular patch, I get
> > another spam bomb within a few hours, which is not much better than
> > the previous one.
> 
> That's definitely a good workflow question.  We tell people to break
> huge patches down into pieces, and that can turn substantial changes
> into long patch series.  Which of the following is your primary concern:
> 
> - That people update the patch series before you're done submitting
>   feedback on it?  That does seem like an annoying problem, and I
>   haven't seen anyone explicitly talk about it; we should document it,
>   and develop some standard boilerplate saying "If you get feedback on
>   patch 04/25, don't send out v2 of the whole series until you get the
>   remaining feedback on the patches".  And until users are reasonably
>   educated on that problem, frequent reviewers may want to include that
>   boilerplate in their patch reviews.

I'm happy to add a boiler plate if that actually makes people to think
about what they are doing. I just doubt that it works.
 
> - That a 30-patch series is painful to review via email?  Agreed
>   completely; I think we need better git-based workflows that don't
>   always have to fall back to the least-common-denominator of patchbomb
>   threads.

No. I rather review a 30+ patch series in email than going through
loops and hoops of git tools and whatever. That's a non starter as
trying to force people to deal with bugzilla.

You are trying again to solve a non technical problem with technical
measures. That does not work. The mail volume is not the issue. I can
easily delete a whole thread with one keystroke.

The issue is the quality and you are not going to fix that by git
based workflows or whatever tools you dream of. A large well thought
of large patch queue is very well manageable in mail.

Crap stays crap no matter what tools you are using to relay it. You
need to fix the crap issue before you even start to think about better
tools.

Thanks,

	tglx

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 4950 bytes --]

On Tue, 2014-05-27 at 20:10 -0400, Martin K. Petersen wrote:
> >>>>> "Jiri" == Jiri Kosina <jkosina@suse.cz> writes:
> 
> Jiri> Are you implying that Linux is still not in a position to force HW
> Jiri> vendor companies to rather invest 30 man-minutes in order to have
> Jiri> a proper changelog and driver merged in Linus' tree compared to
> Jiri> receiving bad public press when they are being rejected
> Jiri> (especially for such negligible reason as changelog text)?
> 
> There are a few companies in the enterprise space that get it. But in
> general it can be quite the challenge to get these vendors to see the
> value of being good Linux citizens. And these companies are much less
> susceptible to phoronix rants than entities producing consumer widgets.

For consumer widgets, it appears that most vendors don't update the
software, so they won't care that any out-of-tree drivers may break on
the next kernel version.  In fact, they're likely to be starting with
their SoC vendor's kernel tree which was forked a few years ago and has
a ton of dirty performance hacks in the core.

The 'enterprise' world seems to be in a comparatively much better state.

> These companies have internal development processes that are often quite
> unfriendly to the way we work. More often than not, upstream drivers are
> trailing internal driver trees by many, many months. Patches are only
> sent upstream when there is a bit of slack in the engineering schedule.
> 
> In many cases upstream Linux is not seen as an important target because
> the driver vendor will provide OEMs that ship their hardware with a
> "value added" outbox driver tarball or CD. The server vendors are
> partially to blame for keeping this dreadful anachronism alive. One
> would think that "It Just Works with RHEL/SLES/OL" would be better story
> than retrofitting tarball drivers and jeopardizing future kernel
> updates.

In my experience working for an 'enterprise' hardware vendor, there was
definitely pressure from OEMs to get the hardware supported in mainline
or in their favoured distribution (and distribution maintainers require
them to be upstream first).  I seem to recall one OEM formally requiring
in-tree drivers for all components aside from graphics.  But they
usually wanted out-of-tree driver packages *as well*, since 'enterprise'
customers do like to run old kernel versions.

> According to the server vendors, however, customers expect to
> install an updated Linux driver just like they do on Windows. Otherwise
> the perception is that Linux is lagging behind or poorly supported.
> *sigh*

Chip/board vendors and OEMs often like to differentiate themselves in a
competitive market by implementing unique features and adding special
applications and driver interfaces to support these.

Linux subsystem maintainers may push back against such interfaces -
often because they are not all that unique, or not likely to remain so.
David Miller is particularly strict about this; other maintainers may be
less so.  This does not block differentiation, but it means those
features may be included only in the OOT driver.  When an OEM is
interested in differentiation, then of course it will encourage use of
the OOT driver.

> Vendor driver release cycles are often tied exclusively to new silicon
> availability and internal firmware release schedules. Many vendors pay
> little to no attention to Linux development cycles. Furthermore, driver
> code is often shared between several operating systems so every patch
> needs to undergo IP/legal review before it can be submitted upstream.
> Internal commit messages often have partner references, bug numbers,
> code names, etc. in them. So it's typically easier to just drop the
> patch description than rewriting it and have the new text reviewed by
> the legal team.
[...]

I never had to go through legal review, though I certainly sanitised
some commit messages based on my own knowledge of what product
information was and wasn't already public.

Other reasons I had for needing to write new commit messages:

- The internal commit message had just a bug number and one line of
summary, because the internal bug tracker had the full explanation
- I combined and/or split some internal commits, because we discovered a
regression before sending them upstream

Driver development for a new generation of hardware may begin long
before it is publicly announced, and may contain many false starts or
temporary hacks to work around pre-release hardware or simulations.
Somehow, that messy development branch needs to be turned into some
semblance of coherent refactoring when sending upstream.  Either that or
you fork and rename the driver for the new generation, and send it
upstream with no history at all.

Ben.

-- 
Ben Hutchings
Absolutum obsoletum. (If it works, it's out of date.) - Stafford Beer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mimi Zohar]

On Wed, 2014-05-28 at 16:31 -0700, josh@joshtriplett.org wrote:
> 
> > That's the real issue. And this needs to be fixed first.
> > 
> > I really started to put breaks into this cycle of hell, where I get
> > spammed with a 30+ patch series in the morning and after I spent some
> > quality time looking at it and replying to a particular patch, I get
> > another spam bomb within a few hours, which is not much better than
> > the previous one.
> 
> That's definitely a good workflow question.  We tell people to break
> huge patches down into pieces, and that can turn substantial changes
> into long patch series.

Sometimes it isn't possible, or desirable, to break up large patch sets,
but for the most part that isn't the case.  The next step would be to
have all of the patches within a patch set be related.

Mimi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Randy Dunlap]

On 05/28/2014 05:39 PM, Mimi Zohar wrote:
> On Wed, 2014-05-28 at 16:31 -0700, josh@joshtriplett.org wrote:
>>
>>> That's the real issue. And this needs to be fixed first.
>>>
>>> I really started to put breaks into this cycle of hell, where I get
>>> spammed with a 30+ patch series in the morning and after I spent some
>>> quality time looking at it and replying to a particular patch, I get
>>> another spam bomb within a few hours, which is not much better than
>>> the previous one.
>>
>> That's definitely a good workflow question.  We tell people to break
>> huge patches down into pieces, and that can turn substantial changes
>> into long patch series.
> 
> Sometimes it isn't possible, or desirable, to break up large patch sets,
> but for the most part that isn't the case.  The next step would be to
> have all of the patches within a patch set be related.

from Documentation/SubmittingPatches:

patch series (where a "patch 
series" is an ordered sequence of multiple, related patches).

Not that anyone reads documentation.


-- 
~Randy

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mimi Zohar]

On Wed, 2014-05-28 at 17:47 -0700, Randy Dunlap wrote: 
> On 05/28/2014 05:39 PM, Mimi Zohar wrote:
> > On Wed, 2014-05-28 at 16:31 -0700, josh@joshtriplett.org wrote:
> >>
> >>> That's the real issue. And this needs to be fixed first.
> >>>
> >>> I really started to put breaks into this cycle of hell, where I get
> >>> spammed with a 30+ patch series in the morning and after I spent some
> >>> quality time looking at it and replying to a particular patch, I get
> >>> another spam bomb within a few hours, which is not much better than
> >>> the previous one.
> >>
> >> That's definitely a good workflow question.  We tell people to break
> >> huge patches down into pieces, and that can turn substantial changes
> >> into long patch series.
> > 
> > Sometimes it isn't possible, or desirable, to break up large patch sets,
> > but for the most part that isn't the case.  The next step would be to
> > have all of the patches within a patch set be related.
> 
> from Documentation/SubmittingPatches:
> 
> patch series (where a "patch 
> series" is an ordered sequence of multiple, related patches).
> 
> Not that anyone reads documentation.

Thanks, Randy, for the reminder.

Mimi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Rob Herring]

On Wed, May 28, 2014 at 1:47 PM, Paul Walmsley <paul@pwsan.com> wrote:
> On Sat, 24 May 2014, James Bottomley wrote:
>
>> I'm sure there are many other things people could suggest.
>
> What's needed is to bring quality reviewers up to the same level of
> recognition and control as maintainers.
>
> Ideally, maintainers would recognize quality reviewers, and list them in
> the MAINTAINERS file - perhaps with an "R:" tag?  Maintainers would be
> expected to designate at least one quality reviewer, but ideally more, for
> a given subsystem.
>
> Then we should require every patch to have at least one "Reviewed-by:",
> aside from the maintainer's "Signed-off-by:" before being merged.  This
> "Reviewed-by:" could come from the maintainer, but ideally would come from
> a quality reviewer.
>
> Patch submitters would need to get their patches reviewed by at least one
> of the recognized reviewers before expecting it to be merged.
>
> Part of the goal here would also be to convert quality reviewers into
> co-maintainers over time, so maintainership duties can be spread among a
> larger group of people.

What really needs to change here as we already essentially have this
today. Getting more reviewer bandwidth is why we have 5 DT binding
maintainers. DT bindings are a bit unique in that almost everything
goes in thru other maintainers trees, so the role is almost entirely
reviews. But what's to say a co-maintainers role is not solely
reviews. How co-maintainers split up the load is really an internal
decision among them.

Do we really have people we trust to review that we wouldn't trust to
be a co-maintainer?

Rob

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Olof Johansson]

On Wed, May 28, 2014 at 7:15 PM, Rob Herring <robherring2@gmail.com> wrote:
> On Wed, May 28, 2014 at 1:47 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> On Sat, 24 May 2014, James Bottomley wrote:
>>
>>> I'm sure there are many other things people could suggest.
>>
>> What's needed is to bring quality reviewers up to the same level of
>> recognition and control as maintainers.
>>
>> Ideally, maintainers would recognize quality reviewers, and list them in
>> the MAINTAINERS file - perhaps with an "R:" tag?  Maintainers would be
>> expected to designate at least one quality reviewer, but ideally more, for
>> a given subsystem.
>>
>> Then we should require every patch to have at least one "Reviewed-by:",
>> aside from the maintainer's "Signed-off-by:" before being merged.  This
>> "Reviewed-by:" could come from the maintainer, but ideally would come from
>> a quality reviewer.
>>
>> Patch submitters would need to get their patches reviewed by at least one
>> of the recognized reviewers before expecting it to be merged.
>>
>> Part of the goal here would also be to convert quality reviewers into
>> co-maintainers over time, so maintainership duties can be spread among a
>> larger group of people.
>
> What really needs to change here as we already essentially have this
> today. Getting more reviewer bandwidth is why we have 5 DT binding
> maintainers. DT bindings are a bit unique in that almost everything
> goes in thru other maintainers trees, so the role is almost entirely
> reviews. But what's to say a co-maintainers role is not solely
> reviews. How co-maintainers split up the load is really an internal
> decision among them.

I honestly think 5 are too many from some perspectives, but I also
understand why it's needed in this case due to sheer volume of
patches.

The main drawback, that I have complained a little about at random
moments lately, is that the 5 maintainers have quite different
feedback to give and it's hard for a contributor to know what to do to
please the union set of reviewers. Preferences shift for everybody
over time, but with 5x the shifting, it can be annoying to deal with.
One maintainer's "good enough, let's pick it up" can be "oh no, you
really need to change this -- everybody got this wrong in the past so
don't look at existing code" for someone else.

> Do we really have people we trust to review that we wouldn't trust to
> be a co-maintainer?

It's not about trust as much as practicalities of shared git repos and
the overhead of getting that going (in some cases).

To me as a patch poster, if I send a patch to a maintainer and he
finds it good, I expect it to show up in linux-next shortly and then
be on the way in for the next merge window. For a reviewer, this might
not be the case since he/she will just reply with an ack and then the
patch might or might not show up in a tree at some point in the
future. That's why I'm somewhat hesitant to label every reviewer a
maintainer since it's hard to tune expectations.


-Olof

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Martin K. Petersen]

>>>>> "Dan" == Dan Williams <dan.j.williams@intel.com> writes:

Dan,

Dan> Isn't the fundamental problem:

Dan> HW Vendor: "Are you saying you may hold up driver updates for an
Dan> indefinite period of time?"

Dan> Kernel community: "Yes"

There's a big difference between a driver bug fix and "value added"
features that are deemed too ugly to be included upstream.


Dan> HW Vendor: "Ok, we'll keep sending our official updates direct to
Dan> customers as an out-of-tree tarball and let that 'upstream-thing'
Dan> happen in the background."

Well, and thus forcing the customer to void their distro support. I
don't think that's doing anyone a favor.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Martin K. Petersen]

>>>>> "Ben" == Ben Hutchings <ben@decadent.org.uk> writes:

Ben> I seem to recall one OEM formally requiring in-tree drivers for all
Ben> components aside from graphics.  But they usually wanted
Ben> out-of-tree driver packages *as well*, since 'enterprise' customers
Ben> do like to run old kernel versions.

Cc: stable :)

Ben> Driver development for a new generation of hardware may begin long
Ben> before it is publicly announced, and may contain many false starts
Ben> or temporary hacks to work around pre-release hardware or
Ben> simulations.  Somehow, that messy development branch needs to be
Ben> turned into some semblance of coherent refactoring when sending
Ben> upstream.  Either that or you fork and rename the driver for the
Ben> new generation, and send it upstream with no history at all.

I agree that the history situation is different for completely new
drivers. However, the problem at hand is code we've had in the kernel
for a long time. It is still widely used but has bitrotted or is in a
shape that would never get accepted into the kernel by today's
standards.

Vendors use arguments like "we developed and tested this outbox driver
version x.y.z thoroughly in a 2.6.15 environment on i386. Didn't see any
problems, therefore it works with every kernel a customer might
run". Such a statement may carry some weight in a stable kernel
interface world but it is completely meaningless in our case.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Williams]

On Wed, May 28, 2014 at 9:01 PM, Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>>>>>> "Dan" == Dan Williams <dan.j.williams@intel.com> writes:
>
> Dan,
>
> Dan> Isn't the fundamental problem:
>
> Dan> HW Vendor: "Are you saying you may hold up driver updates for an
> Dan> indefinite period of time?"
>
> Dan> Kernel community: "Yes"
>
> There's a big difference between a driver bug fix and "value added"
> features that are deemed too ugly to be included upstream.

Yes, ill conceived "value add" is indeed toxic.  Is that the only
contributor factor to indefinite patch acceptance latency?  We're
missing a mechanism to allow for experimentation without 1/ risking
the quality of the rest of the kernel 2/ committing to carrying the
experiment upstream indefinitely.  Upstream acceptance should not be a
precedent setting event.

> Dan> HW Vendor: "Ok, we'll keep sending our official updates direct to
> Dan> customers as an out-of-tree tarball and let that 'upstream-thing'
> Dan> happen in the background."
>
> Well, and thus forcing the customer to void their distro support. I
> don't think that's doing anyone a favor.
>

Assuming everyone consumes enterprise Linux via an enterprise OSV then
there isn't a problem.  Outside of that, the HW vendor is forced to
either cover the gap, or watch their competitor do it.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Wed, 2014-05-28 at 16:31 -0700, josh@joshtriplett.org wrote:
> On Wed, May 28, 2014 at 11:59:02PM +0200, Thomas Gleixner wrote:
> > You need to look at the underlying problem. And that's at least
> > related by the commit statistics. The flood of crappy patches and the
> > amount of review cycles it takes to get even trivial stuff into an
> > acceptable shape is what makes the live of a maintainer and reviewer a
> > nightmare.
> > 
> > The goals of some organizations, to reach a top X contributor level in
> > 201X, results in a frency of half baken "works for me" patches,
> > completely unreviewed inside of the organization and let lose on the
> > maintainers/reviewers who are burdened to educate the submitters.
> 
> While I do think that problem exists, I don't think the presence or
> absence of commit statistics particularly motivate it.  Either way,
> you'll have random driver/patch submissions motivated by "I need to ship
> a product" or "entity X made it a requirement to get my driver
> upstream".

I suspect, but cannot prove, it is somewhat driven by the patch
statistics.  I know in Parallels, I track features accepted into the
kernel, not patches, but we do have a patch metric as well for people
who do incidental bug fixing as they push features, so we're not 100%
pure on this.

When I recently gave a comparison of OpenStack and the Kernel, one thing
that stood out starkly was that they just don't have our volume of one
line, one patch per file hundreds of times and trivial changes, so we
definitely have a problem in this area.

The reason OpenStack has so few trivial and multiply split patches is
mostly grounded in the difficulty of submitting a patch to their tree.
Just preparing it for gerrit takes a long time (around ten minutes per
patch), which really reduces the volume of trivial patches, just because
no-one has that much energy (although some would argue it reduces the
volume too far).

It's human nature to spend more time on the easy problems, so a flood of
trivial patches which are "obviously" correct is easier to review and
include than one patch which alters something deep within mm and needs
careful thought.  At best, excessively split trivial patches are a
dilution of our review effort and at worst, they're actively sucking
people away from tackling the hard problems.

> > That's the real issue. And this needs to be fixed first.
> > 
> > I really started to put breaks into this cycle of hell, where I get
> > spammed with a 30+ patch series in the morning and after I spent some
> > quality time looking at it and replying to a particular patch, I get
> > another spam bomb within a few hours, which is not much better than
> > the previous one.
> 
> That's definitely a good workflow question.  We tell people to break
> huge patches down into pieces, and that can turn substantial changes
> into long patch series.  Which of the following is your primary concern:
> 
> - That people update the patch series before you're done submitting
>   feedback on it?  That does seem like an annoying problem, and I
>   haven't seen anyone explicitly talk about it; we should document it,
>   and develop some standard boilerplate saying "If you get feedback on
>   patch 04/25, don't send out v2 of the whole series until you get the
>   remaining feedback on the patches".  And until users are reasonably
>   educated on that problem, frequent reviewers may want to include that
>   boilerplate in their patch reviews.
> 
> - That people send patch series with too many problems, and you're
>   frustrated having to educate them on a dozen fundamental problems
>   (formatting, standard API usage, etc) rather than just what's specific
>   to your subsystem?  It seems completely reasonable to remind people of
>   the standard tools, improve those standard tools as needed, and
>   ideally set up more automation to run those tools before patches hit
>   reviewers.  You shouldn't need to waste your time reviewing patches
>   that don't build, for instance.
> 
> - That a 30-patch series is painful to review via email?  Agreed
>   completely; I think we need better git-based workflows that don't
>   always have to fall back to the least-common-denominator of patchbomb
>   threads.

This is a taste question.  I really appreciate a well split patch series
tackling a hard problem because it makes working out what changed and
why a lot easier than in a single monolith ... and as a reviewer, that's
gold dust.  I don't appreciate 30 patches changing the same thing in 30
different files.

I see a lot of patches generated by semantic patch scripts (once per
file)  where we add a pattern to the kernel for something (say replacing
kmalloc followed by memset with kzalloc) and people immediately use
semantic patching to change the other thousand places in the kernel
where this pattern was used ... regardless of whether the use they're
changing was correct or not.

I think for the pattern case, we have to agree when adding the pattern
whether it's useful or essential, and in the latter case do a big bang
change if we agree it's so essential that all current uses must change
so as not to have bad pattern examples.


James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Carpenter]

Under my scheme only *real* bug fixes would get a tag.

        fixup aperture and cmap frees (Imre)  <--  bug fix
        fix curly brace around DSPADDR check (Imre)  <-- not a bug fix

Nit picking code is its own reward and doesn't need a special tag.

regards,
dan carpenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Jonathan Cameron]

On 29/05/14 03:15, Rob Herring wrote:
> On Wed, May 28, 2014 at 1:47 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> On Sat, 24 May 2014, James Bottomley wrote:
>>
>>> I'm sure there are many other things people could suggest.
>>
>> What's needed is to bring quality reviewers up to the same level of
>> recognition and control as maintainers.
>>
>> Ideally, maintainers would recognize quality reviewers, and list them in
>> the MAINTAINERS file - perhaps with an "R:" tag?  Maintainers would be
>> expected to designate at least one quality reviewer, but ideally more, for
>> a given subsystem.
>>
>> Then we should require every patch to have at least one "Reviewed-by:",
>> aside from the maintainer's "Signed-off-by:" before being merged.  This
>> "Reviewed-by:" could come from the maintainer, but ideally would come from
>> a quality reviewer.
>>
>> Patch submitters would need to get their patches reviewed by at least one
>> of the recognized reviewers before expecting it to be merged.
>>
>> Part of the goal here would also be to convert quality reviewers into
>> co-maintainers over time, so maintainership duties can be spread among a
>> larger group of people.
>
> What really needs to change here as we already essentially have this
> today. Getting more reviewer bandwidth is why we have 5 DT binding
> maintainers. DT bindings are a bit unique in that almost everything
> goes in thru other maintainers trees, so the role is almost entirely
> reviews. But what's to say a co-maintainers role is not solely
> reviews. How co-maintainers split up the load is really an internal
> decision among them.
>
> Do we really have people we trust to review that we wouldn't trust to
> be a co-maintainer?
There are people who effectively fulfil this role already but for
are perhaps under pressure from their employer not to formally take
on a co-maintainer role? (this happened to me a while back - I
think it might have been something that had to go a few too many steps
up into management...)  Note that the person in question took on most
of the role in reality anyway but isn't formally acknowledged for it.
   
>
> Rob
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Li Zefan]

On 2014/5/29 5:32, Thomas Gleixner wrote:
> On Wed, 28 May 2014, Mark Brown wrote:
> 
>> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
>>
>>>> There's a world of difference between thanking people for review and a
>>>> detailed account of all the changes made in every single iteration of
>>>> the review.
>>
>>> This is already covered in Documentation/SubmittingPatches. Quoting
>>> lines 585-592 (see last sentence):
>>
>> Right, but Daniel is proposing lifting that above the --- and including
>> it in git.
> 
> What you really want is:
> 
> Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
> 
> It's way more useful than any of the v1-n writeups, which are most of
> the time just a completely waste of electrons. Even if written well,
> without the actual review context they are pretty pointless.
> 

A QA asked me about kernel development process. One of his question is,
he found some valuable information in the discussion of the patch often
won't be added to the changelog, so providing the commit how to find
the discussion?

You can simply google the subject or use the link in the changelog,
and the latter is much more convinient. I'd like to see the use of
link is more widely adopted by other maintainers.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Daniel Vetter]

On Thu, May 29, 2014 at 1:31 AM,  <josh@joshtriplett.org> wrote:
>> I really started to put breaks into this cycle of hell, where I get
>> spammed with a 30+ patch series in the morning and after I spent some
>> quality time looking at it and replying to a particular patch, I get
>> another spam bomb within a few hours, which is not much better than
>> the previous one.
>
> That's definitely a good workflow question.  We tell people to break
> huge patches down into pieces, and that can turn substantial changes
> into long patch series.  Which of the following is your primary concern:
>
> - That people update the patch series before you're done submitting
>   feedback on it?  That does seem like an annoying problem, and I
>   haven't seen anyone explicitly talk about it; we should document it,
>   and develop some standard boilerplate saying "If you get feedback on
>   patch 04/25, don't send out v2 of the whole series until you get the
>   remaining feedback on the patches".  And until users are reasonably
>   educated on that problem, frequent reviewers may want to include that
>   boilerplate in their patch reviews.

What I enforce on our mailing list is that for updates for issues in
specific patches submitters should resend only that patch in-reply-to
the old one. It means a bit more work of fiddling out the msg-id and
submitters complain (but meh), but it does nicely keep the discussion
together. And it allows submitters to chase reviewer's feedback while
the review is still ongoing without causing hell on the m-l. This
makes for a bit of fun when applying patches, but if it gets messy I
just ask for a full resend with r-b tags and all included once it's
all done.

Of course patchbombs which are fundamentally wrong need to be resent
hole. And for those I tell submitters I tell them pretty sternly that
they need to wait with resending until the review round concluded if
they fail to do so.

If course

> - That people send patch series with too many problems, and you're
>   frustrated having to educate them on a dozen fundamental problems
>   (formatting, standard API usage, etc) rather than just what's specific
>   to your subsystem?  It seems completely reasonable to remind people of
>   the standard tools, improve those standard tools as needed, and
>   ideally set up more automation to run those tools before patches hit
>   reviewers.  You shouldn't need to waste your time reviewing patches
>   that don't build, for instance.

Imo checkpatch makes this worse since with that you get patches that
look pretty but are still shit. Imo we need to push back on companies
who consistenly submit crap patches and tell them to get some good
internal mentoring going. Or throw money at lf/linaro/whatever to get
such expertise if they completely lack it.

And for me hobbyist/gsoc/... are completely different and I'll give
them much more leeway simply because I've started like that myself.

Of course for both these points Intel mostly gets this and I only have
issues with new people joining upstream from e.g. Android teams If
they don't get it I escalate through their managers _real_ quickly.
Maybe we should require from companies some "you have a loose cannon,
take care of it" contact, at least those in lf/linaro/... to make sure
this gets addressed quickly? Since ime sternly written mails at
engineers is usually not sufficient, escalation is usually required.

> - That a 30-patch series is painful to review via email?  Agreed
>   completely; I think we need better git-based workflows that don't
>   always have to fall back to the least-common-denominator of patchbomb
>   threads.

Nope if you mean tools like gerrit. The android gfx guys use it and I
think it's horrible. Mail-based patchbomb review works, and tool like
gerrit has all the sigils of a working review process but imo
completely misses the point. From a UI perspective the web thing is a
disaster, and everything else has too many indirections until you can
finally look at the patch. The only upside I see is that managers like
it since it helps to them to keep track of things. Not something
upstream cares about.

For our review we're looking into augmenting mail-based review with
stuff like patchwork, but atm patchwork is still lacking a lot. But at
least it's opt-in for everyone (even if a given maintainer uses it)
and doesn't replace the main mail discussions at all.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Christoph Lameter]

On Thu, 29 May 2014, Laurent Pinchart wrote:

> We most certainly do, but if we want that credit to be an incentive, it has to
> have value. A casual review tag could even be seen as having a negative value.
> My opinion is most probably strongly biased on that subject though.


There are patches that straddle multiple subsystems. Maybe we need
something to indicate that the portion relevant to a certain subsystems
have been approved? I often see these with the slab allocators and
sometimes I just ack them to indicate that the slab portions are fine
thinking people know that this is in my role as slab maintainer.

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Laurent Pinchart]

On Thursday 29 May 2014 09:44:11 Christoph Lameter wrote:
> On Thu, 29 May 2014, Laurent Pinchart wrote:
> > We most certainly do, but if we want that credit to be an incentive, it
> > has to have value. A casual review tag could even be seen as having a
> > negative value. My opinion is most probably strongly biased on that
> > subject though.
>
> There are patches that straddle multiple subsystems. Maybe we need
> something to indicate that the portion relevant to a certain subsystems
> have been approved? I often see these with the slab allocators and
> sometimes I just ack them to indicate that the slab portions are fine
> thinking people know that this is in my role as slab maintainer.

I usually reply with a "For driver/subsystem xyz, Acked-by ....". Only the 
Acked-by is recorded in the git history though, but that might not be an issue 
as we can always go back to the mailing list archives if we really want to 
know who is to blame for a too casual review.

On the other hand, this can be an issue for developers and/or maintainers who 
want to ensure that all parts of a patch have received proper review. That's 
why I sometimes split patches that perform a simple change to multiple drivers 
in a series with one patch per driver, and then squash everything into a 
single patch before submitting a pull request. That workflow could probably be 
improved.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Luck, Tony]

> - That a 30-patch series is painful to review via email?  Agreed
>  completely; I think we need better git-based workflows that don't
>  always have to fall back to the least-common-denominator of patchbomb
>  threads.

I'm lucky that nobody sends me 30-patch series for ia64 ...  but if they
did I'd try to manage the follow-up flow by setting up a topic branch in
git and pushing to kernel.org.  Maybe parts 1-5 of the series are OK
and I'd commit them to the branch. Next review cycle would only have
to deal with the remaining 25 patches.

Perhaps the submitter would see this, and spend extra efforts to make
sure all issues in the first 4-8 patches of the remainder are fixed (while keeping
the rest as context for where this whole train is headed). So we'll nibble
the problem down by stages.  The submitter would also see definite
signs of progress.

In some cases we might get to part27 and realize that something was
wrong in part3 ... that's OK ... a topic branch in a maintainer sub-tree
can be rebased as long as all parties to the discussion know that it
is happening.

-Tony

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[H. Peter Anvin]

On 05/28/2014 11:48 AM, Paul Walmsley wrote:
> 
> Also long-overdue is a clarification on exactly what "Acked-by" means.
> Right now it is being used for at least two distinct and
> mutually-incompatible purposes:
> 
> 1. A maintainer A for code affected by a patch, who is distinct from a
> maintainer B queuing a patch, has reviewed the patch and has cleared it as
> being OK for maintainer B to send upstream
> 
> 2. A casual review has been done by someone who is not a maintainer for
> the code in question
> 
> What I would propose is to have the first use replaced by a new tag, 
> "Maintainer-acked-by:", and the second use abolished, along with 
> "Acked-by:", and replaced by "Reviewed-by:".
> 

The distinction is on the right side of the colon.  It's a matter of
reputation.

	-hpa

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Christoph Lameter]

On Thu, 29 May 2014, Laurent Pinchart wrote:

> On the other hand, this can be an issue for developers and/or maintainers who
> want to ensure that all parts of a patch have received proper review. That's
> why I sometimes split patches that perform a simple change to multiple drivers
> in a series with one patch per driver, and then squash everything into a
> single patch before submitting a pull request. That workflow could probably be
> improved.

Well that in turn may lead to breakage if modifications to only some files
are merged. If the modifications in multiple files are not depending on
one another then they could go in as separate patches in the first place.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]

On Thu, May 29, 2014 at 01:35:45AM +0100, Ben Hutchings wrote:
> On Tue, 2014-05-27 at 20:10 -0400, Martin K. Petersen wrote:

> > There are a few companies in the enterprise space that get it. But in
> > general it can be quite the challenge to get these vendors to see the
> > value of being good Linux citizens. And these companies are much less
> > susceptible to phoronix rants than entities producing consumer widgets.

> For consumer widgets, it appears that most vendors don't update the
> software, so they won't care that any out-of-tree drivers may break on
> the next kernel version.  In fact, they're likely to be starting with
> their SoC vendor's kernel tree which was forked a few years ago and has
> a ton of dirty performance hacks in the core.

This is very true, to a good approximation nobody in the consumer
electronics space cares in the slightest about upstream - it's just not
useful as-is.  These days it's likely to be the current Android kernel
at the time the SoC was current.  There is some pressure from some of
the more clueful system integrators to be working with upstream, partly
for bringing things forwards on new devices and partly for leveraging
the review.

> > Vendor driver release cycles are often tied exclusively to new silicon
> > availability and internal firmware release schedules. Many vendors pay
> > little to no attention to Linux development cycles. Furthermore, driver
> > code is often shared between several operating systems so every patch
> > needs to undergo IP/legal review before it can be submitted upstream.
> > Internal commit messages often have partner references, bug numbers,
> > code names, etc. in them. So it's typically easier to just drop the
> > patch description than rewriting it and have the new text reviewed by
> > the legal team.
> [...]

> I never had to go through legal review, though I certainly sanitised
> some commit messages based on my own knowledge of what product
> information was and wasn't already public.

Indeed, particularly around errata.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Thu, May 29, 2014 at 05:28:11PM +0800, Li Zefan wrote:
> On 2014/5/29 5:32, Thomas Gleixner wrote:
> > On Wed, 28 May 2014, Mark Brown wrote:
> > 
> >> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
> >>
> >>>> There's a world of difference between thanking people for review and a
> >>>> detailed account of all the changes made in every single iteration of
> >>>> the review.
> >>
> >>> This is already covered in Documentation/SubmittingPatches. Quoting
> >>> lines 585-592 (see last sentence):
> >>
> >> Right, but Daniel is proposing lifting that above the --- and including
> >> it in git.
> > 
> > What you really want is:
> > 
> > Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
> > 
> > It's way more useful than any of the v1-n writeups, which are most of
> > the time just a completely waste of electrons. Even if written well,
> > without the actual review context they are pretty pointless.
> > 
> 
> A QA asked me about kernel development process. One of his question is,
> he found some valuable information in the discussion of the patch often
> won't be added to the changelog, so providing the commit how to find
> the discussion?

A research group has created a tool that takes a given git commit, finds
the mailing list discussion for that patch.  It was posted to lkml 6 or
so months ago, you should point them at that tool if they want to do
this.

greg k-h

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Theodore Ts'o]

On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> 
> Also long-overdue is a clarification on exactly what "Acked-by" means.
> Right now it is being used for at least two distinct and
> mutually-incompatible purposes:
> 
> 1. A maintainer A for code affected by a patch, who is distinct from a
> maintainer B queuing a patch, has reviewed the patch and has cleared it as
> being OK for maintainer B to send upstream
> 
> 2. A casual review has been done by someone who is not a maintainer for
> the code in question
> 
> What I would propose is to have the first use replaced by a new tag, 
> "Maintainer-acked-by:", and the second use abolished, along with 
> "Acked-by:", and replaced by "Reviewed-by:".

I agree in general, but if we're going to abolish the 2nd use
entirely, then it's much simpler to keep Acked-by for its original
meaning; it's easier to type, after all.

This is basically I do for ext4 patches today; if someone sends me an
acked-by in the #2 sense, I simply won't add it to the s-o-b section,
and I don't let the fact that someone has asserted that they have done
a casual review to give me a false sense of security; if I still have
to do a deep review, I'm going to catch the casual stuff anyway, and
the fact that a casual review doesn't obviate the need for a careful
review.

But if a senior ext4 developer adds a Reviewed-by:, that does lend a
lot of value to me as a maintainer, since I can trust that certain
folks like Jan and Eric and Lukas and others will do a good job doing
the review, and that actually *does* offload significant amounts of
work off my shoulders.

Cheers,

						- Ted

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Thu, May 29, 2014 at 12:31:52PM +0200, Daniel Vetter wrote:
> Of course for both these points Intel mostly gets this and I only have
> issues with new people joining upstream from e.g. Android teams If
> they don't get it I escalate through their managers _real_ quickly.
> Maybe we should require from companies some "you have a loose cannon,
> take care of it" contact, at least those in lf/linaro/... to make sure
> this gets addressed quickly? Since ime sternly written mails at
> engineers is usually not sufficient, escalation is usually required.

The LF does have such a list, email me, or the LF Tech board if you need
such a contact for an issue like this.  We've done it in the past and
have no problem doing it in the future :)

thanks,

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Steven Rostedt]

On Wed, 28 May 2014 23:32:11 +0200 (CEST)
Thomas Gleixner <tglx@linutronix.de> wrote:
\
> What you really want is:
> 
> Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
> 
> It's way more useful than any of the v1-n writeups, which are most of
> the time just a completely waste of electrons. Even if written well,
> without the actual review context they are pretty pointless.

Agreed!

And unfortunately, even that doesn't work because version patches
usually start a new thread, and you lose out on what discussions were
made before it.

Lately, I've been adding more than one Link: tag. It may be overkill,
but when I post a v2 patch series, I try to have changes to the
patches have a Link tag that point to the discussion that caused the
changes.

If I have v5 (which I haven't had more than a v2 lately), I would
recommend having 4 Link tags that point to the posting of all previous
versions of the patch in order to be able to read the entire discussion
in the future.

-- Steve

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Steven Rostedt]

On Thu, 29 May 2014 01:17:39 +0200
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:


> That's one particular point that I have to spend too much time on according to 
> my tastes. When receiving version N+1 of a patch, I need to compare versions N 
> and N+1, open the comments I've sent for version N, and verify that they all 
> have been taken into account. I'd be interested in knowing about how other 
> maintainers automate (part of) that process (if they do).

I'd like to recommend more use of the Link tag here. If you make a v2
of a patch, that should include the Link tag that points back to the v1
posting. That way it becomes easier for a reviewer to go back and see
what the previous comments were with that particular patch.

-- Steve

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Steven Rostedt]

On Thu, 29 May 2014 10:13:21 +0400
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:


> This is a taste question.  I really appreciate a well split patch series
> tackling a hard problem because it makes working out what changed and
> why a lot easier than in a single monolith ... and as a reviewer, that's
> gold dust.  I don't appreciate 30 patches changing the same thing in 30
> different files.


I don't care for those either, but when I do a cross-the-board change,
I sometimes am guilty of this myself. The rational reason is that I
will Cc the maintainer of the change. Usually it's a trivial change on
something tracing related, where I don't really need the Acked-by from
them (appreciated if they do), but a Cc just to let them know it was
changed.

When you deal with 30 different subsystems you don't want one patch
with 30 different Cc's. That will put you over the LKML Cc limit and
the patch never makes it to LKML. Thus, you need to break it up into 30
different patches each with one Cc to the respective maintainer.

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Olof Johansson]

On Thu, May 29, 2014 at 2:03 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> On Thursday, May 29, 2014 02:27:53 PM Theodore Ts'o wrote:
>> On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
>> >
>> > Also long-overdue is a clarification on exactly what "Acked-by" means.
>> > Right now it is being used for at least two distinct and
>> > mutually-incompatible purposes:
>> >
>> > 1. A maintainer A for code affected by a patch, who is distinct from a
>> > maintainer B queuing a patch, has reviewed the patch and has cleared it as
>> > being OK for maintainer B to send upstream
>> >
>> > 2. A casual review has been done by someone who is not a maintainer for
>> > the code in question
>> >
>> > What I would propose is to have the first use replaced by a new tag,
>> > "Maintainer-acked-by:", and the second use abolished, along with
>> > "Acked-by:", and replaced by "Reviewed-by:".
>>
>> I agree in general, but if we're going to abolish the 2nd use
>> entirely, then it's much simpler to keep Acked-by for its original
>> meaning; it's easier to type, after all.
>>
>> This is basically I do for ext4 patches today; if someone sends me an
>> acked-by in the #2 sense, I simply won't add it to the s-o-b section,
>> and I don't let the fact that someone has asserted that they have done
>> a casual review to give me a false sense of security; if I still have
>> to do a deep review, I'm going to catch the casual stuff anyway, and
>> the fact that a casual review doesn't obviate the need for a careful
>> review.
>>
>> But if a senior ext4 developer adds a Reviewed-by:, that does lend a
>> lot of value to me as a maintainer, since I can trust that certain
>> folks like Jan and Eric and Lukas and others will do a good job doing
>> the review, and that actually *does* offload significant amounts of
>> work off my shoulders.
>
> Well, perhaps we can reserve the Acked-by for maintainers and add
> something like Supported-by for the 2nd meaning.

What are we really trying to fix here? Is the current process really
broken or are we trying to create more process that's not needed for
some other reason?

As a maintainer for arm-soc, I know which subsystem maintainers I
should get an Acked-by before I'm ok merging in a branch with, say,
some driver changes from a platform maintainer. I mostly know because
we keep intersecting with the same subsystems, but for new ones
there's one natural place I go to look: MAINTAINERS. Figuring out
merge paths and when something should go through our tree instead of
the maintainers tree is always going to be something where people
actually need to talk to each other and make a decision, I don't think
we can make tools and process for it.

Renaming the tag that they use isn't going to change the due diligence
I have to make (I still need to make sure they're actually the right
person to give it out, etc). And I'm definitely not worried by the
possible conflict of the "I gave this a casual review and I think we
should let it go in" acks since a maintainer is unlikely to give out
those kind of acks to code that he would otherwise merge himself.


-Olof

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Rafael J. Wysocki]

On Thursday, May 29, 2014 02:27:53 PM Theodore Ts'o wrote:
> On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> > 
> > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > Right now it is being used for at least two distinct and
> > mutually-incompatible purposes:
> > 
> > 1. A maintainer A for code affected by a patch, who is distinct from a
> > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> > being OK for maintainer B to send upstream
> > 
> > 2. A casual review has been done by someone who is not a maintainer for
> > the code in question
> > 
> > What I would propose is to have the first use replaced by a new tag, 
> > "Maintainer-acked-by:", and the second use abolished, along with 
> > "Acked-by:", and replaced by "Reviewed-by:".
> 
> I agree in general, but if we're going to abolish the 2nd use
> entirely, then it's much simpler to keep Acked-by for its original
> meaning; it's easier to type, after all.
> 
> This is basically I do for ext4 patches today; if someone sends me an
> acked-by in the #2 sense, I simply won't add it to the s-o-b section,
> and I don't let the fact that someone has asserted that they have done
> a casual review to give me a false sense of security; if I still have
> to do a deep review, I'm going to catch the casual stuff anyway, and
> the fact that a casual review doesn't obviate the need for a careful
> review.
> 
> But if a senior ext4 developer adds a Reviewed-by:, that does lend a
> lot of value to me as a maintainer, since I can trust that certain
> folks like Jan and Eric and Lukas and others will do a good job doing
> the review, and that actually *does* offload significant amounts of
> work off my shoulders.

Well, perhaps we can reserve the Acked-by for maintainers and add
something like Supported-by for the 2nd meaning.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Frank Rowand]

On 5/29/2014 9:46 AM, Mark Brown wrote:
> On Thu, May 29, 2014 at 01:35:45AM +0100, Ben Hutchings wrote:
>> On Tue, 2014-05-27 at 20:10 -0400, Martin K. Petersen wrote:
> 
>>> There are a few companies in the enterprise space that get it. But in
>>> general it can be quite the challenge to get these vendors to see the
>>> value of being good Linux citizens. And these companies are much less
>>> susceptible to phoronix rants than entities producing consumer widgets.
> 
>> For consumer widgets, it appears that most vendors don't update the
>> software, so they won't care that any out-of-tree drivers may break on
>> the next kernel version.  In fact, they're likely to be starting with
>> their SoC vendor's kernel tree which was forked a few years ago and has
>> a ton of dirty performance hacks in the core.
> 
> This is very true, to a good approximation nobody in the consumer
> electronics space cares in the slightest about upstream - it's just not
> useful as-is.  These days it's likely to be the current Android kernel
> at the time the SoC was current.  There is some pressure from some of
> the more clueful system integrators to be working with upstream, partly
> for bringing things forwards on new devices and partly for leveraging
> the review.

I must be nobody then.  For many years I have been pushing for being as
close to mainline as possible.  Even to the point of using an -rc1
release as the beginning point of new product development, knowing that
we would need to follow the churn through the rc versions.

And we encourage and are supportive of our SoC vendors working in mainline
instead of vendor trees.

-Frank Rowand
(for random parts of Sony)

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On Thu, May 29, 2014 at 02:57:38PM -0700, Frank Rowand wrote:
> On 5/29/2014 9:46 AM, Mark Brown wrote:

> > This is very true, to a good approximation nobody in the consumer
> > electronics space cares in the slightest about upstream - it's just not
> > useful as-is.  These days it's likely to be the current Android kernel

> I must be nobody then.  For many years I have been pushing for being as
> close to mainline as possible.  Even to the point of using an -rc1
> release as the beginning point of new product development, knowing that
> we would need to follow the churn through the rc versions.

> And we encourage and are supportive of our SoC vendors working in mainline
> instead of vendor trees.

Right, there's a very small number of people doing this sort of thing
(hence the "approximation") and you're not the only ones but it really
is very rare.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Greg KH]

On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> What are we really trying to fix here? Is the current process really
> broken or are we trying to create more process that's not needed for
> some other reason?

I think the latter.

Somehow, we seem to be constantly increasing our rate of change, are
people thinking we are having problems here?  If so, exactly where?
This thread has taken an odd turn into trying to make some new kind of
process for an unknown issue (i.e. the people on this list are not going
to recognize the reviewers more, it's up to you to educate your managers
/ company more.)

Are maintainers who deal with huge number of patches (i.e. 1000+ a year)
crying out for help that this is somehow going to reduce their burden?

I don't think so, but hey, what do I know about this development
process...

grumble,

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Thu, May 29, 2014 at 10:13:21AM +0400, James Bottomley wrote:
> I suspect, but cannot prove, it is somewhat driven by the patch
> statistics.  I know in Parallels, I track features accepted into the
> kernel, not patches, but we do have a patch metric as well for people
> who do incidental bug fixing as they push features, so we're not 100%
> pure on this.
> 
> When I recently gave a comparison of OpenStack and the Kernel, one thing
> that stood out starkly was that they just don't have our volume of one
> line, one patch per file hundreds of times and trivial changes, so we
> definitely have a problem in this area.
> 
> The reason OpenStack has so few trivial and multiply split patches is
> mostly grounded in the difficulty of submitting a patch to their tree.

And I would argue that this is going to be a long-term problem for them.

> Just preparing it for gerrit takes a long time (around ten minutes per
> patch), which really reduces the volume of trivial patches, just because
> no-one has that much energy (although some would argue it reduces the
> volume too far).
> 
> It's human nature to spend more time on the easy problems, so a flood of
> trivial patches which are "obviously" correct is easier to review and
> include than one patch which alters something deep within mm and needs
> careful thought.  At best, excessively split trivial patches are a
> dilution of our review effort and at worst, they're actively sucking
> people away from tackling the hard problems.

I have not heard of these "cleanup" patches taking anyone's review
cycles up for the "harder" patches, do you have examples of it?

In fact, I would argue the fact that we do so easily accept cleanups
across the board that keeps our developer community engaged and new
people joining in easy.  It's _really_ hard to get involved in
OpenStack, by design.  The kernel is not that way, and hopefully never
will be, as we want to succeed...

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Martin K. Petersen]

>>>>> "Dan" == Dan Williams <dan.j.williams@intel.com> writes:

Dan> Yes, ill conceived "value add" is indeed toxic.  Is that the only
Dan> contributor factor to indefinite patch acceptance latency?  We're
Dan> missing a mechanism to allow for experimentation without 1/ risking
Dan> the quality of the rest of the kernel 2/ committing to carrying the
Dan> experiment upstream indefinitely. 

Another problem in the value-add department is what hw vendor FOO is
unlikely to collaborate with main competitor BAR on a common
interface. Friendly collaboration works for us Linux folks but is rare
(although it does happen) in the driver space.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Williams]

On Thu, May 29, 2014 at 4:56 PM, Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>>>>>> "Dan" == Dan Williams <dan.j.williams@intel.com> writes:
>
> Dan> Yes, ill conceived "value add" is indeed toxic.  Is that the only
> Dan> contributor factor to indefinite patch acceptance latency?  We're
> Dan> missing a mechanism to allow for experimentation without 1/ risking
> Dan> the quality of the rest of the kernel 2/ committing to carrying the
> Dan> experiment upstream indefinitely.
>
> Another problem in the value-add department is what hw vendor FOO is
> unlikely to collaborate with main competitor BAR on a common
> interface. Friendly collaboration works for us Linux folks but is rare
> (although it does happen) in the driver space.

If both vendors put their "experiments" upstream, maybe we as a
community have a better chance of finding commonality to up-level to
libraries?

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Paul Walmsley]

On Wed, 28 May 2014, Rob Herring wrote:

> What really needs to change here as we already essentially have this
> today. Getting more reviewer bandwidth is why we have 5 DT binding
> maintainers.

Yeah, under the change that was proposed earlier, most of those people 
would be "officially designated reviewers" if they're not testing patches 
and sending them upstream.

> DT bindings are a bit unique in that almost everything goes in thru 
> other maintainers trees, so the role is almost entirely reviews. But 
> what's to say a co-maintainers role is not solely reviews. How 
> co-maintainers split up the load is really an internal decision among 
> them.
> 
> Do we really have people we trust to review that we wouldn't trust to
> be a co-maintainer?

Maintainers have different responsibilities than reviewers:

1. Maintainers batch up patches, resolve conflicts with other trees, and 
   send pull requests upstream

2. Leaf maintainers should be testing everything they send upstream, at 
   least in some basic fashion

3. Maintainers are responsible for setting some kind of architectural 
   direction for the parts of the kernel that they maintain


- Paul

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Paul Walmsley]

On Wed, 28 May 2014, Rob Herring wrote:

> What really needs to change here as we already essentially have this
> today. Getting more reviewer bandwidth is why we have 5 DT binding
> maintainers.

Iin the current process, obviously you guys are doing well in terms of 
designating others to share the blame and/or glory - you're in the top 
three.


- Paul

Maintainers per "kernel entity," as of v3.15-rc7

10	INTEL ETHERNET DRIVERS (e100/e1000/e1000e/igb/igbvf/ixgb/ixgbe/ixgbevf/i40e/i40evf)
6	LTP (Linux Test Project)
5	OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS
5	NETWORKING [IPv4/IPv6]
5	CISCO VIC ETHERNET NIC DRIVER
4	X86 ARCHITECTURE (32-BIT AND 64-BIT)
4	STAGING - SPEAKUP CONSOLE SPEECH DRIVER
4	STAGING - CRYSTAL HD VIDEO DECODER
4	RADOS BLOCK DEVICE (RBD)
4	QLOGIC QLGE 10Gb ETHERNET DRIVER
4	PSTORE FILESYSTEM
4	PERFORMANCE EVENTS SUBSYSTEM
4	PARAVIRT_OPS INTERFACE
4	MEMORY RESOURCE CONTROLLER
4	KPROBES
4	DRM DRIVERS FOR EXYNOS
4	COCCINELLE/Semantic Patches (SmPL)
4	BROADCOM BRCM80211 IEEE802.11n WIRELESS DRIVER
4	ARM/Marvell Armada 370 and Armada XP SOC support
3	XEN HYPERVISOR INTERFACE
3	VME SUBSYSTEM
3	VIDEOBUF2 FRAMEWORK
3	TRACING
3	TPM DEVICE DRIVER
3	SUSPEND TO RAM
3	STAGING - OLPC SECONDARY DISPLAY CONTROLLER (DCON)
3	SLAB ALLOCATOR
3	SERVER ENGINES 10Gbps NIC - BladeEngine 2 DRIVER
3	SELINUX SECURITY MODULE
3	SAMSUNG SXGBE DRIVERS
3	S390 NETWORK DRIVERS
3	S390
3	RTL8187 WIRELESS DRIVER
3	QLOGIC QLA3XXX NETWORK DRIVER
3	PXA2xx/PXA3xx SUPPORT
3	NFC SUBSYSTEM
3	NETXEN (1/10) GbE SUPPORT
3	NETWORKING [IPSEC]
3	NETFILTER/IPTABLES
3	LSILOGIC MPT FUSION DRIVERS (FC/SAS/SPI)
3	KERNEL VIRTUAL MACHINE for s390 (KVM/s390)
3	IPVS
3	INTEL WIRELESS WIFI LINK (iwlwifi)
3	INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
3	INTEL C600 SERIES SAS CONTROLLER DRIVER
3	INOTIFY
3	INFINIBAND SUBSYSTEM
3	INDUSTRY PACK SUBSYSTEM (IPACK)
3	EXYNOS MIPI DISPLAY DRIVERS
3	EXT3 FILE SYSTEM
3	EMBEDDED LINUX
3	EFI VARIABLE FILESYSTEM
3	EDAC-CORE
3	DEVICE-MAPPER  (LVM)
3	DC395x SCSI driver
3	CISCO FCOE HBA DRIVER
3	BONDING DRIVER
3	BLUETOOTH SUBSYSTEM
3	BLUETOOTH DRIVERS
3	BATMAN ADVANCED
3	BACKLIGHT CLASS/SUBSYSTEM
3	ATHEROS ATH5K WIRELESS DRIVER
3	ARM/STI ARCHITECTURE
3	ARM/SAMSUNG S5P SERIES Multi Format Codec (MFC) SUPPORT
3	ARM/QUALCOMM MSM MACHINE SUPPORT
3	ARM/Marvell Dove/Kirkwood/MV78xx0/Orion SOC support
3	ARM/EZX SMARTPHONES (A780, A910, A1200, E680, ROKR E2 and ROKR E6)
3	ARM/ATMEL AT91RM9200 AND AT91SAM ARM ARCHITECTURES
3	ALPHA PORT
3	ACPI COMPONENT ARCHITECTURE (ACPICA)
3	9P FILE SYSTEM
2	ZSMALLOC COMPRESSED SLAB MEMORY ALLOCATOR
2	ZRAM COMPRESSED RAM BLOCK DEVICE DRVIER
2	ZD1211RW WIRELESS DRIVER
2	XILINX AXI ETHERNET DRIVER
2	XFS FILESYSTEM
2	XEN NETWORK BACKEND DRIVER
2	X86 MCE INFRASTRUCTURE
2	WM97XX TOUCHSCREEN DRIVERS
2	WIMAX STACK
2	VOLTAGE AND CURRENT REGULATOR FRAMEWORK
2	VMWARE VMXNET3 ETHERNET DRIVER
2	VMware PVSCSI driver
2	VIRTIO CORE, NET AND BLOCK DRIVERS
2	VIA SD/MMC CARD CONTROLLER DRIVER
2	USERSPACE I/O (UIO)
2	USER-MODE LINUX (UML)
2	USB ATTACHED SCSI
2	UNIVERSAL FLASH STORAGE HOST CONTROLLER DRIVER
2	UNISYS S-PAR DRIVERS
2	UBI FILE SYSTEM (UBIFS)
2	TURBOCHANNEL SUBSYSTEM
2	TTY LAYER
2	TOMOYO SECURITY MODULE
2	TLG2300 VIDEO4LINUX-2 DRIVER
2	TIPC NETWORK LAYER
2	TIMEKEEPING, CLOCKSOURCE CORE, NTP
2	TI LM49xxx FAMILY ASoC CODEC DRIVERS
2	TI DAVINCI MACHINE SUPPORT
2	THERMAL
2	TENSILICA XTENSA PORT (xtensa)
2	TEGRA KBC DRIVER
2	TEGRA CLOCK DRIVER
2	TEGRA ARCHITECTURE SUPPORT
2	TCP LOW PRIORITY MODULE
2	SYNOPSYS DESIGNWARE MMC/SD/SDIO DRIVER
2	SYNOPSYS DESIGNWARE DMAC DRIVER
2	STAGING - SLICOSS
2	STAGING - REALTEK RTL8712U DRIVERS
2	STAGING - NVIDIA COMPLIANT EMBEDDED CONTROLLER INTERFACE (nvec)
2	STAGING - ECHO CANCELLER
2	STAGING - COMEDI
2	SPIDERNET NETWORK DRIVER for CELL
2	SPEAR PLATFORM SUPPORT
2	SOUND - SOC LAYER / DYNAMIC AUDIO POWER MANAGEMENT (ASoC)
2	SOUND
2	SMARTREFLEX DRIVERS FOR ADAPTIVE VOLTAGE SCALING (AVS)
2	SLEEPABLE READ-COPY UPDATE (SRCU)
2	SGI XP/XPC/XPNET DRIVER
2	SFC NETWORK DRIVER
2	SCTP PROTOCOL
2	SCORE ARCHITECTURE
2	SCHEDULER
2	SAMSUNG S5P/EXYNOS4 SOC SERIES CAMERA SUBSYSTEM DRIVERS
2	SAMSUNG S5K5BAF CAMERA DRIVER
2	SAMSUNG S5C73M3 CAMERA DRIVER
2	S390 ZFCP DRIVER
2	S390 ZCRYPT DRIVER
2	S390 PCI SUBSYSTEM
2	S390 IUCV NETWORK LAYER
2	S390 DASD DRIVER
2	S390 COMMON I/O LAYER
2	RTL8192CE WIRELESS DRIVER
2	READ-COPY UPDATE (RCU)
2	RCUTORTURE MODULE
2	RAPIDIO SUBSYSTEM
2	RALINK RT2X00 WIRELESS LAN DRIVER
2	RADEON DRM DRIVERS
2	QLOGIC QLCNIC (1/10)Gb ETHERNET DRIVER
2	QLOGIC QLA4XXX iSCSI DRIVER
2	PTRACE SUPPORT
2	POWER SUPPLY CLASS/SUBSYSTEM and DRIVERS
2	PNP SUPPORT
2	PMC SIERRA PM8001 DRIVER
2	PIN CONTROLLER - SAMSUNG
2	PER-CPU MEMORY ALLOCATOR
2	PCI DRIVER FOR SYNOPSIS DESIGNWARE
2	PCI DRIVER FOR MVEBU (Marvell Armada 370 and Armada XP SOC support)
2	PCI DRIVER FOR IMX6
2	PARISC ARCHITECTURE
2	PANASONIC MN10300/AM33/AM34 PORT
2	OSD LIBRARY and FILESYSTEM
2	ORACLE CLUSTER FILESYSTEM 2 (OCFS2)
2	OPEN FIRMWARE AND FLATTENED DEVICE TREE
2	OMAP POWERDOMAIN SOC ADAPTATION LAYER SUPPORT
2	OMAP HWMOD SUPPORT
2	OMAP GPIO DRIVER
2	OMAP DEVICE TREE SUPPORT
2	OMAP AUDIO SUPPORT
2	NINJA SCSI-32Bi/UDE PCI/CARDBUS SCSI HOST ADAPTER DRIVER
2	MULTIMEDIA CARD (MMC), SECURE DIGITAL (SD) AND SDIO SUBSYSTEM
2	MULTIFUNCTION DEVICES (MFD)
2	MMP SUPPORT
2	MEMORY TECHNOLOGY DEVICES (MTD)
2	MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2)
2	MAC80211 PID RATE CONTROL
2	LogFS
2	LOCKDEP AND LOCKSTAT
2	LINUX FOR POWERPC EMBEDDED PPC8XX
2	LINUX FOR POWERPC EMBEDDED PPC4XX
2	LINUX FOR POWERPC (32-BIT AND 64-BIT)
2	LED SUBSYSTEM
2	KMEMCHECK
2	KEYS-TRUSTED
2	KEYS-ENCRYPTED
2	KERNEL VIRTUAL MACHINE (KVM) FOR ARM
2	KERNEL VIRTUAL MACHINE (KVM)
2	KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)
2	KDUMP
2	JOURNALLING LAYER FOR BLOCK DEVICES (JBD)
2	ISCSI EXTENSIONS FOR RDMA (ISER) INITIATOR
2	iSCSI BOOT FIRMWARE TABLE (iBFT) DRIVER
2	IRQCHIP DRIVERS
2	IPWIRELESS DRIVER
2	IP1000A 10/100/1000 GIGABIT ETHERNET DRIVER
2	INTEL WIRELESS WIMAX CONNECTION 2400
2	INTEL I/OAT DMA DRIVER
2	INTEL DRM DRIVERS (excluding Poulsbo, Moorestown and derivative chipsets)
2	INTEGRITY MEASUREMENT ARCHITECTURE (IMA)
2	INPUT (KEYBOARD, MOUSE, JOYSTICK, TOUCHSCREEN) DRIVERS
2	IKANOS/ADI EAGLE ADSL USB DRIVER
2	IEEE 802.15.4 SUBSYSTEM
2	IBM Power in-Nest Crypto Acceleration
2	IA64 (Itanium) PLATFORM
2	I2C/SMBUS ISMT DRIVER
2	Hyper-V CORE AND DRIVERS
2	HIBERNATION (aka Software Suspend, aka swsusp)
2	HARDWARE RANDOM NUMBER GENERATOR CORE
2	HARDWARE MONITORING
2	GPIO SUBSYSTEM
2	GIGASET ISDN DRIVERS
2	FUJITSU M-5MO LS CAMERA ISP DRIVER
2	FREEZER
2	FREESCALE SOC FS_ENET DRIVER
2	FREESCALE DMA DRIVER
2	FRAMEBUFFER LAYER
2	FLASH ADAPTER DRIVER (IBM Flash Adapter 900GB Full Height PCI Flash Card)
2	FILE LOCKING (flock() and fcntl()/lockf())
2	EXTERNAL CONNECTOR SUBSYSTEM (EXTCON)
2	EXT4 FILE SYSTEM
2	EHCA (IBM GX bus InfiniBand adapter) DRIVER
2	EDAC-I82975X
2	EDAC-E752X
2	EDAC-CAVIUM
2	EDAC-CALXEDA
2	EDAC-AMD64
2	DRM DRIVERS FOR NVIDIA TEGRA
2	DMA GENERIC OFFLOAD ENGINE SUBSYSTEM
2	DISTRIBUTED LOCK MANAGER (DLM)
2	DEVICE FREQUENCY (DEVFREQ)
2	DC390/AM53C974 SCSI driver
2	CRYPTO API
2	CRIS PORT
2	CPU POWER MONITORING SUBSYSTEM
2	CPUIDLE DRIVERS
2	CPUIDLE DRIVER - ARM BIG LITTLE
2	CPU FREQUENCY DRIVERS - ARM BIG LITTLE
2	CPU FREQUENCY DRIVERS
2	CONTROL GROUPS (CGROUPS)
2	CODA FILE SYSTEM
2	CMPC ACPI DRIVER
2	CLOCKSOURCE, CLOCKEVENT DRIVERS
2	CIRRUS LOGIC AUDIO CODEC DRIVERS
2	CHECKPATCH
2	CHAR and MISC DRIVERS
2	CAN NETWORK DRIVERS
2	CALGARY x86-64 IOMMU
2	C6X ARCHITECTURE
2	BTRFS FILE SYSTEM
2	BROCADE BFA FC SCSI DRIVER
2	BROADCOM TG3 GIGABIT ETHERNET DRIVER
2	BROADCOM BCM281XX/BCM11XXX/BCM216XX ARM ARCHITECTURE
2	B43LEGACY WIRELESS DRIVER
2	AVR32/AT32AP MACHINE SUPPORT
2	AVR32 ARCHITECTURE
2	ATLX ETHERNET DRIVERS
2	ARM/TOSA MACHINE SUPPORT
2	ARM/SHMOBILE ARM ARCHITECTURE
2	ARM/SAMSUNG S5P SERIES TV SUBSYSTEM SUPPORT
2	ARM/SAMSUNG S5P SERIES 2D GRAPHICS ACCELERATION (G2D) SUPPORT
2	ARM/SAMSUNG ARM ARCHITECTURES
2	ARM/QUALCOMM SUPPORT
2	ARM/NOMADIK ARCHITECTURE
2	ARM/INTEL XSC3 (MANZANO) ARM CORE
2	ARM/INTEL IXP4XX ARM ARCHITECTURE
2	ARM/INTEL IQ81342EX MACHINE SUPPORT
2	ARM/INTEL IOP32X ARM ARCHITECTURE
2	ARM/INTEL IOP13XX ARM ARCHITECTURE
2	ARM/IGEP MACHINE SUPPORT
2	ARM/H4700 (HP IPAQ HX4700) MACHINE SUPPORT
2	ARM/FREESCALE IMX / MXC ARM ARCHITECTURE
2	ARM/CIRRUS LOGIC EP93XX ARM ARCHITECTURE
2	ARM64 PORT (AARCH64 ARCHITECTURE)
2	AMSO1100 RNIC DRIVER
2	ACPI
2	8169 10/100/1000 GIGABIT ETHERNET DRIVER
1	ZSWAP COMPRESSED SWAP CACHING
1	ZS DECSTATION Z85C30 SERIAL DRIVER
1	ZBUD COMPRESSED PAGE ALLOCATOR
1	Z8530 DRIVER FOR AX.25
1	YEALINK PHONE DRIVER
1	YAM DRIVER FOR AX.25
1	XTENSA XTFPGA PLATFORM SUPPORT
1	XILINX UARTLITE SERIAL DRIVER
1	XEN SWIOTLB SUBSYSTEM
1	XEN PCI SUBSYSTEM
1	XEN HYPERVISOR ARM64
1	XEN HYPERVISOR ARM
1	XC2028/3028 TUNER DRIVER
1	X86 PLATFORM DRIVERS
1	X.25 NETWORK LAYER
1	WORKQUEUE
1	WL3501 WIRELESS PCMCIA CARD DRIVER
1	WISTRON LAPTOP BUTTON DRIVER
1	WINBOND CIR DRIVER
1	WILOCITY WIL6210 WIRELESS DRIVER
1	WIIMOTE HID DRIVER
1	WD7000 SCSI DRIVER
1	WATCHDOG DEVICE DRIVERS
1	W83L51xD SD/MMC CARD INTERFACE DRIVER
1	W83795 HARDWARE MONITORING DRIVER
1	W83793 HARDWARE MONITORING DRIVER
1	W83791D HARDWARE MONITORING DRIVER
1	W1 DALLAS'S 1-WIRE BUS
1	VUB300 USB to SDIO/SD/MMC bridge chip
1	VT8231 HARDWARE MONITOR DRIVER
1	VT1211 HARDWARE MONITOR DRIVER
1	VMWARE HYPERVISOR INTERFACE
1	VLYNQ BUS
1	VLAN (802.1Q)
1	VIVI VIRTUAL VIDEO DRIVER
1	VIRTIO HOST (VHOST)
1	VIRTIO CONSOLE DRIVER
1	VIA VELOCITY NETWORK DRIVER
1	VIA UNICHROME(PRO)/CHROME9 FRAMEBUFFER DRIVER
1	VIA RHINE NETWORK DRIVER
1	VFIO DRIVER
1	VFAT/FAT/MSDOS FILESYSTEM
1	UVESAFB DRIVER
1	UTIL-LINUX PACKAGE
1	USB ZR364XX DRIVER
1	USB XHCI DRIVER
1	USB WIRELESS RNDIS DRIVER (rndis_wlan)
1	USB WEBCAM GADGET
1	USB VISION DRIVER
1	USB VIDEO CLASS
1	USB "USBNET" DRIVER FRAMEWORK
1	USB UHCI DRIVER
1	USB SUBSYSTEM
1	USB SN9C1xx DRIVER
1	USB SMSC95XX ETHERNET DRIVER
1	USB SMSC75XX ETHERNET DRIVER
1	USB SERIAL SUBSYSTEM
1	USB RTL8150 DRIVER
1	USB PRINTER DRIVER (usblp)
1	USB PHY LAYER
1	USB PEGASUS DRIVER
1	USB OPTION-CARD DRIVER
1	USB OHCI DRIVER
1	USB MIDI DRIVER
1	USB MASS STORAGE DRIVER
1	USB KAWASAKI LSI DRIVER
1	USB ISP116X DRIVER
1	USB HID/HIDBP DRIVERS (USB KEYBOARDS, MICE, REMOTE CONTROLS, ...)
1	USB GADGET/PERIPHERAL SUBSYSTEM
1	USB EHCI DRIVER
1	USB DIAMOND RIO500 DRIVER
1	USB DAVICOM DM9601 DRIVER
1	USB CYPRESS C67X00 DRIVER
1	USB CDC ETHERNET DRIVER
1	USB AR5523 WIRELESS DRIVER
1	USB ACM DRIVER
1	UNSORTED BLOCK IMAGES (UBI) Fastmap
1	UNSORTED BLOCK IMAGES (UBI)
1	UNIFORM CDROM DRIVER
1	UNIFDEF
1	UNICORE32 ARCHITECTURE:
1	UHID USERSPACE HID IO DRIVER:
1	UFS FILESYSTEM
1	UDF FILESYSTEM
1	UCLINUX (AND M68KNOMMU)
1	U14-34F SCSI DRIVER
1	TUN/TAP driver
1	TULIP NETWORK DRIVERS
1	TUA9001 MEDIA DRIVER
1	TRIVIAL PATCHES
1	TOSHIBA SMM DRIVER
1	TOPSTAR LAPTOP EXTRAS DRIVER
1	TMPFS (SHMEM FILESYSTEM)
1	TMP401 HARDWARE MONITOR DRIVER
1	TMIO MMC DRIVER
1	TM6000 VIDEO4LINUX DRIVER
1	TLAN NETWORK DRIVER
1	TI TWL4030 SERIES SOC CODEC DRIVER
1	TI LP8788 MFD DRIVER
1	TI LP8727 CHARGER DRIVER
1	TI LP855x BACKLIGHT DRIVER
1	TILE ARCHITECTURE
1	TI FLASH MEDIA INTERFACE DRIVER
1	TI DAVINCI SERIES MEDIA DRIVER
1	TI BANDGAP AND THERMAL DRIVER
1	THINKPAD ACPI EXTRAS DRIVER
1	THINGM BLINK(1) USB RGB LED DRIVER
1	THE REST
1	THANKO'S RAREMONO AM/FM/SW RADIO RECEIVER USB DRIVER
1	Telecom Clock Driver for MCPL0010
1	TEHUTI ETHERNET DRIVER
1	TEGRA SPI DRIVER
1	TEGRA SERIAL DRIVER
1	TEGRA PWM DRIVER
1	TEGRA PINCTRL DRIVER
1	TEGRA IOMMU DRIVERS
1	TEGRA I2C DRIVER
1	TEGRA GPIO DRIVER
1	TEGRA DMA DRIVER
1	TEGRA ASOC DRIVER
1	TECHNOTREND USB IR RECEIVER
1	TECHNOLOGIC SYSTEMS TS-5500 PLATFORM SUPPORT
1	TEAM DRIVER
1	TEA6420 MEDIA DRIVER
1	TEA6415C MEDIA DRIVER
1	TEA5767 TUNER DRIVER
1	TEA5761 TUNER DRIVER
1	TDA9840 MEDIA DRIVER
1	TDA8290 MEDIA DRIVER
1	TDA827x MEDIA DRIVER
1	TDA18271 MEDIA DRIVER
1	TDA18218 MEDIA DRIVER
1	TDA18212 MEDIA DRIVER
1	TDA10071 MEDIA DRIVER
1	TC CLASSIFIER
1	TASKSTATS STATISTICS INTERFACE
1	TARGET SUBSYSTEM
1	SYSV FILESYSTEM
1	SYNOPSYS ARC ARCHITECTURE
1	SWIOTLB SUBSYSTEM
1	SVGA HANDLING
1	SUNDANCE NETWORK DRIVER
1	SUN3/3X
1	STMMAC ETHERNET DRIVER
1	STK1160 USB VIDEO CAPTURE DRIVER
1	STARFIRE/DURALAN NETWORK DRIVER
1	STAGING - XGI Z7,Z9,Z11 PCI DISPLAY DRIVER
1	STAGING - WINBOND IS89C35 WLAN USB DRIVER
1	STAGING - VIA VT665X DRIVERS
1	STAGING - USB ENE SM/MS CARD READER DRIVER
1	STAGING - TI DSP BRIDGE DRIVERS
1	STAGING SUBSYSTEM
1	STAGING - SOFTLOGIC 6x10 MPEG CODEC
1	STAGING - SILICON MOTION SM7XX FRAME BUFFER DRIVER
1	STAGING - PARALLEL LCD/KEYPAD PANEL DRIVER
1	STAGING - OZMO DEVICES USB OVER WIFI DRIVER
1	STAGING - LIRC (LINUX INFRARED REMOTE CONTROL) DRIVERS
1	STAGING - INDUSTRIAL IO
1	STAGING - GO7007 MPEG CODEC
1	STAGING - FRONTIER TRANZPORT AND ALPHATRACK
1	STAGING - FLARION FT1000 DRIVERS
1	STAGING - ET131X NETWORK DRIVER
1	STAGING - ASUS OLED
1	STAGING - AGERE HERMES II and II.5 WIRELESS DRIVERS
1	STABLE BRANCH
1	SRM (Alpha) environment access
1	SQUASHFS FILE SYSTEM
1	SPU FILE SYSTEM
1	SPI SUBSYSTEM
1	SPEAR CLOCK FRAMEWORK SUPPORT
1	SPARSE CHECKER
1	SPARC + UltraSPARC (sparc/sparc64)
1	SPARC SERIAL DRIVERS
1	SOUND - DMAENGINE HELPERS
1	SOUND - COMPRESSED AUDIO
1	SONY VAIO CONTROL DEVICE DRIVER
1	SONY MEMORYSTICK STANDARD SUPPORT
1	SONY MEMORYSTICK CARD SUPPORT
1	SONICS SILICON BACKPLANE DRIVER (SSB)
1	SONIC NETWORK DRIVER
1	SOFTWARE RAID (Multiple Disks) SUPPORT
1	SOEKRIS NET48XX LED SUPPORT
1	SOC-CAMERA V4L2 SUBSYSTEM
1	SMSC UFX6000 and UFX7000 USB to VGA DRIVER
1	SMSC SCH5627 HARDWARE MONITOR DRIVER
1	SMSC EMC2103 HARDWARE MONITOR DRIVER
1	SMSC9420 PCI ETHERNET DRIVER
1	SMSC911x ETHERNET DRIVER
1	SMSC47B397 HARDWARE MONITOR DRIVER
1	SMM665 HARDWARE MONITOR DRIVER
1	SMIA AND SMIA++ IMAGE SENSOR DRIVER
1	SMC91x ETHERNET DRIVER
1	SMACK SECURITY MODULE
1	SIS USB2VGA DRIVER
1	SIS FRAMEBUFFER DRIVER
1	SIS 900/7016 FAST ETHERNET DRIVER
1	SIS 190 ETHERNET DRIVER
1	SIMTEC EB2410ITX (BAST)
1	SIMTEC EB110ATX (Chalice CATS)
1	SIMPLE FIRMWARE INTERFACE (SFI)
1	SIANO DVB DRIVER
1	SI4713 FM RADIO TRANSMITTER USB DRIVER
1	SI4713 FM RADIO TRANSMITTER PLATFORM DRIVER
1	SI4713 FM RADIO TRANSMITTER I2C DRIVER
1	SI470X FM RADIO RECEIVER USB DRIVER
1	SI470X FM RADIO RECEIVER I2C DRIVER
1	SGI SN-IA64 (Altix) SERIAL CONSOLE DRIVER
1	SGI GRU DRIVER
1	SERVER ENGINES 10Gbps iSCSI - BladeEngine 2 DRIVER
1	SERIAL DRIVERS
1	SERIAL ATA (SATA) SUBSYSTEM
1	SENSABLE PHANTOM
1	SECURITY SUBSYSTEM
1	SECURITY CONTACT
1	SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) ST SPEAR DRIVER
1	SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) SAMSUNG DRIVER
1	SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) DRIVER
1	SECURE DIGITAL HOST CONTROLLER INTERFACE, OPEN FIRMWARE BINDINGS (SDHCI-OF)
1	SDRICOH_CS MMC/SD HOST CONTROLLER INTERFACE DRIVER
1	SCx200 HRT CLOCKSOURCE DRIVER
1	SCx200 GPIO DRIVER
1	SCx200 CPU SUPPORT
1	SCSI TAPE DRIVER
1	SCSI SUBSYSTEM
1	SCSI SG DRIVER
1	SCSI RDMA PROTOCOL (SRP) INITIATOR
1	SCSI CDROM DRIVER
1	SC1200 WDT DRIVER
1	SAMSUNG SOC CLOCK DRIVERS
1	SAMSUNG S3C24XX/S3C64XX SOC SERIES CAMIF DRIVER
1	SAMSUNG MULTIFUNCTION DEVICE DRIVERS
1	SAMSUNG LAPTOP DRIVER
1	SAMSUNG FRAMEBUFFER DRIVER
1	SAMSUNG AUDIO (ASoC) DRIVERS
1	SAA7146 VIDEO4LINUX-2 DRIVER
1	SAA7134 VIDEO4LINUX DRIVER
1	SAA6588 RDS RECEIVER DRIVER
1	S3 SAVAGE FRAMEBUFFER DRIVER
1	S3C24XX SD/MMC Driver
1	RTL8180 WIRELESS DRIVER
1	RTL2832_SDR MEDIA DRIVER
1	RTL2832 MEDIA DRIVER
1	RTL2830 MEDIA DRIVER
1	ROSE NETWORK LAYER
1	ROCCAT DRIVERS
1	RICOH SMARTMEDIA/XD DRIVER
1	RICOH R5C592 MEMORYSTICK DRIVER
1	RFKILL
1	REMOTE PROCESSOR (REMOTEPROC) SUBSYSTEM
1	REMOTE PROCESSOR MESSAGING (RPMSG) SUBSYSTEM
1	REGISTER MAP ABSTRACTION
1	REAL TIME CLOCK (RTC) SUBSYSTEM
1	RDS - RELIABLE DATAGRAM SOCKETS
1	RDC R6040 FAST ETHERNET DRIVER
1	RDC R-321X SoC
1	RCUTORTURE TEST FRAMEWORK
1	RANDOM NUMBER DRIVER
1	RAMDISK RAM BLOCK DEVICE DRIVER
1	RAGE128 FRAMEBUFFER DISPLAY DRIVER
1	RADIOSHARK RADIO DRIVER
1	RADIOSHARK2 RADIO DRIVER
1	RADEON FRAMEBUFFER DISPLAY DRIVER
1	QUICKCAM PARALLEL PORT WEBCAMS
1	QUALCOMM WCN36XX WIRELESS DRIVER
1	QUALCOMM HEXAGON ARCHITECTURE
1	QUALCOMM ATHEROS ATH9K WIRELESS DRIVER
1	QUALCOMM ATHEROS ATH10K WIRELESS DRIVER
1	QT1010 MEDIA DRIVER
1	QNX4 FILESYSTEM
1	QLOGIC QLA2XXX FC-SCSI DRIVER
1	QLOGIC QLA1280 SCSI DRIVER
1	QIB DRIVER
1	PXA RTC DRIVER
1	PXA3xx NAND FLASH DRIVER
1	PWM SUBSYSTEM
1	PWC WEBCAM DRIVER
1	PVRUSB2 VIDEO4LINUX DRIVER
1	PTP HARDWARE CLOCK SUPPORT
1	PS3VRAM DRIVER
1	PS3 PLATFORM SUPPORT
1	PS3 NETWORK SUPPORT
1	PROMISE SATA TX2/TX4 CONTROLLER LIBATA DRIVER
1	PRISM54 WIRELESS DRIVER
1	PREEMPTIBLE KERNEL
1	PPTP DRIVER
1	PPS SUPPORT
1	PPP PROTOCOL DRIVERS AND COMPRESSORS
1	PPP OVER L2TP
1	PPP OVER ETHERNET
1	PPP OVER ATM (RFC 2364)
1	POSIX CLOCKS and TIMERS
1	PNXxxxx I2C DRIVER
1	PMC SIERRA MaxRAID DRIVER
1	PMBUS HARDWARE MONITORING DRIVERS
1	PKUNITY SOC DRIVERS
1	PKTCDVD DRIVER
1	PIN CONTROL SUBSYSTEM
1	PIN CONTROLLER - ST SPEAR
1	PIN CONTROLLER - ATMEL AT91
1	PICOXCELL SUPPORT
1	PICOLCD HID DRIVER
1	PHRAM MTD DRIVER
1	PHONET PROTOCOL
1	PER-TASK DELAY ACCOUNTING
1	PERSONALITY HANDLING
1	PCRYPT PARALLEL CRYPTO ENGINE
1	PCNET32 NETWORK DRIVER
1	PCI SUBSYSTEM
1	PCI ERROR RECOVERY
1	PCI DRIVER FOR SAMSUNG EXYNOS
1	PCI DRIVER FOR RENESAS R-CAR
1	PCI DRIVER FOR NVIDIA TEGRA
1	PCDP - PRIMARY CONSOLE AND DEBUG PORT
1	PCA9541 I2C BUS MASTER SELECTOR DRIVER
1	PCA9532 LED DRIVER
1	PC87427 HARDWARE MONITORING DRIVER
1	PC8736x GPIO DRIVER
1	PC87360 HARDWARE MONITORING DRIVER
1	PA SEMI SMBUS DRIVER
1	PA SEMI ETHERNET DRIVER
1	PARIDE DRIVERS FOR PARALLEL PORT IDE DEVICES
1	PANASONIC LAPTOP ACPI EXTRAS DRIVER
1	PADATA PARALLEL EXECUTION MECHANISM
1	P54 WIRELESS DRIVER
1	OPROFILE
1	OPL4 DRIVER
1	OPENVSWITCH
1	OPENRISC ARCHITECTURE
1	OPENCORES I2C BUS DRIVER
1	ONSTREAM SCSI TAPE DRIVER
1	ONENAND FLASH DRIVER
1	OMNIVISION OV7670 SENSOR DRIVER
1	OMNIKEY CARDMAN 4040 DRIVER
1	OMNIKEY CARDMAN 4000 DRIVER
1	OMFS FILESYSTEM
1	OMAP USB SUPPORT
1	OMAP SUPPORT
1	OMAP RANDOM NUMBER GENERATOR SUPPORT
1	OMAP POWER MANAGEMENT SUPPORT
1	OMAP/NEWFLOW NANOBONE MACHINE SUPPORT
1	OMAP MMC SUPPORT
1	OMAP IMAGE SIGNAL PROCESSOR (ISP)
1	OMAP HWMOD DATA FOR OMAP4-BASED DEVICES
1	OMAP HS MMC SUPPORT
1	OMAP HARDWARE SPINLOCK SUPPORT
1	OMAP FRAMEBUFFER SUPPORT
1	OMAP DISPLAY SUBSYSTEM and FRAMEBUFFER SUPPORT (DSS2)
1	OMAP CLOCK FRAMEWORK SUPPORT
1	NXP TDA998X DRM DRIVER
1	NVM EXPRESS DRIVER
1	NVIDIA (rivafb and nvidiafb) FRAMEBUFFER DRIVER
1	NTFS FILESYSTEM
1	NTB DRIVER
1	NINJA SCSI-3 / NINJA SCSI-32Bi (16bit/CardBus) PCMCIA SCSI HOST ADAPTER DRIVER
1	NILFS2 FILESYSTEM
1	NFS, SUNRPC, AND LOCKD CLIENTS
1	NETWORKING [WIRELESS]
1	NETWORKING [LABELED] (NetLabel, CIPSO, Labeled IPsec, SECMARK)
1	NETWORKING [GENERAL]
1	NETWORK DROP MONITOR
1	NETWORK BLOCK DEVICE (NBD)
1	NETROM NETWORK LAYER
1	NETLABEL
1	NETERION 10GbE DRIVERS (s2io/vxge)
1	NETEM NETWORK EMULATOR
1	NETEFFECT IWARP RNIC DRIVER (IW_NES)
1	NCT6775 HARDWARE MONITOR DRIVER
1	NCR DUAL 700 SCSI DRIVER (MICROCHANNEL)
1	NCP FILESYSTEM
1	NATIVE INSTRUMENTS USB SOUND INTERFACE DRIVER
1	MYRICOM MYRI-10G 10GbE DRIVER (MYRI10GE)
1	MXL5007T MEDIA DRIVER
1	MUSB MULTIPOINT HIGH SPEED DUAL-ROLE CONTROLLER
1	MULTISOUND SOUND DRIVER
1	MT9V032 APTINA CAMERA SENSOR
1	MT9T001 APTINA CAMERA SENSOR
1	MT9P031 APTINA CAMERA SENSOR
1	MT9M032 APTINA SENSOR DRIVER
1	MSI WMI SUPPORT
1	MSI LAPTOP SUPPORT
1	MSI3101 MEDIA DRIVER
1	MSI001 MEDIA DRIVER
1	MR800 AVERMEDIA USB FM RADIO DRIVER
1	MOXA SMARTIO/INDUSTIO/INTELLIO SERIAL CARD
1	MODULE SUPPORT
1	MIROSOUND PCM20 FM RADIO RECEIVER DRIVER
1	MIPS
1	MICROTEK X6 SCANNER
1	MICROBLAZE ARCHITECTURE
1	METAG ARCHITECTURE
1	MEN CHAMELEON BUS (mcb)
1	MEN A21 WATCHDOG DRIVER
1	Mellanox MLX5 IB driver
1	Mellanox MLX5 core VPI driver
1	MELLANOX ETHERNET DRIVER (mlx4_en)
1	MEGARAID SCSI DRIVERS
1	MEDIAVISION PRO MOVIE STUDIO DRIVER
1	MEDIA INPUT INFRASTRUCTURE (V4L/DVB)
1	MAXIRADIO FM RADIO RECEIVER DRIVER
1	MAX6697 HARDWARE MONITOR DRIVER
1	MAX6650 HARDWARE MONITOR AND FAN CONTROLLER DRIVER
1	MAX16065 HARDWARE MONITOR DRIVER
1	MARVELL SOC MMC/SD/SDIO CONTROLLER DRIVER
1	MARVELL MWL8K WIRELESS DRIVER
1	MARVELL MWIFIEX WIRELESS DRIVER
1	MARVELL MVNETA ETHERNET DRIVER
1	MARVELL MV643XX ETHERNET DRIVER
1	MARVELL ARMADA DRM SUPPORT
1	MAN-PAGES: MANUAL PAGES FOR LINUX -- Sections 2, 3, 4, 5, and 7
1	MACVLAN DRIVER
1	MAC80211
1	MA901 MASTERKIT USB FM RADIO DRIVER
1	M88TS2022 MEDIA DRIVER
1	M88RS2000 MEDIA DRIVER
1	M88DS3103 MEDIA DRIVER
1	M68K ON HP9000/300
1	M68K ON APPLE MACINTOSH
1	M68K ARCHITECTURE
1	M32R ARCHITECTURE
1	LTC4261 HARDWARE MONITOR DRIVER
1	LSILOGIC/SYMBIOS/NCR 53C8XX and 53C1010 PCI-SCSI drivers
1	LPC32XX MACHINE SUPPORT
1	LOGICAL DISK MANAGER SUPPORT (LDM, Windows 2000/XP/Vista Dynamic Disks)
1	LME2510 MEDIA DRIVER
1	LM95234 HARDWARE MONITOR DRIVER
1	LM90 HARDWARE MONITOR DRIVER
1	LM83 HARDWARE MONITOR DRIVER
1	LM78 HARDWARE MONITOR DRIVER
1	LM73 HARDWARE MONITOR DRIVER
1	LLC (802.2)
1	LIS3LV02D ACCELEROMETER DRIVER
1	LINUX SECURITY MODULE (LSM) FRAMEWORK
1	LINUX FOR POWERPC PA SEMI PWRFICIENT
1	LINUX FOR POWERPC EMBEDDED PPC83XX AND PPC85XX
1	LINUX FOR POWERPC EMBEDDED MPC5XXX
1	LINUX FOR POWER MACINTOSH
1	LINUX FOR IBM pSERIES (RS/6000)
1	LIBLOCKDEP
1	LGUEST
1	LGDT3305 MEDIA DRIVER
1	LG2160 MEDIA DRIVER
1	LEGO USB Tower driver
1	LEGACY EEPROM DRIVER
1	LASI 53c700 driver for PARISC
1	KTAP
1	KS0108 LCD CONTROLLER DRIVER
1	KMEMLEAK
1	KGDB / KDB /debug_core
1	KEYS/KEYRINGS:
1	KEXEC
1	KERNEL VIRTUAL MACHINE (KVM) FOR POWERPC
1	KERNEL VIRTUAL MACHINE (KVM) FOR AMD-V
1	KERNEL VIRTUAL MACHINE For Itanium (KVM/IA64)
1	KERNEL NFSD, SUNRPC, AND LOCKD SERVERS
1	KERNEL BUILD + files below scripts/ (unless maintained elsewhere)
1	KERNEL AUTOMOUNTER v4 (AUTOFS4)
1	KEENE FM RADIO TRANSMITTER DRIVER
1	KCONFIG
1	K8TEMP HARDWARE MONITORING DRIVER
1	K10TEMP HARDWARE MONITORING DRIVER
1	JSM Neo PCI based serial card
1	JOURNALLING LAYER FOR BLOCK DEVICES (JBD2)
1	JOURNALLING FLASH FILE SYSTEM V2 (JFFS2)
1	JME NETWORK DRIVER
1	JFS FILESYSTEM
1	JC42.4 TEMPERATURE SENSOR DRIVER
1	IX2505V MEDIA DRIVER
1	IVTV VIDEO4LINUX DRIVER
1	IT913X MEDIA DRIVER
1	IT87 HARDWARE MONITORING DRIVER
1	ISDN SUBSYSTEM (Eicon active card driver)
1	ISDN SUBSYSTEM
1	ISCSI
1	ISA RADIO MODULE
1	ISAPNP
1	IRQ SUBSYSTEM
1	IRQ DOMAINS (IRQ NUMBER MAPPING LIBRARY)
1	IRDA SUBSYSTEM
1	IPX NETWORK LAYER
1	IPS SCSI RAID DRIVER
1	IPMI SUBSYSTEM
1	IP MASQUERADING
1	IPATH DRIVER
1	IOMMU DRIVERS
1	IOC3 SERIAL DRIVER
1	IOC3 ETHERNET DRIVER
1	INTEL WIRELESS 3945ABG/BG, 4965AGN (iwlegacy)
1	INTEL PRO/WIRELESS 2100, 2200BG, 2915ABG NETWORK CONNECTION SUPPORT
1	INTEL-MID GPIO DRIVER
1	INTEL MENLOW THERMAL DRIVER
1	INTEL MANAGEMENT ENGINE (mei)
1	INTEL IXP4XX RANDOM NUMBER GENERATOR SUPPORT
1	INTEL IXP4XX QMGR, NPE, ETHERNET and HSS SUPPORT
1	INTEL IOP-ADMA DMA DRIVER
1	INTEL IOMMU (VT-d)
1	INTEL IDLE DRIVER
1	INTEL IA32 MICROCODE UPDATE SUPPORT
1	INTEL FRAMEBUFFER DRIVER (excluding 810 and 815)
1	INTEL 810/815 FRAMEBUFFER DRIVER
1	INPUT MULTITOUCH (MT) PROTOCOL
1	INA2XX HARDWARE MONITOR DRIVER
1	INA209 HARDWARE MONITOR DRIVER
1	IIO SUBSYSTEM AND DRIVERS
1	IGUANAWORKS USB IR TRANSCEIVER
1	IDLE-I7300
1	IDE SUBSYSTEM
1	IDE/ATAPI DRIVERS
1	IDEAPAD LAPTOP SLIDEBAR DRIVER
1	IDEAPAD LAPTOP EXTRAS DRIVER
1	ICH LPC AND GPIO DRIVER
1	IBM ServeRAID RAID DRIVER
1	IBM Power Virtual SCSI/FC Device Drivers
1	IBM Power Virtual Ethernet Device Driver
1	IBM Power Linux RAID adapter
1	IBM Power 842 compression accelerator
1	i386 SETUP CODE / CPU ERRATA WORKAROUNDS
1	i386 BOOT CODE
1	I2C-TINY-USB DRIVER
1	I2C-TAOS-EVM DRIVER
1	I2C SUBSYSTEM
1	I2C/SMBUS STUB DRIVER
1	I2C/SMBUS CONTROLLER DRIVERS FOR PC
1	I2C OVER PARALLEL PORT
1	HWPOISON MEMORY FAILURE HANDLING
1	HUGETLB FILESYSTEM
1	HTCPEN TOUCHSCREEN DRIVER
1	HSR NETWORK PROTOCOL
1	HSO 3G MODEM DRIVER
1	HSI SUBSYSTEM
1	HPFS FILESYSTEM
1	HPET:	High Precision Event Timers driver
1	HP100:	Driver for HP 10/100 Mbit/s Voice Grade Network Adapter Series
1	HOST AP DRIVER
1	HIPPI
1	HIGH-RESOLUTION TIMERS, CLOCKEVENTS, DYNTICKS
1	HIGHPOINT ROCKETRAID 3xxx RAID DRIVER
1	HID CORE LAYER
1	HGA FRAMEBUFFER DRIVER
1	HEWLETT-PACKARD SMART CISS RAID DRIVER (cciss)
1	HEWLETT-PACKARD SMART ARRAY RAID DRIVER (hpsa)
1	HEWLETT-PACKARD SMART2 RAID DRIVER
1	HDPVR USB VIDEO ENCODER DRIVER
1	HD29L2 MEDIA DRIVER
1	HARDWARE SPINLOCK CORE
1	HARD DRIVE ACTIVE PROTECTION SYSTEM (HDAPS) DRIVER
1	GUID PARTITION TABLE (GPT)
1	GSPCA USB WEBCAM DRIVER
1	GSPCA T613 SUBDRIVER
1	GSPCA SN9C20X SUBDRIVER
1	GSPCA PAC207 SONIXB SUBDRIVER
1	GSPCA M5602 SUBDRIVER
1	GSPCA GL860 SUBDRIVER
1	GSPCA FINEPIX SUBDRIVER
1	GRETH 10/100/1G Ethernet MAC device driver
1	GRE DEMULTIPLEXER DRIVER
1	GFS2 FILE SYSTEM
1	GENERIC UIO DRIVER FOR PCI DEVICES
1	GENERIC PHY FRAMEWORK
1	GENERIC INCLUDE/ASM HEADER FILES
1	GENERIC HDLC (WAN) DRIVERS
1	GENERIC GPIO I2C MULTIPLEXER DRIVER
1	GENERIC GPIO I2C DRIVER
1	GEMTEK FM RADIO RECEIVER DRIVER
1	GDT SCSI DISK ARRAY CONTROLLER DRIVER
1	GCOV BASED KERNEL PROFILING
1	FUTURE DOMAIN TMC-16x0 SCSI DRIVER (16-bit)
1	FUSE: FILESYSTEM IN USERSPACE
1	FUJITSU TABLET EXTRAS
1	FUJITSU LAPTOP EXTRAS
1	FUJITSU FR-V (FRV) PORT
1	FS-CACHE: LOCAL CACHING FOR NETWORK FILESYSTEMS
1	FRONTSWAP API
1	FREEVXFS FILESYSTEM
1	FREESCALE USB PERIPHERAL DRIVERS
1	FREESCALE SOC SOUND DRIVERS
1	FREESCALE QUICC ENGINE UCC UART DRIVER
1	FREESCALE QUICC ENGINE UCC ETHERNET DRIVER
1	FREESCALE IMX / MXC FRAMEBUFFER DRIVER
1	FREESCALE I2C CPM DRIVER
1	FREESCALE DIU FRAMEBUFFER DRIVER
1	FPU EMULATOR
1	FMC SUBSYSTEM
1	FLOPPY DRIVER
1	FIRMWARE LOADER (request_firmware)
1	FIREWIRE SUBSYSTEM
1	FIREWIRE SBP-2 TARGET
1	FIREWIRE MEDIA DRIVERS (firedtv)
1	FIREWIRE AUDIO DRIVERS
1	FINTEK F75375S HARDWARE MONITOR AND FAN CONTROLLER DRIVER
1	FILESYSTEMS (VFS and infrastructure)
1	FCOE SUBSYSTEM (libfc, libfcoe, fcoe)
1	FC2580 MEDIA DRIVER
1	FC0011 TUNER DRIVER
1	FAULT INJECTION SUPPORT
1	FARSYNC SYNCHRONOUS DRIVER
1	FANOTIFY
1	F71805F HARDWARE MONITORING DRIVER
1	F2FS FILE SYSTEM
1	EXYNOS DP DRIVER
1	EXTENSIBLE FIRMWARE INTERFACE (EFI)
1	Extended Verification Module (EVM)
1	EXT2 FILE SYSTEM
1	ETHERNET PHY LIBRARY
1	ETHERNET BRIDGE
1	EPSON S1D13XXX FRAMEBUFFER DRIVER
1	ENHANCED ERROR HANDLING (EEH)
1	ENE KB2426 (ENE0100/ENE020XX) INFRARED RECEIVER
1	ENE CB710 FLASH CARD READER DRIVER
1	EMULEX LPFC FC SCSI DRIVER
1	EM28XX VIDEO4LINUX DRIVER
1	EHEA (IBM pSeries eHEA 10Gb ethernet adapter) DRIVER
1	EFIFB FRAMEBUFFER DRIVER
1	EDIROL UA-101/UA-1000 DRIVER
1	EDAC-SBRIDGE
1	EDAC-R82600
1	EDAC-PASEMI
1	EDAC-MPC85XX
1	EDAC-I82443BXGX
1	EDAC-I7CORE
1	EDAC-I7300
1	EDAC-I5400
1	EDAC-I5000
1	EDAC-I3000
1	EDAC-GHES
1	EDAC-E7XXX
1	ECRYPT FILE SYSTEM
1	EC100 MEDIA DRIVER
1	EBTABLES
1	EATA-PIO SCSI DRIVER
1	EATA ISA/EISA/PCI SCSI DRIVER
1	EATA-DMA SCSI DRIVER
1	E4000 MEDIA DRIVER
1	DZ DECSTATION DZ11 SERIAL DRIVER
1	DYNAMIC DEBUG
1	DVB_USB_V2 MEDIA DRIVER
1	DVB_USB_RTL28XXU MEDIA DRIVER
1	DVB_USB_MXL111SF MEDIA DRIVER
1	DVB_USB_GL861 MEDIA DRIVER
1	DVB_USB_EC168 MEDIA DRIVER
1	DVB_USB_CXUSB MEDIA DRIVER
1	DVB_USB_CE6230 MEDIA DRIVER
1	DVB_USB_AU6610 MEDIA DRIVER
1	DVB_USB_ANYSEE MEDIA DRIVER
1	DVB_USB_AF9035 MEDIA DRIVER
1	DVB_USB_AF9015 MEDIA DRIVER
1	DSCC4 DRIVER
1	DSBR100 USB FM RADIO DRIVER
1	DRM PANEL DRIVERS
1	DRM DRIVERS
1	DRIVER CORE, KOBJECTS, DEBUGFS AND SYSFS
1	DRBD DRIVER
1	DPT_I2O SCSI RAID DRIVER
1	DOUBLETALK DRIVER
1	DOCUMENTATION
1	DOCKING STATION DRIVER
1	DME1737 HARDWARE MONITOR DRIVER
1	DMA BUFFER SHARING FRAMEWORK
1	DISPLAYLINK USB 2.0 FRAMEBUFFER DRIVER (UDLFB)
1	DISKQUOTA
1	DISK GEOMETRY AND PARTITION HANDLING
1	DIRECTORY NOTIFICATION (DNOTIFY)
1	DIOLAN U2C-12 I2C DRIVER
1	DIGI NEO AND CLASSIC PCI PRODUCTS
1	DIGI EPCA PCI PRODUCTS
1	DIALOG SEMICONDUCTOR DRIVERS
1	DEVICE NUMBER REGISTRY
1	DESIGNWARE USB3 DRD IP DRIVER
1	DESIGNWARE USB2 DRD IP DRIVER
1	DELL WMI EXTRAS DRIVER
1	DELL SYSTEMS MANAGEMENT BASE DRIVER (dcdbas)
1	DELL LAPTOP SMM DRIVER
1	DELL LAPTOP DRIVER
1	DEFXX FDDI NETWORK DRIVER
1	DCCP PROTOCOL
1	DAMA SLAVE for AX.25
1	CYTTSP TOUCHSCREEN DRIVER
1	CYPRESS_FIRMWARE MEDIA DRIVER
1	CYBERPRO FB DRIVER
1	CXGB4VF ETHERNET DRIVER (CXGB4VF)
1	CXGB4 IWARP RNIC DRIVER (IW_CXGB4)
1	CXGB4 ETHERNET DRIVER (CXGB4)
1	CXGB3 IWARP RNIC DRIVER (IW_CXGB3)
1	CXGB3 ETHERNET DRIVER (CXGB3)
1	CXD2820R MEDIA DRIVER
1	CX88 VIDEO4LINUX DRIVER
1	CX2341X MPEG ENCODER HELPER MODULE
1	CX18 VIDEO4LINUX DRIVER
1	CW1200 WLAN driver
1	CS5535 Audio ALSA driver
1	CRYPTOGRAPHIC RANDOM NUMBER GENERATOR
1	CPUSETS
1	CPUID/MSR DRIVER
1	CPMAC ETHERNET DRIVER
1	COSA/SRP SYNC SERIAL DRIVER
1	CORETEMP HARDWARE MONITORING DRIVER
1	CONNECTOR
1	CONFIGFS
1	CONEXANT ACCESSRUNNER USB DRIVER
1	COMPAL LAPTOP SUPPORT
1	COMPACTPCI HOTPLUG ZIATECH ZT5550 DRIVER
1	COMPACTPCI HOTPLUG GENERIC DRIVER
1	COMPACTPCI HOTPLUG CORE
1	COMMON INTERNET FILE SYSTEM (CIFS)
1	COMMON CLK FRAMEWORK
1	C-MEDIA CMI8788 DRIVER
1	CLK API
1	CLEANCACHE API
1	CISCO VIC LOW LATENCY NIC DRIVER
1	CIRRUS LOGIC EP93XX OHCI USB HOST DRIVER
1	CIRRUS LOGIC EP93XX ETHERNET DRIVER
1	CHROME HARDWARE PLATFORM SUPPORT
1	CHIPIDEA USB HIGH SPEED DUAL ROLE CONTROLLER
1	CHINESE DOCUMENTATION
1	CFG80211 and NL80211
1	CFAG12864B LCD DRIVER
1	CFAG12864BFB LCD FRAMEBUFFER DRIVER
1	CEPH DISTRIBUTED FILE SYSTEM CLIENT
1	CELL BROADBAND ENGINE ARCHITECTURE
1	CARL9170 LINUX COMMUNITY WIRELESS DRIVER
1	CAPABILITIES
1	CAN NETWORK LAYER
1	CAIF NETWORK LAYER
1	CAFE CMOS INTEGRATED CAMERA CONTROLLER DRIVER
1	CADET FM/AM RADIO RECEIVER DRIVER
1	CACHEFILES: FS-CACHE BACKEND FOR CACHING ON MOUNTED FILESYSTEMS
1	BUSLOGIC SCSI DRIVER
1	BTTV VIDEO4LINUX DRIVER
1	BT8XXGPIO DRIVER
1	BT87X AUDIO DRIVER
1	BSG (block layer generic sg v4 driver)
1	BROCADE BNA 10 GIGABIT ETHERNET DRIVER
1	BROADCOM SPECIFIC AMBA DRIVER (BCMA)
1	BROADCOM KONA GPIO DRIVER
1	BROADCOM GENET ETHERNET DRIVER
1	BROADCOM BNX2X 10 GIGABIT ETHERNET DRIVER
1	BROADCOM BNX2I 1/10 GIGABIT iSCSI DRIVER
1	BROADCOM BNX2 GIGABIT ETHERNET DRIVER
1	BROADCOM BNX2FC 10 GIGABIT FCOE DRIVER
1	BROADCOM BCM5301X ARM ARCHICTURE
1	BROADCOM BCM2835 ARM ARCHICTURE
1	BROADCOM B44 10/100 ETHERNET DRIVER
1	BLOCK LAYER
1	BLOCK2MTD DRIVER
1	BLINKM RGB LED DRIVER
1	BLACKFIN SERIAL DRIVER
1	BLACKFIN SDH DRIVER
1	BLACKFIN MEDIA DRIVER
1	BLACKFIN I2C TWI DRIVER
1	BLACKFIN ARCHITECTURE
1	BFS FILE SYSTEM
1	BCACHE (BLOCK LAYER CACHE)
1	BAYCOM/HDLCDRV DRIVERS FOR AX.25
1	B43 WIRELESS DRIVER
1	AZTECH FM RADIO RECEIVER DRIVER
1	AZ6007 DVB DRIVER
1	AX.25 NETWORK LAYER
1	AUXILIARY DISPLAY DRIVERS
1	AUDIT SUBSYSTEM
1	ATTO EXPRESSSAS SAS/SATA RAID SCSI DRIVER
1	ATMEL WIRELESS DRIVER
1	ATMEL USBA UDC DRIVER
1	ATMEL TSADCC DRIVER
1	ATMEL Timer Counter (TC) AND CLOCKSOURCE DRIVERS
1	ATMEL SPI DRIVER
1	ATMEL MACB ETHERNET DRIVER
1	ATMEL LCDFB DRIVER
1	ATMEL ISI DRIVER
1	ATMEL I2C DRIVER
1	ATMEL DMA DRIVER
1	ATMEL AT91 / AT32 SERIAL DRIVER
1	ATMEL AT91 / AT32 MCI DRIVER
1	ATM
1	ATK0110 HWMON DRIVER
1	ATI_REMOTE2 DRIVER
1	ATHEROS ATH GENERIC UTILITIES
1	ATHEROS ATH6KL WIRELESS DRIVER
1	ATA OVER ETHERNET (AOE) DRIVER
1	AT24 EEPROM DRIVER
1	ASYNCHRONOUS TRANSFERS/TRANSFORMS (IOAT) API
1	ASUS NOTEBOOKS AND EEEPC ACPI/WMI EXTRAS DRIVERS
1	ASC7621 HARDWARE MONITOR DRIVER
1	AS3645A LED FLASH CONTROLLER DRIVER
1	ARM/ZYNQ ARCHITECTURE
1	ARM/ZIPIT Z2 SUPPORT
1	ARM/VT8500 ARM ARCHITECTURE
1	ARM/VOIPAC PXA270 SUPPORT
1	ARM/VFP SUPPORT
1	ARM/Ux500 CLOCK FRAMEWORK SUPPORT
1	ARM/Ux500 ARM ARCHITECTURE
1	ARM/U300 MACHINE SUPPORT
1	ARM/THECUS N2100 MACHINE SUPPORT
1	ARM/TEXAS INSTRUMENT KEYSTONE ARCHITECTURE
1	ARM/TETON BGA MACHINE SUPPORT
1	ARM/TECHNOLOGIC SYSTEMS TS7250 MACHINE SUPPORT
1	ARM/SOCFPGA CLOCK FRAMEWORK SUPPORT
1	ARM/SOCFPGA ARCHITECTURE
1	ARM SMMU DRIVER
1	ARM/SAMSUNG MOBILE MACHINE SUPPORT
1	ARM/S5P EXYNOS ARM ARCHITECTURES
1	ARM/Rockchip SoC support
1	ARM/RISCPC ARCHITECTURE
1	ARM/RADISYS ENP2611 MACHINE SUPPORT
1	ARM/PT DIGITAL BOARD PORT
1	ARM PRIMECELL UART PL010 AND PL011 DRIVERS
1	ARM PRIMECELL MMCI PL180/1 DRIVER
1	ARM PRIMECELL KMI PL050 DRIVER
1	ARM PRIMECELL CLCD PL110 DRIVER
1	ARM PRIMECELL BUS SUPPORT
1	ARM PRIMECELL AACI PL041 DRIVER
1	ARM PORT
1	ARM PMU PROFILING AND DEBUGGING
1	ARM/PLEB SUPPORT
1	ARM/PALMZ72 SUPPORT
1	ARM/PALMTX,PALMT5,PALMLD,PALMTE2,PALMTC SUPPORT
1	ARM/PALM TREO SUPPORT
1	ARM/Orion SoC/Technologic Systems TS-78xx platform support
1	ARM/OPENMOKO NEO FREERUNNER (GTA02) MACHINE SUPPORT
1	ARM/NUVOTON W90X900 ARM ARCHITECTURE
1	ARM/NEC MOBILEPRO 900/c MACHINE SUPPORT
1	ARM/MIOA701 MACHINE SUPPORT
1	ARM/MICREL KS8695 ARCHITECTURE
1	ARM MFM AND FLOPPY DRIVERS
1	ARM/Marvell Berlin SoC support
1	ARM/MAGICIAN MACHINE SUPPORT
1	ARM/LOGICPD PXA270 MACHINE SUPPORT
1	ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT
1	ARM/INTEL RESEARCH IMOTE/STARGATE 2 MACHINE SUPPORT
1	ARM/INTEL IXDP2850 MACHINE SUPPORT
1	ARM/INTEL IOP33X ARM ARCHITECTURE
1	ARM/INCOME PXA270 SUPPORT
1	ARM/HP JORNADA 7XX MACHINE SUPPORT
1	ARM/GUMSTIX MACHINE SUPPORT
1	ARM/GLOMATION GESBC9312SX MACHINE SUPPORT
1	ARM/FREESCALE MXS ARM ARCHITECTURE
1	ARM/FOOTBRIDGE ARCHITECTURE
1	ARM/FARADAY FA526 PORT
1	ARM/ENERGY MICRO (SILICON LABS) EFM32 SUPPORT
1	ARM/EBSA110 MACHINE SUPPORT
1	ARM/CSR SIRFPRIMA2 MACHINE SUPPORT
1	ARM/CORTINA SYSTEMS GEMINI ARM ARCHITECTURE
1	ARM/CORGI MACHINE SUPPORT
1	ARM/CONTEC MICRO9 MACHINE SUPPORT
1	ARM/COMPULAB CM-X270/EM-X270 and CM-X300 MACHINE SUPPORT
1	ARM/CLKDEV SUPPORT
1	ARM/CIRRUS LOGIC EDB9315A MACHINE SUPPORT
1	ARM/CIRRUS LOGIC CLPS711X ARM ARCHITECTURE
1	ARM/CAVIUM NETWORKS CNS3XXX MACHINE SUPPORT
1	ARM/CALXEDA HIGHBANK ARCHITECTURE
1	ARM/Allwinner SoC Clock Support
1	ARM/Allwinner A1X SoC support
1	ARM/AJECO 1ARM MACHINE SUPPORT
1	ARM/AFEB9260 MACHINE SUPPORT
1	ARM/ADS SPHERE MACHINE SUPPORT
1	ARC FRAMEBUFFER DRIVER
1	ARASAN COMPACT FLASH PATA CONTROLLER
1	APTINA CAMERA SENSOR PLL
1	APPLETALK NETWORK LAYER
1	APPLE SMC DRIVER
1	APPLE BCM5974 MULTITOUCH DRIVER
1	APPARMOR SECURITY MODULE
1	APM DRIVER
1	AOA (Apple Onboard Audio) ALSA DRIVER
1	ANALOG DEVICES INC ASOC CODEC DRIVERS
1	ANALOG DEVICES INC ADV7842 DRIVER
1	ANALOG DEVICES INC ADV7604 DRIVER
1	ANALOG DEVICES INC ADV7511 DRIVER
1	ANALOG DEVICES INC AD9389B DRIVER
1	AMS (Apple Motion Sensor) DRIVER
1	AMD MICROCODE UPDATE SUPPORT
1	AMD IOMMU (AMD-VI)
1	AMD GEODE CS5536 USB DEVICE CONTROLLER DRIVER
1	AMD FAM15H PROCESSOR POWER MONITORING DRIVER
1	AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER
1	ALTERA UART/JTAG UART SERIAL DRIVERS
1	ALTERA TRIPLE SPEED ETHERNET DRIVER
1	ALI1563 I2C DRIVER
1	ALCHEMY AU1XX0 MMC DRIVER
1	ALCATEL SPEEDTOUCH USB DRIVER
1	AIO
1	AIMSLAB FM RADIO RECEIVER DRIVER
1	AIC7XXX / AIC79XX SCSI DRIVER
1	AHA152X SCSI DRIVER
1	AGPGART DRIVER
1	AFS FILESYSTEM & AF_RXRPC SOCKET DOMAIN
1	AF9033 MEDIA DRIVER
1	AF9013 MEDIA DRIVER
1	AEDSP16 DRIVER
1	ADXL34X THREE-AXIS DIGITAL ACCELEROMETER DRIVER (ADXL345/ADXL346)
1	ADVANSYS SCSI DRIVER
1	ADT7475 HARDWARE MONITOR DRIVER
1	ADT746X FAN DRIVER
1	ADS1015 HARDWARE MONITOR DRIVER
1	ADP8860 BACKLIGHT DRIVER (ADP8860/ADP8861/ADP8863)
1	ADP5588 QWERTY KEYPAD AND IO EXPANDER DRIVER (ADP5588/ADP5587)
1	ADP5520 BACKLIGHT DRIVER WITH IO EXPANDER (ADP5520/ADP5501)
1	ADP1653 FLASH CONTROLLER DRIVER
1	ADM1029 HARDWARE MONITOR DRIVER
1	ADM1025 HARDWARE MONITOR DRIVER
1	ADDRESS SPACE LAYOUT RANDOMIZATION (ASLR)
1	AD7879 TOUCHSCREEN DRIVER (AD7879/AD7889)
1	AD7877 TOUCHSCREEN DRIVER
1	AD714X CAPACITANCE TOUCH SENSOR DRIVER (AD7142/3/7/8/7A)
1	AD5398 CURRENT REGULATOR DRIVER (AD5398/AD5821)
1	AD525X ANALOG DEVICES DIGITAL POTENTIOMETERS DRIVER
1	AD1889 ALSA SOUND DRIVER
1	ACPI VIDEO DRIVER
1	ACPI THERMAL DRIVER
1	ACPI FAN DRIVER
1	ACER WMI LAPTOP EXTRAS
1	ACER ASPIRE ONE TEMPERATURE AND FAN DRIVER
1	ACENIC DRIVER
1	ABIT UGURU 3 HARDWARE MONITOR DRIVER
1	ABIT UGURU 1,2 HARDWARE MONITOR DRIVER
1	AACRAID SCSI RAID DRIVER
1	A8293 MEDIA DRIVER
1	8250/16?50 (AND CLONE UARTS) SERIAL DRIVER
1	6PACK NETWORK DRIVER FOR AX.25
1	53C700 AND 53C700-66 SCSI DRIVER
1	3WARE SAS/SATA-RAID SCSI DRIVERS (3W-XXXX, 3W-9XXX, 3W-SAS)
1	3CR990 NETWORK DRIVER
1	3C59X NETWORK DRIVER

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Paul Walmsley]

On Wed, 28 May 2014, Olof Johansson wrote:

> That's why I'm somewhat hesitant to label every reviewer a
> maintainer since it's hard to tune expectations.

Yes - the designated reviewers would be expected to follow the 
architecture or guidelines set out by the maintainers.

- Paul

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Paul Walmsley]

On Thu, 29 May 2014, Olof Johansson wrote:

> On Thu, May 29, 2014 at 2:03 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > On Thursday, May 29, 2014 02:27:53 PM Theodore Ts'o wrote:
> >> On Wed, May 28, 2014 at 06:48:47PM +0000, Paul Walmsley wrote:
> >> >
> >> > Also long-overdue is a clarification on exactly what "Acked-by" means.
> >> > Right now it is being used for at least two distinct and
> >> > mutually-incompatible purposes:
> >> >
> >> > 1. A maintainer A for code affected by a patch, who is distinct from a
> >> > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> >> > being OK for maintainer B to send upstream
> >> >
> >> > 2. A casual review has been done by someone who is not a maintainer for
> >> > the code in question
> >> >
> >> > What I would propose is to have the first use replaced by a new tag,
> >> > "Maintainer-acked-by:", and the second use abolished, along with
> >> > "Acked-by:", and replaced by "Reviewed-by:".
> >>
> >> I agree in general, but if we're going to abolish the 2nd use
> >> entirely, then it's much simpler to keep Acked-by for its original
> >> meaning; it's easier to type, after all.
> >>
> >> This is basically I do for ext4 patches today; if someone sends me an
> >> acked-by in the #2 sense, I simply won't add it to the s-o-b section,
> >> and I don't let the fact that someone has asserted that they have done
> >> a casual review to give me a false sense of security; if I still have
> >> to do a deep review, I'm going to catch the casual stuff anyway, and
> >> the fact that a casual review doesn't obviate the need for a careful
> >> review.
> >>
> >> But if a senior ext4 developer adds a Reviewed-by:, that does lend a
> >> lot of value to me as a maintainer, since I can trust that certain
> >> folks like Jan and Eric and Lukas and others will do a good job doing
> >> the review, and that actually *does* offload significant amounts of
> >> work off my shoulders.
> >
> > Well, perhaps we can reserve the Acked-by for maintainers and add
> > something like Supported-by for the 2nd meaning.
> 
> What are we really trying to fix here? Is the current process really
> broken or are we trying to create more process that's not needed for
> some other reason?

Where's the extra process that you're objecting to?  All this proposal is 
intended to do is to clarify the semantics of Acked-by. 

> And I'm definitely not worried by the possible conflict of the "I gave 
> this a casual review and I think we should let it go in" acks since a 
> maintainer is unlikely to give out those kind of acks to code that he 
> would otherwise merge himself.

Non-maintainers send Acked-by:s regularly, to indicate their "casual 
review".


- Paul

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Paul Walmsley]

On Thu, 29 May 2014, Greg KH wrote:

> On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > What are we really trying to fix here? Is the current process really
> > broken or are we trying to create more process that's not needed for
> > some other reason?
> 
> I think the latter.
> 
> Somehow, we seem to be constantly increasing our rate of change, are
> people thinking we are having problems here?  If so, exactly where?

If "increasing our rate of change" was the only metric that we cared 
about, we wouldn't be discussing how to attract more reviewers.

But, on the other hand, if "increasing our rate of change" is the primary 
metric that's important to some folks, it might explain why we keep 
asking ourselves how to increase the review bandwidth every year, with 
little apparent positive result.

> This thread has taken an odd turn into trying to make some new kind of 
> process for an unknown issue (i.e. the people on this list are not going 
> to recognize the reviewers more, it's up to you to educate your managers 
> / company more.)

There's no additional process suggested for the Acked-by: discussion - so 
I guess you're not referring to the $SUBJECT thread, but rather to the 
"designated reviewer" proposal.

In that case, the issue seems fairly obvious:

   How do we encourage and recognize people who take the time to review 
   code, and who do it well from the point of view of maintainers?

...

Regarding the "managers/company" comment: I don't think it matters whether 
good reviews come from hobbyists or corporate employees - humans like 
being formally recognized for their contributions.


- Paul

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Li Zefan]

On 2014/5/30 7:34, Greg KH wrote:
> On Thu, May 29, 2014 at 10:13:21AM +0400, James Bottomley wrote:
>> I suspect, but cannot prove, it is somewhat driven by the patch
>> statistics.  I know in Parallels, I track features accepted into the
>> kernel, not patches, but we do have a patch metric as well for people
>> who do incidental bug fixing as they push features, so we're not 100%
>> pure on this.
>>
>> When I recently gave a comparison of OpenStack and the Kernel, one thing
>> that stood out starkly was that they just don't have our volume of one
>> line, one patch per file hundreds of times and trivial changes, so we
>> definitely have a problem in this area.
>>
>> The reason OpenStack has so few trivial and multiply split patches is
>> mostly grounded in the difficulty of submitting a patch to their tree.
> 
> And I would argue that this is going to be a long-term problem for them.
> 
>> Just preparing it for gerrit takes a long time (around ten minutes per
>> patch), which really reduces the volume of trivial patches, just because
>> no-one has that much energy (although some would argue it reduces the
>> volume too far).
>>
>> It's human nature to spend more time on the easy problems, so a flood of
>> trivial patches which are "obviously" correct is easier to review and
>> include than one patch which alters something deep within mm and needs
>> careful thought.  At best, excessively split trivial patches are a
>> dilution of our review effort and at worst, they're actively sucking
>> people away from tackling the hard problems.
> 
> I have not heard of these "cleanup" patches taking anyone's review
> cycles up for the "harder" patches, do you have examples of it?
> 
> In fact, I would argue the fact that we do so easily accept cleanups
> across the board that keeps our developer community engaged and new
> people joining in easy.  It's _really_ hard to get involved in
> OpenStack, by design.  The kernel is not that way, and hopefully never
> will be, as we want to succeed...
> 

Yeah, we've been encouraging our people working on upstream kernel, and
many of them started by making cleanup patches, but then they moved to
bug fixes and other work. However it's hard to ask them to join a subsystem
development by also reviewing patches. I can think of some possible
reasons:

- Not confident enough?
- Language barrier.
- Asian culture issue. We don't like discussions, so we tend to avoid
discussing and arguing with other kernel developers and maintainers.
- Credit and the feeling of achievement is not high in reviewing patches.
- We are busy with internal jobs. :)

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Li Zefan]

On 2014/5/30 1:41, Greg KH wrote:
> On Thu, May 29, 2014 at 05:28:11PM +0800, Li Zefan wrote:
>> On 2014/5/29 5:32, Thomas Gleixner wrote:
>>> On Wed, 28 May 2014, Mark Brown wrote:
>>>
>>>> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
>>>>
>>>>>> There's a world of difference between thanking people for review and a
>>>>>> detailed account of all the changes made in every single iteration of
>>>>>> the review.
>>>>
>>>>> This is already covered in Documentation/SubmittingPatches. Quoting
>>>>> lines 585-592 (see last sentence):
>>>>
>>>> Right, but Daniel is proposing lifting that above the --- and including
>>>> it in git.
>>>
>>> What you really want is:
>>>
>>> Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
>>>
>>> It's way more useful than any of the v1-n writeups, which are most of
>>> the time just a completely waste of electrons. Even if written well,
>>> without the actual review context they are pretty pointless.
>>>
>>
>> A QA asked me about kernel development process. One of his question is,
>> he found some valuable information in the discussion of the patch often
>> won't be added to the changelog, so providing the commit how to find
>> the discussion?
> 
> A research group has created a tool that takes a given git commit, finds
> the mailing list discussion for that patch.  It was posted to lkml 6 or
> so months ago, you should point them at that tool if they want to do
> this.
> 

Thanks for the information!

So I believe not many people know about that tool and I have to try it
out to see how well it works.

I think many people want to do this, and the link tag is better than
an out-of-tree tool, which people just don't know about.

We may also use the link tag for stable commits. So for those commits
that were not backported automatically but requested specifically,
people can easily find the history about why it got merged.

If it were there, our previous backporting work for 3.4.x would have
been eaiser.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Thu, 2014-05-29 at 16:34 -0700, Greg KH wrote:
> On Thu, May 29, 2014 at 10:13:21AM +0400, James Bottomley wrote:
> > I suspect, but cannot prove, it is somewhat driven by the patch
> > statistics.  I know in Parallels, I track features accepted into the
> > kernel, not patches, but we do have a patch metric as well for people
> > who do incidental bug fixing as they push features, so we're not 100%
> > pure on this.
> > 
> > When I recently gave a comparison of OpenStack and the Kernel, one thing
> > that stood out starkly was that they just don't have our volume of one
> > line, one patch per file hundreds of times and trivial changes, so we
> > definitely have a problem in this area.
> > 
> > The reason OpenStack has so few trivial and multiply split patches is
> > mostly grounded in the difficulty of submitting a patch to their tree.
> 
> And I would argue that this is going to be a long-term problem for them.

Well, I think they make it too hard, but I think they also demonstrate
that some friction in the process is a useful refiner.

> > Just preparing it for gerrit takes a long time (around ten minutes per
> > patch), which really reduces the volume of trivial patches, just because
> > no-one has that much energy (although some would argue it reduces the
> > volume too far).
> > 
> > It's human nature to spend more time on the easy problems, so a flood of
> > trivial patches which are "obviously" correct is easier to review and
> > include than one patch which alters something deep within mm and needs
> > careful thought.  At best, excessively split trivial patches are a
> > dilution of our review effort and at worst, they're actively sucking
> > people away from tackling the hard problems.
> 
> I have not heard of these "cleanup" patches taking anyone's review
> cycles up for the "harder" patches, do you have examples of it?

Yes, me.

> In fact, I would argue the fact that we do so easily accept cleanups
> across the board that keeps our developer community engaged and new
> people joining in easy.

OK, so this is a difference in style.  I prefer the find a bug and fix
it approach.  The thing that worries me about cleanup for its own sake
is that we're developing ever more elaborate tools to detect pattern
changes, so it's spawning an industry in its own right and some people
see it as an end in itself, not a precursor to graduating to substantive
changes.

I also have no objections to cleanup coupled with substantive changes.
In fact, when someone wants to clean up one of the old and crufty
drivers, I say yes, provided they can maintain it.  We've had a couple
of successes in this regard.

James

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Fri, May 30, 2014 at 08:26:13AM +0400, James Bottomley wrote:
> On Thu, 2014-05-29 at 16:34 -0700, Greg KH wrote:
> > > Just preparing it for gerrit takes a long time (around ten minutes per
> > > patch), which really reduces the volume of trivial patches, just because
> > > no-one has that much energy (although some would argue it reduces the
> > > volume too far).
> > > 
> > > It's human nature to spend more time on the easy problems, so a flood of
> > > trivial patches which are "obviously" correct is easier to review and
> > > include than one patch which alters something deep within mm and needs
> > > careful thought.  At best, excessively split trivial patches are a
> > > dilution of our review effort and at worst, they're actively sucking
> > > people away from tackling the hard problems.
> > 
> > I have not heard of these "cleanup" patches taking anyone's review
> > cycles up for the "harder" patches, do you have examples of it?
> 
> Yes, me.

Then flat out reject them.  Some subsystem maintainers do.  Or, push
back, like you said later with a "I'll take this if you will maintain
the driver from now on." :)

No one is forcing you to take these drivers, but as someone who has
really old code, I can see how it would get annoying with people hitting
you with cleanup patches.  Maybe you should just do a "coding style
cleanup" set of patches yourself one release to get it all in shape then
not have to worry about it anymore.  That's what I did years ago for the
USB drivers...

thanks,

greg k-h

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Greg KH]

On Fri, May 30, 2014 at 01:12:06AM +0000, Paul Walmsley wrote:
> On Thu, 29 May 2014, Greg KH wrote:
> 
> > On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > > What are we really trying to fix here? Is the current process really
> > > broken or are we trying to create more process that's not needed for
> > > some other reason?
> > 
> > I think the latter.
> > 
> > Somehow, we seem to be constantly increasing our rate of change, are
> > people thinking we are having problems here?  If so, exactly where?
> 
> If "increasing our rate of change" was the only metric that we cared 
> about, we wouldn't be discussing how to attract more reviewers.

I'm not saying it's the only metric, but just pointing out that while we
constantly ask for more reviewers, it doesn't seem to be slowing us
down.

> But, on the other hand, if "increasing our rate of change" is the primary 
> metric that's important to some folks, it might explain why we keep 
> asking ourselves how to increase the review bandwidth every year, with 
> little apparent positive result.

I have more review help now.  Some of it happened organically, and some
I specifically asked for.  If you need help, ask other developers in the
area to help out.  The worse they can do is say no and you are back to
where you were before, no harm done.

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[James Bottomley]

On Thu, 2014-05-29 at 22:02 -0700, Greg KH wrote:
> On Fri, May 30, 2014 at 08:26:13AM +0400, James Bottomley wrote:
> > On Thu, 2014-05-29 at 16:34 -0700, Greg KH wrote:
> > > > Just preparing it for gerrit takes a long time (around ten minutes per
> > > > patch), which really reduces the volume of trivial patches, just because
> > > > no-one has that much energy (although some would argue it reduces the
> > > > volume too far).
> > > > 
> > > > It's human nature to spend more time on the easy problems, so a flood of
> > > > trivial patches which are "obviously" correct is easier to review and
> > > > include than one patch which alters something deep within mm and needs
> > > > careful thought.  At best, excessively split trivial patches are a
> > > > dilution of our review effort and at worst, they're actively sucking
> > > > people away from tackling the hard problems.
> > > 
> > > I have not heard of these "cleanup" patches taking anyone's review
> > > cycles up for the "harder" patches, do you have examples of it?
> > 
> > Yes, me.
> 
> Then flat out reject them.  Some subsystem maintainers do.  Or, push
> back, like you said later with a "I'll take this if you will maintain
> the driver from now on." :)

I do (I believe I already said that).

> No one is forcing you to take these drivers, but as someone who has
> really old code, I can see how it would get annoying with people hitting
> you with cleanup patches.  Maybe you should just do a "coding style
> cleanup" set of patches yourself one release to get it all in shape then
> not have to worry about it anymore.  That's what I did years ago for the
> USB drivers...

I'm not necessarily promoting my own position, I'm questioning the wider
assertion that all these clean up patches are a good way of getting into
kernel coding.  What worries me the most is that I don't see evidence
that after hundreds of clean up patches anyone moves on to more
substantive issues.  Once we incent people with kudos for patches, why
would you move on to more substantive changes? they're harder and take
more time.  You can turn out many more cleanup type patches in the time
it takes to argue for and produce a well thought out substantive change.
On these metrics we're actually encouraging people not to graduate ...
that's really a bad thing.

We need to design programs for new people that draw them in, not keep
them on the periphery.  That's why I think bug fixing is a better
activity: it means they have to understand some of the code; the change
can be small and isolated, so they don't have to understand all the code
and it often encourages people to find out more about whatever they're
fixing, hence draws them in.

We're way off the topic of encouraging reviewers, but the bottom line
for me is I don't believe starting people out with cleanup type changes
is a good way to get wider engagement.  It is a good way to increase the
number of overall patches, all of which need to be reviewed, hence the
need for more reviewers.

James

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[James Bottomley]

On Thu, 2014-05-29 at 22:04 -0700, Greg KH wrote:
> On Fri, May 30, 2014 at 01:12:06AM +0000, Paul Walmsley wrote:
> > On Thu, 29 May 2014, Greg KH wrote:
> > 
> > > On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > > > What are we really trying to fix here? Is the current process really
> > > > broken or are we trying to create more process that's not needed for
> > > > some other reason?
> > > 
> > > I think the latter.
> > > 
> > > Somehow, we seem to be constantly increasing our rate of change, are
> > > people thinking we are having problems here?  If so, exactly where?
> > 
> > If "increasing our rate of change" was the only metric that we cared 
> > about, we wouldn't be discussing how to attract more reviewers.
> 
> I'm not saying it's the only metric, but just pointing out that while we
> constantly ask for more reviewers, it doesn't seem to be slowing us
> down.

It depends whether you believe every patch has to be reviewed.  If you
do (and I certainly believe we have to in order to maintain code
quality), we have to increase our review bandwidth at the same rate we
increase our commits, otherwise the imbalance causes a backlog of
unreviewed patches.

What I see at the moment is that the number of patches exceeds the
number of reviewers quite substantially.  By and large the reviewers
mostly get through the backlog between merge windows (but the process is
an exhausting one which will lead to reviewer burn out), but I can see a
time arriving soon when they can't and we start wrapping ourselves
around the axle because of too few reviewers.

James

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Lukáš Czerner]

On Thu, 29 May 2014, Greg KH wrote:

> Date: Thu, 29 May 2014 16:30:04 -0700
> From: Greg KH <greg@kroah.com>
> To: Olof Johansson <olof@lixom.net>
> Cc: James Bottomley <James.Bottomley@hansenpartnership.com>,
>     "ksummit-discuss@lists.linuxfoundation.org"
>     <ksummit-discuss@lists.linuxfoundation.org>
> Subject: Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging
>      more reviewers)
> 
> On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > What are we really trying to fix here? Is the current process really
> > broken or are we trying to create more process that's not needed for
> > some other reason?
> 
> I think the latter.
> 
> Somehow, we seem to be constantly increasing our rate of change, are
> people thinking we are having problems here?  If so, exactly where?
> This thread has taken an odd turn into trying to make some new kind of
> process for an unknown issue (i.e. the people on this list are not going
> to recognize the reviewers more, it's up to you to educate your managers
> / company more.)

This is not only about managers / company. Reviewers does not seem
to have much recognition in upstream community either. For example
we do take into account s-o-b when creating preliminary list of
people to get invited to kernel summit, but we do _not_ take into
account reviewd-by (or has anything changed?)...

> 
> Are maintainers who deal with huge number of patches (i.e. 1000+ a year)
> crying out for help that this is somehow going to reduce their burden?

It is going to help. More reviews is less work for a maintainer to
do on it's own.

-Lukas

> 
> I don't think so, but hey, what do I know about this development
> process...
> 
> grumble,
> 
> greg k-h
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> 

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Laurent Pinchart]

On Thursday 29 May 2014 11:33:15 Christoph Lameter wrote:
> On Thu, 29 May 2014, Laurent Pinchart wrote:
> > On the other hand, this can be an issue for developers and/or maintainers
> > who want to ensure that all parts of a patch have received proper review.
> > That's why I sometimes split patches that perform a simple change to
> > multiple drivers in a series with one patch per driver, and then squash
> > everything into a single patch before submitting a pull request. That
> > workflow could probably be improved.
> 
> Well that in turn may lead to breakage if modifications to only some files
> are merged.

This can of course only be done when the patches don't depend on each other in 
a circular way.

> If the modifications in multiple files are not depending on one another then
> they could go in as separate patches in the first place.

They could, but some maintainers prefer them not to as they find it easier to 
handle this kind of change in a single patch.

-- 
Regards,

Laurent Pinchart

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Carpenter]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

I use a script to help review cleanup patches.  It chops out the white
space, the comment changes and bracing changes.  If there is a variable
renamed then I can strip that out as well when you
`cat patch | rename_rev.pl OldName new_name`.

Attached.

regards,
dan carpenter


[-- Attachment #2: rename_rev.pl --]
[-- Type: text/x-perl, Size: 5104 bytes --]

#!/usr/bin/perl

# This is a tool to help review variable rename patches. The goal is
# to strip out the automatic sed renames and the white space changes
# and leaves the interesting code changes.
#
# Example 1: A patch renames openInfo to open_info:
#     cat diff | rename_review.pl openInfo open_info
#
# Example 2: A patch swaps the first two arguments to some_func():
#     cat diff | rename_review.pl \
#                    -e 's/some_func\((.*?),(.*?),/some_func\($2, $1,/'
#
# Example 3: A patch removes the xkcd_ prefix from some but not all the
# variables.  Instead of trying to figure out which variables were renamed
# just remove the prefix from them all:
#     cat diff | rename_review.pl -ea 's/xkcd_//g'
#
# Example 4: A patch renames 20 CamelCase variables.  To review this let's
# just ignore all case changes and all '_' chars.
#     cat diff | rename_review -ea 'tr/[A-Z]/[a-z]/' -ea 's/_//g'
#
# The other arguments are:
# -nc removes comments
# -ns removes '\' chars if they are at the end of the line.

use strict;
use File::Temp qw/ :mktemp  /;

sub usage() {
    print "usage: cat diff | $0 old new old new old new...\n";
    print "   or: cat diff | $0 -e 's/old/new/g'\n";
    print " -e : execute on old lines\n";
    print " -ea: execute on all lines\n";
    print " -nc: no comments\n";
    print " -nb: no unneeded braces\n";
    print " -ns: no slashes at the end of a line\n";
    exit(1);
}
my @subs;
my @cmds;
my $strip_comments;
my $strip_braces;
my $strip_slashes;

sub filter($) {
    my $_ = shift();
    my $old = 0;
    if ($_ =~ /^-/) {
        $old = 1;
    }
    # remove the first char
    s/^[ +-]//;
    if ($strip_comments) {
        s/\/\*.*?\*\///g;
        s/\/\/.*//;
    }
    foreach my $cmd (@cmds) {
        if ($old || $cmd->[0] =~ /^-ea$/) {
            eval $cmd->[1];
        }
    }
    foreach my $sub (@subs) {
        if ($old) {
            s/$sub->[0]/$sub->[1]/g;
        }
    }

    # remove the newline so we can move curly braces here if we want.
    s/\n//;
    return $_;
}

while (my $param1 = shift()) {
    if ($param1 =~ /^-nc$/) {
        $strip_comments = 1;
        next;
    }
    if ($param1 =~ /^-nb$/) {
        $strip_braces = 1;
        next;
    }
    if ($param1 =~ /^-ns$/) {
        $strip_slashes = 1;
        next;
    }
    my $param2 = shift();
    if ($param2 =~ /^$/) {
        usage();
    }
    if ($param1 =~ /^-e(a|)$/) {
        push @cmds, [$param1, $param2];
        next;
    }
    push @subs, [$param1, $param2];
}

my ($oldfh, $oldfile) = mkstemp("/tmp/oldXXXXX");
my ($newfh, $newfile) = mkstemp("/tmp/newXXXXX");

my $started = 0;
my $output;

#recreate an old file and a new file
while (<>) {
    if ($_ =~ /^(---|\+\+\+)/) {
        next;
    }
    if ($_ =~ /^@/) {
        $started = 1;
    }
    if ($started && !($_ =~ /^[- @+]/)) {
        last;
    }
    $output = filter($_);

    if ($strip_braces && $_ =~ /^(\+|-)\W+{/) {
        $output =~ s/^[\t ]+(.*)/ $1/;
    } else {
        $output = "\n" . $output;
    }

    if ($_ =~ /^-/) {
        print $oldfh $output;
        next;
    }
    if ($_ =~ /^\+/) {
        print $newfh $output;
        next;
    }
    print $oldfh $output;
    print $newfh $output;

}
print $oldfh "\n";
print $newfh "\n";
# git diff puts a -- and version at the end of the diff.  put the -- into the
# new file as well so it's ignored
if ($output =~ /\n-/) {
    print $newfh "-\n";
}

my $hunk;
my $old_txt;
my $new_txt;

open diff, "diff -uw $oldfile $newfile |";
while (<diff>) {
    if ($_ =~ /^(---|\+\+\+)/) {
        next;
    }

    if ($_ =~ /^@/) {

        if ($strip_comments) {
            $old_txt =~ s/\/\*.*?\*\///g;
            $new_txt =~ s/\/\*.*?\*\///g;
        }
        if ($strip_braces) {
            $old_txt =~ s/{([^;{]*?);}/$1;/g;
            $new_txt =~ s/{([^;{]*?);}/$1;/g;
            # this is a hack because i don't know how to replace nested
            # unneeded curly braces.
            $old_txt =~ s/{([^;{]*?);}/$1;/g;
            $new_txt =~ s/{([^;{]*?);}/$1;/g;
        }

        if ($old_txt ne $new_txt) {
            print $hunk;
            print $_;
        }
        $hunk = "";
        $old_txt = "";
        $new_txt = "";
        next;
    }

    $hunk = $hunk . $_;

    if ($strip_slashes) {
        s/\\$//;
    }

    if ($_ =~ /^-/) {
        s/-//;
        s/[ \t\n]//g;
        $old_txt = $old_txt . $_;
        next;
    }
    if ($_ =~ /^\+/) {
        s/\+//;
        s/[ \t\n]//g;
        $new_txt = $new_txt . $_;
        next;
    }
    if ($_ =~ /^ /) {
        s/^ //;
        s/[ \t\n]//g;
        $old_txt = $old_txt . $_;
        $new_txt = $new_txt . $_;
    }
}

if ($old_txt ne $new_txt) {
    if ($strip_comments) {
        $old_txt =~ s/\/\*.*?\*\///g;
        $new_txt =~ s/\/\*.*?\*\///g;
    }
    if ($strip_braces) {
        $old_txt =~ s/{([^;{]*?);}/$1;/g;
        $new_txt =~ s/{([^;{]*?);}/$1;/g;
        $old_txt =~ s/{([^;{]*?);}/$1;/g;
        $new_txt =~ s/{([^;{]*?);}/$1;/g;
    }

    print $hunk;
}

unlink($oldfile);
unlink($newfile);

print "\ndone.\n";

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Daniel Vetter]

On Fri, May 30, 2014 at 7:39 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> What I see at the moment is that the number of patches exceeds the
> number of reviewers quite substantially.  By and large the reviewers
> mostly get through the backlog between merge windows (but the process is
> an exhausting one which will lead to reviewer burn out), but I can see a
> time arriving soon when they can't and we start wrapping ourselves
> around the axle because of too few reviewers.

Since I merge > 1000 patches a year I seem to qualify according to
Greg. Percentage of patches with r-b tag in drm/i915: roughly 70% And
that percentage is climbing (because I'm a jerk about it) and the 30%
isn't all me since I have a co-maintainer now.

Overall kernel: roughly 15%

Require review and it will happen.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Jan Kara]

On Fri 30-05-14 12:08:12, Lukáš Czerner wrote:
> On Thu, 29 May 2014, Greg KH wrote:
> 
> > Date: Thu, 29 May 2014 16:30:04 -0700
> > From: Greg KH <greg@kroah.com>
> > To: Olof Johansson <olof@lixom.net>
> > Cc: James Bottomley <James.Bottomley@hansenpartnership.com>,
> >     "ksummit-discuss@lists.linuxfoundation.org"
> >     <ksummit-discuss@lists.linuxfoundation.org>
> > Subject: Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging
> >      more reviewers)
> > 
> > On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > > What are we really trying to fix here? Is the current process really
> > > broken or are we trying to create more process that's not needed for
> > > some other reason?
> > 
> > I think the latter.
> > 
> > Somehow, we seem to be constantly increasing our rate of change, are
> > people thinking we are having problems here?  If so, exactly where?
> > This thread has taken an odd turn into trying to make some new kind of
> > process for an unknown issue (i.e. the people on this list are not going
> > to recognize the reviewers more, it's up to you to educate your managers
> > / company more.)
> 
> This is not only about managers / company. Reviewers does not seem
> to have much recognition in upstream community either. For example
> we do take into account s-o-b when creating preliminary list of
> people to get invited to kernel summit, but we do _not_ take into
> account reviewd-by (or has anything changed?)...
  Reviewed-by *is* taken into account for KS selection. It is even
positively biased against s-o-b AFAIK.

							Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Lukáš Czerner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2169 bytes --]

On Fri, 30 May 2014, Jan Kara wrote:

> Date: Fri, 30 May 2014 15:07:28 +0200
> From: Jan Kara <jack@suse.cz>
> To: Lukáš Czerner <lczerner@redhat.com>
> Cc: Greg KH <greg@kroah.com>,
>     James Bottomley <James.Bottomley@hansenpartnership.com>,
>     "ksummit-discuss@lists.linuxfoundation.org"
>     <ksummit-discuss@lists.linuxfoundation.org>
> Subject: Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging
>      more reviewers)
> 
> On Fri 30-05-14 12:08:12, Lukáš Czerner wrote:
> > On Thu, 29 May 2014, Greg KH wrote:
> > 
> > > Date: Thu, 29 May 2014 16:30:04 -0700
> > > From: Greg KH <greg@kroah.com>
> > > To: Olof Johansson <olof@lixom.net>
> > > Cc: James Bottomley <James.Bottomley@hansenpartnership.com>,
> > >     "ksummit-discuss@lists.linuxfoundation.org"
> > >     <ksummit-discuss@lists.linuxfoundation.org>
> > > Subject: Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging
> > >      more reviewers)
> > > 
> > > On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > > > What are we really trying to fix here? Is the current process really
> > > > broken or are we trying to create more process that's not needed for
> > > > some other reason?
> > > 
> > > I think the latter.
> > > 
> > > Somehow, we seem to be constantly increasing our rate of change, are
> > > people thinking we are having problems here?  If so, exactly where?
> > > This thread has taken an odd turn into trying to make some new kind of
> > > process for an unknown issue (i.e. the people on this list are not going
> > > to recognize the reviewers more, it's up to you to educate your managers
> > > / company more.)
> > 
> > This is not only about managers / company. Reviewers does not seem
> > to have much recognition in upstream community either. For example
> > we do take into account s-o-b when creating preliminary list of
> > people to get invited to kernel summit, but we do _not_ take into
> > account reviewd-by (or has anything changed?)...
>   Reviewed-by *is* taken into account for KS selection. It is even
> positively biased against s-o-b AFAIK.

Good to know :) Thanks!

-Lukas

> 
> 							Honza
> 

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[John W. Linville]

On Fri, May 30, 2014 at 09:33:18AM +0400, James Bottomley wrote:

> We need to design programs for new people that draw them in, not keep
> them on the periphery.  That's why I think bug fixing is a better
> activity: it means they have to understand some of the code; the change
> can be small and isolated, so they don't have to understand all the code
> and it often encourages people to find out more about whatever they're
> fixing, hence draws them in.
> 
> We're way off the topic of encouraging reviewers, but the bottom line
> for me is I don't believe starting people out with cleanup type changes
> is a good way to get wider engagement.  It is a good way to increase the
> number of overall patches, all of which need to be reviewed, hence the
> need for more reviewers.

You may have a good point -- people that have spent time finding
and fixing bugs may actually have an incentive to keep new bugs out.
But someone looking to boost their patch count may be perfectly happy
to observe _some_ (hopefully only minor) things slip through only
for the observer to later post a patch to correct them.

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[John W. Linville]

On Thu, May 29, 2014 at 04:30:04PM -0700, Greg KH wrote:
> On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > What are we really trying to fix here? Is the current process really
> > broken or are we trying to create more process that's not needed for
> > some other reason?
> 
> I think the latter.
> 
> Somehow, we seem to be constantly increasing our rate of change, are
> people thinking we are having problems here?  If so, exactly where?
> This thread has taken an odd turn into trying to make some new kind of
> process for an unknown issue (i.e. the people on this list are not going
> to recognize the reviewers more, it's up to you to educate your managers
> / company more.)
> 
> Are maintainers who deal with huge number of patches (i.e. 1000+ a year)
> crying out for help that this is somehow going to reduce their burden?
> 
> I don't think so, but hey, what do I know about this development
> process...

I think I'm in that group.  FWIW, all this talk of new and vaguely
differentiated "Foo-by" tags gives me heartburn.  I try to be
dilligent about collecting any such tags that get posted before a
patch is merged, but it can be a bit painful.

Those tags still come down to trust anyway -- just
because someone replies with a Reviewed-by, Tested-by,
Built-and-run-with-ten-other-pathes-by or whatnot doesn't provide
any guarantee at all unless you know the person and trust what they
are saying in the first place.

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Wed, 28 May 2014 15:15:14 -0400
"John W. Linville" <linville@tuxdriver.com> wrote:

> On Wed, May 28, 2014 at 03:11:55PM -0400, Mimi Zohar wrote:
> > On Wed, 2014-05-28 at 18:48 +0000, Paul Walmsley wrote: 
> > > Also long-overdue is a clarification on exactly what "Acked-by" means.
> > > Right now it is being used for at least two distinct and
> > > mutually-incompatible purposes:
> > > 
> > > 1. A maintainer A for code affected by a patch, who is distinct from a
> > > maintainer B queuing a patch, has reviewed the patch and has cleared it as
> > > being OK for maintainer B to send upstream
> > > 
> > > 2. A casual review has been done by someone who is not a maintainer for
> > > the code in question
> > > 
> > > What I would propose is to have the first use replaced by a new tag, 
> > > "Maintainer-acked-by:", and the second use abolished, along with 
> > > "Acked-by:", and replaced by "Reviewed-by:".
> > 
> > Agreed, "Acked-by" is ambiguous and should be dis-ambiguated.
> > "Reviewed-by:" is too much of a barrier for people to feel comfortable
> > using.  Just as the "Maintainer-acked-by:" would imply a subset of the
> > patch related to the subsystem, "Reviewed-by" needs something similar to
> > limit its scope.
> 
> I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
> 

Yeah, I wouldn't want to type that. What about:

Approved-by: ...

That is reserved for maintainers only?

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Thu, 29 May 2014 23:03:28 +0200
"Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:


> Well, perhaps we can reserve the Acked-by for maintainers and add
> something like Supported-by for the 2nd meaning.

Or add a new tag for maintainers. I like:

 Approved-by:

As that is what the meaning of the maintainer acked-by is.  I sometimes
use Acked-by when I'm not the maintainer of the code but the code
changes something related to tracing, or something else I dabble in
(interrupts, scheduler, etc). Code that I may not maintain, but
something I do work with. Acked-by to me isn't limited to a maintainer,
but for those that deal with the code, and agree with the change.
Supported-by gives the meaning that I'm funding that change.

Where as, a maintainer could pass in an "Approved-by" which means "Yes,
I approve this change and it may go through another tree instead of
mine".

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[John W. Linville]

On Fri, May 30, 2014 at 10:59:38AM -0400, Steven Rostedt wrote:
> On Wed, 28 May 2014 15:15:14 -0400
> "John W. Linville" <linville@tuxdriver.com> wrote:

> > I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
> > 
> 
> Yeah, I wouldn't want to type that. What about:
> 
> Approved-by: ...
> 
> That is reserved for maintainers only?

If we need such a tag, I like this verion better.

Thanks,

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Thu, 29 May 2014 14:03:16 -0700
Olof Johansson <olof@lixom.net> wrote:

> What are we really trying to fix here? Is the current process really
> broken or are we trying to create more process that's not needed for
> some other reason?

The only reason I'm actually for adding another tag is that I get
confused in what to tag something with. I get Cc'd on most things
related to tracing or x86 or scheduling. Yes, I'm the maintainer of
tracing, but the code I'm Cc'd on is about tracing a particular
subsystem that I do not have the expertise to really review the patch
that is changing. I'm assuming they just want me to ack that the
tracing parts are correct. I don't add a reviewed-by because I'm not
saying that I review the guts of the change, just the tracing parts.

I send an Acked-by, which you could argue that as maintainer of tracing
it fits. But really, the code is completely self contained in some
particular subsystem that I have no connection with, besides that they
are using my tracing infrastructure.

The maintainers may want my Acked-by to give them a warm fuzzy feeling
that they did the tracing correct, but I don't want others to think
that I'm in any way a maintainer of their system. This is why I would
like to see a maintainer only tag. For others that are reviewing the
change logs, they can see, "oh this person must be important to require
the approved-by tag on this commit".

Then I get Cc'd on x86 and scheduling because I do work there, but I'm
far from a maintainer of that code. If I get a chance, I will do a
quick review and send a Acked-by, especially if it touches the code
I've changed. But I may not have time to review it enough to designate
a reviewed-by tag.

> 
> As a maintainer for arm-soc, I know which subsystem maintainers I
> should get an Acked-by before I'm ok merging in a branch with, say,
> some driver changes from a platform maintainer. I mostly know because
> we keep intersecting with the same subsystems, but for new ones
> there's one natural place I go to look: MAINTAINERS. Figuring out
> merge paths and when something should go through our tree instead of
> the maintainers tree is always going to be something where people
> actually need to talk to each other and make a decision, I don't think
> we can make tools and process for it.
> 
> Renaming the tag that they use isn't going to change the due diligence
> I have to make (I still need to make sure they're actually the right
> person to give it out, etc). And I'm definitely not worried by the
> possible conflict of the "I gave this a casual review and I think we
> should let it go in" acks since a maintainer is unlikely to give out
> those kind of acks to code that he would otherwise merge himself.

You also are forgetting that you push to Linus. Now this may be a
question to Linus if he would like a separate tag. If he notices that
you included a change from another subsystem, he will need to check if
you got all the necessary acked-bys from the maintainers. If there's a
bunch of Acked-bys he needs to make sure the maintainer is one of them.
Having a single Approved-by may facilitate this for him.

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Fri, 30 May 2014 15:07:28 +0200
Jan Kara <jack@suse.cz> wrote:

> > This is not only about managers / company. Reviewers does not seem
> > to have much recognition in upstream community either. For example
> > we do take into account s-o-b when creating preliminary list of
> > people to get invited to kernel summit, but we do _not_ take into
> > account reviewd-by (or has anything changed?)...
>   Reviewed-by *is* taken into account for KS selection. It is even
> positively biased against s-o-b AFAIK.

Yes, it is very much taken into account. But so is all other
activities. How much you participate is a key factor in KS selection.
If you submit a 1000 patches in your little subsystem but don't make
any attempt to review other patches or get involved with the rest of
the kernel, you are very unlikely to be invited.

-- Steve

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[mark gross]

On Mon, May 26, 2014 at 08:41:37AM -0700, Guenter Roeck wrote:
> On 05/24/2014 09:57 PM, James Bottomley wrote:
> >On Sat, 2014-05-24 at 12:18 -0700, Guenter Roeck wrote:
> >>>
> >>>The thing I'd like to see way more in the Linux ecosystem:
> >>>
> >>>Paid reviewers/maintainers (selected people, no hiring offers). The
> >>>number of developers increases faster than the number of quality
> >>>keepers. So, the latter should be given the chance to focus on it, if
> >>>they want to.
> >>>
> >>
> >>Problem with that is that in most company hierarchies code reviewers
> >>get little  if no credit for their work.
> >
> >I could see this in start up type companies.  Older companies learned
> >long ago that customers value quality over features so they tend to have
> >elaborate review processes. (As an aside, customers say they value
> >features, but if you deliver one with a regression, it's the regression
> >you'll hear about the whole time).
> >
> 
> I am not sure if all those companies learned the lesson. Agreed, many
> of them do, but I have seen the opposite. But that is not really
> the point here.
> 
> You can actually take the Linux kernel at a case in point: Let's assume
> someone wants to hire a kernel engineer and looks up kernel commits for
> reference. What do you think that person will look for ? Patch authors
> or  "Reviewed-by" tags ? I would argue it is going to be patch authors.
> 
> Really, again, the point (or question) here is how much credit an engineer
> gets for doing code reviews (or fixing bugs, for that matter) vs. for
> writing code. I would argue that there is very little incentive for
> senior engineers (ie those who are best suited to do code reviews)
> to actually _do_ code reviews more or less for a living, or at least
> for a substantial amount of their time.
>
I got a significant promotion at intel this year largely because I do a lot of
internal code reviews.  I think you are looking at things in a short sited
manner.

Doing a lot of reviews and getting pretty good at it will make one a leader in
setting the priorities and quality metrics for the code written by everyone else.
This is a significant power over any organization.  It sets you apart as a leader
and mentor, how could you not be recognized for it.  If you keep it up long
enough to pay off.  Which is a challenge.

A solid code reviewer has project global sway over how things get designed and
implemented and over time grows trust and confidence between the reviewer and the
organization.  From a world dominion point of view this is vector I would
recommend to anyone wanting to be important.

Further I find arguments like the above to be pathetic and distasteful.  Profit
motives are getting out of control IMO.  (yeah, I know its too easy for me to say
while getting well paid)  But, still if that is your motive you are still doing
it wrong if you ignore reviewer value.  Its easier to get money through having
authority than any other way.

--mark

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Theodore Ts'o]

On Fri, May 30, 2014 at 09:33:18AM +0400, James Bottomley wrote:
> We need to design programs for new people that draw them in, not keep
> them on the periphery.  That's why I think bug fixing is a better
> activity: it means they have to understand some of the code; the change
> can be small and isolated, so they don't have to understand all the code
> and it often encourages people to find out more about whatever they're
> fixing, hence draws them in.

When I need to bring a Noogler (or intern) up to speed, and they are
new to kernel coding or new to the storage subsystem, the two
activities that seem to work the best is (a) bug fixing, and (b)
participating in a rebase to forward port patches to a newer upstream
kernel.

Obviously, the latter isn't applicable for upstream development (but
probably has at least some relevance for companies maintaining
community or enterprise distros).  Also note that one key enabling
technology to let relatively experienced developers do rebases is that
every single commit that they need to rebase is annotated with either
the name of the automated test or test suite, or manual testing
instructions.

We don't even *bother* having Nooglers or interns work on whitespace
or spelling fix ups, even as a "my first patch" so they can learn how
to use Gerrit and other internal tools.  They can learn all of that
while doing work that actually adds real value, such as working on a
backlogged bug report or by forward porting patches.

Cheers,

					- Ted

> We're way off the topic of encouraging reviewers, but the bottom line
> for me is I don't believe starting people out with cleanup type changes
> is a good way to get wider engagement.  It is a good way to increase the
> number of overall patches, all of which need to be reviewed, hence the
> need for more reviewers.

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Theodore Ts'o]

On Fri, May 30, 2014 at 12:40:05PM -0400, Theodore Ts'o wrote:
> 
> Obviously, the latter isn't applicable for upstream development (but
> probably has at least some relevance for companies maintaining
> community or enterprise distros).  Also note that one key enabling
> technology to let relatively experienced developers do rebases is that
> every single commit that they need to rebase is annotated with either
> the name of the automated test or test suite, or manual testing
> instructions.

Sigh,  s/experienced/inexperienced/

					- Ted

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Guenter Roeck]

On 05/30/2014 09:05 AM, mark gross wrote:
> On Mon, May 26, 2014 at 08:41:37AM -0700, Guenter Roeck wrote:
>> On 05/24/2014 09:57 PM, James Bottomley wrote:
>>> On Sat, 2014-05-24 at 12:18 -0700, Guenter Roeck wrote:
>>>>>
>>>>> The thing I'd like to see way more in the Linux ecosystem:
>>>>>
>>>>> Paid reviewers/maintainers (selected people, no hiring offers). The
>>>>> number of developers increases faster than the number of quality
>>>>> keepers. So, the latter should be given the chance to focus on it, if
>>>>> they want to.
>>>>>
>>>>
>>>> Problem with that is that in most company hierarchies code reviewers
>>>> get little  if no credit for their work.
>>>
>>> I could see this in start up type companies.  Older companies learned
>>> long ago that customers value quality over features so they tend to have
>>> elaborate review processes. (As an aside, customers say they value
>>> features, but if you deliver one with a regression, it's the regression
>>> you'll hear about the whole time).
>>>
>>
>> I am not sure if all those companies learned the lesson. Agreed, many
>> of them do, but I have seen the opposite. But that is not really
>> the point here.
>>
>> You can actually take the Linux kernel at a case in point: Let's assume
>> someone wants to hire a kernel engineer and looks up kernel commits for
>> reference. What do you think that person will look for ? Patch authors
>> or  "Reviewed-by" tags ? I would argue it is going to be patch authors.
>>
>> Really, again, the point (or question) here is how much credit an engineer
>> gets for doing code reviews (or fixing bugs, for that matter) vs. for
>> writing code. I would argue that there is very little incentive for
>> senior engineers (ie those who are best suited to do code reviews)
>> to actually _do_ code reviews more or less for a living, or at least
>> for a substantial amount of their time.
>>
> I got a significant promotion at intel this year largely because I do a lot of
> internal code reviews.  I think you are looking at things in a short sited
> manner.
>
Good for you and for Intel. However, I think you may have misunderstood my comments.

> Doing a lot of reviews and getting pretty good at it will make one a leader in
> setting the priorities and quality metrics for the code written by everyone else.
> This is a significant power over any organization.  It sets you apart as a leader
> and mentor, how could you not be recognized for it.  If you keep it up long
> enough to pay off.  Which is a challenge.
>
> A solid code reviewer has project global sway over how things get designed and
> implemented and over time grows trust and confidence between the reviewer and the
> organization.  From a world dominion point of view this is vector I would
> recommend to anyone wanting to be important.
>
Excellent for Intel if that is the mentality and culture there. Unfortunately,
that culture does not necessarily apply to other companies.

> Further I find arguments like the above to be pathetic and distasteful.  Profit
> motives are getting out of control IMO.  (yeah, I know its too easy for me to say
> while getting well paid)  But, still if that is your motive you are still doing
> it wrong if you ignore reviewer value.  Its easier to get money through having
> authority than any other way.
>

Shoot the messenger ?

I did not say or suggest that I agree that not rewarding code reviewers would
be a good idea. In fact, I strongly disagree. What I said was an observation;
if anything, the argument behind it was that there is a management problem
at many companies with recognizing code reviewers, which would need to change
to encourage more and better code reviews. It may be considered pathetic
and/or distasteful that such a management problem exists (though I would not
use those words - they are a bit close to getting personal in my opinion),
but I fail to see how _reporting_ that such a problem exists would in any
way or form be pathetic or distasteful.

Guenter

[Ksummit-discuss] More productive uses of enthusiastic new kernel developers (was: Re: [TOPIC] Encouraging more reviewers)

[Theodore Ts'o]

Thinking about this some more, what if instead of, or in addition to,
having newcomers work on cleanup patches, what if we encouraged some
of them to help to manually backport patches to the stable kernels
that don't apply automatically?

Currently the stable maintainers will typically send a notification
subsystem maintainer / mailing lists that a particular patch didn't
apply and that it's up some subsystem developers to handle resolving
the patch conflict.

What if those patches were also cc'ed to a separate mailing list which
was also monitored by patchwork, and newcomers were encouraged to grab
patches and try to make a determination whether it's really needed,
and how to handle doing the backport, and once the patch was properly
backported they could cc the subsystem mailing list asking for
approval before the patch could get fed into a long-term stable tree?

It does mean more patches to reviews, but I'd much rather review a
patch going into a long-term stable tree that would otherwise get
dropped (and thus avoid whiny entitled users and/or embedded engineers
who think it's the maintainer's job to backport to N different stable
kernels), rather than review whitespace patches.

If this worked out, I could also imagine encouraging more advanced
newcomers to look for bug fix patches that didn't have an explicit cc:
stable@vger.kernel.org, and look to see if they should be backported.
This would help for certain subsystems that have elected not to
automatically mark their patches.  There are subsystems which don't
like anyone but their experienced developers to handle doing
backports, but I suspect there would also be many subsystems that
would welcome the extra help, especially if it was much more likely to
result in additional reviewers / subsystem developers.

Cheers,

						- Ted

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Paul E. McKenney]

On Thu, May 29, 2014 at 10:41:22AM -0700, Greg KH wrote:
> On Thu, May 29, 2014 at 05:28:11PM +0800, Li Zefan wrote:
> > On 2014/5/29 5:32, Thomas Gleixner wrote:
> > > On Wed, 28 May 2014, Mark Brown wrote:
> > > 
> > >> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
> > >>
> > >>>> There's a world of difference between thanking people for review and a
> > >>>> detailed account of all the changes made in every single iteration of
> > >>>> the review.
> > >>
> > >>> This is already covered in Documentation/SubmittingPatches. Quoting
> > >>> lines 585-592 (see last sentence):
> > >>
> > >> Right, but Daniel is proposing lifting that above the --- and including
> > >> it in git.
> > > 
> > > What you really want is:
> > > 
> > > Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
> > > 
> > > It's way more useful than any of the v1-n writeups, which are most of
> > > the time just a completely waste of electrons. Even if written well,
> > > without the actual review context they are pretty pointless.
> > > 
> > 
> > A QA asked me about kernel development process. One of his question is,
> > he found some valuable information in the discussion of the patch often
> > won't be added to the changelog, so providing the commit how to find
> > the discussion?
> 
> A research group has created a tool that takes a given git commit, finds
> the mailing list discussion for that patch.  It was posted to lkml 6 or
> so months ago, you should point them at that tool if they want to do
> this.

This one?  https://lkml.org/lkml/2014/3/11/2

							Thanx, Paul

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers (was: Re: [TOPIC] Encouraging more reviewers)

[Shuah Khan]

On Fri, May 30, 2014 at 10:56 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> Thinking about this some more, what if instead of, or in addition to,
> having newcomers work on cleanup patches, what if we encouraged some
> of them to help to manually backport patches to the stable kernels
> that don't apply automatically?
>
> Currently the stable maintainers will typically send a notification
> subsystem maintainer / mailing lists that a particular patch didn't
> apply and that it's up some subsystem developers to handle resolving
> the patch conflict.
>
> What if those patches were also cc'ed to a separate mailing list which
> was also monitored by patchwork, and newcomers were encouraged to grab
> patches and try to make a determination whether it's really needed,
> and how to handle doing the backport, and once the patch was properly
> backported they could cc the subsystem mailing list asking for
> approval before the patch could get fed into a long-term stable tree?
>
> It does mean more patches to reviews, but I'd much rather review a
> patch going into a long-term stable tree that would otherwise get
> dropped (and thus avoid whiny entitled users and/or embedded engineers
> who think it's the maintainer's job to backport to N different stable
> kernels), rather than review whitespace patches.
>
> If this worked out, I could also imagine encouraging more advanced
> newcomers to look for bug fix patches that didn't have an explicit cc:
> stable@vger.kernel.org, and look to see if they should be backported.
> This would help for certain subsystems that have elected not to
> automatically mark their patches.  There are subsystems which don't
> like anyone but their experienced developers to handle doing
> backports, but I suspect there would also be many subsystems that
> would welcome the extra help, especially if it was much more likely to
> result in additional reviewers / subsystem developers.
>

In addition to the above, how about keeping a maintainer and sub-maintainer
wish list of tasks and work they would like to see get done in their areas and
don't have time do it themselves. It could be maintained on kernelnewbies site.
Having something like this will help guide and give direction to people looking
to get started in a new area and/or any area of the kernel.

-- Shuah

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers (was: Re: [TOPIC] Encouraging more reviewers)

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On Fri, 2014-05-30 at 12:56 -0400, Theodore Ts'o wrote:
> Thinking about this some more, what if instead of, or in addition to,
> having newcomers work on cleanup patches, what if we encouraged some
> of them to help to manually backport patches to the stable kernels
> that don't apply automatically?

Hm, my concern with that is that reviewing a patch that's been ported by
a newbie is probably *more* work than just doing the backport oneself.

When a backport doesn't trivially apply, there's usually an "obvious"
answer you start from, either dropping a hunk that doesn't apply because
there's nowhere for it to go, or inserting the same lines even though
the context is different, etc.

And then you apply your knowledge of the code, and perhaps refer to the
git history of the changes between then and now, and you then make sure
it's *correct*.

A newcomer is going to be able to do the simple part of that, but rarely
will they have the experience to do the interesting part. And if they
try, that just ends up with you trying to do a three-way merge of old,
new and their version when you review it. Surely if you have the time
for that, you had the time to do the backport in the first place?

It's a nice idea, but in practice I suspect that backports like this
aren't the best use of newcomers' time.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[James Bottomley]

On Fri, 2014-05-30 at 11:10 -0400, John W. Linville wrote:
> On Fri, May 30, 2014 at 10:59:38AM -0400, Steven Rostedt wrote:
> > On Wed, 28 May 2014 15:15:14 -0400
> > "John W. Linville" <linville@tuxdriver.com> wrote:
> 
> > > I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
> > > 
> > 
> > Yeah, I wouldn't want to type that. What about:
> > 
> > Approved-by: ...
> > 
> > That is reserved for maintainers only?
> 
> If we need such a tag, I like this verion better.

Bikeshedded-by:?

Discussed-in-circles-by:?

Maintainer-penguin-pee-blessed-by:?

OK, it's late here; but for the record, I think Acked-by is perfectly
clear and we don't need a new tag.

James

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Rafael J. Wysocki]

On Friday, May 30, 2014 11:06:40 AM Steven Rostedt wrote:
> On Thu, 29 May 2014 23:03:28 +0200
> "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> 
> 
> > Well, perhaps we can reserve the Acked-by for maintainers and add
> > something like Supported-by for the 2nd meaning.
> 
> Or add a new tag for maintainers. I like:
> 
>  Approved-by:
> 
> As that is what the meaning of the maintainer acked-by is.  I sometimes
> use Acked-by when I'm not the maintainer of the code but the code
> changes something related to tracing, or something else I dabble in
> (interrupts, scheduler, etc). Code that I may not maintain, but
> something I do work with. Acked-by to me isn't limited to a maintainer,
> but for those that deal with the code, and agree with the change.
> Supported-by gives the meaning that I'm funding that change.

Right.

> Where as, a maintainer could pass in an "Approved-by" which means "Yes,
> I approve this change and it may go through another tree instead of
> mine".

That's what an ACK means, though, isn't it?

To me, really, Acked-by from a non-maintainer is something like a "+1" in G+.

I would prefer the new tag to suggest that meaning more directly.  Something
like "Fine-by", maybe?

Rafael

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Fri, 30 May 2014 23:26:38 +0200
"Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:

 
> > Where as, a maintainer could pass in an "Approved-by" which means "Yes,
> > I approve this change and it may go through another tree instead of
> > mine".
> 
> That's what an ACK means, though, isn't it?

I would argue that it means that as well as other meanings.

> 
> To me, really, Acked-by from a non-maintainer is something like a "+1" in G+.
> 
> I would prefer the new tag to suggest that meaning more directly.  Something
> like "Fine-by", maybe?

I actually disagree with this analysis. I like to see Acked-by from
those that may not be the maintainer, but have some stake in the change
that is being made. A Acked-by from a random person that I don't know
is more of a "+1" check, which I agree is useless and I don't even
bother adding them.

Thus, my point is to distinguish between a "Acked-by" that comes from
someone that has a stake in the code and approves the change (which to
me is the real meaning of Acked-by) and a stronger "Approved-by" which
comes from a maintainer that is stating that they are OK with you
pushing it through your tree even though it touches their code.

I'm not saying people have to use "Approved-by", but I would like to be
able to us it myself when people make a change in my code and I would
like to let the maintainer of the tree it is going in to know they have
my OK to pull it.

I ask for Acked-bys from people if they can't give me a Reviewed-by
but they still have a stake in the code that is being modified, and I
want to know it's still OK with them. But they are not a maintainer
telling me it's OK to take their code into my tree.

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Steven Rostedt]

On Fri, 30 May 2014 17:10:53 -0400
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:


> 
> Bikeshedded-by:?
> 
> Discussed-in-circles-by:?
> 
> Maintainer-penguin-pee-blessed-by:?
> 
> OK, it's late here; but for the record, I think Acked-by is perfectly
> clear and we don't need a new tag.

Pissed-off-James-Bottomely-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Rafael J. Wysocki]

On Friday, May 30, 2014 05:29:21 PM Steven Rostedt wrote:
> On Fri, 30 May 2014 23:26:38 +0200
> "Rafael J. Wysocki" <rjw@rjwysocki.net> wrote:
> 
>  
> > > Where as, a maintainer could pass in an "Approved-by" which means "Yes,
> > > I approve this change and it may go through another tree instead of
> > > mine".
> > 
> > That's what an ACK means, though, isn't it?
> 
> I would argue that it means that as well as other meanings.
> 
> > 
> > To me, really, Acked-by from a non-maintainer is something like a "+1" in G+.
> > 
> > I would prefer the new tag to suggest that meaning more directly.  Something
> > like "Fine-by", maybe?
> 
> I actually disagree with this analysis. I like to see Acked-by from
> those that may not be the maintainer, but have some stake in the change
> that is being made. A Acked-by from a random person that I don't know
> is more of a "+1" check, which I agree is useless and I don't even
> bother adding them.
> 
> Thus, my point is to distinguish between a "Acked-by" that comes from
> someone that has a stake in the code and approves the change (which to
> me is the real meaning of Acked-by) and a stronger "Approved-by" which
> comes from a maintainer that is stating that they are OK with you
> pushing it through your tree even though it touches their code.
> 
> I'm not saying people have to use "Approved-by", but I would like to be
> able to us it myself when people make a change in my code and I would
> like to let the maintainer of the tree it is going in to know they have
> my OK to pull it.
> 
> I ask for Acked-bys from people if they can't give me a Reviewed-by
> but they still have a stake in the code that is being modified, and I
> want to know it's still OK with them. But they are not a maintainer
> telling me it's OK to take their code into my tree.

That makes sense.

Rafael

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Greg KH]

On Fri, May 30, 2014 at 09:39:06AM +0400, James Bottomley wrote:
> On Thu, 2014-05-29 at 22:04 -0700, Greg KH wrote:
> > On Fri, May 30, 2014 at 01:12:06AM +0000, Paul Walmsley wrote:
> > > On Thu, 29 May 2014, Greg KH wrote:
> > > 
> > > > On Thu, May 29, 2014 at 02:03:16PM -0700, Olof Johansson wrote:
> > > > > What are we really trying to fix here? Is the current process really
> > > > > broken or are we trying to create more process that's not needed for
> > > > > some other reason?
> > > > 
> > > > I think the latter.
> > > > 
> > > > Somehow, we seem to be constantly increasing our rate of change, are
> > > > people thinking we are having problems here?  If so, exactly where?
> > > 
> > > If "increasing our rate of change" was the only metric that we cared 
> > > about, we wouldn't be discussing how to attract more reviewers.
> > 
> > I'm not saying it's the only metric, but just pointing out that while we
> > constantly ask for more reviewers, it doesn't seem to be slowing us
> > down.
> 
> It depends whether you believe every patch has to be reviewed.

Well, aside from people I "trust", yes, the patch better be reviewed,
that's what a subsystem maintainer's job is.

> If you do (and I certainly believe we have to in order to maintain
> code quality), we have to increase our review bandwidth at the same
> rate we increase our commits, otherwise the imbalance causes a backlog
> of unreviewed patches.
> 
> What I see at the moment is that the number of patches exceeds the
> number of reviewers quite substantially.  By and large the reviewers
> mostly get through the backlog between merge windows (but the process is
> an exhausting one which will lead to reviewer burn out), but I can see a
> time arriving soon when they can't and we start wrapping ourselves
> around the axle because of too few reviewers.

As was pointed out, push back, and ask for help.  We have people asking
for what they can do to help out, give them something to do!

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Fri, May 30, 2014 at 10:28:58AM -0700, Paul E. McKenney wrote:
> On Thu, May 29, 2014 at 10:41:22AM -0700, Greg KH wrote:
> > On Thu, May 29, 2014 at 05:28:11PM +0800, Li Zefan wrote:
> > > On 2014/5/29 5:32, Thomas Gleixner wrote:
> > > > On Wed, 28 May 2014, Mark Brown wrote:
> > > > 
> > > >> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
> > > >>
> > > >>>> There's a world of difference between thanking people for review and a
> > > >>>> detailed account of all the changes made in every single iteration of
> > > >>>> the review.
> > > >>
> > > >>> This is already covered in Documentation/SubmittingPatches. Quoting
> > > >>> lines 585-592 (see last sentence):
> > > >>
> > > >> Right, but Daniel is proposing lifting that above the --- and including
> > > >> it in git.
> > > > 
> > > > What you really want is:
> > > > 
> > > > Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
> > > > 
> > > > It's way more useful than any of the v1-n writeups, which are most of
> > > > the time just a completely waste of electrons. Even if written well,
> > > > without the actual review context they are pretty pointless.
> > > > 
> > > 
> > > A QA asked me about kernel development process. One of his question is,
> > > he found some valuable information in the discussion of the patch often
> > > won't be added to the changelog, so providing the commit how to find
> > > the discussion?
> > 
> > A research group has created a tool that takes a given git commit, finds
> > the mailing list discussion for that patch.  It was posted to lkml 6 or
> > so months ago, you should point them at that tool if they want to do
> > this.
> 
> This one?  https://lkml.org/lkml/2014/3/11/2

Yes, that's it.

greg k-h

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Greg KH]

On Fri, May 30, 2014 at 09:33:18AM +0400, James Bottomley wrote:
> On Thu, 2014-05-29 at 22:02 -0700, Greg KH wrote:
> > On Fri, May 30, 2014 at 08:26:13AM +0400, James Bottomley wrote:
> > > On Thu, 2014-05-29 at 16:34 -0700, Greg KH wrote:
> > > > > Just preparing it for gerrit takes a long time (around ten minutes per
> > > > > patch), which really reduces the volume of trivial patches, just because
> > > > > no-one has that much energy (although some would argue it reduces the
> > > > > volume too far).
> > > > > 
> > > > > It's human nature to spend more time on the easy problems, so a flood of
> > > > > trivial patches which are "obviously" correct is easier to review and
> > > > > include than one patch which alters something deep within mm and needs
> > > > > careful thought.  At best, excessively split trivial patches are a
> > > > > dilution of our review effort and at worst, they're actively sucking
> > > > > people away from tackling the hard problems.
> > > > 
> > > > I have not heard of these "cleanup" patches taking anyone's review
> > > > cycles up for the "harder" patches, do you have examples of it?
> > > 
> > > Yes, me.
> > 
> > Then flat out reject them.  Some subsystem maintainers do.  Or, push
> > back, like you said later with a "I'll take this if you will maintain
> > the driver from now on." :)
> 
> I do (I believe I already said that).
> 
> > No one is forcing you to take these drivers, but as someone who has
> > really old code, I can see how it would get annoying with people hitting
> > you with cleanup patches.  Maybe you should just do a "coding style
> > cleanup" set of patches yourself one release to get it all in shape then
> > not have to worry about it anymore.  That's what I did years ago for the
> > USB drivers...
> 
> I'm not necessarily promoting my own position, I'm questioning the wider
> assertion that all these clean up patches are a good way of getting into
> kernel coding.  What worries me the most is that I don't see evidence
> that after hundreds of clean up patches anyone moves on to more
> substantive issues.

Well, my sample size might be biased, but I know of at least 5 people
that started out with staging "clean up" patches that went on to get
"real" jobs being kernel developers at different companies.  Some of
them disappeared into those companies, and left me with less overall
help, but that's fine with me.  We helped someone out in a concrete way,
which I view is a success, and worth me doing this type of work.

I'm sure there are thousands of other examples where the person didn't
go on to do "real" work on the kernel, but how do you know who is going
to be that 1% that does help out?  I'm not willing to take that chance.

And again, push back and refuse these types of patches for your
subsystem if they are bothering and annoying you.  Have them go through
the trivial tree instead, that's what it is there for.  No one is
forcing you to deal with them.

> Once we incent people with kudos for patches, why
> would you move on to more substantive changes? they're harder and take
> more time.  You can turn out many more cleanup type patches in the time
> it takes to argue for and produce a well thought out substantive change.
> On these metrics we're actually encouraging people not to graduate ...
> that's really a bad thing.

I have not found that to be the case, ever, that someone continues to do
cleanup patches because they like the "numbers".  Employers, and others,
aren't dumb enough to just look at raw numbers.  If they are, well, they
deserve what they get...

> We need to design programs for new people that draw them in, not keep
> them on the periphery.

I totally agree, which is why I have done some work in this area, and it
is slowly paying off, but it's not something that I can talk about at
this point in time.

If others have ideas of things to do to get people into kernel
development in ways that are more "sticky", I'm really interested in
hearing them.

> That's why I think bug fixing is a better activity: it means they have
> to understand some of the code; the change can be small and isolated,
> so they don't have to understand all the code and it often encourages
> people to find out more about whatever they're fixing, hence draws
> them in.

I used to assign bug reports to people as part of "here's something to
learn" when getting new developers at different companies.  It almost
always failed as a person has to be "invested" in wanting to fix the bug
(i.e. it's bothering them, their hardware, etc.)  Even for people who
were made it part of their job, a horrible job was done and they hated
it.

Now if you have someone report a bug, asking them to help fix it is a
much better thing to do and we have had success with that.  But again,
just telling people to go off and scrape through bugzilla is mean, and
in my experience, not worth it at all.

> We're way off the topic of encouraging reviewers, but the bottom line
> for me is I don't believe starting people out with cleanup type changes
> is a good way to get wider engagement.  It is a good way to increase the
> number of overall patches, all of which need to be reviewed, hence the
> need for more reviewers.

It also gets code in staging cleaned up from being so crummy, which I
like, so I'll keep encouraging it in my part of the kernel.  Again, feel
free to reject it from yours.

greg k-h

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Li Zefan]

On 2014/5/30 23:22, Steven Rostedt wrote:
> On Fri, 30 May 2014 15:07:28 +0200
> Jan Kara <jack@suse.cz> wrote:
> 
>>> This is not only about managers / company. Reviewers does not seem
>>> to have much recognition in upstream community either. For example
>>> we do take into account s-o-b when creating preliminary list of
>>> people to get invited to kernel summit, but we do _not_ take into
>>> account reviewd-by (or has anything changed?)...
>>   Reviewed-by *is* taken into account for KS selection. It is even
>> positively biased against s-o-b AFAIK.
> 
> Yes, it is very much taken into account. But so is all other
> activities. How much you participate is a key factor in KS selection.
> If you submit a 1000 patches in your little subsystem but don't make
> any attempt to review other patches or get involved with the rest of
> the kernel, you are very unlikely to be invited.

Do we account the activity on stable kernels? I guess the answer is no,
because you can't tell if a stable commit was backported automatically
or it was backported due to someone's request/help, and the latter
should be credited.

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Li Zefan]

On 2014/5/31 0:56, Theodore Ts'o wrote:
> Thinking about this some more, what if instead of, or in addition to,
> having newcomers work on cleanup patches, what if we encouraged some
> of them to help to manually backport patches to the stable kernels
> that don't apply automatically?
> 

This is similar to my idea in "stable issues" to find a co-maintainer
for Greg to do this work. While I sugguest to find a experienced and
trustful person, you sugguest newcomers.

I share the same concern with David, and I doubt newcompers are
enthusiastic in doing backport for stable trees.

> Currently the stable maintainers will typically send a notification
> subsystem maintainer / mailing lists that a particular patch didn't
> apply and that it's up some subsystem developers to handle resolving
> the patch conflict.
> 
> What if those patches were also cc'ed to a separate mailing list which
> was also monitored by patchwork, and newcomers were encouraged to grab
> patches and try to make a determination whether it's really needed,
> and how to handle doing the backport, and once the patch was properly
> backported they could cc the subsystem mailing list asking for
> approval before the patch could get fed into a long-term stable tree?
> 
> It does mean more patches to reviews, but I'd much rather review a
> patch going into a long-term stable tree that would otherwise get
> dropped (and thus avoid whiny entitled users and/or embedded engineers
> who think it's the maintainer's job to backport to N different stable
> kernels), rather than review whitespace patches.
> 
> If this worked out, I could also imagine encouraging more advanced
> newcomers to look for bug fix patches that didn't have an explicit cc:
> stable@vger.kernel.org, and look to see if they should be backported.
> This would help for certain subsystems that have elected not to
> automatically mark their patches.  There are subsystems which don't
> like anyone but their experienced developers to handle doing
> backports, but I suspect there would also be many subsystems that
> would welcome the extra help, especially if it was much more likely to
> result in additional reviewers / subsystem developers.
> 

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Guenter Roeck]

On 05/30/2014 06:44 PM, Li Zefan wrote:
> On 2014/5/31 0:56, Theodore Ts'o wrote:
>> Thinking about this some more, what if instead of, or in addition to,
>> having newcomers work on cleanup patches, what if we encouraged some
>> of them to help to manually backport patches to the stable kernels
>> that don't apply automatically?
>>
>
> This is similar to my idea in "stable issues" to find a co-maintainer
> for Greg to do this work. While I sugguest to find a experienced and
> trustful person, you sugguest newcomers.
>
> I share the same concern with David, and I doubt newcompers are
> enthusiastic in doing backport for stable trees.
>

I don't think this is a matter of being enthusiastic or not. Much more
one of expertise. As a user of stable kernels, I would very much prefer
to have it maintained by an expert (or by experts), not by a newcomer.
Of course, that may be just me.

Guenter

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Theodore Ts'o]

On Sat, May 31, 2014 at 09:44:17AM +0800, Li Zefan wrote:
> This is similar to my idea in "stable issues" to find a co-maintainer
> for Greg to do this work. While I sugguest to find a experienced and
> trustful person, you sugguest newcomers.
> 
> I share the same concern with David, and I doubt newcompers are
> enthusiastic in doing backport for stable trees.

All I can say is that I've seen it work.  Granted, it was in a
corporate environment, where if the person gets stuck, they can easily
call for help from a teammate at an adjacent desk.  The commits were
also in generally well annotated with testing instructions, and all of
the patches went through a code review process.  And, while these
people may have been newcomers to kernel programming, they had all
passed the Google hiring interview process.

Admittedly, for someone who just shows up on a mailing list, we don't
know as much about their technical capabilities, and how both
accountability and motivation works is slightly different in a work
environment compared to a volunteer/hobbyist setting.

Cheers,

					- Ted

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Theodore Ts'o]

On Fri, May 30, 2014 at 06:54:07PM -0700, Guenter Roeck wrote:
> 
> I don't think this is a matter of being enthusiastic or not. Much more
> one of expertise. As a user of stable kernels, I would very much prefer
> to have it maintained by an expert (or by experts), not by a newcomer.
> Of course, that may be just me.

The stable kernel tree would still be maintained by experts.  Even if
newcomers supplied some of the backports, they would still get
reviewed by experts.  Granted, for the first few patches it might not
save me that much time compared to my just doing the backport myself,
but it's IMHO much more likely to result in a capable programmer in
the long run (at least compared to people who are just encouraged to
do drive-by whitespace / spelling patch submissions). 

And if we have newcomers looking for bug fix patches that weren't
properly marked with "cc: stable@vger.kernel.org", and calling those
patches to the stable tree and subsystem maintainers, again, I think
it's healthier and certainly more productive than traning them to look
for whitespace and checkpatch.pl warnings.

If the argument is that newcomers aren't motivated to do this sort of
grunt work, recall how much grunt work maintainers have to do.  How
much time do you think most maintainers spend writing new sexy
features?  As opposed to the less fun bits, such as reviewing code and
running integration tests and bisecting through submitted patches to
find the buggy "cleanup" patch someone sent me?  

If I can find people who don't mind sharing in some of the more less
glory-filled aspects of being an open source developer, and who isn't
afraid to do some of the less fun but still vitally important bits,
speaking personally, those are the people that I would be more
motivated to mentor.  In contrast, someone who sends thousands of
whitespace patches is ***not*** someone I'm personally going to be
much inclined to spend time mentoring.  And if the goal is to make
sure we can groom people who can step in when maintainers get hit by a
bus or retire, which sort of people do you want to be groomed to
become the new maintainers?

Cheers,

						- Ted

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Greg KH]

On Fri, May 30, 2014 at 10:07:02PM -0400, Theodore Ts'o wrote:
> On Sat, May 31, 2014 at 09:44:17AM +0800, Li Zefan wrote:
> > This is similar to my idea in "stable issues" to find a co-maintainer
> > for Greg to do this work. While I sugguest to find a experienced and
> > trustful person, you sugguest newcomers.
> > 
> > I share the same concern with David, and I doubt newcompers are
> > enthusiastic in doing backport for stable trees.
> 
> All I can say is that I've seen it work.  Granted, it was in a
> corporate environment, where if the person gets stuck, they can easily
> call for help from a teammate at an adjacent desk.  The commits were
> also in generally well annotated with testing instructions, and all of
> the patches went through a code review process.  And, while these
> people may have been newcomers to kernel programming, they had all
> passed the Google hiring interview process.

I think you just listed all of the reasons why this is a totally
different environment from the "normal" stable kernel patch backport
process.

Not to mention the fact that we don't usually have tests for any
specific patch, to ensure that they fix what they say they do, and that
nothing got messed up in the backport.

thanks,

greg k-h

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Guenter Roeck]

On 05/30/2014 08:52 PM, Greg KH wrote:
> On Fri, May 30, 2014 at 10:07:02PM -0400, Theodore Ts'o wrote:
>> On Sat, May 31, 2014 at 09:44:17AM +0800, Li Zefan wrote:
>>> This is similar to my idea in "stable issues" to find a co-maintainer
>>> for Greg to do this work. While I sugguest to find a experienced and
>>> trustful person, you sugguest newcomers.
>>>
>>> I share the same concern with David, and I doubt newcompers are
>>> enthusiastic in doing backport for stable trees.
>>
>> All I can say is that I've seen it work.  Granted, it was in a
>> corporate environment, where if the person gets stuck, they can easily
>> call for help from a teammate at an adjacent desk.  The commits were
>> also in generally well annotated with testing instructions, and all of
>> the patches went through a code review process.  And, while these
>> people may have been newcomers to kernel programming, they had all
>> passed the Google hiring interview process.
>
> I think you just listed all of the reasons why this is a totally
> different environment from the "normal" stable kernel patch backport
> process.
>
> Not to mention the fact that we don't usually have tests for any
> specific patch, to ensure that they fix what they say they do, and that
> nothing got messed up in the backport.
>

Exactly - I could not have said it better.

Guenter

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Geert Uytterhoeven]

On Wed, May 28, 2014 at 11:59 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> I really started to put breaks into this cycle of hell, where I get
> spammed with a 30+ patch series in the morning and after I spent some
> quality time looking at it and replying to a particular patch, I get
> another spam bomb within a few hours, which is not much better than
> the previous one.

That's indeed an issue. Some large series are immediately reposted for every
single (sometimes trivial) change suggested by a commenter.
This also leads to inflated patch version numbers. I'm already getting nervous
when I have to send out a v3 or v4 myself, but from a quick search, v29
seems to be the current record on lkml...

Does git send-email remember when the previous version of a series was
sent out?
If yes, it could refuse to resend it too soon. The back off period would
depend on the size of the patch series (large patch series need more delay).

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Dan Carpenter]

[-- Attachment #1: Type: text/plain, Size: 102 bytes --]

Oops.  That version had a bug in it so it chopped off too much of the
patch.

regards,
dan carpenter


[-- Attachment #2: rename_rev.pl --]
[-- Type: text/x-perl, Size: 5171 bytes --]

#!/usr/bin/perl

# This is a tool to help review variable rename patches. The goal is
# to strip out the automatic sed renames and the white space changes
# and leaves the interesting code changes.
#
# Example 1: A patch renames openInfo to open_info:
#     cat diff | rename_review.pl openInfo open_info
#
# Example 2: A patch swaps the first two arguments to some_func():
#     cat diff | rename_review.pl \
#                    -e 's/some_func\((.*?),(.*?),/some_func\($2, $1,/'
#
# Example 3: A patch removes the xkcd_ prefix from some but not all the
# variables.  Instead of trying to figure out which variables were renamed
# just remove the prefix from them all:
#     cat diff | rename_review.pl -ea 's/xkcd_//g'
#
# Example 4: A patch renames 20 CamelCase variables.  To review this let's
# just ignore all case changes and all '_' chars.
#     cat diff | rename_review -ea 'tr/[A-Z]/[a-z]/' -ea 's/_//g'
#
# The other arguments are:
# -nc removes comments
# -ns removes '\' chars if they are at the end of the line.

use strict;
use File::Temp qw/ :mktemp  /;

sub usage() {
    print "usage: cat diff | $0 old new old new old new...\n";
    print "   or: cat diff | $0 -e 's/old/new/g'\n";
    print " -e : execute on old lines\n";
    print " -ea: execute on all lines\n";
    print " -nc: no comments\n";
    print " -nb: no unneeded braces\n";
    print " -ns: no slashes at the end of a line\n";
    exit(1);
}
my @subs;
my @cmds;
my $strip_comments;
my $strip_braces;
my $strip_slashes;

sub filter($) {
    my $_ = shift();
    my $old = 0;
    if ($_ =~ /^-/) {
        $old = 1;
    }
    # remove the first char
    s/^[ +-]//;
    if ($strip_comments) {
        s/\/\*.*?\*\///g;
        s/\/\/.*//;
    }
    foreach my $cmd (@cmds) {
        if ($old || $cmd->[0] =~ /^-ea$/) {
            eval $cmd->[1];
        }
    }
    foreach my $sub (@subs) {
        if ($old) {
            s/$sub->[0]/$sub->[1]/g;
        }
    }

    # remove the newline so we can move curly braces here if we want.
    s/\n//;
    return $_;
}

while (my $param1 = shift()) {
    if ($param1 =~ /^-nc$/) {
        $strip_comments = 1;
        next;
    }
    if ($param1 =~ /^-nb$/) {
        $strip_braces = 1;
        next;
    }
    if ($param1 =~ /^-ns$/) {
        $strip_slashes = 1;
        next;
    }
    my $param2 = shift();
    if ($param2 =~ /^$/) {
        usage();
    }
    if ($param1 =~ /^-e(a|)$/) {
        push @cmds, [$param1, $param2];
        next;
    }
    push @subs, [$param1, $param2];
}

my ($oldfh, $oldfile) = mkstemp("/tmp/oldXXXXX");
my ($newfh, $newfile) = mkstemp("/tmp/newXXXXX");

my $started = 0;
my $output;

#recreate an old file and a new file
while (<>) {
    if ($_ =~ /^(---|\+\+\+)/) {
        next;
    }
    if ($_ =~ /^@/) {
        $started = 1;
    }
# Oops.  This isn't the right way to know that a diff has ended
#    if ($started && !($_ =~ /^[- @+]/)) {
#        last;
#    }
    $output = filter($_);

    if ($strip_braces && $_ =~ /^(\+|-)\W+{/) {
        $output =~ s/^[\t ]+(.*)/ $1/;
    } else {
        $output = "\n" . $output;
    }

    if ($_ =~ /^-/) {
        print $oldfh $output;
        next;
    }
    if ($_ =~ /^\+/) {
        print $newfh $output;
        next;
    }
    print $oldfh $output;
    print $newfh $output;

}
print $oldfh "\n";
print $newfh "\n";
# git diff puts a -- and version at the end of the diff.  put the -- into the
# new file as well so it's ignored
if ($output =~ /\n-/) {
    print $newfh "-\n";
}

my $hunk;
my $old_txt;
my $new_txt;

open diff, "diff -uw $oldfile $newfile |";
while (<diff>) {
    if ($_ =~ /^(---|\+\+\+)/) {
        next;
    }

    if ($_ =~ /^@/) {

        if ($strip_comments) {
            $old_txt =~ s/\/\*.*?\*\///g;
            $new_txt =~ s/\/\*.*?\*\///g;
        }
        if ($strip_braces) {
            $old_txt =~ s/{([^;{]*?);}/$1;/g;
            $new_txt =~ s/{([^;{]*?);}/$1;/g;
            # this is a hack because i don't know how to replace nested
            # unneeded curly braces.
            $old_txt =~ s/{([^;{]*?);}/$1;/g;
            $new_txt =~ s/{([^;{]*?);}/$1;/g;
        }

        if ($old_txt ne $new_txt) {
            print $hunk;
            print $_;
        }
        $hunk = "";
        $old_txt = "";
        $new_txt = "";
        next;
    }

    $hunk = $hunk . $_;

    if ($strip_slashes) {
        s/\\$//;
    }

    if ($_ =~ /^-/) {
        s/-//;
        s/[ \t\n]//g;
        $old_txt = $old_txt . $_;
        next;
    }
    if ($_ =~ /^\+/) {
        s/\+//;
        s/[ \t\n]//g;
        $new_txt = $new_txt . $_;
        next;
    }
    if ($_ =~ /^ /) {
        s/^ //;
        s/[ \t\n]//g;
        $old_txt = $old_txt . $_;
        $new_txt = $new_txt . $_;
    }
}

if ($old_txt ne $new_txt) {
    if ($strip_comments) {
        $old_txt =~ s/\/\*.*?\*\///g;
        $new_txt =~ s/\/\*.*?\*\///g;
    }
    if ($strip_braces) {
        $old_txt =~ s/{([^;{]*?);}/$1;/g;
        $new_txt =~ s/{([^;{]*?);}/$1;/g;
        $old_txt =~ s/{([^;{]*?);}/$1;/g;
        $new_txt =~ s/{([^;{]*?);}/$1;/g;
    }

    print $hunk;
}

unlink($oldfile);
unlink($newfile);

print "\ndone.\n";

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers

[Rafael J. Wysocki]

On Friday, May 30, 2014 10:21:06 PM Theodore Ts'o wrote:
> On Fri, May 30, 2014 at 06:54:07PM -0700, Guenter Roeck wrote:
> > 
> > I don't think this is a matter of being enthusiastic or not. Much more
> > one of expertise. As a user of stable kernels, I would very much prefer
> > to have it maintained by an expert (or by experts), not by a newcomer.
> > Of course, that may be just me.
> 
> The stable kernel tree would still be maintained by experts.  Even if
> newcomers supplied some of the backports, they would still get
> reviewed by experts.  Granted, for the first few patches it might not
> save me that much time compared to my just doing the backport myself,
> but it's IMHO much more likely to result in a capable programmer in
> the long run (at least compared to people who are just encouraged to
> do drive-by whitespace / spelling patch submissions). 
> 
> And if we have newcomers looking for bug fix patches that weren't
> properly marked with "cc: stable@vger.kernel.org", and calling those
> patches to the stable tree and subsystem maintainers, again, I think
> it's healthier and certainly more productive than traning them to look
> for whitespace and checkpatch.pl warnings.
> 
> If the argument is that newcomers aren't motivated to do this sort of
> grunt work, recall how much grunt work maintainers have to do.  How
> much time do you think most maintainers spend writing new sexy
> features?  As opposed to the less fun bits, such as reviewing code and
> running integration tests and bisecting through submitted patches to
> find the buggy "cleanup" patch someone sent me?  
> 
> If I can find people who don't mind sharing in some of the more less
> glory-filled aspects of being an open source developer, and who isn't
> afraid to do some of the less fun but still vitally important bits,
> speaking personally, those are the people that I would be more
> motivated to mentor.  In contrast, someone who sends thousands of
> whitespace patches is ***not*** someone I'm personally going to be
> much inclined to spend time mentoring.

I totally agree.

Rafael

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Randy Dunlap]

On 05/30/2014 10:28 AM, Paul E. McKenney wrote:
> On Thu, May 29, 2014 at 10:41:22AM -0700, Greg KH wrote:
>> On Thu, May 29, 2014 at 05:28:11PM +0800, Li Zefan wrote:
>>> On 2014/5/29 5:32, Thomas Gleixner wrote:
>>>> On Wed, 28 May 2014, Mark Brown wrote:
>>>>
>>>>> On Wed, May 28, 2014 at 05:44:41PM +0000, Luck, Tony wrote:
>>>>>
>>>>>>> There's a world of difference between thanking people for review and a
>>>>>>> detailed account of all the changes made in every single iteration of
>>>>>>> the review.
>>>>>
>>>>>> This is already covered in Documentation/SubmittingPatches. Quoting
>>>>>> lines 585-592 (see last sentence):
>>>>>
>>>>> Right, but Daniel is proposing lifting that above the --- and including
>>>>> it in git.
>>>>
>>>> What you really want is:
>>>>
>>>> Link: https://lkml.kernel.org/r/MESSAGE_ID_OF_PATCH
>>>>
>>>> It's way more useful than any of the v1-n writeups, which are most of
>>>> the time just a completely waste of electrons. Even if written well,
>>>> without the actual review context they are pretty pointless.
>>>>
>>>
>>> A QA asked me about kernel development process. One of his question is,
>>> he found some valuable information in the discussion of the patch often
>>> won't be added to the changelog, so providing the commit how to find
>>> the discussion?
>>
>> A research group has created a tool that takes a given git commit, finds
>> the mailing list discussion for that patch.  It was posted to lkml 6 or
>> so months ago, you should point them at that tool if they want to do
>> this.
> 
> This one?  https://lkml.org/lkml/2014/3/11/2

Thanks, I didn't find it.


-- 
~Randy

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Takashi Iwai]

At Sat, 31 May 2014 18:49:51 +0200,
Geert Uytterhoeven wrote:
> 
> On Wed, May 28, 2014 at 11:59 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > I really started to put breaks into this cycle of hell, where I get
> > spammed with a 30+ patch series in the morning and after I spent some
> > quality time looking at it and replying to a particular patch, I get
> > another spam bomb within a few hours, which is not much better than
> > the previous one.
> 
> That's indeed an issue. Some large series are immediately reposted for every
> single (sometimes trivial) change suggested by a commenter.
> This also leads to inflated patch version numbers. I'm already getting nervous
> when I have to send out a v3 or v4 myself, but from a quick search, v29
> seems to be the current record on lkml...
> 
> Does git send-email remember when the previous version of a series was
> sent out?
> If yes, it could refuse to resend it too soon. The back off period would
> depend on the size of the patch series (large patch series need more delay).

IMO, it's dangerous to let a basic tool like git blocking the
immediate resend, too.

I agree that digesting a large patchset is one of most painful tasks
as a maintainer.  In such a case, however, I'd rather suggest to split
the large patch series into multiple staged patchsets.


Takashi

Re: [Ksummit-discuss] [TOPIC] Encouraging more reviewers

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]


> Further I find arguments like the above to be pathetic and distasteful.  Profit
> motives are getting out of control IMO.

Just to make sure: My original suggestion was not to create motivation
but to back off people who already do this work (on their own). Yes,
there are obstacles in implementing this, but I'd think this is worth
it.

> Its easier to get money through having authority than any other way.

True.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Randy Dunlap]

On 05/30/2014 02:10 PM, James Bottomley wrote:
> On Fri, 2014-05-30 at 11:10 -0400, John W. Linville wrote:
>> On Fri, May 30, 2014 at 10:59:38AM -0400, Steven Rostedt wrote:
>>> On Wed, 28 May 2014 15:15:14 -0400
>>> "John W. Linville" <linville@tuxdriver.com> wrote:
>>
>>>> I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
>>>>
>>>
>>> Yeah, I wouldn't want to type that. What about:
>>>
>>> Approved-by: ...
>>>
>>> That is reserved for maintainers only?
>>
>> If we need such a tag, I like this verion better.
> 
> Bikeshedded-by:?
> 
> Discussed-in-circles-by:?
> 
> Maintainer-penguin-pee-blessed-by:?
> 
> OK, it's late here; but for the record, I think Acked-by is perfectly
> clear and we don't need a new tag.

Acked-by: me


-- 
~Randy

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1067 bytes --]

On Sun, 01 Jun 2014 19:43:42 -0700 Randy Dunlap <rdunlap@infradead.org> wrote:

> On 05/30/2014 02:10 PM, James Bottomley wrote:
> > On Fri, 2014-05-30 at 11:10 -0400, John W. Linville wrote:
> >> On Fri, May 30, 2014 at 10:59:38AM -0400, Steven Rostedt wrote:
> >>> On Wed, 28 May 2014 15:15:14 -0400
> >>> "John W. Linville" <linville@tuxdriver.com> wrote:
> >>
> >>>> I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
> >>>>
> >>>
> >>> Yeah, I wouldn't want to type that. What about:
> >>>
> >>> Approved-by: ...
> >>>
> >>> That is reserved for maintainers only?
> >>
> >> If we need such a tag, I like this verion better.
> > 
> > Bikeshedded-by:?
> > 
> > Discussed-in-circles-by:?
> > 
> > Maintainer-penguin-pee-blessed-by:?
> > 
> > OK, it's late here; but for the record, I think Acked-by is perfectly
> > clear and we don't need a new tag.
> 
> Acked-by: me
> 
> 

Are you just saying that you agree, or are you affirming this in your status
as MAINTAINER of the documentation?

- Confused


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [Ksummit-discuss] Reforming Acked-by (was Re: [TOPIC] Encouraging more reviewers)

[Randy Dunlap]

On 06/01/2014 07:53 PM, NeilBrown wrote:
> On Sun, 01 Jun 2014 19:43:42 -0700 Randy Dunlap <rdunlap@infradead.org> wrote:
> 
>> On 05/30/2014 02:10 PM, James Bottomley wrote:
>>> On Fri, 2014-05-30 at 11:10 -0400, John W. Linville wrote:
>>>> On Fri, May 30, 2014 at 10:59:38AM -0400, Steven Rostedt wrote:
>>>>> On Wed, 28 May 2014 15:15:14 -0400
>>>>> "John W. Linville" <linville@tuxdriver.com> wrote:
>>>>
>>>>>> I hate to bikeshed this, but "Maintainer-acked-by" seems too long to type...
>>>>>>
>>>>>
>>>>> Yeah, I wouldn't want to type that. What about:
>>>>>
>>>>> Approved-by: ...
>>>>>
>>>>> That is reserved for maintainers only?
>>>>
>>>> If we need such a tag, I like this verion better.
>>>
>>> Bikeshedded-by:?
>>>
>>> Discussed-in-circles-by:?
>>>
>>> Maintainer-penguin-pee-blessed-by:?
>>>
>>> OK, it's late here; but for the record, I think Acked-by is perfectly
>>> clear and we don't need a new tag.
>>
>> Acked-by: me
>>
>>
> 
> Are you just saying that you agree, or are you affirming this in your status
> as MAINTAINER of the documentation?
> 
> - Confused
> 

Yes (I agree with James).

-- 
~Randy

Re: [Ksummit-discuss] More productive uses of enthusiastic new kernel developers (was: Re: [TOPIC] Encouraging more reviewers)

[Jason Cooper]

On Fri, May 30, 2014 at 01:54:49PM -0600, Shuah Khan wrote:
> On Fri, May 30, 2014 at 10:56 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> > Thinking about this some more, what if instead of, or in addition to,
> > having newcomers work on cleanup patches, what if we encouraged some
> > of them to help to manually backport patches to the stable kernels
> > that don't apply automatically?
> >
> > Currently the stable maintainers will typically send a notification
> > subsystem maintainer / mailing lists that a particular patch didn't
> > apply and that it's up some subsystem developers to handle resolving
> > the patch conflict.
> >
> > What if those patches were also cc'ed to a separate mailing list which
> > was also monitored by patchwork, and newcomers were encouraged to grab
> > patches and try to make a determination whether it's really needed,
> > and how to handle doing the backport, and once the patch was properly
> > backported they could cc the subsystem mailing list asking for
> > approval before the patch could get fed into a long-term stable tree?
> >
> > It does mean more patches to reviews, but I'd much rather review a
> > patch going into a long-term stable tree that would otherwise get
> > dropped (and thus avoid whiny entitled users and/or embedded engineers
> > who think it's the maintainer's job to backport to N different stable
> > kernels), rather than review whitespace patches.
> >
> > If this worked out, I could also imagine encouraging more advanced
> > newcomers to look for bug fix patches that didn't have an explicit cc:
> > stable@vger.kernel.org, and look to see if they should be backported.
> > This would help for certain subsystems that have elected not to
> > automatically mark their patches.  There are subsystems which don't
> > like anyone but their experienced developers to handle doing
> > backports, but I suspect there would also be many subsystems that
> > would welcome the extra help, especially if it was much more likely to
> > result in additional reviewers / subsystem developers.
> >
> 
> In addition to the above, how about keeping a maintainer and sub-maintainer
> wish list of tasks and work they would like to see get done in their areas and
> don't have time do it themselves. It could be maintained on kernelnewbies site.
> Having something like this will help guide and give direction to people looking
> to get started in a new area and/or any area of the kernel.

There was some previous discussion on this that I can't locate atm.  In
short, the staging tree already has TODO files in-tree.  The advantage
being the patch handling the todo item should also delete the item from
the list.

Keeping it on a separate website just means additional out-of-workflow
maintenance for people who are already saturated.

thx,

Jason.