Re: [Ksummit-discuss] [CORE TOPIC] Recruitment (Reviewers, Testers, Maintainers, Hobbyists)

[Guenter Roeck <linux@roeck-us.net>]

On Thu, Jul 09, 2015 at 04:13:15PM -0400, Theodore Ts'o wrote:
> No matter what we ask people to type, the system can be gamed so long
> as we (a) use automated systems, and (b) we're transparent what the
> automated system uses as a signal about what's a valid review versus a
> garbage review.  This is basically the same problem that Google has
> when trying to deal with SEO folks who are trying to game the search
> algorithm, and it's not really a soluble problem unless you have a lot
> of people constantly working on refining the system and adjusting in
> response to humans who are trying to attack the system.
> 
> And if there are companies who are using reviews, or signed-off-by
> counts as a basis of performance reviews, humans *will* be
> incentivized to game the system, regardless of what the kernel summit
> program committee might be doing to use these systems.
> 
While this may be correct in some circumstances, there are also
companies which do just the opposite, especially when it comes
to reviews. At least for my part I still have to encounter a
company where "provides excellent and thorough code reviews"
can be found as positive statement in a performance review.

Earlier it was discussed how to improve the recognition of reviewers.
Your comments seems to suggest the opposite, and may actually discourage
reviewers. Why should I review Linux kernel code if that is seen
by some as me trying to "game" the system ?

I used to submit some more or less random cleanup patches into various
areas of the kernel, just for fun, because I liked doing it, and because
I needed some distraction. I stopped doing it after I realized that this
may be seen as "gaming the system". Nowadays I only submit such random
patches to fix build errors or real bugs. Would I continue to do that
if I get accused of trying to game the system ? Probably not.

I am not much if at all concerned with people gaming the system
to improve their standing, whereever that may be. I am much more
concerned that we may drive people off by accusing them to just
submit patches or reviews to game the system. At the end this is our
loss, not theirs. If their work results in better code, who cares
what their motivation might be ? And who are we to judge their
motivation ?

> (Although I will say there is one person who I've consistently
> downgraded *because* it's obvious s/he has been sending lots of
> trivial patches, and while that person may (or may not) have changed,
> it's still human nature that I assume that this person's scores are
> inflated.  And this is true of whoever sends huge number of checkpatch
> or coccinelle generated or inspired patches.  So people should be
> aware that attempts to game the system can backfire, when people start
> imposing informal "manual penalties" to their evaluation of that
> particular person.  Which is exactly what happens in the Search Engine
> Optimization world, by the way....)
> 
That may be true for some people, but at the same time I think statements
like the above might discourage people who just like cleaning up code for
fun. There are several of those working on cleaning up the Linux kernel,
and I truly appreciate their efforts.

Guenter