Re: [Ksummit-discuss] [CORE TOPIC] Recruitment (Reviewers, Testers, Maintainers, Hobbyists)

[Theodore Ts'o <tytso@mit.edu>]

No matter what we ask people to type, the system can be gamed so long
as we (a) use automated systems, and (b) we're transparent what the
automated system uses as a signal about what's a valid review versus a
garbage review.  This is basically the same problem that Google has
when trying to deal with SEO folks who are trying to game the search
algorithm, and it's not really a soluble problem unless you have a lot
of people constantly working on refining the system and adjusting in
response to humans who are trying to attack the system.

And if there are companies who are using reviews, or signed-off-by
counts as a basis of performance reviews, humans *will* be
incentivized to game the system, regardless of what the kernel summit
program committee might be doing to use these systems.

(Although I will say there is one person who I've consistently
downgraded *because* it's obvious s/he has been sending lots of
trivial patches, and while that person may (or may not) have changed,
it's still human nature that I assume that this person's scores are
inflated.  And this is true of whoever sends huge number of checkpatch
or coccinelle generated or inspired patches.  So people should be
aware that attempts to game the system can backfire, when people start
imposing informal "manual penalties" to their evaluation of that
particular person.  Which is exactly what happens in the Search Engine
Optimization world, by the way....)

							- Ted