Re: [MAINTAINERS SUMMIT] Maintainer burnout

[Alexei Starovoitov <alexei.starovoitov@gmail.com>]

On Thu, Aug 17, 2023 at 8:20 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 17 Aug 2023 14:36:31 +0200 Andrew Lunn wrote:
> > > > In so far as making it possible to get b) to help, my current excitement
> > > > surrounds around what Song Liu mentioned to me at LSFMM and then
> > > > quickly demonstrated that the eBPF folks are doing with patchwork.
> > > > Get the patches to be tested automatically, and *immediately*
> > > > patch reviewers and maintainers can get feedback if something is not even
> > > > worth reviewing.
> > >
> > > This is interesting, do you have any link to share to related resources
> > > ?
> >
> > I'm guessing, but i think that is referring to the "Checks" section in
> > a patchworks status page. Picking a couple of patches at random:
> >
> > https://patchwork.kernel.org/project/netdevbpf/patch/20230816210049.54733-9-saeed@kernel.org/
> >
> > https://patchwork.kernel.org/project/netdevbpf/patch/20230816210049.54733-2-saeed@kernel.org/
> >
> > Jakub can tell you more.
>
> FWIW BPF runs more stuff, they spin up VMs and run the actual selftests,
> so looking at a BPF patch will be more informative:
>
> https://patchwork.kernel.org/project/netdevbpf/patch/20230816045959.358059-3-houtao@huaweicloud.com/
>
> Exact details are to my knowledge in flux, the system is constantly
> being improved.

Thanks for raising awareness of BPF CI.
I have to highlight that maintaining BPF CI is a full time job on its own.
We have engineers oncall who monitor failures in CI itself
(not failures in bpf selftests caused by new patches).
CI automation breaks often. Packages missing, VMs too slow, etc.

The link above is an example where bpf test_maps fails on s390 and
passes on arm64 and x86.
We've been trying to root cause it for a long time. So far it looks
to be an odd CPU virtualization artifact on that particular architecture.

There is a long list of CI features that we're working on.
CI framework is open sourced, of course.

I'd like to bring the thread back to Josef's point:

> This thread has sort of wandered off into the "how to do automation" weeds. > I think that automation is a good solution for a subset of the problems that
> maintainers face, but it's not my main focus.

BPF has solid CI that helps a lot, but the maintainer burnout is acutely felt.
The main reason for burnout is patch flood.
The maintainer's day looks like non-stop code review.
The patch backlog just doesn't end.
We're trying to encourage active developers to be code reviewers as well
via positive/negative scores:
https://lore.kernel.org/bpf/ZJx8sBW%2FQPOBswNF@google.com/

It doesn't help much yet. All incoming kernel contributors assume
that it's a maintainer's job to do code reviews.
Developers just send patches and wait. It doesn't occur to them that
reviewing other patches will help them land their own.

To address maintainer burnout we need to change the culture of the community
and transform active developers to active code reviewers.
We're looking for ideas on how to do that.