RE: [MAINTAINERS / KERNEL SUMMIT] AI patch review tools

["Bird, Tim" <Tim.Bird@sony.com>]

> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> On Wed, Oct 08, 2025 at 08: 38: 45PM +0000, Bird, Tim wrote: > > -----Original Message----- > > From: James Bottomley
> <James. Bottomley@ HansenPartnership. com> > > On Wed, 2025-10-08 at 19: 50 +0000, Bird, Tim wrote: >
> On Wed, Oct 08, 2025 at 08:38:45PM +0000, Bird, Tim wrote:
> > > -----Original Message-----
> > > From: James Bottomley <James.Bottomley@HansenPartnership.com>
> > > On Wed, 2025-10-08 at 19: 50 +0000, Bird, Tim wrote:
> > > > > -----Original Message-----
> > > From: Laurent Pinchart <laurent. pinchart@ideasonboard.com>
> > > On Wed, 2025-10-08 at 19:50 +0000, Bird, Tim wrote:
> > > > > -----Original Message-----
> > > > > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > On Wed, Oct 08, 2025 at 09:08:33PM +0200, Andrew Lunn wrote:
> > > > > > > My goal for KS/MS is to discuss how to enable maintainers to
> > > > > > > use review automation tools to lower their workload.
> > > > > >
> > > > > > Maintainers will want to use these tools, if they prove to be
> > > > > > useful. But ideally, we want the developers to use these tools
> > > > > > and fix the issues before they post code for review. That reduces
> > > > > > the maintainers workload even more. So Maintainers just need to
> > > > > > run the tools to prove that the developers have run the tools and
> > > > > > have already fixed the problems.
> > > > > >
> > > > > > So i'm not sure your goal is the correct long term goal. It
> > > > > > should be a tool for everybody, not just maintainers.
> > > > >
> > > > > This raises the interesting and important question of how to get
> > > > > patch submitters to follow a recommended workflow. We routinely get
> > > > > patches that produce checkpatch errors that are clearly not false
> > > > > positives. Rob Herring implemented a bot to run checks on device
> > > > > tree bindings and device tree sources because lots of patches fail
> > > > > those checks. I'm sure there are lots of other examples that have
> > > > > led maintainers to automate checks on the receiver's side, through
> > > > > various types of standard CIs or hand-made solutions. Submitters
> > > > > should run more tests, how to get them to do so is a broader
> > > > > question.
> > > >
> > > > Maybe it would be worthwhile to annotate patch submissions with tags
> > > > indicating what tools have been run on them.  I know we're trying to
> > > > avoid overuse of commit tags, but maybe we could automate this a bit,
> > > > and/or' reuse the 'Reviewed-by:' tag in the commit message.  I could
> > > > envision, in some future workflow utopia, where a missing 'Reviewed-
> > > > by: checkpatch.pl AND claude AI review' would be grounds for
> > > > requesting these before human review.
> > >
> > > Realistically, we can't even get some submitters to run checkpatch, so
> > > I don't think the additional tag would help. What does help is having
> > > the 0day bot take a feed of the mailing list, select the [PATCH] tag
> > > and run checkpatch.pl as part of the tests, so someone could do the
> > > same with whatever AI acceptance tests are agreed.
> >
> > There's no question that 0day automation of checkpatch.pl feedback
> > has been a great thing.  I suspect that more submitters would run
> > checkpatch before sending their patches, if failure to do so resulted
> > in automatic rejection of the patch.  This is more of a process backbone
> > issue than anything else.
> >
> > > Although the problem with AI acceptance testing is that these models
> > > and the inferencing required to run them doesn't come for free so
> > > someone is going to end up paying for all this AI ... unless we can get
> > > some cloud company to donate it, of course ...
> >
> > Indeed.  All the more reason to enforce it at the source.  It then becomes
> > a cost for the contributor instead of the upstream community, which is
> > going to scale better.
> 
> Forcing contributors to pay for access to proprietary tools is not
> acceptable. Forcing contributors to even run proprietary tools is not
> acceptable. If maintainers want contributions to go through any
> proprietary tooling before submission, then this has to run on the
> maintainer side (be it on a maintainer's machine, in some form of CI, or
> somewhere else).

I agree.
We don't even have the resolve to get contributors to run checkpatch.pl
(a freely available and imminently accessible tool).
I think it would be illustrative to figure out what the barriers are to that, or what a proper
incentive structure would be (and why we're missing it), before tackling the AI review issue.
Then we have to deal with the cost/proprietary tool issue for AI review.

There are open-source, locally runnable models out there.  They currently don't yield
results that are as good as the large proprietary models, but they'll get better over time.

I did say that this was in the context of a future vision of a utopian workflow.

What bugs me about the current discussion is the implication that we're willing to just
add more cost to maintainers or the workflow infrastructure, when the cost should
IMHO, be born by contributors.

> 
> You're right that cost would then be a problem. I can certainly imagine
> $popular_ai_company sponsoring this short term, until we're
> vendor-locked and they stop sponsorship. I don't think rushing in that
> direction is a good idea.
Good point about needing to avoid lockin.
 -- Tim