Re: [MAINTAINERS SUMMIT] The role of AI and LLMs in the kernel process

[Luis Chamberlain <mcgrof@kernel.org>]

On Thu, Aug 07, 2025 at 02:00:56PM +0100, Lorenzo Stoakes wrote:
> On Thu, Aug 07, 2025 at 01:25:23PM +0100, Mark Brown wrote:
> > On Wed, Aug 06, 2025 at 08:26:41PM +0100, Lorenzo Stoakes wrote:
> >
> > > - Was the commit message of this patch generated in large part by an LLM
> > >   (excluding non-native speakers using an LLM to simply assist writing it
> > >   in english)?
> >
> > Easiest way to say that is probably to say "written by" and "translated
> > by".  I think for all these things we should just talk about tooling
> > rather than specifically LLMs, as well as avoiding any rules lawyering
> > about how precisely a given tool is implemented it's probably useful to
> > know about tools whatever the technology.
> 
> That's a great idea!!
> 
> And agreed on language/rules lawyering, I think we have to have something
> _simple_ and robust at least to begin with.

I've been using for years the tag "Generated-by" starting wit Coccinelle:

git log --oneline --author="mcgrof" --grep "Generated-by"| wc -l
31

And it seems like I'm not the only one:

git log --oneline --grep "Generated-by"| wc -l
49

For other projects such as kdevops where I *only* use LLMs to write new
code now, we have been using:

Generated-by: ChatGPT Codex
Generated-by: Claude AI

We use this even if it was partially AI. I think that gives
maintainers sufficient information to make judgement calls.

Other than this practice, if we're to slowly and carefully welcome LLM
generated code on the kernel I'd recommend we evaluate a context
intialization file.  For Claude that's CLAUDE.md, you can look at
kdevops's file for an example [0]. Having one makes adoption easier, and you
can provide strict rules. The context is limited though, you want about
~40 KiB. However I'm not sure if a generic one may be so easily
agreeable, so fortunatley the bots can also look for your ~/CLAUDE.md.
But I can vouch for the fact that its proven useful for kdevops.

Other than this, another best pratice we've adopted on kdevops is to
to grade commits based on LLM prompts so to keep tabs on how well LLMs improve
overtime with example prompts, and to track them with full prompts, we have
PROMPTS.md [1]. These can help LLMs as well.

The grammatical evolution on kdevops is what makes LLMs adoption today
easily possible [2]. I don't think its as solid yet for kernel development in
agreement with recent findings [3], however its only getting better so
best is we prepare for it and learn from existing projects' use cases.

So for testing -- clearly its a win.  For other things, here's a few
things we can evaluate future success over time:

  * fix syzbot bugs
  * take on maintenance for orphaned drivers
  * help maintainers with patch review / testing

[0] https://github.com/linux-kdevops/kdevops/blob/main/CLAUDE.md
[1] https://github.com/linux-kdevops/kdevops/blob/main/PROMPTS.md
[2] https://github.com/linux-kdevops/kdevops?tab=readme-ov-file#generative-ai-usage-on-kdevops
[3] https://neurips.cc/virtual/2024/poster/97426

  Luis