Re: [MAINTAINERS SUMMIT] The role of AI and LLMs in the kernel process

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Tue, 2025-08-05 at 17:03 +0100, Lorenzo Stoakes wrote:
> Unavoidably, LLMs are the hot topic in tech right now, and are here
> to stay.
> 
> This poses unique problems:
> 
> * Never before have people been able to generate as much content that
> may, on a surface reading, seem valid whilst in reality being quite
> the opposite.
> 
> * Equally, LLM's can introduce very subtle mistakes that humans find
> difficult to pick up upon - humans implicitly assume that the classes
> of errors they will encounter are the kinds other humans would make -
> AI defeats that instinct.

Do you have any examples of this?  I've found the opposite to be true:
AI is capable of really big stupid mistakes when it hasn't seen enough
of the pattern, but I can't recall seeing it make something you'd
classify as a subtle mistake (I assume it could copy subtle mistakes
from wrong training data, so I'm not saying it can't, just that I
haven't seen any).

I think the big mistakes could possibly be avoided by asking people who
submit patches to also append the AI confidence score:

https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept/accuracy-confidence?view=doc-intel-4.0.0

So we know how much similar training the model has seen before coming
to any conclusion about the value of the output.

> * The kernel is uniquely sensitive to erroneous (especially subtly
> erroneous) code - even small errors can be highly consequential. We
> use a programming language that can almost be defined by its lack of
> any kind   of safety, and in some subsystems patches are simply taken
> if no obvious problems exist, making us rather vulnerable to this.

I think that's really overlooking the fact that if properly trained (a
somewhat big *if* depending on the model) AI should be very good at
writing safe code in unsafe languages.  However it takes C specific
training to do this, so any LLM that's absorbed a load of rust, python
and javascript from the internet will be correspondingly bad at writing
safe C code.  Hence the origin of the LLM and its training corpus would
be a key factor in deciding to trust it.

> * On the other hand, there are use cases which are useful - test
> data/code generation, summarisation, smart auto-complete - so it'd
> perhaps be foolish to entirely dismiss AI.

Patch backporting is another such nice use.

> A very important non-technical point we must consider is that, the
> second we even appear to be open to AI submission of _any_ kind, the
> press will inevitably report on it gleefully, likely with
> oversimplified headlines like 'Linux accepts AI patches'.

Oh, I think simply accepting AI patches is old news:

https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html

> The moment that happens, we are likely to see a significant uptick in
> AI submissions whether we like it or not.
> 
> I propose that we establish the broad rules as they pertain to the
> kernel, and would like to bring the discussion to the Maintainer's
> Summit so we can determine what those should be.
> 
> It's important to get a sense of how maintainers feel about this -
> whether what is proposed is opt-in or opt-out - and how we actually
> implement this.
> 
> There has been discussion on-list about this (see [0]), with many
> suggestions made including a 'traffic light' system per-subsystem,
> however many open questions remain - the devil is in the details.
> 
> [0]:
> https://lore.kernel.org/all/20250727195802.2222764-1-sashal@kernel.or
> g/

We're already getting AI generated bug reports from what I can tell. 
It would be really helpful to see the AI confidence score for them as
well.

Regards,

James