Re: [MAINTAINERS SUMMIT] The role of AI and LLMs in the kernel process

[Stephen Hemminger <stephen@networkplumber.org>]

On Tue, 05 Aug 2025 12:43:38 -0400
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:

> On Tue, 2025-08-05 at 17:03 +0100, Lorenzo Stoakes wrote:
> > Unavoidably, LLMs are the hot topic in tech right now, and are here
> > to stay.
> > 
> > This poses unique problems:
> > 
> > * Never before have people been able to generate as much content that
> > may, on a surface reading, seem valid whilst in reality being quite
> > the opposite.
> > 
> > * Equally, LLM's can introduce very subtle mistakes that humans find
> > difficult to pick up upon - humans implicitly assume that the classes
> > of errors they will encounter are the kinds other humans would make -
> > AI defeats that instinct.  
> 
> Do you have any examples of this?  I've found the opposite to be true:
> AI is capable of really big stupid mistakes when it hasn't seen enough
> of the pattern, but I can't recall seeing it make something you'd
> classify as a subtle mistake (I assume it could copy subtle mistakes
> from wrong training data, so I'm not saying it can't, just that I
> haven't seen any).
> 
> I think the big mistakes could possibly be avoided by asking people who
> submit patches to also append the AI confidence score:
> 
> https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept/accuracy-confidence?view=doc-intel-4.0.0
> 
> So we know how much similar training the model has seen before coming
> to any conclusion about the value of the output.
> 
> > * The kernel is uniquely sensitive to erroneous (especially subtly
> > erroneous) code - even small errors can be highly consequential. We
> > use a programming language that can almost be defined by its lack of
> > any kind   of safety, and in some subsystems patches are simply taken
> > if no obvious problems exist, making us rather vulnerable to this.  
> 
> I think that's really overlooking the fact that if properly trained (a
> somewhat big *if* depending on the model) AI should be very good at
> writing safe code in unsafe languages.  However it takes C specific
> training to do this, so any LLM that's absorbed a load of rust, python
> and javascript from the internet will be correspondingly bad at writing
> safe C code.  Hence the origin of the LLM and its training corpus would
> be a key factor in deciding to trust it.
> 
> > * On the other hand, there are use cases which are useful - test
> > data/code generation, summarisation, smart auto-complete - so it'd
> > perhaps be foolish to entirely dismiss AI.  
> 
> Patch backporting is another such nice use.
> 
> > A very important non-technical point we must consider is that, the
> > second we even appear to be open to AI submission of _any_ kind, the
> > press will inevitably report on it gleefully, likely with
> > oversimplified headlines like 'Linux accepts AI patches'.  
> 
> Oh, I think simply accepting AI patches is old news:
> 
> https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html
> 
> > The moment that happens, we are likely to see a significant uptick in
> > AI submissions whether we like it or not.
> > 
> > I propose that we establish the broad rules as they pertain to the
> > kernel, and would like to bring the discussion to the Maintainer's
> > Summit so we can determine what those should be.
> > 
> > It's important to get a sense of how maintainers feel about this -
> > whether what is proposed is opt-in or opt-out - and how we actually
> > implement this.
> > 
> > There has been discussion on-list about this (see [0]), with many
> > suggestions made including a 'traffic light' system per-subsystem,
> > however many open questions remain - the devil is in the details.
> > 
> > [0]:
> > https://lore.kernel.org/all/20250727195802.2222764-1-sashal@kernel.or
> > g/  
> 
> We're already getting AI generated bug reports from what I can tell. 
> It would be really helpful to see the AI confidence score for them as
> well.
> 
> Regards,
> 
> James
> 
> 
> 

I have been toying with AI for some things that got stuck on my TODO
list for years (like writing tests). Have to agree with James, the errors
in that code are not subtle. Mostly they are things that never work because
AI has no context about what it is doing. Especially when porting between environments.