Re: [MAINTAINERS SUMMIT] The role of AI and LLMs in the kernel process

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Tue, 2025-08-05 at 20:15 +0100, Lorenzo Stoakes wrote:
> On Tue, Aug 05, 2025 at 02:39:20PM -0400, Sasha Levin wrote:
> > On Tue, Aug 05, 2025 at 06:55:29PM +0100, Lorenzo Stoakes wrote:
> > > On Tue, Aug 05, 2025 at 12:43:38PM -0400, James Bottomley wrote:
> > > > I think that's really overlooking the fact that if properly
> > > > trained (a somewhat big *if* depending on the model) AI should
> > > > be very good at writing safe code in unsafe languages.  However
> > > > it takes C specific
> > > 
> > > I fundamentally disagree.
> > > 
> > > The consequences of even extremely small mistakes can be very
> > > serious in C, as the language does little to nothing for you.
> > > 
> > > No matter how much data it absorbs it cannot span the entire
> > > space of all possible programs or even anywhere close.
> > 
> > Neither can a human :)
> > 
> > I think that this is where we see things differently: I don't think
> > that AI needs to be perfect, I just want it to be at the same lever
> > (or better) than a human.
> 
> Not at all, none of my objections are about perfection. I use LLMs
> myself, in appropriate circumstances where the expected failings are
> not problematic.
> 
> My objections are to do with the the kinds of errors one can
> encounter with statistical inference like this.
> 
> Humans do not confidently hallucinate in the absence of concrete
> data, rather we infer and model.

Might I refer you to pretty much any white house press briefing for
counter examples ...

>  This is dynamics vs. statistics (I genuinely recommend the article I
> linked to James, it's a fascinating insight - [0]).
> 
> It's the _nature_ of these errors that I am concerned about in
> conjunction with unsafe development tooling and highly consequential
> results of even subtle errors that makes the kernel especially
> problematic in my view.

You know that's an argument for not allowing teenagers to learn to
drive (at any age), or over operate heavy machinery or ...

The point being that with enough training human society thinks the
probability of error is remote enough (for some value of enough) to
become an acceptable risk.

> > Humans aren't great at writing C code. There's a reason we're
> > looking at using Rust for the kernel, and there's a reason that LTS
> > trees exist - they're living evidence of just how many mistakes
> > humans make.
> 
> Humans make human-like errors. and not at industrial scale :)

I've an infinite number of monkeys^WSet of Far Eastern Call centres for
appliance repair that would beg to disagree.

> > Look at the contents of LTS trees or the CVEs that get assigned:
> > most of them are fairly simple memory safety issues, off-by-one,
> > use-after-free, etc...
> 
> Absolutely.
> 
> > 
> > I don't think we should expect a bar for AI that is higher than the
> > one we set for humans.
> 
> I'm not, rather I'm saying let's be aware of the kinds of issues we
> might encounter from LLMs and take them into account when
> establishing policy.

Well, if we set a policy, it should be flexible enough to adapt as the
AI does and not be locked to what would prevent the AI mistakes I can
find today from happening.  If we're going to codify this rigidly we
could arguably have a policy not to accept patches from humans who
might be (and often are) wrong as well.

I think we should stick to indicators of trustworthiness that AI is
already generating at let that guide maintainer taste without
necessarily having something more detailed.

Regards,

James