Re: [MAINTAINERS / KERNEL SUMMIT] AI patch review tools

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Thu, 9 Oct 2025 11:56:23 -0700
Linus Torvalds <torvalds@linux-foundation.org> escreveu:

> On Thu, 9 Oct 2025 at 11:43, Chris Mason <clm@meta.com> wrote:
> >
> > I think it's also important to remember that AI is sometimes wildly
> > wrong.  Having the reviews show up on a list where more established
> > developers can call bullshit definitely helps protect against wasting
> > people's time.  
> 
> I really want any AI effort to be primarily geared towards
> maintainers, not end developers.
> 
> And this is a large part of the reason. I think we've all seen the
> garbage end of AI, and how it can generate more work rather than less.
> 
> We see the downsides on the security list where people use AI to
> generate questionable reports, we see it in various general bug
> handling, and we've seen it on the infrastructure where the AI tools
> cause huge issues.

I recently read an article about LLM Needle-In-a-Haystack issues
with current models(*). The problem is that, even when they do support
big input tokens (with is proportional to the number of statements at
the program code), current AI models usually loses needles (for instance 
a bad code) when the context is bigger than just a small percentage
of the maximum context(*).

(*) I didn't save the link of the article I read, but this one
    https://arxiv.org/html/2406.10149v2 is somewhat similar.
    Basically, even when the model supports large contexts,
    the probability of losing a needle in a stack increases
    with the input size in a way that, even a relatively small
    input can cause false negatives. See this comment at figure
    1:

    "GPT-4 effectively uses only about 10% of the full 128K window."

So, yeah, if one uses AI to review patches, there is a high risk
of mistakes, meaning a maintainer or an experienced developer is needed 
to review "AI reviews".

> Fairly recently Jens gave up maintaining his own git server just
> because of the horror that is AI scraping tools.

This is a different issue. I needed solve such issues on linuxTV.org 
as well. The problem there doesn't seem due to someone giving AI some
prompt, but, instead, due to bad automation used by LLM training
that behaves like a DDoS attack, trying to retrieve all possible
existing and previously-existing pages continuously, using
multiple connections and from different ip addresses.
> 
> Yes, that scraping issue a "secondary" kind of "more work rather than
> less", but it's very much a real issue nonetheless. Just the
> infrastructure load is a real thing.
> 
> Some long term goal may well be to help end developers, but I really
> think it should not be seen as any kind of primary goal for now and
> shouldn't even be on anybody's radar as a design issue.
> 
> Exactly because I think we need that "established developers can call
> bullshit" without being overwhelmed by said BS from external sources.
> 
> So I think that only once any AI tools are actively helping
> maintainers in a day-to-day workflow should people even *look* at
> having non-maintainers use them.

For bug reviews, agreed. AI can still be helpful for developers to
do things like writing unit tests and more trivial tasks, provided
that will be reviewed/fixed by the develper using such tools.
> 
> And I say that as somebody who does think it's going to be a big help.
> I know there are lots of less optimistic people around.
> 
>              Linus
>