[RFC] AI review prompt updates

[Chris Mason <clm@meta.com>]

Hi everyone,

If you haven't been playing with my AI review prompts locally, this
probably isn't relevant.

The prompts are: https://github.com/masoncl/review-prompts

For anyone that is using the prompts, I'm hoping to get some feedback on
changes I pushed today that break the review up into individual tasks.
The previous prompts are all still there, so we can directly compare the
two in terms of time, token cost, and effectiveness.

Using tasks allows us to break up large diffs into smaller chunks, and
review each chunk individually.  This ends up using fewer tokens a lot
of the time, because we're not sending context back and forth for the
entire diff with every turn.  It also catches more bugs all around.

I'm using a python script to break the changes up, and do bulk
processing to pull out modified functions, types, and call graphs.  This
also helps reduce token usage, we're just telling AI to load the
processed results in bulk instead of making it discover the same
information on its own.  It's probably buggy, I'm sure I'll have to
refine the python from here.

All those tokens saved hopefully helps make up for extra tokens spent.
Each task has its own context window, so research done while processing
one file in the diff has to be redone if another file in the diff needs
the same functions loaded.  AI providers cache tokens so it kind of
works out, but I'm looking for feedback on this part especially.

The new setup has tasks for:

- Reviewing chunks of code
- Checking past lore threads (if semcode is available)
- Checking Fixes: tags
- Deep dive into syzkaller fixes (more on that below)
- Making the final report

To use the old prompts:

cd some_linux_dir
claude> read the prompt <path to review-prompts>/kernel/review-core.md
and run it on the HEAD commit

To use the new prompts

cp kernel/scripts/create_changes.py somewhere in your PATH,
or run it by hand:

cd some_linux_dir
# the prompts will run create_changes.py if you don't
kernel/scripts/create_changes.py HEAD

claude> read the prompt <path to review-prompts>/kernel/agent/orc.md and
run it on the HEAD commit

I'm sure proper gemini support will need more changes in the prompts,
please feel free to send fixes/improvements.

re: syzkaller, the prompts focus on finding bugs in proposed changes,
but they don't try to decide if the thesis behind a patch is correct.
With syzkaller, it helps to have some extra rigor around making sure the
bug described is real.

So, any time a commit message mentions syzbot/syzkaller, it gets turn
through extra validation.  I'm sure this will need a lot of fine tuning,
but it does catch some good issues. (thanks to Johannes Weiner for the
suggestion!)

-chris