[LSF/MM/BPF TOPIC] Evaluating Rust for XArray

[Daniel Gomez <da.gomez@kernel.org>]

We propose a topic to discuss the potential use of Rust in core Linux
kernel data structures.

The XArray unified the kernel's radix trees into one API back in v4.20
(2018). It sits under the page cache, block layer, filesystems, GPU
drivers, and dozens of other subsystems.

We looked at XArray bugs since 2018. About half are the usual suspects:
races, resource leaks, encoding and overflow mistakes. The kind of bugs
Greg says make up 60-80% of kernel security bugs [1]. Most of the rest
are complex logic bugs where Rust's type system may help but cannot
eliminate entirely.

The motivation for our interest in evaluating Rust for use in the XArray
started with a bug in mm/filemap.c that corrupted page cache data [2].
Consumer code called xa_get_order() without the lock, the tree changed,
and pages from wrong files ended up in the cache. No CVE was assigned
to this issue*, and no reproducer outside production existed before the
fix (a synthetic reproducer was only created [5][6] after the issue was
escalated). Kairui Song's refactoring and optimization patches [7][8][9]
happened to eliminate the issue.

In Rust, xa_get_order() would be a method on the lock guard. XArray
users would not be able to call it without holding the lock because
the type system would not let them. That does not help with the complex
logic bugs, but it eliminates the class of mistakes where the API is
used outside its required context.

We have a Rust XArray prototype that implements core operations designed
around Rust's ownership model. By the conference we aim to benchmark
it against tools/testing/radix-tree/ and use it to back the Rust null
block driver for A/B testing performance against the C implementation.
In the longer term, we plan to deploy the Rust implementation in the
page cache for A/B testing performance of generic workloads. We invite
the community to take a look at the code (when published), which at the
moment is rather simple as it only covers the core operations.

We know that Rust is not magic. The first kernel Rust CVE was a race
in an unsafe block where the safety comment was wrong [10]. However, we
believe that constraining potential memory safety issues to small unsafe
blocks provides a significant advantage over writing in C, where the
entire program text is to be considered one big unsafe block.

With Rust now part of the kernel's core infrastructure [11], we would
like to explore whether Rust is applicable for use in core MM data
structures. We invite the community to this exploration and we would
like to start the discussion. Specifically, we seek insights into:

 - Which workloads set the performance bar?
 - Would a type-safe API reduce consumer bugs, or would they just move
elsewhere?
 - Would rewriting core data structures introduce new, previously
unseen, bugs**?

*Under the Linux kernel CNA's CVE assignment policy, data corruption
issues do not meet the cve.org definition of a vulnerability and are
therefore not eligible for CVE assignment [3][4].

**Our thought: The existing test suite covers years of real edge cases.
A second implementation running against it is not just a rewrite, it is
differential testing.

Link: https://osskorea2025.sched.com/event/296sD/keynote-rust-in-the-linux-kernel-why-greg-kroah-hartman-linux-kernel-maintainer-and-fellow-the-linux-foundation [1]
Link: https://lore.kernel.org/all/A5A976CB-DB57-4513-A700-656580488AB6@flyingcircus.io [2]
Link: https://www.cve.org/ResourcesSupport/Glossary#glossaryVulnerability [3]
Link: http://www.kroah.com/log/blog/2026/02/16/linux-cve-assignment-process [4]
Link: https://lore.kernel.org/all/d4a1cca4-96b8-4692-81f0-81c512f55ccf@meta.com [5]
Link: https://lore.kernel.org/all/5bee194c-9cd3-47e7-919b-9f352441f855@kernel.dk [6]
Link: https://git.kernel.org/torvalds/c/de60fd8ddeda2 [7]
Link: https://git.kernel.org/torvalds/c/a4864671ca0bf [8]
Link: https://git.kernel.org/torvalds/c/6758c1128ceb4 [9]
Link: https://lore.kernel.org/all/2025121614-CVE-2025-68260-558d@gregkh [10]
Link: https://lore.kernel.org/all/20251213000042.23072-1-ojeda@kernel.org [11]