[PATCH 06/13] rust: start supporting several compiler versions

[PATCH 06/13] rust: start supporting several compiler versions

[Miguel Ojeda]

It is time to start supporting several Rust compiler versions and thus
establish a minimum Rust version.

We may still want to upgrade the minimum sometimes in the beginning since
there may be important features coming into the language that improve
how we write code (e.g. field projections), which may or may not make
sense to support conditionally.

We will start with a window of two stable releases, and widen it over
time. Thus this patch does not move the current minimum (1.78.0), but
instead adds support for the recently released 1.79.0.

This should already be enough for kernel developers in distributions that
provide recent Rust compiler versions routinely, such as Arch Linux,
Debian Unstable (outside the freeze period), Fedora Linux, Gentoo
Linux (especially the testing channel), Nix (unstable) and openSUSE
Tumbleweed. See the documentation patch about it later in this series.

In addition, Rust for Linux is now being built-tested in Rust's pre-merge
CI [1]. That is, every change that is attempting to land into the Rust
compiler is tested against the kernel, and it is merged only if it passes
-- thanks to the Rust project for that!

Thus, with the pre-merge CI in place, both projects hope to avoid
unintentional changes to Rust that break the kernel. This means that,
in general, apart from intentional changes on their side (that we will
need to workaround conditionally on our side), the upcoming Rust compiler
versions should generally work.

For instance, currently, the beta (1.80.0) and nightly (1.81.0) branches
work as well.

Of course, the Rust for Linux CI job in the Rust toolchain may still need
to be temporarily disabled for different reasons, but the intention is
to help bring Rust for Linux into stable Rust.

Link: https://github.com/rust-lang/rust/pull/125209 [1]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
---
 Documentation/process/changes.rst  | 4 +---
 Documentation/rust/quick-start.rst | 9 ++++-----
 scripts/rust_is_available.sh       | 8 --------
 scripts/rust_is_available_test.py  | 5 -----
 4 files changed, 5 insertions(+), 21 deletions(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 5685d7bfe4d0..0d0b7120792b 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -88,9 +88,7 @@ docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`.
 Rust (optional)
 ---------------
 
-A particular version of the Rust toolchain is required. Newer versions may or
-may not work because the kernel depends on some unstable Rust features, for
-the moment.
+A recent version of the Rust compiler is required.
 
 Each Rust toolchain comes with several "components", some of which are required
 (like ``rustc``) and some that are optional. The ``rust-src`` component (which
diff --git a/Documentation/rust/quick-start.rst b/Documentation/rust/quick-start.rst
index cc3f11e0d441..139a8a536838 100644
--- a/Documentation/rust/quick-start.rst
+++ b/Documentation/rust/quick-start.rst
@@ -29,16 +29,15 @@ if that is the case.
 rustc
 *****
 
-A particular version of the Rust compiler is required. Newer versions may or
-may not work because, for the moment, the kernel depends on some unstable
-Rust features.
+A recent version of the Rust compiler is required.
 
 If ``rustup`` is being used, enter the kernel build directory (or use
-``--path=<build-dir>`` argument to the ``set`` sub-command) and run::
+``--path=<build-dir>`` argument to the ``set`` sub-command) and run,
+for instance::
 
 	rustup override set $(scripts/min-tool-version.sh rustc)
 
-This will configure your working directory to use the correct version of
+This will configure your working directory to use the given version of
 ``rustc`` without affecting your default toolchain.
 
 Note that the override applies to the current working directory (and its
diff --git a/scripts/rust_is_available.sh b/scripts/rust_is_available.sh
index 117018946b57..67cb900124cc 100755
--- a/scripts/rust_is_available.sh
+++ b/scripts/rust_is_available.sh
@@ -117,14 +117,6 @@ if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then
 	echo >&2 "***"
 	exit 1
 fi
-if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
-	echo >&2 "***"
-	echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
-	echo >&2 "***   Your version:     $rust_compiler_version"
-	echo >&2 "***   Expected version: $rust_compiler_min_version"
-	echo >&2 "***"
-	warning=1
-fi
 
 # Check that the Rust bindings generator is suitable.
 #
diff --git a/scripts/rust_is_available_test.py b/scripts/rust_is_available_test.py
index 57613fe5ed75..a255f79aafc2 100755
--- a/scripts/rust_is_available_test.py
+++ b/scripts/rust_is_available_test.py
@@ -193,11 +193,6 @@ else:
         result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
         self.assertIn(f"Rust compiler '{rustc}' is too old.", result.stderr)
 
-    def test_rustc_new_version(self):
-        rustc = self.generate_rustc("rustc 1.999.0 (a8314ef7d 2099-06-27)")
-        result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "RUSTC": rustc })
-        self.assertIn(f"Rust compiler '{rustc}' is too new. This may or may not work.", result.stderr)
-
     def test_bindgen_nonexecutable(self):
         result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.nonexecutable })
         self.assertIn(f"Running '{self.nonexecutable}' to check the Rust bindings generator version failed with", result.stderr)
-- 
2.45.2

[PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Miguel Ojeda]

Now that we are starting to support several Rust compiler and `bindgen`
versions, there is a good chance some Linux distributions work out of
the box.

Thus, provide some instructions on how to set the toolchain up for a
few major Linux distributions. This simplifies the setup users need to
build the kernel.

In addition, add an introduction to the document so that it is easier
to understand its structure. We may want to reorganize it or split it
in the future, but I wanted to focus this commit on the new information
added about each particular distribution.

Finally, remove the `rustup`'s components mention in `changes.rst` since
users do not need it if they install the toolchain via the distributions
(and anyway it was too detailed for that main document).

Cc: Jan Alexander Steffens <heftig@archlinux.org>
Cc: Johannes Löthberg <johannes@kyriasis.com>
Cc: Fabian Grünbichler <debian@fabian.gruenbichler.email>
Cc: Josh Stone <jistone@redhat.com>
Cc: Randy Barlow <randy@electronsweatshop.com>
Cc: Anna (navi) Figueiredo Gomes <navi@vlhl.dev>
Cc: Matoro Mahri <matoro_gentoo@matoro.tk>
Cc: Ryan Scheel <ryan.havvy@gmail.com>
Cc: figsoda <figsoda@pm.me>
Cc: Jörg Thalheim <joerg@thalheim.io>
Cc: Theodore Ni <43ngvg@masqt.com>
Cc: Winter <nixos@winter.cafe>
Cc: William Brown <wbrown@suse.de>
Cc: Xiaoguang Wang <xiaoguang.wang@suse.com>
Cc: Andrea Righi <andrea.righi@canonical.com>
Cc: Zixing Liu <zixing.liu@canonical.com>
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
---
Note that openSUSE Tumbleweed is adding the Rust standard library source
code, `rustfmt` and Clippy, so we may be able to drop that small note in
their section in a few days (possibly before this gets applied) --
thanks William!

 Documentation/process/changes.rst  |  5 --
 Documentation/rust/quick-start.rst | 83 ++++++++++++++++++++++++++++--
 2 files changed, 78 insertions(+), 10 deletions(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 0d0b7120792b..0ce96ae2588c 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -90,11 +90,6 @@ Rust (optional)
 
 A recent version of the Rust compiler is required.
 
-Each Rust toolchain comes with several "components", some of which are required
-(like ``rustc``) and some that are optional. The ``rust-src`` component (which
-is optional) needs to be installed to build the kernel. Other components are
-useful for developing.
-
 Please see Documentation/rust/quick-start.rst for instructions on how to
 satisfy the build requirements of Rust support. In particular, the ``Makefile``
 target ``rustavailable`` is useful to check why the Rust toolchain may not
diff --git a/Documentation/rust/quick-start.rst b/Documentation/rust/quick-start.rst
index f624b913ff89..65a34a9a7089 100644
--- a/Documentation/rust/quick-start.rst
+++ b/Documentation/rust/quick-start.rst
@@ -5,17 +5,90 @@ Quick Start
 
 This document describes how to get started with kernel development in Rust.
 
+There are a few ways to install a Rust toolchain needed for kernel development.
+A simple way is to use the packages from your Linux distribution if they are
+suitable -- the first section below explains this approach. An advantage of this
+approach is that, typically, the distribution will match the LLVM used by Rust
+and Clang.
+
+Alternatively, the next two "Requirements" sections explain each component and
+how to install them through ``rustup``, the standalone installers from Rust
+and/or building them.
+
+The rest of the document explains other aspects on how to get started.
+
+
+Distributions
+-------------
+
+Arch Linux
+**********
+
+Arch Linux provides recent Rust releases and thus it should generally work out
+of the box, e.g.::
+
+	pacman -S rust rust-src rust-bindgen
+
+
+Debian
+******
+
+Debian Unstable (Sid), outside of the freeze period, provides recent Rust
+release and thus it should generally work out of the box, e.g.::
+
+	apt install rustc rust-src bindgen rustfmt rust-clippy
+
+
+Fedora Linux
+************
+
+Fedora Linux provides recent Rust releases and thus it should generally work out
+of the box, e.g.::
+
+	dnf install rust rust-src bindgen-cli rustfmt clippy
+
+
+Gentoo Linux
+************
+
+Gentoo Linux (and especially the testing branch) provides recent Rust releases
+and thus it should generally work out of the box, e.g.::
+
+	USE='rust-src rustfmt clippy' emerge dev-lang/rust dev-util/bindgen
+
+``LIBCLANG_PATH`` may need to be set.
+
+
+Nix
+***
+
+Nix (unstable channel) provides recent Rust releases and thus it should
+generally work out of the box, e.g.::
+
+	{ pkgs ? import <nixpkgs> {} }:
+	pkgs.mkShell {
+	  nativeBuildInputs = with pkgs; [ rustc rust-bindgen rustfmt clippy ];
+	  RUST_LIB_SRC = "${pkgs.rust.packages.stable.rustPlatform.rustLibSrc}";
+	}
+
+
+openSUSE
+********
+
+openSUSE Tumbleweed provides recent Rust releases and thus it should generally
+work out of the box, e.g.::
+
+	zypper install rust rust-bindgen clang
+
+The Rust standard library source code, ``rustfmt`` and Clippy are not packaged
+(yet), thus currently they need to be installed separately.
+
 
 Requirements: Building
 ----------------------
 
 This section explains how to fetch the tools needed for building.
 
-Some of these requirements might be available from Linux distributions
-under names like ``rustc``, ``rust-src``, ``rust-bindgen``, etc. However,
-at the time of writing, they are likely not to be recent enough unless
-the distribution tracks the latest releases.
-
 To easily check whether the requirements are met, the following target
 can be used::
 
-- 
2.45.2

Re: [PATCH 06/13] rust: start supporting several compiler versions

[Miguel Ojeda]

On Thu, Jul 4, 2024 at 4:45 PM Finn Behrens <me@kloenk.dev> wrote:
>
> One idea below, but fine with or without it.
>
> Reviewed-by: Finn Behrens <me@kloenk.dev>

Thanks Finn!

> Would it make sense to not advise the min version but the “max” or just latest stable version? Min version should work as well latest could potentially be a bit more optimised if both work either way?

Yeah, that is a good idea -- we could just say `stable` there.

In fact, it makes more sense, because this is using `rustup`, i.e. the
Rust-provided binaries, and the Rust project only supports the latest
version (so far -- there are proposals about LTSs there).

Cheers,
Miguel

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Andrea Righi]

On Mon, Jul 01, 2024 at 08:36:23PM +0200, Miguel Ojeda wrote:
..
> +Debian
> +******
> +
> +Debian Unstable (Sid), outside of the freeze period, provides recent Rust
> +release and thus it should generally work out of the box, e.g.::
> +
> +	apt install rustc rust-src bindgen rustfmt rust-clippy

This implicitly covers Ubuntu, since packages are sync'ed with Debian.

In addition to that Ubuntu also provides versioned packages (such as
rustc-1.74, bindgen-0.65, etc.), so in case of special requirements
(e.g., older kernels) users should be able to install the required
version(s) using the packages provided by the distro.

Maybe it's worth mentioning as a little note, so that users are aware of
these extra packages.

Thanks,
-Andrea

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Fabian Grünbichler]

On Fri, Jul 05, 2024 at 08:19:01AM GMT, Andrea Righi wrote:
> On Mon, Jul 01, 2024 at 08:36:23PM +0200, Miguel Ojeda wrote:
> ..
> > +Debian
> > +******
> > +
> > +Debian Unstable (Sid), outside of the freeze period, provides recent Rust
> > +release and thus it should generally work out of the box, e.g.::
> > +
> > +	apt install rustc rust-src bindgen rustfmt rust-clippy
> 
> This implicitly covers Ubuntu, since packages are sync'ed with Debian.
> 
> In addition to that Ubuntu also provides versioned packages (such as
> rustc-1.74, bindgen-0.65, etc.), so in case of special requirements
> (e.g., older kernels) users should be able to install the required
> version(s) using the packages provided by the distro.

Debian (for building firefox and chromium), and uses a -web suffix for
that:

https://tracker.debian.org/pkg/rustc-web
https://tracker.debian.org/pkg/rust-cbindgen-web

cheers,
Fabian

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Miguel Ojeda]

On Fri, Jul 5, 2024 at 8:19 AM Andrea Righi <righi.andrea@gmail.com> wrote:
>
> This implicitly covers Ubuntu, since packages are sync'ed with Debian.

Do you mean the names (i.e. command) is the same, or that the newer
versions from e.g. Sid can be used in Ubuntu? If the latter, that
would be definitely worth adding, yeah -- is that supported / expected
to work?

> In addition to that Ubuntu also provides versioned packages (such as
> rustc-1.74, bindgen-0.65, etc.), so in case of special requirements
> (e.g., older kernels) users should be able to install the required
> version(s) using the packages provided by the distro.
>
> Maybe it's worth mentioning as a little note, so that users are aware of
> these extra packages.

I thought about adding a section for Ubuntu to mention those -- so far
I only added the distributions/commands that were likely to work with
the versions supported by the kernel, with the idea of expand later as
time passes. So I didn't add the versioned ones since the latest is
1.76, so it wouldn't work for the current kernel.

But it may be useful to state it nevertheless, since some people may
only look at the latest docs. (Or perhaps Ubuntu can add Rust 1.79? :)

Thanks!

Cheers,
Miguel

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Miguel Ojeda]

On Fri, Jul 5, 2024 at 8:47 AM Fabian Grünbichler
<debian@fabian.gruenbichler.email> wrote:
>
> Debian (for building firefox and chromium), and uses a -web suffix for
> that:
>
> https://tracker.debian.org/pkg/rustc-web

Like for Ubuntu's versioned ones, it seems not recent enough at the
moment (i.e. for the current kernel), if I understand correctly.

Also, it is a single version, i.e. 1.70, right? We didn't use that
particular version in any old kernel version (we moved from 1.68.2 to
1.71.1 in commit 89eed1ab1161 ("rust: upgrade to Rust 1.71.1") back
then). So I am not sure how useful it would be, but if you think it
will be in the future, perhaps it is worth mentioning.

> https://tracker.debian.org/pkg/rust-cbindgen-web

We may use `cbindgen` too in the future, but not currently.

Is something like `bindgen-web` also available for stable?

Thanks!

Cheers,
Miguel

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Andrea Righi]

On Fri, Jul 05, 2024 at 12:50:35PM +0200, Miguel Ojeda wrote:
> On Fri, Jul 5, 2024 at 8:19 AM Andrea Righi <righi.andrea@gmail.com> wrote:
> >
> > This implicitly covers Ubuntu, since packages are sync'ed with Debian.
> 
> Do you mean the names (i.e. command) is the same, or that the newer
> versions from e.g. Sid can be used in Ubuntu? If the latter, that
> would be definitely worth adding, yeah -- is that supported / expected
> to work?

Command and package names are the same. We may also have different
(newer) versions than Sid, but this is up to foundations team.

Zixing, what do you think?

> 
> > In addition to that Ubuntu also provides versioned packages (such as
> > rustc-1.74, bindgen-0.65, etc.), so in case of special requirements
> > (e.g., older kernels) users should be able to install the required
> > version(s) using the packages provided by the distro.
> >
> > Maybe it's worth mentioning as a little note, so that users are aware of
> > these extra packages.
> 
> I thought about adding a section for Ubuntu to mention those -- so far
> I only added the distributions/commands that were likely to work with
> the versions supported by the kernel, with the idea of expand later as
> time passes. So I didn't add the versioned ones since the latest is
> 1.76, so it wouldn't work for the current kernel.
> 
> But it may be useful to state it nevertheless, since some people may
> only look at the latest docs. (Or perhaps Ubuntu can add Rust 1.79? :)

Actually, considering that this doc will go in newer kernels, probably
it's not worth to mention the *old* packages.

About Rust 1.79, calling out Zixing again... :) He may have newer Rust
packages in his ppa at some point, but I don't know how much we can
consider that ppa "official".

-Andrea

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Fabian Grünbichler]

On Fri, Jul 05, 2024 at 02:52:44PM GMT, Miguel Ojeda wrote:
> On Fri, Jul 5, 2024 at 8:47 AM Fabian Grünbichler
> <debian@fabian.gruenbichler.email> wrote:
> >
> > Debian (for building firefox and chromium), and uses a -web suffix for
> > that:
> >
> > https://tracker.debian.org/pkg/rustc-web
> 
> Like for Ubuntu's versioned ones, it seems not recent enough at the
> moment (i.e. for the current kernel), if I understand correctly.

yep.

> Also, it is a single version, i.e. 1.70, right? We didn't use that
> particular version in any old kernel version (we moved from 1.68.2 to
> 1.71.1 in commit 89eed1ab1161 ("rust: upgrade to Rust 1.71.1") back
> then). So I am not sure how useful it would be, but if you think it
> will be in the future, perhaps it is worth mentioning.

right now it is updated whenever its rdeps (firefox and chromium in
stable) need newer versions. once e.g. a stable Debian kernel has
similar requirements, I guess the same would apply there. I am not sure
upstream kernel development on Debian stable would be enough of an
argument to update it (or provide similar packages), but I am not a
member of the teams that would make that decision.

we currently don't provide multiple versions in parallel (like GCC or
LLVM though) besides this special affordance for browsers. I am not sure
this would be a good idea either given rustc's release cadence ;)

as discussed off-list, -backports might be a better place for providing
more recent toolchain packages on Debian stable, and independent from
this thread, I have pondered providing them there in the past already.
backports would only ship one version as well though, and at most the
one in testing (so it would also be affected by the freeze period, just
like unstable and testing).

> > https://tracker.debian.org/pkg/rust-cbindgen-web
> 
> We may use `cbindgen` too in the future, but not currently.

it's used by firefox :)

> Is something like `bindgen-web` also available for stable?

not yet, but could conceivably be provided once/iff there is a need (see
above).

Re: [PATCH 13/13] docs: rust: quick-start: add section on Linux distributions

[Miguel Ojeda]

On Fri, Jul 5, 2024 at 3:09 PM Fabian Grünbichler
<debian@fabian.gruenbichler.email> wrote:
>
> right now it is updated whenever its rdeps (firefox and chromium in
> stable) need newer versions. once e.g. a stable Debian kernel has
> similar requirements, I guess the same would apply there. I am not sure
> upstream kernel development on Debian stable would be enough of an
> argument to update it (or provide similar packages), but I am not a
> member of the teams that would make that decision.

That is reasonable.

> as discussed off-list, -backports might be a better place for providing
> more recent toolchain packages on Debian stable, and independent from
> this thread, I have pondered providing them there in the past already.
> backports would only ship one version as well though, and at most the
> one in testing (so it would also be affected by the freeze period, just
> like unstable and testing).

I imagine that could be useful for some (kernel or not) developers on
stable, even with the freeze period.

Thanks!

Cheers,
Miguel

Re: [PATCH 06/13] rust: start supporting several compiler versions

[Miguel Ojeda]

On Wed, Jul 3, 2024 at 7:20 PM Andreas Hindborg <nmi@metaspace.dk> wrote:
>
> This does not set a max version. Will older versions of the kernel
> always compile with newer releases of `rustc`?

That depends on how many changes Rust introduces around the remaining
unstable features (as well as plain bugs in new versions, like for any
other tool). With the kernel being tested in the Rust pre-merge CI,
hopefully we will reduce those to a minimum.

It is also why we need to keep reducing the unstable features and get
into stable Rust as soon as possible. The Rust project is starting to
support us more here, and they are setting up a project goal for
2024h2 on their side to resolve the biggest blockers for the kernel:

    https://rust-lang.github.io/rust-project-goals/2024h2/rfl_stable.html

Meanwhile, on our side, we can do our best to backport any workarounds
for new versions into supported kernels, i.e. stable and LTSs.

But, at least, this series should already help kernel developers (they
will be able to compile the kernel with Rust enabled using their
distribution) and Linux distributions (they will have more freedom to
pick a Rust toolchain version and avoid having to package another
version just for the kernel).

Cheers,
Miguel

Re: [PATCH 06/13] rust: start supporting several compiler versions

[Andreas Hindborg]

Miguel Ojeda <ojeda@kernel.org> writes:

[...]

> diff --git a/scripts/rust_is_available.sh b/scripts/rust_is_available.sh
> index 117018946b57..67cb900124cc 100755
> --- a/scripts/rust_is_available.sh
> +++ b/scripts/rust_is_available.sh
> @@ -117,14 +117,6 @@ if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then
>  	echo >&2 "***"
>  	exit 1
>  fi
> -if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
> -	echo >&2 "***"
> -	echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
> -	echo >&2 "***   Your version:     $rust_compiler_version"
> -	echo >&2 "***   Expected version: $rust_compiler_min_version"
> -	echo >&2 "***"
> -	warning=1
> -fi

This does not set a max version. Will older versions of the kernel
always compile with newer releases of `rustc`?

Best regards,
Andreas