* [PATCH v4 0/2] scripts: introduce containerized builds
@ 2026-01-22 14:06 Guillaume Tucker
2026-01-22 14:06 ` [PATCH v4 1/2] scripts: add tool to run " Guillaume Tucker
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Guillaume Tucker @ 2026-01-22 14:06 UTC (permalink / raw)
To: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Onur Özkan
Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
linux-kbuild, automated-testing, workflows, llvm
This proposal emerged from discussions over email and after a talk at
Plumbers 2024:
https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
The aim is to facilitate reproducing builds for CI bots as well as
developers using containers. Here's an illustrative example with a
kernel.org toolchain in a Docker image from tuxmake:
$ scripts/container -i docker.io/tuxmake/korg-clang-21 make LLVM=1 defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
[...]
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'x86_64_defconfig'
#
# configuration written to .config
#
This patch series also includes a documentation page with all the
relevant details and further examples about how to use the tool.
To go one step further, I'm in the process of preparing reference
container images with kernel.org toolchains and no third-party
dependencies other than the base Debian distro. See this thread for
more details and options to host them in an upstream way:
https://lore.kernel.org/all/cc737636-2a43-4a97-975e-4725733f7ee4@gtucker.io/
Say, to run KUnit using the latest kernel.org GCC toolchain:
scripts/container --shell \
-i registry.gitlab.com/gtucker/korg-containers/gcc:kunit -- \
tools/testing/kunit/kunit.py \
run \
--arch=x86_64 \
--cross_compile=x86_64-linux-
---
Changes in v4:
- Add entries to MAINTAINERS for the new script and docs
- Give priority to Podman over Docker when no -r option given
- Update help message and docs regarding Podman priority
- Add note and workaround for out-of-tree builds in the docs
- Mention TuxMake prebuilt images more explicitly in the docs
Changes in v3:
- Refactor common code for Docker and Podman
- Add docs.kernel.org URL in help message
- Use pathlib Python package
- Handle signals in parent process by default
- Add --shell option to use an interactive shell
- Tweak debug messages in verbose mode
- Specify Python 3.10 as minimum version in the docs
- Provide an example env file in the docs
- Update docs regarding interactive shell usage
Changes in v2:
- Drop default image but make -i option required
- Look for Docker and Podman if no runtime specified
- Catch SIGINT from user to abort container with Docker
- Explicitly name each container with a UUID
- Update documentation accordingly
---
Guillaume Tucker (2):
scripts: add tool to run containerized builds
Documentation: dev-tools: add container.rst page
Documentation/dev-tools/container.rst | 227 ++++++++++++++++++++++++++
Documentation/dev-tools/index.rst | 1 +
MAINTAINERS | 7 +
scripts/container | 199 ++++++++++++++++++++++
4 files changed, 434 insertions(+)
create mode 100644 Documentation/dev-tools/container.rst
create mode 100755 scripts/container
--
2.47.3
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 14:06 [PATCH v4 0/2] scripts: introduce containerized builds Guillaume Tucker
@ 2026-01-22 14:06 ` Guillaume Tucker
2026-01-22 14:29 ` Onur Özkan
2026-01-22 14:07 ` [PATCH v4 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
2026-01-30 0:10 ` [PATCH v4 0/2] scripts: introduce containerized builds Nathan Chancellor
2 siblings, 1 reply; 10+ messages in thread
From: Guillaume Tucker @ 2026-01-22 14:06 UTC (permalink / raw)
To: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Onur Özkan
Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
linux-kbuild, automated-testing, workflows, llvm
Add a 'scripts/container' tool written in Python to run any command in
the source tree from within a container. This can typically be used
to call 'make' with a compiler toolchain image to run reproducible
builds but any arbitrary command can be run too. Only Docker and
Podman are supported in this initial version.
Add a new entry to MAINTAINERS accordingly.
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nicolas Schier <nsc@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Link: https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
MAINTAINERS | 6 ++
scripts/container | 199 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 205 insertions(+)
create mode 100755 scripts/container
diff --git a/MAINTAINERS b/MAINTAINERS
index da9dbc1a4019..affd55ff05e0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6384,6 +6384,11 @@ S: Supported
F: drivers/video/console/
F: include/linux/console*
+CONTAINER BUILD SCRIPT
+M: Guillaume Tucker <gtucker@gtucker.io>
+S: Maintained
+F: scripts/container
+
CONTEXT TRACKING
M: Frederic Weisbecker <frederic@kernel.org>
M: "Paul E. McKenney" <paulmck@kernel.org>
@@ -13676,6 +13681,7 @@ F: scripts/Makefile*
F: scripts/bash-completion/
F: scripts/basic/
F: scripts/clang-tools/
+F: scripts/container
F: scripts/dummy-tools/
F: scripts/include/
F: scripts/mk*
diff --git a/scripts/container b/scripts/container
new file mode 100755
index 000000000000..09663eccb8d3
--- /dev/null
+++ b/scripts/container
@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2025 Guillaume Tucker
+
+"""Containerized builds"""
+
+import abc
+import argparse
+import logging
+import os
+import pathlib
+import shutil
+import subprocess
+import sys
+import uuid
+
+
+class ContainerRuntime(abc.ABC):
+ """Base class for a container runtime implementation"""
+
+ name = None # Property defined in each implementation class
+
+ def __init__(self, args, logger):
+ self._uid = args.uid or os.getuid()
+ self._gid = args.gid or args.uid or os.getgid()
+ self._env_file = args.env_file
+ self._shell = args.shell
+ self._logger = logger
+
+ @classmethod
+ def is_present(cls):
+ """Determine whether the runtime is present on the system"""
+ return shutil.which(cls.name) is not None
+
+ @abc.abstractmethod
+ def _do_run(self, image, cmd, container_name):
+ """Runtime-specific handler to run a command in a container"""
+
+ @abc.abstractmethod
+ def _do_abort(self, container_name):
+ """Runtime-specific handler to abort a running container"""
+
+ def run(self, image, cmd):
+ """Run a command in a runtime container"""
+ container_name = str(uuid.uuid4())
+ self._logger.debug("container: %s", container_name)
+ try:
+ return self._do_run(image, cmd, container_name)
+ except KeyboardInterrupt:
+ self._logger.error("user aborted")
+ self._do_abort(container_name)
+ return 1
+
+
+class CommonRuntime(ContainerRuntime):
+ """Common logic for Docker and Podman"""
+
+ def _do_run(self, image, cmd, container_name):
+ cmdline = [self.name, 'run']
+ cmdline += self._get_opts(container_name)
+ cmdline.append(image)
+ cmdline += cmd
+ self._logger.debug('command: %s', ' '.join(cmdline))
+ return subprocess.call(cmdline)
+
+ def _get_opts(self, container_name):
+ opts = [
+ '--name', container_name,
+ '--rm',
+ '--volume', f'{pathlib.Path.cwd()}:/src',
+ '--workdir', '/src',
+ ]
+ if self._env_file:
+ opts += ['--env-file', self._env_file]
+ if self._shell:
+ opts += ['--interactive', '--tty']
+ return opts
+
+ def _do_abort(self, container_name):
+ subprocess.call([self.name, 'kill', container_name])
+
+
+class DockerRuntime(CommonRuntime):
+ """Run a command in a Docker container"""
+
+ name = 'docker'
+
+ def _get_opts(self, container_name):
+ return super()._get_opts(container_name) + [
+ '--user', f'{self._uid}:{self._gid}'
+ ]
+
+
+class PodmanRuntime(CommonRuntime):
+ """Run a command in a Podman container"""
+
+ name = 'podman'
+
+ def _get_opts(self, container_name):
+ return super()._get_opts(container_name) + [
+ '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
+ ]
+
+
+class Runtimes:
+ """List of all supported runtimes"""
+
+ runtimes = [PodmanRuntime, DockerRuntime]
+
+ @classmethod
+ def get_names(cls):
+ """Get a list of all the runtime names"""
+ return list(runtime.name for runtime in cls.runtimes)
+
+ @classmethod
+ def get(cls, name):
+ """Get a single runtime class matching the given name"""
+ for runtime in cls.runtimes:
+ if runtime.name == name:
+ if not runtime.is_present():
+ raise ValueError(f"runtime not found: {name}")
+ return runtime
+ raise ValueError(f"unknown runtime: {runtime}")
+
+ @classmethod
+ def find(cls):
+ """Find the first runtime present on the system"""
+ for runtime in cls.runtimes:
+ if runtime.is_present():
+ return runtime
+ raise ValueError("no runtime found")
+
+
+def _get_logger(verbose):
+ """Set up a logger with the appropriate level"""
+ logger = logging.getLogger('container')
+ handler = logging.StreamHandler()
+ handler.setFormatter(logging.Formatter(
+ fmt='[container {levelname}] {message}', style='{'
+ ))
+ logger.addHandler(handler)
+ logger.setLevel(logging.DEBUG if verbose is True else logging.INFO)
+ return logger
+
+
+def main(args):
+ """Main entry point for the container tool"""
+ logger = _get_logger(args.verbose)
+ try:
+ cls = Runtimes.get(args.runtime) if args.runtime else Runtimes.find()
+ except ValueError as ex:
+ logger.error(ex)
+ return 1
+ logger.debug("runtime: %s", cls.name)
+ logger.debug("image: %s", args.image)
+ return cls(args, logger).run(args.image, args.cmd)
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(
+ 'container',
+ description="See the documentation for more details: "
+ "https://docs.kernel.org/dev-tools/container.html"
+ )
+ parser.add_argument(
+ '-e', '--env-file',
+ help="Path to an environment file to load in the container."
+ )
+ parser.add_argument(
+ '-g', '--gid',
+ help="Group ID to use inside the container."
+ )
+ parser.add_argument(
+ '-i', '--image', required=True,
+ help="Container image name."
+ )
+ parser.add_argument(
+ '-r', '--runtime', choices=Runtimes.get_names(),
+ help="Container runtime name. If not specified, the first one found "
+ "on the system will be used i.e. Podman if present, otherwise Docker."
+ )
+ parser.add_argument(
+ '-s', '--shell', action='store_true',
+ help="Run the container in an interactive shell."
+ )
+ parser.add_argument(
+ '-u', '--uid',
+ help="User ID to use inside the container. If the -g option is not "
+ "specified, the user ID will also be set as the group ID."
+ )
+ parser.add_argument(
+ '-v', '--verbose', action='store_true',
+ help="Enable verbose output."
+ )
+ parser.add_argument(
+ 'cmd', nargs='+',
+ help="Command to run in the container"
+ )
+ sys.exit(main(parser.parse_args(sys.argv[1:])))
--
2.47.3
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 2/2] Documentation: dev-tools: add container.rst page
2026-01-22 14:06 [PATCH v4 0/2] scripts: introduce containerized builds Guillaume Tucker
2026-01-22 14:06 ` [PATCH v4 1/2] scripts: add tool to run " Guillaume Tucker
@ 2026-01-22 14:07 ` Guillaume Tucker
2026-01-22 14:22 ` Onur Özkan
2026-01-30 0:10 ` [PATCH v4 0/2] scripts: introduce containerized builds Nathan Chancellor
2 siblings, 1 reply; 10+ messages in thread
From: Guillaume Tucker @ 2026-01-22 14:07 UTC (permalink / raw)
To: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Onur Özkan
Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
linux-kbuild, automated-testing, workflows, llvm
Add a dev-tools/container.rst documentation page for the
scripts/container tool. This covers the basic usage with additional
information about environment variables and user IDs. It also
includes a number of practical examples with a reference to the
experimental kernel.org toolchain images.
Update MAINTAINERS accordingly with a reference to the added file.
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nicolas Schier <nsc@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
Documentation/dev-tools/container.rst | 227 ++++++++++++++++++++++++++
Documentation/dev-tools/index.rst | 1 +
MAINTAINERS | 1 +
3 files changed, 229 insertions(+)
create mode 100644 Documentation/dev-tools/container.rst
diff --git a/Documentation/dev-tools/container.rst b/Documentation/dev-tools/container.rst
new file mode 100644
index 000000000000..452415b64662
--- /dev/null
+++ b/Documentation/dev-tools/container.rst
@@ -0,0 +1,227 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2025 Guillaume Tucker
+
+====================
+Containerized Builds
+====================
+
+The ``container`` tool can be used to run any command in the kernel source tree
+from within a container. Doing so facilitates reproducing builds across
+various platforms, for example when a test bot has reported an issue which
+requires a specific version of a compiler or an external test suite. While
+this can already be done by users who are familiar with containers, having a
+dedicated tool in the kernel tree lowers the barrier to entry by solving common
+problems once and for all (e.g. user id management). It also makes it easier
+to share an exact command line leading to a particular result. The main use
+case is likely to be kernel builds but virtually anything can be run: KUnit,
+checkpatch etc. provided a suitable image is available.
+
+
+Options
+=======
+
+Command line syntax::
+
+ scripts/container -i IMAGE [OPTION]... CMD...
+
+Available options:
+
+``-e, --env-file ENV_FILE``
+
+ Path to an environment file to load in the container.
+
+``-g, --gid GID``
+
+ Group id to use inside the container.
+
+``-i, --image IMAGE``
+
+ Container image name (required).
+
+``-r, --runtime RUNTIME``
+
+ Container runtime name. Supported runtimes: ``docker``, ``podman``.
+
+ If not specified, the first one found on the system will be used
+ i.e. Podman if present, otherwise Docker.
+
+``-s, --shell``
+
+ Run the container in an interactive shell.
+
+``-u, --uid UID``
+
+ User id to use inside the container.
+
+ If the ``-g`` option is not specified, the user id will also be used for
+ the group id.
+
+``-v, --verbose``
+
+ Enable verbose output.
+
+``-h, --help``
+
+ Show the help message and exit.
+
+
+Usage
+=====
+
+It's entirely up to the user to choose which image to use and the ``CMD``
+arguments are passed directly as an arbitrary command line to run in the
+container. The tool will take care of mounting the source tree as the current
+working directory and adjust the user and group id as needed.
+
+The container image which would typically include a compiler toolchain is
+provided by the user and selected via the ``-i`` option. The container runtime
+can be selected with the ``-r`` option, which can be either ``docker`` or
+``podman``. If none is specified, the first one found on the system will be
+used while giving priority to Podman. Support for other runtimes may be added
+later depending on their popularity among users.
+
+By default, commands are run non-interactively. The user can abort a running
+container with SIGINT (Ctrl-C). To run commands interactively with a TTY, the
+``--shell`` or ``-s`` option can be used. Signals will then be received by the
+shell directly rather than the parent ``container`` process. To exit an
+interactive shell, use Ctrl-D or ``exit``.
+
+.. note::
+
+ The only host requirement aside from a container runtime is Python 3.10 or
+ later.
+
+.. note::
+
+ Out-of-tree builds are not fully supported yet. The ``O=`` option can
+ however already be used with a relative path inside the source tree to keep
+ separate build outputs. A workaround to build outside the tree is to use
+ ``mount --bind``, see the examples section further down.
+
+
+Environment Variables
+=====================
+
+Environment variables are not propagated to the container so they have to be
+either defined in the image itself or via the ``-e`` option using an
+environment file. In some cases it makes more sense to have them defined in
+the Containerfile used to create the image. For example, a Clang-only compiler
+toolchain image may have ``LLVM=1`` defined.
+
+The local environment file is more useful for user-specific variables added
+during development. It is passed as-is to the container runtime so its format
+may vary. Typically, it will look like the output of ``env``. For example::
+
+ INSTALL_MOD_STRIP=1
+ SOME_RANDOM_TEXT=One upon a time
+
+Please also note that ``make`` options can still be passed on the command line,
+so while this can't be done since the first argument needs to be the
+executable::
+
+ scripts/container -i docker.io/tuxmake/korg-clang LLVM=1 make # won't work
+
+this will work::
+
+ scripts/container -i docker.io/tuxmake/korg-clang make LLVM=1
+
+
+User IDs
+========
+
+This is an area where the behaviour will vary slightly depending on the
+container runtime. The goal is to run commands as the user invoking the tool.
+With Podman, a namespace is created to map the current user id to a different
+one in the container (1000 by default). With Docker, while this is also
+possible with recent versions it requires a special feature to be enabled in
+the daemon so it's not used here for simplicity. Instead, the container is run
+with the current user id directly. In both cases, this will provide the same
+file permissions for the kernel source tree mounted as a volume. The only
+difference is that when using Docker without a namespace, the user id may not
+be the same as the default one set in the image.
+
+Say, we're using an image which sets up a default user with id 1000 and the
+current user calling the ``container`` tool has id 1234. The kernel source
+tree was checked out by this same user so the files belong to user 1234. With
+Podman, the container will be running as user id 1000 with a mapping to id 1234
+so that the files from the mounted volume appear to belong to id 1000 inside
+the container. With Docker and no namespace, the container will be running
+with user id 1234 which can access the files in the volume but not in the user
+1000 home directory. This shouldn't be an issue when running commands only in
+the kernel tree but it is worth highlighting here as it might matter for
+special corner cases.
+
+.. note::
+
+ Podman's `Docker compatibility
+ <https://podman-desktop.io/docs/migrating-from-docker/managing-docker-compatibility>`__
+ mode to run ``docker`` commands on top of a Podman backend is more complex
+ and not fully supported yet. As such, Podman will take priority if both
+ runtimes are available on the system.
+
+
+Examples
+========
+
+The TuxMake project provides a variety of prebuilt container images available
+on `Docker Hub <https://hub.docker.com/u/tuxmake>`__. Here's the shortest
+example to build a kernel using a TuxMake Clang image::
+
+ scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 defconfig
+ scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 -j$(nproc)
+
+.. note::
+
+ When running a command with options within the container, it should be
+ separated with a double dash ``--`` to not confuse them with the
+ ``container`` tool options. Plain commands with no options don't strictly
+ require the double dashes e.g.::
+
+ scripts/container -i docker.io/tuxmake/korg-clang make mrproper
+
+To run ``checkpatch.pl`` in a ``patches`` directory with a generic Perl image::
+
+ scripts/container -i perl:slim-trixie scripts/checkpatch.pl patches/*
+
+As an alternative to the TuxMake images, the examples below refer to
+``kernel.org`` images which are based on the `kernel.org compiler toolchains
+<https://mirrors.edge.kernel.org/pub/tools/>`__. These aren't (yet) officially
+available in any public registry but users can build their own locally instead
+using this `experimental repository
+<https://gitlab.com/gtucker/korg-containers>`__ by running ``make
+PREFIX=kernel.org/``.
+
+To build just ``bzImage`` using Clang::
+
+ scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
+
+Same with GCC 15 as a particular version tag::
+
+ scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
+
+For an out-of-tree build, a trick is to bind-mount the destination directory to
+a relative path inside the source tree::
+
+ mkdir -p $HOME/tmp/my-kernel-build
+ mkdir -p build
+ sudo mount --bind $HOME/tmp/my-kernel-build build
+ scripts/container -i kernel.org/gcc -- make mrproper
+ scripts/container -i kernel.org/gcc -- make O=build defconfig
+ scripts/container -i kernel.org/gcc -- make O=build -j$(nproc)
+
+To run KUnit in an interactive shell and get the full output::
+
+ scripts/container -s -i kernel.org/gcc:kunit -- \
+ tools/testing/kunit/kunit.py \
+ run \
+ --arch=x86_64 \
+ --cross_compile=x86_64-linux-
+
+To just start an interactive shell::
+
+ scripts/container -si kernel.org/gcc bash
+
+To build the HTML documentation, which requires the ``kdocs`` image built with
+``make PREFIX=kernel.org/ extra`` as it's not a compiler toolchain::
+
+ scripts/container -i kernel.org/kdocs make htmldocs
diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
index 4b8425e348ab..527a0e4cf2ed 100644
--- a/Documentation/dev-tools/index.rst
+++ b/Documentation/dev-tools/index.rst
@@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
gpio-sloppy-logic-analyzer
autofdo
propeller
+ container
.. only:: subproject and html
diff --git a/MAINTAINERS b/MAINTAINERS
index affd55ff05e0..4e82dba3bd25 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6387,6 +6387,7 @@ F: include/linux/console*
CONTAINER BUILD SCRIPT
M: Guillaume Tucker <gtucker@gtucker.io>
S: Maintained
+F: Documentation/dev-tools/container.rst
F: scripts/container
CONTEXT TRACKING
--
2.47.3
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 2/2] Documentation: dev-tools: add container.rst page
2026-01-22 14:07 ` [PATCH v4 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
@ 2026-01-22 14:22 ` Onur Özkan
0 siblings, 0 replies; 10+ messages in thread
From: Onur Özkan @ 2026-01-22 14:22 UTC (permalink / raw)
To: Guillaume Tucker
Cc: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
On Thu, 22 Jan 2026 15:07:00 +0100
Guillaume Tucker <gtucker@gtucker.io> wrote:
> Add a dev-tools/container.rst documentation page for the
> scripts/container tool. This covers the basic usage with additional
> information about environment variables and user IDs. It also
> includes a number of practical examples with a reference to the
> experimental kernel.org toolchain images.
>
> Update MAINTAINERS accordingly with a reference to the added file.
>
> Cc: Nathan Chancellor <nathan@kernel.org>
> Cc: Nicolas Schier <nsc@kernel.org>
> Cc: Miguel Ojeda <ojeda@kernel.org>
> Cc: David Gow <davidgow@google.com>
> Cc: "Onur Özkan" <work@onurozkan.dev>
> Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
> ---
> Documentation/dev-tools/container.rst | 227
> ++++++++++++++++++++++++++ Documentation/dev-tools/index.rst |
> 1 + MAINTAINERS | 1 +
> 3 files changed, 229 insertions(+)
> create mode 100644 Documentation/dev-tools/container.rst
>
> diff --git a/Documentation/dev-tools/container.rst
> b/Documentation/dev-tools/container.rst new file mode 100644
> index 000000000000..452415b64662
> --- /dev/null
> +++ b/Documentation/dev-tools/container.rst
> @@ -0,0 +1,227 @@
> +.. SPDX-License-Identifier: GPL-2.0-only
> +.. Copyright (C) 2025 Guillaume Tucker
> +
> +====================
> +Containerized Builds
> +====================
> +
> +The ``container`` tool can be used to run any command in the kernel
> source tree +from within a container. Doing so facilitates
> reproducing builds across +various platforms, for example when a test
> bot has reported an issue which +requires a specific version of a
> compiler or an external test suite. While +this can already be done
> by users who are familiar with containers, having a +dedicated tool
> in the kernel tree lowers the barrier to entry by solving common
> +problems once and for all (e.g. user id management). It also makes
> it easier +to share an exact command line leading to a particular
> result. The main use +case is likely to be kernel builds but
> virtually anything can be run: KUnit, +checkpatch etc. provided a
> suitable image is available. + +
> +Options
> +=======
> +
> +Command line syntax::
> +
> + scripts/container -i IMAGE [OPTION]... CMD...
> +
> +Available options:
> +
> +``-e, --env-file ENV_FILE``
> +
> + Path to an environment file to load in the container.
> +
> +``-g, --gid GID``
> +
> + Group id to use inside the container.
> +
> +``-i, --image IMAGE``
> +
> + Container image name (required).
> +
> +``-r, --runtime RUNTIME``
> +
> + Container runtime name. Supported runtimes: ``docker``,
> ``podman``. +
> + If not specified, the first one found on the system will be used
> + i.e. Podman if present, otherwise Docker.
> +
> +``-s, --shell``
> +
> + Run the container in an interactive shell.
> +
> +``-u, --uid UID``
> +
> + User id to use inside the container.
> +
> + If the ``-g`` option is not specified, the user id will also be
> used for
> + the group id.
> +
> +``-v, --verbose``
> +
> + Enable verbose output.
> +
> +``-h, --help``
> +
> + Show the help message and exit.
> +
> +
> +Usage
> +=====
> +
> +It's entirely up to the user to choose which image to use and the
> ``CMD`` +arguments are passed directly as an arbitrary command line
> to run in the +container. The tool will take care of mounting the
> source tree as the current +working directory and adjust the user and
> group id as needed. +
> +The container image which would typically include a compiler
> toolchain is +provided by the user and selected via the ``-i``
> option. The container runtime +can be selected with the ``-r``
> option, which can be either ``docker`` or +``podman``. If none is
> specified, the first one found on the system will be +used while
> giving priority to Podman. Support for other runtimes may be added
> +later depending on their popularity among users. +
> +By default, commands are run non-interactively. The user can abort
> a running +container with SIGINT (Ctrl-C). To run commands
> interactively with a TTY, the +``--shell`` or ``-s`` option can be
> used. Signals will then be received by the +shell directly rather
> than the parent ``container`` process. To exit an +interactive
> shell, use Ctrl-D or ``exit``. +
> +.. note::
> +
> + The only host requirement aside from a container runtime is
> Python 3.10 or
> + later.
> +
> +.. note::
> +
> + Out-of-tree builds are not fully supported yet. The ``O=``
> option can
> + however already be used with a relative path inside the source
> tree to keep
> + separate build outputs. A workaround to build outside the tree
> is to use
> + ``mount --bind``, see the examples section further down.
> +
> +
> +Environment Variables
> +=====================
> +
> +Environment variables are not propagated to the container so they
> have to be +either defined in the image itself or via the ``-e``
> option using an +environment file. In some cases it makes more sense
> to have them defined in +the Containerfile used to create the image.
> For example, a Clang-only compiler +toolchain image may have
> ``LLVM=1`` defined. +
> +The local environment file is more useful for user-specific
> variables added +during development. It is passed as-is to the
> container runtime so its format +may vary. Typically, it will look
> like the output of ``env``. For example:: +
> + INSTALL_MOD_STRIP=1
> + SOME_RANDOM_TEXT=One upon a time
> +
> +Please also note that ``make`` options can still be passed on the
> command line, +so while this can't be done since the first argument
> needs to be the +executable::
> +
> + scripts/container -i docker.io/tuxmake/korg-clang LLVM=1 make #
> won't work +
> +this will work::
> +
> + scripts/container -i docker.io/tuxmake/korg-clang make LLVM=1
> +
> +
> +User IDs
> +========
> +
> +This is an area where the behaviour will vary slightly depending on
> the +container runtime. The goal is to run commands as the user
> invoking the tool. +With Podman, a namespace is created to map the
> current user id to a different +one in the container (1000 by
> default). With Docker, while this is also +possible with recent
> versions it requires a special feature to be enabled in +the daemon
> so it's not used here for simplicity. Instead, the container is run
> +with the current user id directly. In both cases, this will provide
> the same +file permissions for the kernel source tree mounted as a
> volume. The only +difference is that when using Docker without a
> namespace, the user id may not +be the same as the default one set in
> the image. + +Say, we're using an image which sets up a default user
> with id 1000 and the +current user calling the ``container`` tool has
> id 1234. The kernel source +tree was checked out by this same user
> so the files belong to user 1234. With +Podman, the container will
> be running as user id 1000 with a mapping to id 1234 +so that the
> files from the mounted volume appear to belong to id 1000 inside +the
> container. With Docker and no namespace, the container will be
> running +with user id 1234 which can access the files in the volume
> but not in the user +1000 home directory. This shouldn't be an issue
> when running commands only in +the kernel tree but it is worth
> highlighting here as it might matter for +special corner cases. +
> +.. note::
> +
> + Podman's `Docker compatibility
> +
> <https://podman-desktop.io/docs/migrating-from-docker/managing-docker-compatibility>`__
> + mode to run ``docker`` commands on top of a Podman backend is
> more complex
> + and not fully supported yet. As such, Podman will take priority
> if both
> + runtimes are available on the system.
> +
> +
> +Examples
> +========
> +
> +The TuxMake project provides a variety of prebuilt container images
> available +on `Docker Hub <https://hub.docker.com/u/tuxmake>`__.
> Here's the shortest +example to build a kernel using a TuxMake Clang
> image:: +
> + scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1
> defconfig
> + scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1
> -j$(nproc) +
> +.. note::
> +
> + When running a command with options within the container, it
> should be
> + separated with a double dash ``--`` to not confuse them with the
> + ``container`` tool options. Plain commands with no options don't
> strictly
> + require the double dashes e.g.::
> +
> + scripts/container -i docker.io/tuxmake/korg-clang make mrproper
> +
> +To run ``checkpatch.pl`` in a ``patches`` directory with a generic
> Perl image:: +
> + scripts/container -i perl:slim-trixie scripts/checkpatch.pl
> patches/* +
> +As an alternative to the TuxMake images, the examples below refer to
> +``kernel.org`` images which are based on the `kernel.org compiler
> toolchains +<https://mirrors.edge.kernel.org/pub/tools/>`__. These
> aren't (yet) officially +available in any public registry but users
> can build their own locally instead +using this `experimental
> repository +<https://gitlab.com/gtucker/korg-containers>`__ by
> running ``make +PREFIX=kernel.org/``.
> +
> +To build just ``bzImage`` using Clang::
> +
> + scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
> +
> +Same with GCC 15 as a particular version tag::
> +
> + scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
> +
> +For an out-of-tree build, a trick is to bind-mount the destination
> directory to +a relative path inside the source tree::
> +
> + mkdir -p $HOME/tmp/my-kernel-build
> + mkdir -p build
> + sudo mount --bind $HOME/tmp/my-kernel-build build
> + scripts/container -i kernel.org/gcc -- make mrproper
> + scripts/container -i kernel.org/gcc -- make O=build defconfig
> + scripts/container -i kernel.org/gcc -- make O=build -j$(nproc)
> +
> +To run KUnit in an interactive shell and get the full output::
> +
> + scripts/container -s -i kernel.org/gcc:kunit -- \
> + tools/testing/kunit/kunit.py \
> + run \
> + --arch=x86_64 \
> + --cross_compile=x86_64-linux-
> +
> +To just start an interactive shell::
> +
> + scripts/container -si kernel.org/gcc bash
> +
> +To build the HTML documentation, which requires the ``kdocs`` image
> built with +``make PREFIX=kernel.org/ extra`` as it's not a compiler
> toolchain:: +
> + scripts/container -i kernel.org/kdocs make htmldocs
> diff --git a/Documentation/dev-tools/index.rst
> b/Documentation/dev-tools/index.rst index 4b8425e348ab..527a0e4cf2ed
> 100644 --- a/Documentation/dev-tools/index.rst
> +++ b/Documentation/dev-tools/index.rst
> @@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
> gpio-sloppy-logic-analyzer
> autofdo
> propeller
> + container
>
>
> .. only:: subproject and html
> diff --git a/MAINTAINERS b/MAINTAINERS
> index affd55ff05e0..4e82dba3bd25 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -6387,6 +6387,7 @@ F: include/linux/console*
> CONTAINER BUILD SCRIPT
> M: Guillaume Tucker <gtucker@gtucker.io>
> S: Maintained
> +F: Documentation/dev-tools/container.rst
> F: scripts/container
>
> CONTEXT TRACKING
This looks great.
Reviewed-by: Onur Özkan <work@onurozkan.dev>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 14:06 ` [PATCH v4 1/2] scripts: add tool to run " Guillaume Tucker
@ 2026-01-22 14:29 ` Onur Özkan
2026-01-22 14:59 ` Guillaume Tucker
0 siblings, 1 reply; 10+ messages in thread
From: Onur Özkan @ 2026-01-22 14:29 UTC (permalink / raw)
To: Guillaume Tucker
Cc: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
Hi Guillaume,
Just 2 notes from my end.
On Thu, 22 Jan 2026 15:06:59 +0100
Guillaume Tucker <gtucker@gtucker.io> wrote:
> Add a 'scripts/container' tool written in Python to run any command in
> the source tree from within a container. This can typically be used
> to call 'make' with a compiler toolchain image to run reproducible
> builds but any arbitrary command can be run too. Only Docker and
> Podman are supported in this initial version.
>
> Add a new entry to MAINTAINERS accordingly.
>
> Cc: Nathan Chancellor <nathan@kernel.org>
> Cc: Nicolas Schier <nsc@kernel.org>
> Cc: Miguel Ojeda <ojeda@kernel.org>
> Cc: David Gow <davidgow@google.com>
> Cc: "Onur Özkan" <work@onurozkan.dev>
> Link:
> https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
> Signed-off-by: Guillaume Tucker <gtucker@gtucker.io> ---
> MAINTAINERS | 6 ++
> scripts/container | 199
> ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 205
> insertions(+) create mode 100755 scripts/container
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index da9dbc1a4019..affd55ff05e0 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -6384,6 +6384,11 @@ S: Supported
> F: drivers/video/console/
> F: include/linux/console*
>
> +CONTAINER BUILD SCRIPT
> +M: Guillaume Tucker <gtucker@gtucker.io>
> +S: Maintained
> +F: scripts/container
> +
> CONTEXT TRACKING
> M: Frederic Weisbecker <frederic@kernel.org>
> M: "Paul E. McKenney" <paulmck@kernel.org>
> @@ -13676,6 +13681,7 @@ F: scripts/Makefile*
> F: scripts/bash-completion/
> F: scripts/basic/
> F: scripts/clang-tools/
> +F: scripts/container
> F: scripts/dummy-tools/
> F: scripts/include/
> F: scripts/mk*
> diff --git a/scripts/container b/scripts/container
> new file mode 100755
> index 000000000000..09663eccb8d3
> --- /dev/null
> +++ b/scripts/container
> @@ -0,0 +1,199 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Copyright (C) 2025 Guillaume Tucker
> +
> +"""Containerized builds"""
> +
> +import abc
> +import argparse
> +import logging
> +import os
> +import pathlib
> +import shutil
> +import subprocess
> +import sys
> +import uuid
> +
> +
> +class ContainerRuntime(abc.ABC):
> + """Base class for a container runtime implementation"""
> +
> + name = None # Property defined in each implementation class
> +
> + def __init__(self, args, logger):
> + self._uid = args.uid or os.getuid()
> + self._gid = args.gid or args.uid or os.getgid()
> + self._env_file = args.env_file
> + self._shell = args.shell
> + self._logger = logger
> +
> + @classmethod
> + def is_present(cls):
> + """Determine whether the runtime is present on the system"""
> + return shutil.which(cls.name) is not None
> +
> + @abc.abstractmethod
> + def _do_run(self, image, cmd, container_name):
> + """Runtime-specific handler to run a command in a
> container""" +
> + @abc.abstractmethod
> + def _do_abort(self, container_name):
> + """Runtime-specific handler to abort a running container"""
> +
> + def run(self, image, cmd):
> + """Run a command in a runtime container"""
> + container_name = str(uuid.uuid4())
> + self._logger.debug("container: %s", container_name)
> + try:
> + return self._do_run(image, cmd, container_name)
> + except KeyboardInterrupt:
> + self._logger.error("user aborted")
> + self._do_abort(container_name)
> + return 1
> +
> +
> +class CommonRuntime(ContainerRuntime):
> + """Common logic for Docker and Podman"""
> +
> + def _do_run(self, image, cmd, container_name):
> + cmdline = [self.name, 'run']
> + cmdline += self._get_opts(container_name)
> + cmdline.append(image)
> + cmdline += cmd
> + self._logger.debug('command: %s', ' '.join(cmdline))
> + return subprocess.call(cmdline)
> +
> + def _get_opts(self, container_name):
> + opts = [
> + '--name', container_name,
> + '--rm',
> + '--volume', f'{pathlib.Path.cwd()}:/src',
> + '--workdir', '/src',
> + ]
> + if self._env_file:
> + opts += ['--env-file', self._env_file]
> + if self._shell:
> + opts += ['--interactive', '--tty']
> + return opts
> +
> + def _do_abort(self, container_name):
> + subprocess.call([self.name, 'kill', container_name])
> +
> +
> +class DockerRuntime(CommonRuntime):
> + """Run a command in a Docker container"""
> +
> + name = 'docker'
> +
> + def _get_opts(self, container_name):
> + return super()._get_opts(container_name) + [
> + '--user', f'{self._uid}:{self._gid}'
> + ]
> +
> +
> +class PodmanRuntime(CommonRuntime):
> + """Run a command in a Podman container"""
> +
> + name = 'podman'
> +
> + def _get_opts(self, container_name):
> + return super()._get_opts(container_name) + [
> + '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
> + ]
> +
> +
> +class Runtimes:
> + """List of all supported runtimes"""
> +
> + runtimes = [PodmanRuntime, DockerRuntime]
> +
> + @classmethod
> + def get_names(cls):
> + """Get a list of all the runtime names"""
> + return list(runtime.name for runtime in cls.runtimes)
> +
> + @classmethod
> + def get(cls, name):
> + """Get a single runtime class matching the given name"""
> + for runtime in cls.runtimes:
> + if runtime.name == name:
> + if not runtime.is_present():
> + raise ValueError(f"runtime not found: {name}")
> + return runtime
> + raise ValueError(f"unknown runtime: {runtime}")
> +
I think you meant to use "{name}" not "{runtime}" inside ValueError.
> + @classmethod
> + def find(cls):
> + """Find the first runtime present on the system"""
> + for runtime in cls.runtimes:
> + if runtime.is_present():
> + return runtime
> + raise ValueError("no runtime found")
> +
nit: We could extend the error message like: "Couldn't find any runtime.
Use -r <runtime> to specify one manually". What do you think?
> +
> +def _get_logger(verbose):
> + """Set up a logger with the appropriate level"""
> + logger = logging.getLogger('container')
> + handler = logging.StreamHandler()
> + handler.setFormatter(logging.Formatter(
> + fmt='[container {levelname}] {message}', style='{'
> + ))
> + logger.addHandler(handler)
> + logger.setLevel(logging.DEBUG if verbose is True else
> logging.INFO)
> + return logger
> +
> +
> +def main(args):
> + """Main entry point for the container tool"""
> + logger = _get_logger(args.verbose)
> + try:
> + cls = Runtimes.get(args.runtime) if args.runtime else
> Runtimes.find()
> + except ValueError as ex:
> + logger.error(ex)
> + return 1
> + logger.debug("runtime: %s", cls.name)
> + logger.debug("image: %s", args.image)
> + return cls(args, logger).run(args.image, args.cmd)
> +
> +
> +if __name__ == '__main__':
> + parser = argparse.ArgumentParser(
> + 'container',
> + description="See the documentation for more details: "
> + "https://docs.kernel.org/dev-tools/container.html"
> + )
> + parser.add_argument(
> + '-e', '--env-file',
> + help="Path to an environment file to load in the container."
> + )
> + parser.add_argument(
> + '-g', '--gid',
> + help="Group ID to use inside the container."
> + )
> + parser.add_argument(
> + '-i', '--image', required=True,
> + help="Container image name."
> + )
> + parser.add_argument(
> + '-r', '--runtime', choices=Runtimes.get_names(),
> + help="Container runtime name. If not specified, the first
> one found "
> + "on the system will be used i.e. Podman if present,
> otherwise Docker."
> + )
> + parser.add_argument(
> + '-s', '--shell', action='store_true',
> + help="Run the container in an interactive shell."
> + )
> + parser.add_argument(
> + '-u', '--uid',
> + help="User ID to use inside the container. If the -g option
> is not "
> + "specified, the user ID will also be set as the group ID."
> + )
> + parser.add_argument(
> + '-v', '--verbose', action='store_true',
> + help="Enable verbose output."
> + )
> + parser.add_argument(
> + 'cmd', nargs='+',
> + help="Command to run in the container"
> + )
> + sys.exit(main(parser.parse_args(sys.argv[1:])))
The rest LGTM.
Regards,
Onur
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 14:29 ` Onur Özkan
@ 2026-01-22 14:59 ` Guillaume Tucker
2026-01-22 16:39 ` Onur Özkan
2026-01-22 18:49 ` Nathan Chancellor
0 siblings, 2 replies; 10+ messages in thread
From: Guillaume Tucker @ 2026-01-22 14:59 UTC (permalink / raw)
To: Onur Özkan
Cc: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
Hi Onur,
On 22/01/2026 15:29, Onur Özkan wrote:
> Hi Guillaume,
>
> Just 2 notes from my end.
>
> On Thu, 22 Jan 2026 15:06:59 +0100
> Guillaume Tucker <gtucker@gtucker.io> wrote:
>
>> Add a 'scripts/container' tool written in Python to run any command in
>> the source tree from within a container. This can typically be used
>> to call 'make' with a compiler toolchain image to run reproducible
>> builds but any arbitrary command can be run too. Only Docker and
>> Podman are supported in this initial version.
>>
>> Add a new entry to MAINTAINERS accordingly.
>>
>> Cc: Nathan Chancellor <nathan@kernel.org>
>> Cc: Nicolas Schier <nsc@kernel.org>
>> Cc: Miguel Ojeda <ojeda@kernel.org>
>> Cc: David Gow <davidgow@google.com>
>> Cc: "Onur Özkan" <work@onurozkan.dev>
>> Link:
>> https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
>> Signed-off-by: Guillaume Tucker <gtucker@gtucker.io> ---
>> MAINTAINERS | 6 ++
>> scripts/container | 199
>> ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 205
>> insertions(+) create mode 100755 scripts/container
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index da9dbc1a4019..affd55ff05e0 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -6384,6 +6384,11 @@ S: Supported
>> F: drivers/video/console/
>> F: include/linux/console*
>>
>> +CONTAINER BUILD SCRIPT
>> +M: Guillaume Tucker <gtucker@gtucker.io>
>> +S: Maintained
>> +F: scripts/container
>> +
>> CONTEXT TRACKING
>> M: Frederic Weisbecker <frederic@kernel.org>
>> M: "Paul E. McKenney" <paulmck@kernel.org>
>> @@ -13676,6 +13681,7 @@ F: scripts/Makefile*
>> F: scripts/bash-completion/
>> F: scripts/basic/
>> F: scripts/clang-tools/
>> +F: scripts/container
>> F: scripts/dummy-tools/
>> F: scripts/include/
>> F: scripts/mk*
>> diff --git a/scripts/container b/scripts/container
>> new file mode 100755
>> index 000000000000..09663eccb8d3
>> --- /dev/null
>> +++ b/scripts/container
>> @@ -0,0 +1,199 @@
>> +#!/usr/bin/env python3
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +# Copyright (C) 2025 Guillaume Tucker
>> +
>> +"""Containerized builds"""
>> +
>> +import abc
>> +import argparse
>> +import logging
>> +import os
>> +import pathlib
>> +import shutil
>> +import subprocess
>> +import sys
>> +import uuid
>> +
>> +
>> +class ContainerRuntime(abc.ABC):
>> + """Base class for a container runtime implementation"""
>> +
>> + name = None # Property defined in each implementation class
>> +
>> + def __init__(self, args, logger):
>> + self._uid = args.uid or os.getuid()
>> + self._gid = args.gid or args.uid or os.getgid()
>> + self._env_file = args.env_file
>> + self._shell = args.shell
>> + self._logger = logger
>> +
>> + @classmethod
>> + def is_present(cls):
>> + """Determine whether the runtime is present on the system"""
>> + return shutil.which(cls.name) is not None
>> +
>> + @abc.abstractmethod
>> + def _do_run(self, image, cmd, container_name):
>> + """Runtime-specific handler to run a command in a
>> container""" +
>> + @abc.abstractmethod
>> + def _do_abort(self, container_name):
>> + """Runtime-specific handler to abort a running container"""
>> +
>> + def run(self, image, cmd):
>> + """Run a command in a runtime container"""
>> + container_name = str(uuid.uuid4())
>> + self._logger.debug("container: %s", container_name)
>> + try:
>> + return self._do_run(image, cmd, container_name)
>> + except KeyboardInterrupt:
>> + self._logger.error("user aborted")
>> + self._do_abort(container_name)
>> + return 1
>> +
>> +
>> +class CommonRuntime(ContainerRuntime):
>> + """Common logic for Docker and Podman"""
>> +
>> + def _do_run(self, image, cmd, container_name):
>> + cmdline = [self.name, 'run']
>> + cmdline += self._get_opts(container_name)
>> + cmdline.append(image)
>> + cmdline += cmd
>> + self._logger.debug('command: %s', ' '.join(cmdline))
>> + return subprocess.call(cmdline)
>> +
>> + def _get_opts(self, container_name):
>> + opts = [
>> + '--name', container_name,
>> + '--rm',
>> + '--volume', f'{pathlib.Path.cwd()}:/src',
>> + '--workdir', '/src',
>> + ]
>> + if self._env_file:
>> + opts += ['--env-file', self._env_file]
>> + if self._shell:
>> + opts += ['--interactive', '--tty']
>> + return opts
>> +
>> + def _do_abort(self, container_name):
>> + subprocess.call([self.name, 'kill', container_name])
>> +
>> +
>> +class DockerRuntime(CommonRuntime):
>> + """Run a command in a Docker container"""
>> +
>> + name = 'docker'
>> +
>> + def _get_opts(self, container_name):
>> + return super()._get_opts(container_name) + [
>> + '--user', f'{self._uid}:{self._gid}'
>> + ]
>> +
>> +
>> +class PodmanRuntime(CommonRuntime):
>> + """Run a command in a Podman container"""
>> +
>> + name = 'podman'
>> +
>> + def _get_opts(self, container_name):
>> + return super()._get_opts(container_name) + [
>> + '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
>> + ]
>> +
>> +
>> +class Runtimes:
>> + """List of all supported runtimes"""
>> +
>> + runtimes = [PodmanRuntime, DockerRuntime]
>> +
>> + @classmethod
>> + def get_names(cls):
>> + """Get a list of all the runtime names"""
>> + return list(runtime.name for runtime in cls.runtimes)
>> +
>> + @classmethod
>> + def get(cls, name):
>> + """Get a single runtime class matching the given name"""
>> + for runtime in cls.runtimes:
>> + if runtime.name == name:
>> + if not runtime.is_present():
>> + raise ValueError(f"runtime not found: {name}")
>> + return runtime
>> + raise ValueError(f"unknown runtime: {runtime}")
>> +
>
> I think you meant to use "{name}" not "{runtime}" inside ValueError.
Ah yes, sorry. The parser already checks that the runtime name is in
the list so I've never hit this error. We would probably need some
unit tests at some point.
So yes this fixes the issue, which can be reproduced with a small
hack in the script to relax the -r option checks:
--- a/scripts/container
+++ b/scripts/container
@@ -120,7 +120,7 @@ class Runtimes:
if not runtime.is_present():
raise ValueError(f"runtime not found: {name}")
return runtime
- raise ValueError(f"unknown runtime: {runtime}")
+ raise ValueError(f"unknown runtime: {name}")
@classmethod
def find(cls):
Nathan, would you be OK with folding this in or should I send a v5?
>> + @classmethod
>> + def find(cls):
>> + """Find the first runtime present on the system"""
>> + for runtime in cls.runtimes:
>> + if runtime.is_present():
>> + return runtime
>> + raise ValueError("no runtime found")
>> +
>
> nit: We could extend the error message like: "Couldn't find any runtime.
> Use -r <runtime> to specify one manually". What do you think?
I'm all for improving the user experience. It's good to keep the
implementation logic separate from the command line interface though.
Maybe this is something I could improve in a follow-up? There are a
few other potential things to rework in this area; a more detailed
error could be logged in main().
>> +
>> +def _get_logger(verbose):
>> + """Set up a logger with the appropriate level"""
>> + logger = logging.getLogger('container')
>> + handler = logging.StreamHandler()
>> + handler.setFormatter(logging.Formatter(
>> + fmt='[container {levelname}] {message}', style='{'
>> + ))
>> + logger.addHandler(handler)
>> + logger.setLevel(logging.DEBUG if verbose is True else
>> logging.INFO)
>> + return logger
>> +
>> +
>> +def main(args):
>> + """Main entry point for the container tool"""
>> + logger = _get_logger(args.verbose)
>> + try:
>> + cls = Runtimes.get(args.runtime) if args.runtime else
>> Runtimes.find()
>> + except ValueError as ex:
>> + logger.error(ex)
>> + return 1
>> + logger.debug("runtime: %s", cls.name)
>> + logger.debug("image: %s", args.image)
>> + return cls(args, logger).run(args.image, args.cmd)
>> +
>> +
>> +if __name__ == '__main__':
>> + parser = argparse.ArgumentParser(
>> + 'container',
>> + description="See the documentation for more details: "
>> + "https://docs.kernel.org/dev-tools/container.html"
>> + )
>> + parser.add_argument(
>> + '-e', '--env-file',
>> + help="Path to an environment file to load in the container."
>> + )
>> + parser.add_argument(
>> + '-g', '--gid',
>> + help="Group ID to use inside the container."
>> + )
>> + parser.add_argument(
>> + '-i', '--image', required=True,
>> + help="Container image name."
>> + )
>> + parser.add_argument(
>> + '-r', '--runtime', choices=Runtimes.get_names(),
>> + help="Container runtime name. If not specified, the first
>> one found "
>> + "on the system will be used i.e. Podman if present,
>> otherwise Docker."
>> + )
>> + parser.add_argument(
>> + '-s', '--shell', action='store_true',
>> + help="Run the container in an interactive shell."
>> + )
>> + parser.add_argument(
>> + '-u', '--uid',
>> + help="User ID to use inside the container. If the -g option
>> is not "
>> + "specified, the user ID will also be set as the group ID."
>> + )
>> + parser.add_argument(
>> + '-v', '--verbose', action='store_true',
>> + help="Enable verbose output."
>> + )
>> + parser.add_argument(
>> + 'cmd', nargs='+',
>> + help="Command to run in the container"
>> + )
>> + sys.exit(main(parser.parse_args(sys.argv[1:])))
>
> The rest LGTM.
Thanks for the reviews.
Cheers,
Guillaume
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 14:59 ` Guillaume Tucker
@ 2026-01-22 16:39 ` Onur Özkan
2026-01-22 18:49 ` Nathan Chancellor
1 sibling, 0 replies; 10+ messages in thread
From: Onur Özkan @ 2026-01-22 16:39 UTC (permalink / raw)
To: Guillaume Tucker
Cc: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
On Thu, 22 Jan 2026 15:59:54 +0100
Guillaume Tucker <gtucker@gtucker.io> wrote:
> Hi Onur,
>
> On 22/01/2026 15:29, Onur Özkan wrote:
> > Hi Guillaume,
> >
> > Just 2 notes from my end.
> >
> > On Thu, 22 Jan 2026 15:06:59 +0100
> > Guillaume Tucker <gtucker@gtucker.io> wrote:
> >
> >> Add a 'scripts/container' tool written in Python to run any
> >> command in the source tree from within a container. This can
> >> typically be used to call 'make' with a compiler toolchain image
> >> to run reproducible builds but any arbitrary command can be run
> >> too. Only Docker and Podman are supported in this initial version.
> >>
> >> Add a new entry to MAINTAINERS accordingly.
> >>
> >> Cc: Nathan Chancellor <nathan@kernel.org>
> >> Cc: Nicolas Schier <nsc@kernel.org>
> >> Cc: Miguel Ojeda <ojeda@kernel.org>
> >> Cc: David Gow <davidgow@google.com>
> >> Cc: "Onur Özkan" <work@onurozkan.dev>
> >> Link:
> >> https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
> >> Signed-off-by: Guillaume Tucker <gtucker@gtucker.io> ---
> >> MAINTAINERS | 6 ++
> >> scripts/container | 199
> >> ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 205
> >> insertions(+) create mode 100755 scripts/container
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index da9dbc1a4019..affd55ff05e0 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -6384,6 +6384,11 @@ S: Supported
> >> F: drivers/video/console/
> >> F: include/linux/console*
> >>
> >> +CONTAINER BUILD SCRIPT
> >> +M: Guillaume Tucker <gtucker@gtucker.io>
> >> +S: Maintained
> >> +F: scripts/container
> >> +
> >> CONTEXT TRACKING
> >> M: Frederic Weisbecker <frederic@kernel.org>
> >> M: "Paul E. McKenney" <paulmck@kernel.org>
> >> @@ -13676,6 +13681,7 @@ F: scripts/Makefile*
> >> F: scripts/bash-completion/
> >> F: scripts/basic/
> >> F: scripts/clang-tools/
> >> +F: scripts/container
> >> F: scripts/dummy-tools/
> >> F: scripts/include/
> >> F: scripts/mk*
> >> diff --git a/scripts/container b/scripts/container
> >> new file mode 100755
> >> index 000000000000..09663eccb8d3
> >> --- /dev/null
> >> +++ b/scripts/container
> >> @@ -0,0 +1,199 @@
> >> +#!/usr/bin/env python3
> >> +# SPDX-License-Identifier: GPL-2.0-only
> >> +# Copyright (C) 2025 Guillaume Tucker
> >> +
> >> +"""Containerized builds"""
> >> +
> >> +import abc
> >> +import argparse
> >> +import logging
> >> +import os
> >> +import pathlib
> >> +import shutil
> >> +import subprocess
> >> +import sys
> >> +import uuid
> >> +
> >> +
> >> +class ContainerRuntime(abc.ABC):
> >> + """Base class for a container runtime implementation"""
> >> +
> >> + name = None # Property defined in each implementation class
> >> +
> >> + def __init__(self, args, logger):
> >> + self._uid = args.uid or os.getuid()
> >> + self._gid = args.gid or args.uid or os.getgid()
> >> + self._env_file = args.env_file
> >> + self._shell = args.shell
> >> + self._logger = logger
> >> +
> >> + @classmethod
> >> + def is_present(cls):
> >> + """Determine whether the runtime is present on the
> >> system"""
> >> + return shutil.which(cls.name) is not None
> >> +
> >> + @abc.abstractmethod
> >> + def _do_run(self, image, cmd, container_name):
> >> + """Runtime-specific handler to run a command in a
> >> container""" +
> >> + @abc.abstractmethod
> >> + def _do_abort(self, container_name):
> >> + """Runtime-specific handler to abort a running
> >> container""" +
> >> + def run(self, image, cmd):
> >> + """Run a command in a runtime container"""
> >> + container_name = str(uuid.uuid4())
> >> + self._logger.debug("container: %s", container_name)
> >> + try:
> >> + return self._do_run(image, cmd, container_name)
> >> + except KeyboardInterrupt:
> >> + self._logger.error("user aborted")
> >> + self._do_abort(container_name)
> >> + return 1
> >> +
> >> +
> >> +class CommonRuntime(ContainerRuntime):
> >> + """Common logic for Docker and Podman"""
> >> +
> >> + def _do_run(self, image, cmd, container_name):
> >> + cmdline = [self.name, 'run']
> >> + cmdline += self._get_opts(container_name)
> >> + cmdline.append(image)
> >> + cmdline += cmd
> >> + self._logger.debug('command: %s', ' '.join(cmdline))
> >> + return subprocess.call(cmdline)
> >> +
> >> + def _get_opts(self, container_name):
> >> + opts = [
> >> + '--name', container_name,
> >> + '--rm',
> >> + '--volume', f'{pathlib.Path.cwd()}:/src',
> >> + '--workdir', '/src',
> >> + ]
> >> + if self._env_file:
> >> + opts += ['--env-file', self._env_file]
> >> + if self._shell:
> >> + opts += ['--interactive', '--tty']
> >> + return opts
> >> +
> >> + def _do_abort(self, container_name):
> >> + subprocess.call([self.name, 'kill', container_name])
> >> +
> >> +
> >> +class DockerRuntime(CommonRuntime):
> >> + """Run a command in a Docker container"""
> >> +
> >> + name = 'docker'
> >> +
> >> + def _get_opts(self, container_name):
> >> + return super()._get_opts(container_name) + [
> >> + '--user', f'{self._uid}:{self._gid}'
> >> + ]
> >> +
> >> +
> >> +class PodmanRuntime(CommonRuntime):
> >> + """Run a command in a Podman container"""
> >> +
> >> + name = 'podman'
> >> +
> >> + def _get_opts(self, container_name):
> >> + return super()._get_opts(container_name) + [
> >> + '--userns',
> >> f'keep-id:uid={self._uid},gid={self._gid}',
> >> + ]
> >> +
> >> +
> >> +class Runtimes:
> >> + """List of all supported runtimes"""
> >> +
> >> + runtimes = [PodmanRuntime, DockerRuntime]
> >> +
> >> + @classmethod
> >> + def get_names(cls):
> >> + """Get a list of all the runtime names"""
> >> + return list(runtime.name for runtime in cls.runtimes)
> >> +
> >> + @classmethod
> >> + def get(cls, name):
> >> + """Get a single runtime class matching the given name"""
> >> + for runtime in cls.runtimes:
> >> + if runtime.name == name:
> >> + if not runtime.is_present():
> >> + raise ValueError(f"runtime not found: {name}")
> >> + return runtime
> >> + raise ValueError(f"unknown runtime: {runtime}")
> >> +
> >
> > I think you meant to use "{name}" not "{runtime}" inside ValueError.
>
> Ah yes, sorry. The parser already checks that the runtime name is in
> the list so I've never hit this error. We would probably need some
> unit tests at some point.
>
> So yes this fixes the issue, which can be reproduced with a small
> hack in the script to relax the -r option checks:
>
> --- a/scripts/container
> +++ b/scripts/container
> @@ -120,7 +120,7 @@ class Runtimes:
> if not runtime.is_present():
> raise ValueError(f"runtime not found: {name}")
> return runtime
> - raise ValueError(f"unknown runtime: {runtime}")
> + raise ValueError(f"unknown runtime: {name}")
>
> @classmethod
> def find(cls):
>
>
> Nathan, would you be OK with folding this in or should I send a v5?
>
> >> + @classmethod
> >> + def find(cls):
> >> + """Find the first runtime present on the system"""
> >> + for runtime in cls.runtimes:
> >> + if runtime.is_present():
> >> + return runtime
> >> + raise ValueError("no runtime found")
> >> +
> >
> > nit: We could extend the error message like: "Couldn't find any
> > runtime. Use -r <runtime> to specify one manually". What do you
> > think?
>
> I'm all for improving the user experience. It's good to keep the
> implementation logic separate from the command line interface though.
> Maybe this is something I could improve in a follow-up? There are a
> few other potential things to rework in this area; a more detailed
> error could be logged in main().
>
Sure, sounds good to me.
Thanks,
Onur
> >> +
> >> +def _get_logger(verbose):
> >> + """Set up a logger with the appropriate level"""
> >> + logger = logging.getLogger('container')
> >> + handler = logging.StreamHandler()
> >> + handler.setFormatter(logging.Formatter(
> >> + fmt='[container {levelname}] {message}', style='{'
> >> + ))
> >> + logger.addHandler(handler)
> >> + logger.setLevel(logging.DEBUG if verbose is True else
> >> logging.INFO)
> >> + return logger
> >> +
> >> +
> >> +def main(args):
> >> + """Main entry point for the container tool"""
> >> + logger = _get_logger(args.verbose)
> >> + try:
> >> + cls = Runtimes.get(args.runtime) if args.runtime else
> >> Runtimes.find()
> >> + except ValueError as ex:
> >> + logger.error(ex)
> >> + return 1
> >> + logger.debug("runtime: %s", cls.name)
> >> + logger.debug("image: %s", args.image)
> >> + return cls(args, logger).run(args.image, args.cmd)
> >> +
> >> +
> >> +if __name__ == '__main__':
> >> + parser = argparse.ArgumentParser(
> >> + 'container',
> >> + description="See the documentation for more details: "
> >> + "https://docs.kernel.org/dev-tools/container.html"
> >> + )
> >> + parser.add_argument(
> >> + '-e', '--env-file',
> >> + help="Path to an environment file to load in the
> >> container."
> >> + )
> >> + parser.add_argument(
> >> + '-g', '--gid',
> >> + help="Group ID to use inside the container."
> >> + )
> >> + parser.add_argument(
> >> + '-i', '--image', required=True,
> >> + help="Container image name."
> >> + )
> >> + parser.add_argument(
> >> + '-r', '--runtime', choices=Runtimes.get_names(),
> >> + help="Container runtime name. If not specified, the first
> >> one found "
> >> + "on the system will be used i.e. Podman if present,
> >> otherwise Docker."
> >> + )
> >> + parser.add_argument(
> >> + '-s', '--shell', action='store_true',
> >> + help="Run the container in an interactive shell."
> >> + )
> >> + parser.add_argument(
> >> + '-u', '--uid',
> >> + help="User ID to use inside the container. If the -g
> >> option is not "
> >> + "specified, the user ID will also be set as the group ID."
> >> + )
> >> + parser.add_argument(
> >> + '-v', '--verbose', action='store_true',
> >> + help="Enable verbose output."
> >> + )
> >> + parser.add_argument(
> >> + 'cmd', nargs='+',
> >> + help="Command to run in the container"
> >> + )
> >> + sys.exit(main(parser.parse_args(sys.argv[1:])))
> >
> > The rest LGTM.
>
> Thanks for the reviews.
>
> Cheers,
> Guillaume
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 14:59 ` Guillaume Tucker
2026-01-22 16:39 ` Onur Özkan
@ 2026-01-22 18:49 ` Nathan Chancellor
2026-01-22 19:57 ` Nicolas Schier
1 sibling, 1 reply; 10+ messages in thread
From: Nathan Chancellor @ 2026-01-22 18:49 UTC (permalink / raw)
To: Guillaume Tucker
Cc: Onur Özkan, Nicolas Schier, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
On Thu, Jan 22, 2026 at 03:59:54PM +0100, Guillaume Tucker wrote:
> --- a/scripts/container
> +++ b/scripts/container
> @@ -120,7 +120,7 @@ class Runtimes:
> if not runtime.is_present():
> raise ValueError(f"runtime not found: {name}")
> return runtime
> - raise ValueError(f"unknown runtime: {runtime}")
> + raise ValueError(f"unknown runtime: {name}")
>
> @classmethod
> def find(cls):
>
>
> Nathan, would you be OK with folding this in or should I send a v5?
I can fold that in, thanks.
I will carry forward Nicolas's ack and testing tags from v3, as this
revision is not substantially different from what he looked at and
tested.
Cheers,
Nathan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 1/2] scripts: add tool to run containerized builds
2026-01-22 18:49 ` Nathan Chancellor
@ 2026-01-22 19:57 ` Nicolas Schier
0 siblings, 0 replies; 10+ messages in thread
From: Nicolas Schier @ 2026-01-22 19:57 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Guillaume Tucker, Onur Özkan, Miguel Ojeda, David Gow,
Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
On Thu, Jan 22, 2026 at 11:49:39AM -0700, Nathan Chancellor wrote:
> On Thu, Jan 22, 2026 at 03:59:54PM +0100, Guillaume Tucker wrote:
> > --- a/scripts/container
> > +++ b/scripts/container
> > @@ -120,7 +120,7 @@ class Runtimes:
> > if not runtime.is_present():
> > raise ValueError(f"runtime not found: {name}")
> > return runtime
> > - raise ValueError(f"unknown runtime: {runtime}")
> > + raise ValueError(f"unknown runtime: {name}")
> >
> > @classmethod
> > def find(cls):
> >
> >
> > Nathan, would you be OK with folding this in or should I send a v5?
>
> I can fold that in, thanks.
>
> I will carry forward Nicolas's ack and testing tags from v3, as this
> revision is not substantially different from what he looked at and
> tested.
thanks!
Kind regards,
Nicolas
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 0/2] scripts: introduce containerized builds
2026-01-22 14:06 [PATCH v4 0/2] scripts: introduce containerized builds Guillaume Tucker
2026-01-22 14:06 ` [PATCH v4 1/2] scripts: add tool to run " Guillaume Tucker
2026-01-22 14:07 ` [PATCH v4 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
@ 2026-01-30 0:10 ` Nathan Chancellor
2 siblings, 0 replies; 10+ messages in thread
From: Nathan Chancellor @ 2026-01-30 0:10 UTC (permalink / raw)
To: Nathan Chancellor, Nicolas Schier, Miguel Ojeda, David Gow,
Onur Özkan, Guillaume Tucker
Cc: Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
automated-testing, workflows, llvm
On Thu, 22 Jan 2026 15:06:58 +0100, Guillaume Tucker wrote:
> This proposal emerged from discussions over email and after a talk at
> Plumbers 2024:
>
> https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
>
> The aim is to facilitate reproducing builds for CI bots as well as
> developers using containers. Here's an illustrative example with a
> kernel.org toolchain in a Docker image from tuxmake:
>
> [...]
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux.git kbuild-next
Thanks!
[1/2] scripts: add tool to run containerized builds
(no commit info)
[2/2] Documentation: dev-tools: add container.rst page
(no commit info)
Please look out for regression or issue reports or other follow up
comments, as they may result in the patch/series getting dropped or
reverted. Patches applied to an "unstable" branch are accepted pending
wider testing in -next and any post-commit review; they will generally
be moved to the main branch in a week if no issues are found.
Best regards,
--
Nathan Chancellor <nathan@kernel.org>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-01-30 0:10 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-01-22 14:06 [PATCH v4 0/2] scripts: introduce containerized builds Guillaume Tucker
2026-01-22 14:06 ` [PATCH v4 1/2] scripts: add tool to run " Guillaume Tucker
2026-01-22 14:29 ` Onur Özkan
2026-01-22 14:59 ` Guillaume Tucker
2026-01-22 16:39 ` Onur Özkan
2026-01-22 18:49 ` Nathan Chancellor
2026-01-22 19:57 ` Nicolas Schier
2026-01-22 14:07 ` [PATCH v4 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
2026-01-22 14:22 ` Onur Özkan
2026-01-30 0:10 ` [PATCH v4 0/2] scripts: introduce containerized builds Nathan Chancellor
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox