[PATCH v2 0/2] scripts: introduce containerized builds

[PATCH v2 0/2] scripts: introduce containerized builds

[Guillaume Tucker]

This proposal emerged from an email discussion and a talk at Plumbers
last year:

    https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/

The aim is to facilitate reproducing builds for CI bots as well as
developers using containers.  Here's an illustrative example with a
kernel.org toolchain in a Docker image from tuxmake:

    $ scripts/container -i tuxmake/korg-clang-21 make LLVM=1 defconfig
      HOSTCC  scripts/basic/fixdep
      HOSTCC  scripts/kconfig/conf.o
    [...]
      HOSTCC  scripts/kconfig/util.o
      HOSTLD  scripts/kconfig/conf
    *** Default configuration is based on 'x86_64_defconfig'
    #
    # configuration written to .config
    #

and a follow-up command to build the kernel with the verbose flag
turned on to show DEBUG log messages from the container tool:

    $ scripts/container -i tuxmake/korg-clang-21 -v -- make LLVM=1 -j8
    [container DEBUG] runtime: docker
    [container DEBUG] image: tuxmake/korg-clang-21
    [container DEBUG] container: c5a88761-f55a-4027-84c9-bc3c6dc9c4cd
      GEN     arch/x86/include/generated/asm/orc_hash.h
      HOSTCC  scripts/basic/fixdep
      SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
    [...]
      BUILD   arch/x86/boot/bzImage
    Kernel: arch/x86/boot/bzImage is ready  (#1)

As a next step to make this tool more useful, I'm in the process of
preparing reference container images with kernel.org toolchains and no
third-party dependencies other than the base Debian distro:

    https://gitlab.com/gtucker/korg-containers

Say, to run KUnit using the latest kernel.org GCC toolchain:

    scripts/container \
        -i registry.gitlab.com/gtucker/korg-containers/gcc:kunit -- \
        tools/testing/kunit/kunit.py \
            run \
            --arch=x86_64 \
            --cross_compile=x86_64-linux-

This patch series also include a documentation page with all the
relevant details about how to use the tool and the images currently
available.

---
Changes in v2:
- Drop default image but make -i option required
- Look for Docker and Podman if no runtime specified
- Catch SIGINT from user to abort container with Docker
- Explicitly name each container with a UUID
- Update documentation accordingly

---

Guillaume Tucker (2):
  scripts: add tool to run containerized builds
  Documentation: dev-tools: add container.rst page

 Documentation/dev-tools/container.rst | 175 +++++++++++++++++++++++
 Documentation/dev-tools/index.rst     |   1 +
 scripts/container                     | 194 ++++++++++++++++++++++++++
 3 files changed, 370 insertions(+)
 create mode 100644 Documentation/dev-tools/container.rst
 create mode 100755 scripts/container

-- 
2.47.3

[PATCH v2 1/2] scripts: add tool to run containerized builds

[Guillaume Tucker]

Add a 'scripts/container' tool written in Python to run any command in
the source tree from within a container.  This can typically be used
to call 'make' with a compiler toolchain image to run reproducible
builds but any arbitrary command can be run too.  Only Docker and
Podman are supported for this initial version.

Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Link: https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
 scripts/container | 194 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100755 scripts/container

diff --git a/scripts/container b/scripts/container
new file mode 100755
index 000000000000..2d0143c7d43e
--- /dev/null
+++ b/scripts/container
@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2025 Guillaume Tucker
+
+"""Containerized builds"""
+
+import abc
+import argparse
+import logging
+import os
+import shutil
+import subprocess
+import sys
+import uuid
+
+
+class ContainerRuntime(abc.ABC):
+    """Base class for a container runtime implementation"""
+
+    name = None  # Property defined in each implementation class
+
+    def __init__(self, args, logger):
+        self._uid = args.uid or os.getuid()
+        self._gid = args.gid or args.uid or os.getgid()
+        self._env_file = args.env_file
+        self._logger = logger
+
+    @classmethod
+    def is_present(cls):
+        """Determine whether the runtime is present on the system"""
+        return shutil.which(cls.name) is not None
+
+    @abc.abstractmethod
+    def _do_run(self, image, cmd, container_name):
+        """Runtime-specific handler to run a command in a container"""
+
+    @abc.abstractmethod
+    def _do_abort(self, container_name):
+        """Runtime-specific handler to abort a command in running container"""
+
+    def run(self, image, cmd):
+        """Run a command in a runtime container"""
+        container_name = str(uuid.uuid4())
+        self._logger.debug("container: %s", container_name)
+        try:
+            return self._do_run(image, cmd, container_name)
+        except KeyboardInterrupt:
+            self._logger.error("user aborted")
+            self._do_abort(container_name)
+            return 1
+
+
+class DockerRuntime(ContainerRuntime):
+    """Run a command in a Docker container"""
+
+    name = 'docker'
+
+    def _do_run(self, image, cmd, container_name):
+        cmdline = [
+            'docker', 'run',
+            '--name', container_name,
+            '--rm',
+            '--tty',
+            '--volume', f'{os.getcwd()}:/src',
+            '--workdir', '/src',
+            '--user', f'{self._uid}:{self._gid}'
+        ]
+        if self._env_file:
+            cmdline += ['--env-file', self._env_file]
+        cmdline.append(image)
+        cmdline += cmd
+        return subprocess.call(cmdline)
+
+    def _do_abort(self, container_name):
+        subprocess.call(['docker', 'kill', container_name])
+
+
+class PodmanRuntime(ContainerRuntime):
+    """Run a command in a Podman container"""
+
+    name = 'podman'
+
+    def _do_run(self, image, cmd, container_name):
+        cmdline = [
+            'podman', 'run',
+            '--name', container_name,
+            '--rm',
+            '--tty',
+            '--interactive',
+            '--volume', f'{os.getcwd()}:/src',
+            '--workdir', '/src',
+            '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
+        ]
+        if self._env_file:
+            cmdline += ['--env-file', self._env_file]
+        cmdline.append(image)
+        cmdline += cmd
+        return subprocess.call(cmdline)
+
+    def _do_abort(self, container_name):
+        pass  # Signals are handled by Podman in interactive mode
+
+
+class Runtimes:
+    """List of all supported runtimes"""
+
+    runtimes = [DockerRuntime, PodmanRuntime]
+
+    @classmethod
+    def get_names(cls):
+        """Get a list of all the runtime names"""
+        return list(runtime.name for runtime in cls.runtimes)
+
+    @classmethod
+    def get(cls, name):
+        """Get a single runtime class matching the given name"""
+        for runtime in cls.runtimes:
+            if runtime.name == name:
+                if not runtime.is_present():
+                    raise ValueError(f"runtime not found: {name}")
+                return runtime
+        raise ValueError(f"unknown runtime: {runtime}")
+
+    @classmethod
+    def find(cls):
+        """Find the first runtime present on the system"""
+        for runtime in cls.runtimes:
+            if runtime.is_present():
+                return runtime
+        raise ValueError("no runtime found")
+
+
+def _get_logger(verbose):
+    """Set up a logger with the appropriate level"""
+    logger = logging.getLogger('container')
+    handler = logging.StreamHandler()
+    handler.setFormatter(logging.Formatter(
+        fmt='[container {levelname}] {message}', style='{'
+    ))
+    logger.addHandler(handler)
+    logger.setLevel(logging.DEBUG if verbose is True else logging.INFO)
+    return logger
+
+
+def main(args):
+    """Main entry point for the container tool"""
+    logger = _get_logger(args.verbose)
+    try:
+        cls = Runtimes.get(args.runtime) if args.runtime else Runtimes.find()
+    except ValueError as ex:
+        logger.error(ex)
+        return 1
+    logger.debug("runtime: %s", cls.name)
+    logger.debug("image: %s", args.image)
+    return cls(args, logger).run(args.image, args.cmd)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        'container',
+        description="Containerized builds.  See the dev-tools/container "
+        "kernel documentation section for more details."
+    )
+    parser.add_argument(
+        '-e', '--env-file',
+        help="Path to an environment file to load in the container."
+    )
+    parser.add_argument(
+        '-g', '--gid',
+        help="Group ID to use inside the container."
+    )
+    parser.add_argument(
+        '-i', '--image', required=True,
+        help="Container image name."
+    )
+    parser.add_argument(
+        '-r', '--runtime', choices=Runtimes.get_names(),
+        help="Container runtime name.  If not specified, the first one found "
+        "on the system will be used i.e. Docker if present, otherwise Podman."
+    )
+    parser.add_argument(
+        '-u', '--uid',
+        help="User ID to use inside the container.  If the -g option is not "
+        "specified, the user ID will also be set as the group ID."
+    )
+    parser.add_argument(
+        '-v', '--verbose', action='store_true',
+        help="Enable verbose output."
+    )
+    parser.add_argument(
+        'cmd', nargs='+',
+        help="Command to run in the container"
+    )
+    sys.exit(main(parser.parse_args(sys.argv[1:])))
-- 
2.47.3

[PATCH v2 2/2] Documentation: dev-tools: add container.rst page

[Guillaume Tucker]

Add a dev-tools/container.rst documentation page for the
scripts/container tool.  This covers the basic usage with additional
information about environment variables and user IDs.  It also
includes a number of practical examples with a reference to the
experimental kernel.org toolchain images.

Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
 Documentation/dev-tools/container.rst | 175 ++++++++++++++++++++++++++
 Documentation/dev-tools/index.rst     |   1 +
 2 files changed, 176 insertions(+)
 create mode 100644 Documentation/dev-tools/container.rst

diff --git a/Documentation/dev-tools/container.rst b/Documentation/dev-tools/container.rst
new file mode 100644
index 000000000000..5254feae02c2
--- /dev/null
+++ b/Documentation/dev-tools/container.rst
@@ -0,0 +1,175 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2025 Guillaume Tucker
+
+====================
+Containerized Builds
+====================
+
+The ``container`` tool can be used to run any command in the kernel source tree
+from within a container.  Doing so facilitates reproducing builds across
+various platforms, for example when a test bot has reported an issue which
+requires a specific version of a compiler or an external test suite.  While
+this can already be done by users who are familiar with containers, having a
+dedicated tool in the kernel tree lowers the barrier to entry by solving common
+problems once and for all (e.g. user id management).  It also makes it easier
+to share an exact command line leading to a particular result.  The main use
+case is likely to be kernel builds but virtually anything can be run: KUnit,
+checkpatch etc. provided a suitable image is available.
+
+
+Options
+=======
+
+Command line syntax::
+
+  scripts/container -i IMAGE [OPTION]... CMD...
+
+Available options:
+
+``-e, --env-file ENV_FILE``
+
+    Path to an environment file to load in the container.
+
+``-g, --gid GID``
+
+    Group id to use inside the container.
+
+``-i, --image IMAGE``
+
+    Container image name (required).
+
+``-r, --runtime RUNTIME``
+
+    Container runtime name.  Supported runtimes: ``docker``, ``podman``.
+
+    If not specified, the first one found on the system will be used
+    i.e. Docker if present, otherwise Podman.
+
+``-u, --uid UID``
+
+    User id to use inside the container.
+
+    If the ``-g`` option is not specified, the user id will also be used for
+    the group id.
+
+``-v, --verbose``
+
+    Enable verbose output.
+
+``-h, --help``
+
+    Show the help message and exit.
+
+
+Usage
+=====
+
+It's entirely up to the user to choose which image to use and the ``CMD``
+arguments are passed directly as an arbitrary command line to run in the
+container.  The tool will take care of mounting the source tree as the current
+working directory and adjust the user and group id as needed.
+
+The container image which would typically include a compiler toolchain is
+provided by the user and selected via the ``-i`` option.  The container runtime
+can be selected with the ``-r`` option, which can be either ``docker`` or
+``podman``.  If none is specified, the first one found on the system will be
+used.  Support for other runtimes may be added later depending on their
+popularity among users.
+
+
+Environment Variables
+=====================
+
+Environment variables are not propagated to the container so they have to be
+either defined in the image itself or via the ``-e`` option using an
+environment file.  In some cases it makes more sense to have them defined in
+the Containerfile used to create the image.  For example, a Clang-only compiler
+toolchain image may have ``LLVM=1`` defined.  The local environment file is
+more useful for user-specific variables added during development.
+
+Please note that ``make`` options can still be passed on the command line, so
+while this can't be done since the first argument needs to be the executable::
+
+  scripts/container -i tuxmake/korg-clang LLVM=1 make
+
+this will work::
+
+  scripts/container -i tuxmake/korg-clang make LLVM=1
+
+
+User IDs
+========
+
+This is an area where the behaviour will vary slightly depending on the
+container runtime.  The goal is to run commands as the user invoking the tool.
+With Podman, a namespace is created to map the current user id to a different
+one in the container (1000 by default).  With Docker, while this is also
+possible with recent versions it requires a special feature to be enabled in
+the daemon so it's not used here for simplicity.  Instead, the container is run
+with the current user id directly.  In both cases, this will provide the same
+file permissions for the kernel source tree mounted as a volume.  The only
+difference is that when using Docker without a namespace, the user id may not
+be the same as the default one set in the image.
+
+Say, we're using an image which sets up a default user with id 1000 and the
+current user calling the ``container`` tool has id 1234.  The kernel source
+tree was checked out by this same user so the files belong to user 1234.  With
+Podman, the container will be running as user id 1000 with a mapping to id 1234
+so that the files from the mounted volume appear to belong to id 1000 inside
+the container.  With Docker and no namespace, the container will be running
+with user id 1234 which can access the files in the volume but not in the user
+1000 home directory.  This shouldn't be an issue when running commands only in
+the kernel tree but it is worth highlighting here as it might matter for
+special corner cases.
+
+
+Examples
+========
+
+The shortest example is to run a basic kernel build using Docker and a tuxmake
+Clang image::
+
+  scripts/container -i tuxmake/korg-clang -- make LLVM=1 defconfig
+  scripts/container -i tuxmake/korg-clang -- make LLVM=1 -j$(nproc)
+
+.. note::
+
+   When running a command with options within the container, it should be
+   separated with a double dash ``--`` to not confuse them with the
+   ``container`` tool options.  Plain commands with no options don't strictly
+   require the double dashes e.g.::
+
+     scripts/container -i tuxmake/korg-clang make mrproper
+
+To run ``checkpatch.pl`` in a ``patches`` directory with a generic image::
+
+  scripts/container -i perl:slim-trixie scripts/checkpatch.pl patches/*
+
+The examples below refer to ``kernel.org`` images which are based on the
+`kernel.org compiler toolchains
+<https://mirrors.edge.kernel.org/pub/tools/>`__.  These aren't (yet) officially
+available in any public registry but users can build their own locally instead
+using this `experimental repository
+<https://gitlab.com/gtucker/korg-containers>`__ by running ``make
+PREFIX=kernel.org/``.
+
+To build just ``bzImage`` using Clang::
+
+  scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
+
+Same with GCC 15 as a particular version tag::
+
+  scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
+
+To run KUnit::
+
+  scripts/container -i kernel.org/gcc:kunit -- \
+      tools/testing/kunit/kunit.py \
+          run \
+          --arch=x86_64 \
+          --cross_compile=x86_64-linux-
+
+To build the HTML documentation, which requires the ``kdocs`` image built with
+``make PREFIX=kernel.org/ extra`` as it's not a compiler toolchain::
+
+  scripts/container -i kernel.org/kdocs make htmldocs
diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
index 4b8425e348ab..527a0e4cf2ed 100644
--- a/Documentation/dev-tools/index.rst
+++ b/Documentation/dev-tools/index.rst
@@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
    gpio-sloppy-logic-analyzer
    autofdo
    propeller
+   container
 
 
 .. only::  subproject and html
-- 
2.47.3

Re: [PATCH v2 1/2] scripts: add tool to run containerized builds

[Nathan Chancellor]

On Thu, Dec 18, 2025 at 01:49:52PM +0100, Guillaume Tucker wrote:
...
> +    def __init__(self, args, logger):

Adding something like

    self._args = [
        '--rm',
        '--tty',
        '--volume', f'{os.getcwd()}:/src',
        '--workdir', '/src',
    ]

here then adding an __init__() in the subclasses to append the runtime
specific arguments would allow _do_run() to be moved into
ContainerRuntime(). Otherwise, this looks pretty good and extensible.

> +        self._uid = args.uid or os.getuid()
> +        self._gid = args.gid or args.uid or os.getgid()
> +        self._env_file = args.env_file
> +        self._logger = logger
> +
> +    @classmethod
> +    def is_present(cls):
> +        """Determine whether the runtime is present on the system"""
> +        return shutil.which(cls.name) is not None
> +
> +    @abc.abstractmethod
> +    def _do_run(self, image, cmd, container_name):
> +        """Runtime-specific handler to run a command in a container"""
> +
> +    @abc.abstractmethod
> +    def _do_abort(self, container_name):
> +        """Runtime-specific handler to abort a command in running container"""
> +
> +    def run(self, image, cmd):
> +        """Run a command in a runtime container"""
> +        container_name = str(uuid.uuid4())
> +        self._logger.debug("container: %s", container_name)
> +        try:
> +            return self._do_run(image, cmd, container_name)
> +        except KeyboardInterrupt:
> +            self._logger.error("user aborted")
> +            self._do_abort(container_name)
> +            return 1
> +
> +
> +class DockerRuntime(ContainerRuntime):
> +    """Run a command in a Docker container"""
> +
> +    name = 'docker'
> +
> +    def _do_run(self, image, cmd, container_name):
> +        cmdline = [
> +            'docker', 'run',
> +            '--name', container_name,
> +            '--rm',
> +            '--tty',
> +            '--volume', f'{os.getcwd()}:/src',
> +            '--workdir', '/src',
> +            '--user', f'{self._uid}:{self._gid}'
> +        ]
> +        if self._env_file:
> +            cmdline += ['--env-file', self._env_file]
> +        cmdline.append(image)
> +        cmdline += cmd
> +        return subprocess.call(cmdline)
> +
> +    def _do_abort(self, container_name):
> +        subprocess.call(['docker', 'kill', container_name])
> +
> +
> +class PodmanRuntime(ContainerRuntime):
> +    """Run a command in a Podman container"""
> +
> +    name = 'podman'
> +
> +    def _do_run(self, image, cmd, container_name):
> +        cmdline = [
> +            'podman', 'run',
> +            '--name', container_name,
> +            '--rm',
> +            '--tty',
> +            '--interactive',
> +            '--volume', f'{os.getcwd()}:/src',
> +            '--workdir', '/src',
> +            '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
> +        ]
> +        if self._env_file:
> +            cmdline += ['--env-file', self._env_file]
> +        cmdline.append(image)
> +        cmdline += cmd
> +        return subprocess.call(cmdline)
> +
> +    def _do_abort(self, container_name):
> +        pass  # Signals are handled by Podman in interactive mode

Cheers,
Nathan