Hosting first-party kernel.org container images

Hosting first-party kernel.org container images

[Guillaume Tucker]

Hi Konstantin, Arnd et al,

This is a follow-up from the series about adding a scripts/container
tool [1] to run kernel builds in containers.  As per the discussion
at Plumbers last year and the summary I put in a blog post [2], it
would be great to have container images with kernel.org toolchains
hosted upstream.  This can mean several things, so let's break it
down into a set of potential options to choose from:


* Containerfiles Git repository

There is currently a PoC repository on GitLab with a Makefile and a
number of Containerfiles to build a set of images:

     https://gitlab.com/gtucker/korg-containers

It can be improved in many ways since this is an early PoC.  The key
decision to make here, if we do want to have container images
supported upstream, is how to manage these files or a derived
implementation.

One option is to add it to the kernel tree itself under e.g.
tools/container.

Another option is to add a separate repository on git.kernel.org,
which I believe would be a better approach as there aren't any direct
dependencies on the kernel tree itself.

A third option might be to keep it alongside any recipes used to
produce the existing kernel.org toolchain tarballs although I'm not
entirely sure how that's managed - something for Arnd to judge I
guess.

A last option would be to keep it on GitLab or move it to GitHub
which would provide some CI/CD tools for building the images but I
doubt this is something viable for the kernel community as it would
create some vendor lock-in.


* Container image registry

This is where things get a bit more complicated.  As far as I'm
aware, there aren't any container registries hosted in the kernel.org
infrastructure at the moment.  A classic option would be to push the
images to an established one e.g. Docker Hub (docker.io) or the
Google Artifact Registry.  GitLab and GitHub also provide theirs of
course.  I believe there is still a free plan for community projects
to host images on docker.io and that would be the easiest from a user
point of view e.g. "docker pull kernel.org/gcc".  It comes with some
maintenance burden of course, and Docker Hub has a history of
changing its policies quite unexpectedly so it's not entirely
future-proof.

A classic alternative would be to host a dedicated service
e.g. registry.kernel.org and have the images managed there.  This
would obviously involve higher sysadmin efforts and add scalability
issues but would decouple it from external providers.

Then a third option would be to host the container images as OCI
tarball dumps alongside the toolchain tarballs.  They can then be
downloaded and imported with "docker image load" or any other
container runtime.  The only infrastructure resources needed would be
storage space.  This is of course suboptimal as all the layers get
bundled together and users would have to manage these images
themselves, but it's very effective from a kernel.org sysadmin point
of view.


There are undoubtedly other ways to look at this, I'm curious to know
what people think.  The benefits of having readily-available
container images upstream appear to be pretty clear, several
maintainers have expressed their support already.  It's all down to
how much these benefits can outweigh the upstream maintenance costs.
Maybe this can be done in two steps, first with just the
Containerfiles and later on a full solution to host the actual
images.

Best wishes,
Guillaume

[1] https://lore.kernel.org/all/cover.1766061692.git.gtucker@gtucker.io/
[2] https://gtucker.io/posts/2024-09-30-korg-containers/

Re: Hosting first-party kernel.org container images

[Ben Copeland]

Hello Guillaume,

On Fri, 19 Dec 2025 at 13:37, Guillaume Tucker <gtucker@gtucker.io> wrote:
>
> Hi Konstantin, Arnd et al,
>
> This is a follow-up from the series about adding a scripts/container
> tool [1] to run kernel builds in containers.  As per the discussion
> at Plumbers last year and the summary I put in a blog post [2], it
> would be great to have container images with kernel.org toolchains
> hosted upstream.  This can mean several things, so let's break it
> down into a set of potential options to choose from:
>
>
> * Containerfiles Git repository
>
> There is currently a PoC repository on GitLab with a Makefile and a
> number of Containerfiles to build a set of images:
>
>      https://gitlab.com/gtucker/korg-containers

TuxMake already provides container images with kernel.org toolchains
(korg-gcc 8-15, korg-clang 11-22). The Dockerfiles are maintained at
[1]

Since TuxMake is now part of KernelCI and already referenced in the
kernel documentation you pushed [2], it seems like a natural home for
this rather than starting fresh, or having two places for images.

>
> It can be improved in many ways since this is an early PoC.  The key
> decision to make here, if we do want to have container images
> supported upstream, is how to manage these files or a derived
> implementation.
>
> One option is to add it to the kernel tree itself under e.g.
> tools/container.
>
> Another option is to add a separate repository on git.kernel.org,
> which I believe would be a better approach as there aren't any direct
> dependencies on the kernel tree itself.
>
> A third option might be to keep it alongside any recipes used to
> produce the existing kernel.org toolchain tarballs although I'm not
> entirely sure how that's managed - something for Arnd to judge I
> guess.
>
> A last option would be to keep it on GitLab or move it to GitHub
> which would provide some CI/CD tools for building the images but I
> doubt this is something viable for the kernel community as it would
> create some vendor lock-in.
>
>
> * Container image registry
>
> This is where things get a bit more complicated.  As far as I'm
> aware, there aren't any container registries hosted in the kernel.org
> infrastructure at the moment.  A classic option would be to push the
> images to an established one e.g. Docker Hub (docker.io) or the
> Google Artifact Registry.  GitLab and GitHub also provide theirs of
> course.  I believe there is still a free plan for community projects
> to host images on docker.io and that would be the easiest from a user
> point of view e.g. "docker pull kernel.org/gcc".  It comes with some
> maintenance burden of course, and Docker Hub has a history of
> changing its policies quite unexpectedly so it's not entirely
> future-proof.

The images are hosted on Docker Hub (https://hub.docker.com/u/tuxmake)
with ECR Public as fallback:

      docker pull tuxmake/arm64_korg-gcc-14
      docker pull tuxmake/x86_64_korg-clang-22

Happy to discuss how we can align efforts here. But to me, it makes
sense for KernelCI to be the place for these images (or for us to have
a single place and a single set of images).

[1] https://github.com/kernelci/tuxmake/tree/master/support/docker
[2] https://www.kernel.org/doc/html/next/dev-tools/container.html

Regards,

Ben

>
> A classic alternative would be to host a dedicated service
> e.g. registry.kernel.org and have the images managed there.  This
> would obviously involve higher sysadmin efforts and add scalability
> issues but would decouple it from external providers.
>
> Then a third option would be to host the container images as OCI
> tarball dumps alongside the toolchain tarballs.  They can then be
> downloaded and imported with "docker image load" or any other
> container runtime.  The only infrastructure resources needed would be
> storage space.  This is of course suboptimal as all the layers get
> bundled together and users would have to manage these images
> themselves, but it's very effective from a kernel.org sysadmin point
> of view.
>
>
> There are undoubtedly other ways to look at this, I'm curious to know
> what people think.  The benefits of having readily-available
> container images upstream appear to be pretty clear, several
> maintainers have expressed their support already.  It's all down to
> how much these benefits can outweigh the upstream maintenance costs.
> Maybe this can be done in two steps, first with just the
> Containerfiles and later on a full solution to host the actual
> images.
>
> Best wishes,
> Guillaume
>
> [1] https://lore.kernel.org/all/cover.1766061692.git.gtucker@gtucker.io/
> [2] https://gtucker.io/posts/2024-09-30-korg-containers/