Hosting first-party kernel.org container images

[Guillaume Tucker <gtucker@gtucker.io>]

Hi Konstantin, Arnd et al,

This is a follow-up from the series about adding a scripts/container
tool [1] to run kernel builds in containers.  As per the discussion
at Plumbers last year and the summary I put in a blog post [2], it
would be great to have container images with kernel.org toolchains
hosted upstream.  This can mean several things, so let's break it
down into a set of potential options to choose from:


* Containerfiles Git repository

There is currently a PoC repository on GitLab with a Makefile and a
number of Containerfiles to build a set of images:

     https://gitlab.com/gtucker/korg-containers

It can be improved in many ways since this is an early PoC.  The key
decision to make here, if we do want to have container images
supported upstream, is how to manage these files or a derived
implementation.

One option is to add it to the kernel tree itself under e.g.
tools/container.

Another option is to add a separate repository on git.kernel.org,
which I believe would be a better approach as there aren't any direct
dependencies on the kernel tree itself.

A third option might be to keep it alongside any recipes used to
produce the existing kernel.org toolchain tarballs although I'm not
entirely sure how that's managed - something for Arnd to judge I
guess.

A last option would be to keep it on GitLab or move it to GitHub
which would provide some CI/CD tools for building the images but I
doubt this is something viable for the kernel community as it would
create some vendor lock-in.


* Container image registry

This is where things get a bit more complicated.  As far as I'm
aware, there aren't any container registries hosted in the kernel.org
infrastructure at the moment.  A classic option would be to push the
images to an established one e.g. Docker Hub (docker.io) or the
Google Artifact Registry.  GitLab and GitHub also provide theirs of
course.  I believe there is still a free plan for community projects
to host images on docker.io and that would be the easiest from a user
point of view e.g. "docker pull kernel.org/gcc".  It comes with some
maintenance burden of course, and Docker Hub has a history of
changing its policies quite unexpectedly so it's not entirely
future-proof.

A classic alternative would be to host a dedicated service
e.g. registry.kernel.org and have the images managed there.  This
would obviously involve higher sysadmin efforts and add scalability
issues but would decouple it from external providers.

Then a third option would be to host the container images as OCI
tarball dumps alongside the toolchain tarballs.  They can then be
downloaded and imported with "docker image load" or any other
container runtime.  The only infrastructure resources needed would be
storage space.  This is of course suboptimal as all the layers get
bundled together and users would have to manage these images
themselves, but it's very effective from a kernel.org sysadmin point
of view.


There are undoubtedly other ways to look at this, I'm curious to know
what people think.  The benefits of having readily-available
container images upstream appear to be pretty clear, several
maintainers have expressed their support already.  It's all down to
how much these benefits can outweigh the upstream maintenance costs.
Maybe this can be done in two steps, first with just the
Containerfiles and later on a full solution to host the actual
images.

Best wishes,
Guillaume

[1] https://lore.kernel.org/all/cover.1766061692.git.gtucker@gtucker.io/
[2] https://gtucker.io/posts/2024-09-30-korg-containers/