Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

[Serge Hallyn <serge@hallyn.com>]

On Fri, Jun 09, 2023 at 02:42:47PM +0200, Christian Brauner wrote:
> On Fri, Jun 09, 2023 at 08:20:30AM -0400, Colin Walters wrote:
> > 
> > 
> > On Fri, Jun 9, 2023, at 7:45 AM, Christian Brauner wrote:
> > >
> > > Because the series you sent here touches on a lot of things in terms of
> > > infrastructure alone. That work could very well be rather interesting
> > > independent of PuzzleFS. We might just want to get enough infrastructure
> > > to start porting a tiny existing fs (binderfs or something similar
> > > small) to Rust to see how feasible this is and to wet our appetite for
> > > bigger changes such as accepting a new filesystem driver completely
> > > written in Rust.
> > 
> > (Not a kernel developer, but this argument makes sense to me)
> > 
> > > But aside from the infrastructure discussion:
> > >
> > > This is yet another filesystem for solving the container image problem
> > > in the kernel with the addition of yet another filesystem. We just went
> > > through this excercise with another filesystem. So I'd expect some
> > > reluctance here. Tbh, the container world keeps sending us filesystems
> > > at an alarming rate. That's two within a few months and that leaves a
> > > rather disorganized impression.
> > 
> > I am sure you are aware there's not some "container world"
> > monoculture, there are many organizations, people and companies here
> 
> That submission here explicitly references OCI v2. Composefs explicitly
> advertises 100% compatibility with OCI. So, there's a set of OCI specs

OCI v2 doesn't currently exist :)  There were many design goals for
it, and near as I can tell, after everyone discussed those together,
everyone went off to work on implementing the bits the needed - which
is a right and proper step before coming back and comparing notes
about what went well, etc.

The two main things where puzzlefs is experimenting are the use of
content defined chunking (CDC), and being written in rust (and,
especially, to have the same rust code base be used for the in kernel
driver, the fuse mounter, the builder, and the userspace extractor).
(Well, and reproducible images through a canonical representation,
but...)

It requires a POC in order to really determine whether the CDC will
be worth it, or will have pitfalls.  So far, it looks very promising.
Adding that functionality to composefs one day could be cool.

Likewise, it requires a user in order to push on the infrastructure
required to support a full filesystem in rust in the kernel.  But that
really isn't something we can "add to composefs".  :)

The main goal of this posting, then, was to show the infrastructure
pieces and work on that with the community.  We're definitely not
(currently :) asking for puzzlefs to be included.  However, it is
our (business and personal) justification for the rest of the work.

> including runtime and image. As far as I'm concerned you're all one
> container world under the OCI umbrella.

One big happy family!

> We're not going to push multiple filesystems into the kernel that all do
> slightly different things but all serve the OCI container world and use
> some spec as an argument to move stuff into the kernel.
> 
> The OCI initiative is hailed as unifying the container ecosystem. Hence,
> we can expect coordination.