Re: [RFC PATCH 1/5] misc: introduce FDBox

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Mar 18, 2025 at 11:02:31PM +0000, Pratyush Yadav wrote:

> I suppose we can serialize all FDs when the box is sealed and get rid of
> the struct file. If kexec fails, userspace can unseal the box, and FDs
> will be deserialized into a new struct file. This way, the behaviour
> from userspace perspective also stays the same regardless of whether
> kexec went through or not. This also helps tie FDBox closer to KHO.

I don't think we can do a proper de-serialization without going
through kexec. The new stuff Mike is posting for preserving memory
will not work like that.

I think error recovery wil have to work by just restoring access to
the FD and it's driver state that was never actually destroyed.

> > It sure would be nice if the freezing process could be managed
> > generically somehow.
> >
> > One option for freezing would have the kernel enforce that userspace
> > has closed and idled the FD everywhere (eg check the struct file
> > refcount == 1). If userspace doesn't have access to the FD then it is
> > effectively frozen.
> 
> Yes, that is what I want to do in the next revision. FDBox itself will
> not close the file descriptors when you put a FD in the box. It will
> just grab a reference and let the userspace close the FD. Then when the
> box is sealed, the operation can be refused if refcount != 1.

I'm not sure about this sealed idea..

One of the design points here was to have different phases for the KHO
process and we want to shift alot of work to the earlier phases. Some
of that work should be putting things into the fdbox, freezing them,
and writing out the serialzation as that may be quite time consuming.

The same is true for the deserialize step where we don't want to bulk
deserialize but do it in an ordered way to minimize the critical
downtime.

So I'm not sure if a 'seal' operation that goes and bulk serializes
everything makes sense. I still haven't seen a state flow chart and a
proposal where all the different required steps would have to land to
get any certainty here.

At least in my head I imagined you'd open the KHO FD, put it in
serializing mode and then go through in the right order pushing all
the work and building the serializion data structure as you go.

At the very end you'd finalize the KHO serialization, which just
writes out a little bit more to the FDT and gives you back the FDT
blob for the kexec. It should be a very fast operation.

Jason