Re: [RFC PATCH 1/5] misc: introduce FDBox

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Mar 19, 2025 at 01:35:31PM +0000, Pratyush Yadav wrote:
> On Tue, Mar 18 2025, Jason Gunthorpe wrote:
> 
> > On Tue, Mar 18, 2025 at 11:02:31PM +0000, Pratyush Yadav wrote:
> >
> >> I suppose we can serialize all FDs when the box is sealed and get rid of
> >> the struct file. If kexec fails, userspace can unseal the box, and FDs
> >> will be deserialized into a new struct file. This way, the behaviour
> >> from userspace perspective also stays the same regardless of whether
> >> kexec went through or not. This also helps tie FDBox closer to KHO.
> >
> > I don't think we can do a proper de-serialization without going
> > through kexec. The new stuff Mike is posting for preserving memory
> > will not work like that.
> 
> Why not? If the next kernel can restore the file from the serialized
> content, so can the current kernel. What stops this from working with
> the new memory preservation scheme (which I assume is the idea you
> proposed in [0])? 

It is because the current kernel does not destroy the struct page
before the kexec and the new kernel assumes a zero'd fresh struct page
at restore.

So it would be very easy to corrupt the struct page information if you
attempt to deserialize without going through the kexec step.

There would be a big risk of getting things like refcounts out of
sync.

Then you have the issue that I don't actually imagine shutting down
something like iommufd, I was intending to leave it frozen in place
with all its allocations and so on. If you try to de-serialize you
can't de-serialize into the thing that is frozen, you'd create a new
one from empty. Now you have two things pointing at the same stuff,
what a mess.

> The seal operation does bulk serialize/deserialize for _one_ box. You
> can have multiple boxes and distribute your FDs in the boxes based on
> the serialize or deserialize order you want. Userspace decides when to
> seal or unseal a particular box, which gives it full control over the
> order in which things happen.

Why have more than one box? What is the point? I've been thinking we
should just have a KHO control char dev FD for serializing and you can
do all the operations people have been talking about in sysfs, as well
as record FDs for serializing.

Why do we need more than one fdbox container fd?

> All of this is made easier if each component has its own FDT (or any
> other data structure) and doesn't have to share the same FDT. This is
> the direction we are going in anyway with the next KHO versions.

Yes, I agree with that for sure.

Jason