Re: Kexec HandOver (KHO) discussion recap and open questions

[Pratyush Yadav <pratyush@kernel.org>]

Hi all,

On Thu, Nov 28 2024, David Rientjes wrote:

[...]
> ----->o-----
> Pratyush noted he has been working on supporting persistent tmpfs on top
> of KHO using its own structure that has a "stable" version.  Kirill
> expressed the concern that we need to be careful because it becomes part
> of the kernel ABI.
>
> Pratyush, would it be possible to share your example code that you were
> referring to in chat today in response to this thread?
>

The proof-of-concept can be found in my tree at [0]. I have tried to
very roughly split it into some parts and have left some quick
commentary on the patches. As I mentioned in the call, this is really
hacky code that I put together just to have a proof-of-concept and a
starting point for improving it into a workable feature. So be ready to
see all sorts of ugliness in there. You have been warned!

While the code is not yet in good shape, I would still very much
appreciate thoughts, ideas, and suggestions from people for ways I can
fundamentally change or improve the way I am doing this currently.

You can find an asciinema video of the file system in action at [1].

To try the changes out, boot the kernel with "kho_scratch=200M nokaslr"
added to your kernel command line. Then you can mount the file system
using:

    $ mount -t tmpfs -o kho=test test khofs/ # The string after kho= identifies this mount on later boots

Do some operations:

    $ cd khofs/
    $ echo "Test file" > foo
    $ cd ../

Activate KHO and kexec:

    $ echo 1 > /sys/kernel/kho/active
    $ kexec -l bzImage -s --initrd /boot/initramfs-linux.img --reuse-cmdline
    $ kexec -e

On the next boot, mount the file system again and check file contents:

    $ mount -t tmpfs -o kho=test test khofs/
    $ cat khofs/foo


Here's a quick overview of the patches that are in there. Each patch has
some commentary that goes in a bit more detail.

- mm/mm_init.c: avoid deferred init on MEMBLOCK_SCRATCH pages
- fixup! kexec: Add documentation for KHO
- fixup! kexec: Add Kexec HandOver (KHO) generation helpers

These are minor fixes on top of Alex's KHO patches. Mike, you might be
interested in some of these, depending on how similar your code looks to
Alex's.

- Introduce bitmap_kv[z]alloc() and bitmap_kvfree()
- Add ability to iterate superblocks in more ways
- Introduce {freeze,thaw}_super_locked()
- Introduce a super crude slab allocator

These introduce some features/hacks I need to get the KHO things
working.

- Add TMPFS_KHO config and option

This adds the Kconfig and mount option to enable KHO on a TMPFS mount.

- Introduce KHO inode

This patch might be of interest to people who want to see what that ABI
would look like. It doesn't try to tackle any of the tricky bits yet
though. More detailed commentary in the patch.

- Introduce the bulk of TMPFS-KHO ser-des and inode management logic

This patch adds the bulk of the KHO serialization, deserialization, and
initialization logic. This is grouped together in a somewhat big chunk
simply because I was too lazy to split it into smaller groups of
changes.

- Add support for directories

This patch adds support for creating KHO-persistent directories.

- Add support for adding data to files

This patch adds support for tracking data pages for files.

[0] https://git.kernel.org/pub/scm/linux/kernel/git/pratyush/linux.git/log/?h=tmpfs-kho
[1] https://asciinema.org/a/6IMxMwzlrB08XkmswNaMpUyTx

-- 
Regards,
Pratyush Yadav