Re: [LSF/MM/BPF TOPIC] memory persistence over kexec

[Zhu Yanjun <yanjun.zhu@linux.dev>]

在 2025/1/20 20:42, David Rientjes 写道:
> On Mon, 20 Jan 2025, Jason Gunthorpe wrote:
> 
>> On Mon, Jan 20, 2025 at 09:54:15AM +0200, Mike Rapoport wrote:
>>> Hi,
>>>
>>> I'd like to discuss memory persistence across kexec.
>>>
>>> Currently there is ongoing work on Kexec HandOver (KHO) [1] that allows
>>> serialization and deserialization of kernel data as well as preserving
>>> arbitrary memory ranges across kexec.
>>>
>>> In addition, KHO keeps a physically contiguous memory regions that are
>>> guaranteed to not have any memory that KHO would preserve, but still can be
>>> used by the system. The kexeced kernel bootstraps itself using those
>>> regions and sets all handed over memory as in use. KHO users then can
>>> recover their state from the preserved data. This includes memory
>>> reservations, where the user can either discard or claim reservations.
>>>
>>> KHO can be used as the base layer for implementation of persistence-aware
>>> memory allocator and persistent in-memory filesystem.
>>>
>>> Aside from status update on KHO progress there are a few topics that I would
>>> like to discuss:
>>> * Is it feasible and desirable to enable KHO support in tmpfs and hugetlbfs?
> 
> This is a very timely discussion since the last Linux MM Alignment Session
> on the topic since some use cases, at least for tmpfs, have emerged.  Not
> necessarily a requirement, but more out of convenience.
> 
>>> * Or is it better to implement yet another in-memory filesystem dedicated
>>>    for persistence?
>>> * What is the best way to ensure that the memory we want to persist is not
>>>    scattered all over the place?
>>
>> There is alot of talk about taking *drivers* and having them survive
>> kexec, meaning the driver has to put alot of its state into KHO and
>> then get it back out again.
>>
>> I've been hoping for a model where a driver can be told to "go to KHO"
>> and the KHO code can be largely contained in the driver and regulated
>> to recording the driver state. This implies the state may be
>> fragmented all over memory.
>>
> 
> This sounds fantastic if it is doable!
> 
>> The other direction is that the driver has to start up in some special
>> KHO mode and KHO becomes invasive on all driver paths to use special
>> KHO allocations. This seems like a PITA.
>>
>> You can see this difference just in the discussion around the iommu
>> serialization where one idea was to have KHO be an integral (and
>> invasive!) part of the page table operations from time zero vs some
>> later serialization at kexec time.
>>
>> Regardless, I'm interested in this discussion to bring some
>> concreteness about how drivers work..
>>
> 
> +1, I'm also interested in this discussion.

+1, Hope I can join the meeting to listen to this presentation.

Zhu Yanjun

> 
> As previously mentioned[1], we'll also start a biweekly on hypervisor live
> update to accelerate progress.  The first instance of that meeting will be
> next week, Monday, January 27 at 8am PST (UTC-8).  Calendar invites will
> go out later today for everybody on that email thread, if anybody else is
> interested in attending on a regular basis please email me.  Hoping this
> can be leveraged as well to build up to LSF/MM/BPF.

> 
> [1]
> https://lore.kernel.org/kexec/2908e4ab-abc4-ddd0-b191-fe820856cfb4@google.com/T/#u
>