Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>]

On Thu, Sep 28, 2023 at 06:22:54AM -0700, Dave Hansen wrote:
> On 9/27/23 09:13, Stanislav Kinsburskii wrote:
> > Once deposited, these pages can't be accessed by Linux anymore and thus
> > must be preserved in "used" state across kexec, as hypervisor state is
> > unware of kexec.
> 
> If Linux can't access them, they're not RAM any more.  I'd much rather
> remove them from the memory map and move on with life rather than
> implement a bunch of new ABI that's got to be handed across kernels.

Could you elaborate more on the new ABIs? FDT is handled by x86 already,
and passing it over kexec looks like a natural extension.
Also, adding more state to it also doens't look like a new ABI.
Or does it?

Let me also comment on removing this regions from the memory map. The
major peculiarity here is that hypervisor distinguish between the pages,
deposited for guests to rnu and the pages deposited for the Linux root
partition to keep the guest-related portion of hypervisor state in the
root partition. And the latter is the matter in question.

We can indeed isolate and deposit a excessive amount of memory upfront
in hope that hypervisor will never get into the situation, when it needs
more memory.
However, it's not reliable, as the amount of memory will always be an
estimation, depending on the number of expected guests, guest-attached
devices, etc. And this becomes even a bigger problem when most of the
memory is already removed from the memory map to host guest partitions.
It's also not efficient as the amount of memory required by hypervisor
can grow or shrink depending on the use case or host configuration, and
deposting excessive amount of memory will be a waste.

But, actually, the idea of removing the pages from memory map was
reflected to some extent in the first version of this proposal,
so let me elaborate on it a bit.

Effectively, instead of reserving and depositing a lot of memory to
hypervisor upfront, the memory can be allocated from kernel memory when
needed and then returned back when unused.
This would still require pages removal from the memory map upon kexec,
but that's another problem.

Thanks,
Stanislav