Re: [RFC PATCH 0/4] Allow persistent data on DAX device being used as KMEM

[David Hildenbrand <david@redhat.com>]

On 02.08.22 19:57, Srinivas Aji wrote:
> Linux supports adding a DAX driver managed memory region as system
> memory using the KMEM driver (from version 5.1). We would like to use
> a persistent addressable memory segment as system memory and
> simultaneously for storing some persistent data.
> 
> Motivation: It is already possible to partition an NVDIMM device for
> RAM and storage by creating separate regions on the device and using
> one of them with KMEM and another as fsdax. This patch set is a start
> to trying to get zero copy snapshots of processes which are using the
> DAX device as RAM. That requires dynamically sharing pages between
> process RAM and the storage within a single NVDIMM region.
> 
> To do this, we add a layer for handling the persistent data which does
> the following:
> 
> 1. When a DAX device is added as KMEM, mark all the memory as
>    allocated and pass it up to a module which is aware of the storage
>    layout.
> 
> 2. This module scans the memory, identifies the unused parts, and
>    frees those memory pages.
> 
> 3. Further memory from this device is allocated using the kernel
>    memory allocation API. The memory allocation API currently allows
>    the allocation to be limited only based on NUMA node. So this
>    feature works only when the DAX device used as KMEM is the only
>    memory from its NUMA node.
> 
> 4. Discarding of blocks previously used for persistent data results in
>    those blocks being freed to system memory.

Can you explain how "zero copy snapshots of processes" would work, both

a) From a user space POV
b) From a kernel-internal POV

Especially, what I get is that you have a filesystem on that memory
region, and all memory that is not used for filesystem blocks can be
used as ordinary system RAM (a little like shmem, but restricted to dax
memory regions?).

But how does this interact with zero-copy snapshots?

I feel like I am missing one piece where we really need system RAM as
part of the bigger picture. Hopefully it's not some hack that converts
system RAM to file system blocks :)

-- 
Thanks,

David / dhildenb