Re: [LSF/MM/BPF TOPIC] BoF VM live migration over CXL memory​

[Gregory Price <gregory.price@memverge.com>]

On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote:
> Gregory Price <gregory.price@memverge.com> writes:
> 
> [snip]
> 
> > 2. During the migration process, the memory needs to be forced not to be
> >    migrated to another node by other means (tiering software, swap,
> >    etc).  The obvious way of doing this would be to migrate and
> >    temporarily pin the page... but going back to problem #1 we see that
> >    ZONE_MOVABLE and Pinning are mutually exclusive.  So that's
> >    troublesome.
> 
> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to
> avoid move pages out of CXL.mem node?  Now, there are gaps in tiering,
> but I think it is fixable.
> 
> Best Regards,
> Huang, Ying
> 
> [snip]

That feels like a hack/bodge rather than a proper solution to me.

Maybe this is an affirmative argument for the creation of an EXMEM
zone.  Specifically to allow page pinning, but with far more stringent
controls - i.e. the zone is excluded from use via general allocations.

The point of ZONE_MOVABLE is to allow general allocation of userland
data into hotpluggable memory regions.

This memory region is not for general use, and wants to allow pinning
and be hotpluggable under very controlled circumstances.  That seems
like a reasonable argument for the creation of EXMEM.

~Gregory