Re: [LSF/MM/BPF TOPIC] BoF VM live migration over CXL memory​

["Huang, Ying" <ying.huang@intel.com>]

Gregory Price <gregory.price@memverge.com> writes:

> On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote:
>> Gregory Price <gregory.price@memverge.com> writes:
>> 
>> [snip]
>> 
>> > 2. During the migration process, the memory needs to be forced not to be
>> >    migrated to another node by other means (tiering software, swap,
>> >    etc).  The obvious way of doing this would be to migrate and
>> >    temporarily pin the page... but going back to problem #1 we see that
>> >    ZONE_MOVABLE and Pinning are mutually exclusive.  So that's
>> >    troublesome.
>> 
>> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to
>> avoid move pages out of CXL.mem node?  Now, there are gaps in tiering,
>> but I think it is fixable.
>> 
>> Best Regards,
>> Huang, Ying
>> 
>> [snip]
>
> That feels like a hack/bodge rather than a proper solution to me.
>
> Maybe this is an affirmative argument for the creation of an EXMEM
> zone.

Let's start with requirements.  What is the requirements for a new zone
type?

> Specifically to allow page pinning, but with far more stringent
> controls -

> i.e. the zone is excluded from use via general allocations.

This can also be controlled via memory policy.  The alternative solution
is to add a per node attribute.

> The point of ZONE_MOVABLE is to allow general allocation of userland
> data into hotpluggable memory regions.

IIUC, one typical requirement of CXL.mem is hotpluggable, right?

Best Regards,
Huang, Ying

> This memory region is not for general use, and wants to allow pinning
> and be hotpluggable under very controlled circumstances.  That seems
> like a reasonable argument for the creation of EXMEM.
>
> ~Gregory