Re: [LSF/MM/BPF TOPIC] BoF VM live migration over CXL memory​

[David Hildenbrand <david@redhat.com>]

On 12.04.23 04:54, Huang, Ying wrote:
> Gregory Price <gregory.price@memverge.com> writes:
> 
>> On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote:
>>> Gregory Price <gregory.price@memverge.com> writes:
>>>
>>> [snip]
>>>
>>>> 2. During the migration process, the memory needs to be forced not to be
>>>>     migrated to another node by other means (tiering software, swap,
>>>>     etc).  The obvious way of doing this would be to migrate and
>>>>     temporarily pin the page... but going back to problem #1 we see that
>>>>     ZONE_MOVABLE and Pinning are mutually exclusive.  So that's
>>>>     troublesome.
>>>
>>> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to
>>> avoid move pages out of CXL.mem node?  Now, there are gaps in tiering,
>>> but I think it is fixable.
>>>
>>> Best Regards,
>>> Huang, Ying
>>>
>>> [snip]
>>
>> That feels like a hack/bodge rather than a proper solution to me.
>>
>> Maybe this is an affirmative argument for the creation of an EXMEM
>> zone.
> 
> Let's start with requirements.  What is the requirements for a new zone
> type?

I'm stills scratching my head regarding this. I keep hearing all 
different kind of statements that just add more confusions "we want it 
to be hotunpluggable" "we want to allow for long-term pinning memory" 
"but we still want it to be movable" "we want to place some unmovable 
allocations on it". Huh?

Just to clarify: ZONE_MOVABLE allows for pinning. It just doesn't allow 
for long-term pinning of memory.

For good reason, because long-term pinning of memory is just the worst 
(memory waste, fragmentation, overcommit) and instead of finding new 
ways to *avoid* long-term pinnings, we're coming up with advanced 
concepts to work-around the fundamental property of long-term pinnings.

We want all memory to be long-term pinnable and we want all memory to be 
movable/hotunpluggable. That's not going to work.

If you'd ask me today, my prediction is that ZONE_EXMEM is not going to 
happen.

-- 
Thanks,

David / dhildenb