Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug

[David Hildenbrand <david@redhat.com>]

> ---------------------
> Step 9: Memory Zones.
> ---------------------
> We've alluded to "Memory Zones" in prior sections, with really the only
> detail about these concepts being that there are "Kernel-allocation
> compatible" and "Movable" zones, as well as some relationship between
> memory blocks and memory zones.
> 
> The two zones we really care about are `ZONE_NORMAL` and `ZONE_MOVABLE`.
> 
> For the purpose of this reading we'll consider two basic use-cases:
> - memory block hot-unplug
> - kernel resource allocation
> 
> You can (for the most part) consider these cases incompatible.  If the
> kernel allocates `struct page` memory from a block, then that block cannot
> be hot-unplugged.  This memory is typically unmovable (cannot be migrated),
> and its pages unlikely to be removed from the memory map.
> 
> There are other scenarios, such as page pinning, that can block hot-unplug.
> The individual mechanisms preventing hot-unplug are less important than
> their relationship to memory zones.
> 
> ZONE_NORMAL basically allows any allocations, including things like page
> tables, struct pages, and pinned memory.
> 
> ZONE_MOVABLE, under normal conditions, disallows most kernel allocations.
> 

In essence, only movable allocations (some kernel allcoations are movable).

> ZONE_MOVABLE does NOT make a *strong* guarantee of hut-unplug-ability.
> The kernel and privileged users can cause long-term pinning to occur -
> even in ZONE_MOVABLE.  It should be seen as a best-attempt at providing
> hot-unplug-ability under normal conditions.

Yes and no; actual long-term pinning is disallowed (FOLL_LONGTERM), but 
we have a bunch of cases that need fixing. [1]

Of course, new cases keep popping up. It's a constant fight to make 
hot-unplug as reliable as possible. So yes, we cannot give "strong" 
guarantees, but make it as reliable as possible in sane configurations.

[1] 
https://lkml.kernel.org/r/882b566c-34d6-4e68-9447-6c74a0693f18@redhat.com


-- 
Cheers,

David / dhildenb