Re: [RFC] Virtualizing tagged disaggregated memory capacity (app specific, multi host shared)

[Gregory Price <gourry@gourry.net>]

On Tue, Oct 22, 2024 at 11:33:07AM +0200, David Hildenbrand wrote:
> On 20.09.24 11:06, Gregory Price wrote:
> 
> > The only concern is when insufficient ZONE_NORMAL exists to support
> > ZONE_MOVABLE capacity - but this is unlikely to be the general scenario AND
> > can be mitigated w/ existing mechanisms.
> 
> It might be worthwhile looking at
> Documentation/admin-guide/mm/memory-hotplug.rst "auto-movable" memory
> onlining polciy. It might not fit all sue cases, though (just like
> ZONE_MOVABLE doesn't)
> 

I managed to miss auto-movable in my last pass through there. Though for
our use-case, forcibly preventing ZONE_NORMAL for all CXL is the preferred
option in an effort to keep as much kernel resources out of high latency
memory.

So I think we're just going to end up using memhp_default_state, and that'll
be mostly fine.

> > 
> > Manually onlined capacity defaults to ZONE_MOVABLE.
> > 
> > It would be nice to make this behavior consistent, since the general opinion
> > appears to be that this capacity should default to ZONE_MOVABLE.
> 
> It's much easier to shoot yourself into the foot with ZONE_MOVABLE, that's
> why the default can be adjusted manually using "online_movable" with e.g.,
> memhp_default_state.
> 
> It's all a bit complicated, because there are various use cases and
> mechanisms for memory hotplug ... IIRC RHEL defaults with its udev rules to
> "ZONE_MOVABLE" on bare metal and "ZONE_NORMAL" in VMs. Except on s390, where
> we default to "offline" (standby memory ....).
> 
> I once worked on a systemd unit to make this configuration easier (and avoid
> udev rules), and possibly more "automatic" depending on the detected
> environment.
>

Appreciate the additional context, thanks!

~Gregory
 
> -- 
> Cheers,
> 
> David / dhildenb
>