Re: [LSF/MM/BPF TOPIC] Restricting or migrating unmovable kernel allocations from slow tier

[Hyeonggon Yoo <42.hyeyoo@gmail.com>]

On Sat, Feb 1, 2025 at 11:04 PM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Sat, Feb 01, 2025 at 10:29:23PM +0900, Hyeonggon Yoo wrote:
> > The Linux kernel supports hot-plugging CXL memory via dax/kmem functionality.
> > The hot-plugged memory allows either unmovable kernel allocations
> > (ZONE_NORMAL), or restricts them to movable allocations (ZONE_MOVABLE)
> > depending on the hot-plug policy.
>
> This all seems like a grand waste of time.  Don't do that.  Don't allow
> kernel allocations from CXL at all. Don't build systems that have
> vast quantities of CXL memory (or if you do, expose it as really fast
> swap, not as memory).
>
> All of the CXL topics I see this year are "It really hurts performance
> when ..." and my reaction is "Yes, I told you it would hurt and you did
> it anyway".  Just stop doing it.  CXL is this decade's Infiniband / ATM
> / (name your favourite misguided dead technology here).

Hi, Matthew. Thank you for sharing your opinion.

I don't want to introduce too much complexity to MM due to CXL madness either,
but I think at least we need to guide users who buy CXL hardware to avoid
doing stupid things.

My initial subject was "Clearly documenting the use cases of
memhp_default_state=online{,_kernel}" because at first glance,
it was deemed usable for allowing kernel allocations from CXL,
which turned out to be not after some evaluation.

So there are a few questions from my side:
- Why do we support onlining CXL memory as ZONE_NORMAL then?
- Can we remove the feature completely?
- Or shouldn't we at least warn users adequately about it in the documentation?

I genuinely don't want to see users misusing it either.

Best,
Hyeonggon

> You can't stop other people from doing foolish things, but you don't have to join in.
> And we don't have to take stupid patches.