Re: [LSF/MM/BPF TOPIC] Restricting or migrating unmovable kernel allocations from slow tier

[Matthew Wilcox <willy@infradead.org>]

On Mon, Feb 10, 2025 at 11:33:47AM +0900, Harry (Hyeonggon) Yoo wrote:
> Premise: Some ZONE_NORMAL capacity exists on CXL memory
>          due to its large capacity.

I reject your premise.  None of this is inevitable.  Infiniband and ATM
did not beocme dominant networking technologies.  SOP did not dominate
the storage industry.  Itanium did not become the only CPU architcture
that mattered.

Similarly, CXL is a technically flawed protocol.  Lots of money is
being thrown at making it look inevitable, but fundamentally PCIe is
a high-bandwidth protocol, not a low-latency protocol and it can't do
the job.

> > There's a reason most kernel allocations are not swappable.
> 
> Because most kernel allocations cannot be swapped, with a few exceptions.
> 
> However, there's non-LRU page migration functionality where kernel
> allocations can be migrated.
> 
> I don't understand why we shouldn't introduce more kernel movable memory
> if that turns out to be beneficial?

Because it's adding complexity for a stupid use-case.

If you can make the case for making something migratable that's not
currently without using CXL as a justification, then sure, let's do it.
zsmalloc is migratable, and that makes a lot of sense.  But there's
a reason we only have three movable_operations structs defined in the
kernel today.

(also the whole non-LRU page migration needs overhauling to not use
page->lru, but that's a separate matter.  except it's not a separate
matter because that's needed in order to shrink struct page.)