Re: [LSF/MM/BPF TOPIC] Restricting or migrating unmovable kernel allocations from slow tier

[Gregory Price <gourry@gourry.net>]

On Mon, Feb 10, 2025 at 03:55:47PM +0000, Matthew Wilcox wrote:
> On Mon, Feb 10, 2025 at 10:47:58AM -0500, Gregory Price wrote:
> > I also think someone should actively ask whether `struct page` can be
> > hosted on remote memory without performance loss.  I may look into this.
> 
> Given that it contains a refcount and various flags, some of which
> are quite hot, I would expect performance to suffer.  It also suffers
> contention between different CPUs, so depending on your cache protocol
> (can it do cache-to-cche transfers or does it have to be written back
> to memory first?) it may perform quite poorly.  But this is something
> that can be measured.
> 
> Of course, the question must be asked whetheer we care.  Certainly Intel's
> Apache Pass and similar Optane RAM products put the memmap on the 3DXP
> because there wasn't enough DRAM to put it there.  So the pages are
> slower, but they were slower anyway!
> 

Well, *if* said memory is intended to host cold(er) data, then we may
find the structures to describe those pages aren't particularly hot or
contended.  This is my suspicion - and I'd rather limit kernel resource
allocation on remote memory than try to move kernel resources around.

Plus this would still enables hot-unplug.  Once all the zone movable
regions are clicked off, the page-desc regions are unused... probably.

Would just be nice to have some concrete data on when greater zone
movable capacity becomes a net-negative. We're making the assumption
this this occurs fairly early.

~Gregory