Re: [LSF/MM/BPF TOPIC] Restricting or migrating unmovable kernel allocations from slow tier

[Harry Yoo <harry.yoo@oracle.com>]

On Mon, Feb 10, 2025 at 10:47:58AM -0500, Gregory Price wrote:
> On Mon, Feb 10, 2025 at 04:17:41PM +0900, Byungchul Park wrote:
> > On Mon, Feb 10, 2025 at 01:00:02AM -0500, Gregory Price wrote:
> > > 
> > >    You can probably actually (maybe?) collect data on this today - but
> > >    you still have to contend with #2 and #3.
> > 
> > Ah.  You seem to mean those works should be serialized.  Right?  If it
> > should be for some reason, then it could be sensible.
> > 
> 
> I'm suggesting that there isn't a strong reason (yet) to consider such a
> complicated change.  As Willy has said, it's a fairly fundamental change
> for a single-reason (CXL), which does not bode well for its acceptance.
> 
> Honestly trying to save you some frustration. It would behoove you to
> find stronger reasons (w/ data) or consider different solutions. Right
> now there are stronger, simplers solutions to the ZONE_NORMAL capacity
> issue (struct page resize, huge pages) for possible capacities.

Hi, apologies for my late reply. I recently went through a career change.

I truly appreciate your and Matthew's feedback and thank you for saving us
from frustration. I agree that we need a stronger motivation
and data to introduce such a fundamental change. And I also agree that
it's more appropriate to pursue what can be useful for genral MM users
rather than introducing MM changes just for CXL.

With that context, Byungchul and I agree it's a better direction:
Reducing ZONE_NORMAL cost for ZONE_MOVABLE capacity, which is beneficial
for ZONE_MOVABLE users in general, regardless of whether the user is using
CXL memory or not.

Let me organize a few steps to pursue:

- Willy's shrinking struct page project
  - https://fosdem.org/2025/schedule/event/fosdem-2025-4860-shrinking-memmap/ 
  - https://kernelnewbies.org/MatthewWilcox/Memdescs/Path
  - Side note: Byungchul started working on separating the descriptor
    of the pagepool bump allocator

- Slab Movable Objects: This makes sense even without CXL
  as migrating unreclaimable slab will improve compaction success rate.
  It also has been tried in the past by others, but was suspended
  due to lack of data.
  
  I'm looking for workloads that allocate a decent amount of unreclaimable
  slab AND performs migration frequently - for evaluation.

I might be missing some projects that could be useful,
please feel free to add if there is any.

And for page table migration, while it might be doable even without CXL,
we need strong data that suggests that it's actually makes MM better
to pursue this.

> I also think someone should actively ask whether `struct page` can be
> hosted on remote memory without performance loss.  I may look into this.

Did you have a chance to look at this?

-- 
Cheers,
Harry