Re: Slow-tier Page Promotion discussion recap and open questions

[SeongJae Park <sj@kernel.org>]

On Tue, 17 Dec 2024 20:19:56 -0800 (PST) David Rientjes <rientjes@google.com> wrote:

> Hi everybody,
> 
> We had a very interactive discussion last week led by RaghavendraKT on
> slow-tier page promotion intended for memory tiering platforms, thank
> you!  Thanks as well to everybody who attended and provided great
> questions, suggestions, and feedback.
> 
> The RFC patch series "mm: slowtier page promotion based on PTE A bit"[1]
> is a proposal to allow for asynchronous page promotion based on memory
> accesses as an alternative to NUMA Balancing based promotions.  There was
> widespread interest in this topic and the discussion surfaced multiple
> use cases and requirements, very focused on CXL use cases.

Thank you for keeping the series and this great summary, David :)

[...]
> ----->o-----
> I followed up on a discussion point early in the talk about whether this
> should be virtual address scanning like the current approach, walking
> mm_struct's, or the alternative approach which would be physical address
> scanning.
> 
> Raghu sees this as a fully alternative approach such as what DAMON uses
> that is based on rmap.  The only advantage appears to be avoiding
> scanning on top tier memory completely.

IMHO, there could be more advantages of physical address space based
appraoches.  Easier handling of unmapped pages and short-lived processes,
applying different access monitoring / promotion policies for differnt NUMA
nodes (tiers) are some of those off the top of my head.

> 
> ----->o-----
> Wei noted there was a lot of similarities between the RFC implementation
> and the MGLRU page walk functionality and whether it would make sense to
> try to converge these together or make more generally useful.
> 
> SeongJae noted that if DAMON logic were used for the scanning that we
> could re-use the existing support for controlling the overhead.

Just to clarify.  I added this comment since there were concerns around rmap
overhead for pysical address space-based monitoring approaches.

[...]
> My takeaways:
[...]
>  - I think virtual memory scanning is likely the only viable approach for
>    this purpose and we could store state in the underlying struct page,
>    similar to NUMA Balancing, but that all scanning should be driven by
>    walking the mm_struct's to harvest the Accessed bit

I don't clearly get why you think virtual memory scanning is the only viable
approach.  I'm curious if you have some pros/cons list about virtual vs
physical address based appraoches in your mind, and willing to share.

[...]
> We'll be looking to incorporate this discussion in our upstream Memory
> Tiering Working Group to accelerate alignment and progress on the
> approach.

Thank you again for your efforts on this!


Thanks,
SJ

[...]