Re: [LSF/MM/BPF TOPIC] DAMON Updates and Plans: Monitoring Parameters Auot-tuning and Memory Tiering

[Jonathan Cameron <Jonathan.Cameron@huawei.com>]

On Thu,  2 Jan 2025 14:23:17 -0800
SeongJae Park <sj@kernel.org> wrote:

> Hi all,
> 
> 
> We started sharing and discussing DAMON's current status and future plans from
> LSF/MM/BPF 2023.  Thanks to the constructive feedbacks from the discussions, I
> believe DAMON is continuing the evolution in right or at least not
> controversial directions.  To continue getting the benefit and avoid it becomes
> an unexpected "demon" in early stage, I'd like to share and discuss followup
> updates and new plans for DAMON once again on LSF/MM/BPF 2025.
> 
> Major topics and currently expected materials to share on the session include
> below.  Of course, some changes could happen.
> 
> First, followups on work items that we discussed on last LSF/MM/BPF.
> 
> DAMOS auto-tuning based tiered memory manamgent.  No many progress has made so
> far.  But I recently gained an access to a tiered memory system, and setting up
> test environment.  Hopefully I will be able to share early RFC implementation
> and evaluation results by the session.
> 
> Access/Contiguity-aware Memory Auto-scalging.  No progress has made so far,
> too.  I'm pivoting this into reliable huge contig memory occupation
> mechanism, though, since it is smaller scope that we can make faster.  It can
> also be used for not only memory hotplugging based auto-scaling but also
> contiguous memory allocation.  Hopefully early evaluation of a part of the
> work, probably access-aware partial compaction, will be shared by the session.

Hi SJ,

I couldn't immediately identify the huge contig memory occupation mechanism proposal
in last year's slides. Perhaps a brief summary here?

> 
> Please refer to last year LSF/MM/BPF slides[1] for details of the above two
> projects.
> 
> And new projects that currently in their early stages.
> 
> Page level properties-based access monitoring.  We are extending DAMOS filters
> that works in page level properties to further be useful for monitoring
> purpose.  RFC patch series for the essential part[1] is already available, and
[1] seems to be last year's slides?  Was that your intent?
> user-space tool support[2] is made.  By the session, hopefully the patch series
> will be merged in mm tree, and I will be able to share followup plans for
> making it more lightweight and useful.
> 
> Extending DAMON for memory bandwidth monitoring.  We aim to extend DAMON to
> support not only access pattern snapshot generation, but more general access
> pattern information, like memory bandwidth usage.  I expect only rough idea
> will be shared on the session, to make early alignemnt of the future shape, or
> abortion.


Where does this fit alongside hardware aided solutions (resctl  - MPAM on ARM,
similar stuff on x86?)  Idea to do it at sub process granularity?
What are the use cases for that?

One other topic, can we differentiate read access from write accesses?
The promotion costs for the two may be somewhat different if we can keep the
old translation live until the copy is in place for read only. Taking that
into account in promotion decisions may be useful.

Being able to track separately is one thing the hardware tracking units
do well subject to tracking resource constraints.

Thanks,

Jonathan

> 
> Please let me know if you have interest in status of other projects that I
> shared on last LSF/MM/BPF session or somewhere else.  Depending on that, the
> topics and time portions can be changed.
> 
> Note that I proposed[3] yet another LSF/MM/BPF topic for DAMON.  If both are
> accepted, please schedule this one before the other one.  I believe this
> session can make the other session deduplicated and faster.
> 
> [1] https://github.com/damonitor/talks/blob/master/2024/lsfmmbpf/damon_lsfmmbpf_2024.pdf
> [2] https://damonitor.github.io/posts/damon_sz_filter_passed/
> [3] https://lore.kernel.org/20250101222039.74565-1-sj@kernel.org
> 
> 
> Thanks,
> SJ
>