Re: [LSF/MM/BPF TOPIC] DAMON Requirements for Access-aware MM of Future

[SeongJae Park <sj@kernel.org>]

Hi Yuanchu,

On Wed, 29 Jan 2025 18:15:08 -0800 Yuanchu Xie <yuanchu@google.com> wrote:

> On Mon, Jan 13, 2025 at 7:06 PM Gregory Price <gourry@gourry.net> wrote:
> > 5) Scarce resources
> >
> >    We need to be careful not to consume excessive amounts of resources
> >    in an attempt to track all these identifying mechanisms.  Even 1 byte
> >    per folio is 256MB on a 1TB machine.  This gets out of hand quick.
> >
> >    With task-work, I was able to add no additional resource consumption,
> >    but deferring to a fully async scenario and needing to track things
> >    like last-accessing CPU, timestamps, and etc.
> >
> >    We'll need to examine this closely if we decide to aggregate either
> >    of these mechanisms.
> My concern with physical address space monitoring is fragmentation. I
> ran some numbers on a few prod machines. Grouping by regions with the
> same memcg and ignoring any unmapped memory to be generous, machines
> with higher utilization can have a region/total pages ratio of ~40%,
> and even those with lower utilization (<50%) can also reach 20%.
> Accurately tracking these regions would require quite the region
> metadata, on the order of GBs.

You're right, if we need page level accuracy access monitoring and want to use
DAMON with its regions based mechanism for that, the memory overhead of
damon_region could be high.  That's mainly because DAMON's regions-based
mechanism has not designed for such usage.  It is more for a best-effort
tradeoff between the overhead and the accuracy.

Regions-based mechanism is not necessarily the only mechanism of future DAMON,
though.  If there are use cases that regions-based best-effort accuracy cannot
be used while exactly the page level accuracy is really required, we can think
about optimizing regions based mechanism or developing new one.

But, IMHO, the page level accurate access pattern is not always essential.  In
many cases, being able to distinguish some amount of regions agains others
based on access pattern is practical enough.  Indeed, DAMON has been used on
real-world products with physical address based moitoring mode for years with
no significant problem.  Also I think physical address space based monitoring
results[1] on a real server workload that I shared recently seems not very bad.

Of course your use case could be different from what I have experienced so far.
I'm curious if and why you really need page level accuracy.

[1] https://lore.kernel.org/20250110185232.54907-3-sj@kernel.org


Thanks,
SJ