Re: DAMON testing and benchmarking

[Piyush Sachdeva <piyushs@linux.vnet.ibm.com>]

Dear SeongJae Park,

SeongJae Park <sj@kernel.org> writes:

> Hi Piyush,
>
> On Tue, 13 Jun 2023 12:18:48 +0530 Piyush Sachdeva <piyushs@linux.ibm.com> wrote:
>
>> Dear Mr. SeongJae Park,
>> I hope this email finds you well.
>
> It did, thank you for this email :)
>
>> 
>> For the last few months, I have been looking at DAMON from an end-users
>> perspective and a developer's PoV. Most recently, I was focusing on
>> `lru_sort.c` module that uses the `lru_prio` and `lru_deprio` operations
>> which result in a more precise reclaim. In my understanding, enabling
>> the "lru_sort.c" module would make intelligent decisions based on the
>> access frequency of the pages and end up preventing hot page
>> swaps. Hence, when integrated with an LRU algorithm, it should improve
>> it.
>> 
>> If you could share any test/benchmark that you might have run to verify
>> the above assumption?
>
> Yes, of course.  I will share those soon.
>
>> I did find the result numbers you posted (link below), but that doesn't
>> mention the "plrus-*" scheme numbers. It also doesn't have numbers for
>> running the `pageout` operation on the entire physical address space (paddr)
>> i.e. the `pprcl` scheme. So, if you can link those too, it would be amazing.
>
> We run an automated test[1] every day, against the latest damon/next tree.  And
> the page you linked is the output of the test.  The latet version contains the
> results from `pprcl`[2] and `plrus`[3], but I was too lazy at updating the
> document, sorry.  I will try to clean up the mess as soon as possible.
>
>> 
>> Can you also share any real-world (memory-management specific) workload
>> results that you would have used with DAMON in your experiments? Like
>> either MongoDB or memcached over Parsec3.0 (including splash2x) - which,
>> in my understanding, is less memory intensive and more architecture
>> inclined.
>
> On my personal testing setup, I'm using only parsec3 and splash2x at the
> moment.  We heard some production DB system is using DAMON_RECLAIM and achieved
> about 20% memory footprint reduction, though.
>

Are you aware of the specifications of the DB system that you are
mentioning? 20% is an amazing number and if possible I would like to
know more about the details of the workload.

>> 
>> I also had a question regarding schemes - A scheme is highly tweakable,
>> and it's what the efficiency of DAMON rests upon. The more precise the
>> scheme, the more efficient DAMON will be. Hence, I'd be thankful if you
>> can help me derive a config that would provide the best results.
>
> Very good point.  Unfortunately, repeats of experience and adjustment is the
> only way as of now, like other tuning practices.  Nevertheless, DAMOS supports
> some safeguards such as quotas[4], watermarks[5], and filters[6].  Because
> quotas feature provides prioritization, setting the access pattern a little
> wide, and more focusing on tuning of the quotas might be a good practice.
>
> I'm trying to add some more easy-to-use intuitive tuning knobs, including
> feedback-based quota auto-tuning, which I shared the rough idea at LSFMM[7].
>

I did go through your presentation and the summary article on lwn.net .
The feedback-mechanism based self-tuning does sound ingriguing and
promising to me.

>> Hope to hear from you soon.
>
> Thank you again for this great questions.  Please feel free to ask any question
> or helps :)
>
> [1] https://github.com/awslabs/damon-tests/blob/next/perf/full_run.sh
> [2] https://github.com/awslabs/damon-tests/blob/next/perf/schemes/pdarc_v4_2_2.json
> [3] https://github.com/awslabs/damon-tests/blob/next/perf/schemes/plrus-2.json
> [4] https://www.kernel.org/doc/html/next/mm/damon/design.html#quotas
> [5] https://www.kernel.org/doc/html/next/mm/damon/design.html#watermarks
> [6] https://www.kernel.org/doc/html/next/mm/damon/design.html#filters
> [7] https://lwn.net/Articles/931769/
>
>
> Thanks,
> SJ
>
>> 
>> Test results: https://damonitor.github.io/test/result/perf/latest/html/
>> 
>> --
>> Regards,
>> Piyush Sachdeva
>>

Thank you for the information and all the links. 

--
Regards,
Piyush Sachdeva