Re: [LSF/MM/BPF TOPIC] Locally attached memory tiering

[Bharata B Rao <bharata@amd.com>]

On 5/13/2024 7:19 AM, Davidlohr Bueso wrote:
> On Thu, 09 May 2024, Huang, Ying wrote:
> 
>> With the default configuration, current NUMA balancing based promotion
>> solution will almost try to promote any faulting pages.  To select hot
>> pages to promote and control thrashing between NUMA nodes, the promote
>> rate limit needs to be configured.  For example, via,
>>
>> echo 200 > /proc/sys/kernel/numa_balancing_promote_rate_limit_MBps
>>
>> 200MB hot pages will be selected and promoted every second.  Can you 
>> try it?
> 
> Yes, I've played with this tunnable and, just like the LRU approach, it
> shows nice micro wins (less amount of promotions/demotions) but little for
> actual benchmark improvements at a higher level, merely noise level or
> very sublte wins. In fact, the actual data from that series for this
> parameter was a ~2% pmbench win with the rate limiting, but a 69% promotion
> rate descrease.
> 
> And this is really my point, how much effort do we want to put in 
> optimizing
> software mechanisms for hot page detection? Are there other benchmarks we
> should be using?

Yes, some representative benchmarks to evaluate the effectiveness of hot 
page promotion would be useful.

Recently there was a discussion about the effectiveness of hot page 
detection in the context of a micro-benchmark. More details here:

https://lore.kernel.org/linux-mm/929b22ca-bb51-4307-855f-9b4ae0a102e3@amd.com/T/#m04eb5d9dfb30133156d4dcb33b09b89a4e9299ea

> And perhaps doing the async promotion and not incurring in
> the numa balancing overhead and comparing the cost of migration before
> promoting would yield some better numbers, but that also might be easy to
> get wrong when compared to the relative hotness of the page.

Regards,
Bharata.