Re: Slow-tier Page Promotion discussion recap and open questions

[Raghavendra K T <raghavendra.kt@amd.com>]

+ Gang Li
On 1/2/2025 10:14 AM, David Rientjes wrote:
> On Fri, 20 Dec 2024, Raghavendra K T wrote:
[...]
>>> Wei Xu asked if the scan period should be interpreted as the minimal
>>> interval between scans because kmmscand is single threaded and there are
>>> many processes.  Raghu confirmed this is correct, the minimal delay.
>>> Even if the scan period is 400ms, in reality it could be multiple seconds
>>> based on load.
>>>
>>> Liam Howlett asked how we could have two scans colliding in a time
>>> segment.  Raghu noted if we are able to complete the last scan in less
>>> time than 400ms, then we have this delay to avoid continuously scanning
>>> that results in increased cpu overhead.  Liam further asked if processes
>>> opt into a scan or out of the scan, Raghu noted we always scan every
>>> process.  John Hubbard suggested that we have per-process control.
>>
>> +1 for prctl()
>>
>> Also I want to add that, I will get data on:
>>
>> what is the min and max time required to finish the entire scan for the
>> current micro-benchmark and one of the real workload (such as Redis/
>> Rocksdb...), so that we can check if we are meeting the deadline of
>> scanning with single kthread.
>>
> 
> Do we want more fine-grained per-process control other than just the
> ability to opt out entire processes?  There may be situations where we
> want to always serve latency tolerant jobs from CXL extended memory, we
> don't care to ever promote its memory, but I also think there will be
> processes that are between the two extremes (latency critical and latency
> tolerant).
> 
> I think careful consideration needs to be given to how we handle
> per-process policy for multi-tenant systems that have different levels of
> latency sensitivity.  If kmmscand becomes the standard way of doing page
> promotion in the kernel, the userspace API to inform it of these policy
> decisions is going to be key.  There have been approaches where this was
> primarily driven by BPF that has to solve the same challenge.
> 

Came across prctl() approach for NUMAB by Gang Li,

Link: 
https://lore.kernel.org/lkml/20220224075227.27127-1-ligang.bdlg@bytedance.com/T/

I hope there was no stronger reason against the merging (just to ensure
since it will be almost same for our case here).

[...]

Regards
- Raghu