[LSF/MM/BPF TOPIC] Overhauling hot page detection and promotion based on PTE A bit scanning

[Raghavendra K T <raghavendra.kt@amd.com>]

Bharata and I would like to propose the following topic for LSFMM.

Topic: Overhauling hot page detection and promotion based on PTE A bit scanning.
 
In the Linux kernel, hot page information can potentially be obtained from
multiple sources:
 
a. PROT_NONE faults (NUMA balancing)
b. PTE Access bit (LRU scanning)
c. Hardware provided page hotness info (like AMD IBS)
 
This information is further used to migrate (or promote) pages from slow memory
tier to top tier to increase performance.

In the current hot page promotion mechanism, all the activities including the
process address space scanning, NUMA hint fault handling and page migration are
performed in the process context. i.e., scanning overhead is borne by the
applications.
 
I had recently posted a patch [1] to improve this in the context of slow-tier
page promotion. Here, Scanning is done by a global kernel thread which routinely
scans all the processes' address spaces and checks for accesses by reading the
PTE A bit. The hot pages thus identified are maintained in list and subsequently
are promoted to a default top-tier node. Thus, the approach pushes overhead of
scanning, NUMA hint faults and migrations off from process context.

The topic was presented in the MM alignment session hosted by David Rientjes [2].
The topic also finds a mention in S J Park's LSFMM proposal [3].
 
Here is the list of potential discussion points:
1. Other improvements and enhancements to PTE A bit scanning approach. Use of
multiple kernel threads, throttling improvements, promotion policies, per-process
opt-in via prctl, virtual vs physical address based scanning, tuning hot page
detection algorithm etc.

2. Possibility of maintaining single source of truth for page hotness that would
maintain hot page information from multiple sources and let other sub-systems
use that info.

3. Discuss how hardware provided hotness info (like AMD IBS) can further aid
promotion. Bharata had posted an RFC [4] on this a while back.

4. Overlap with DAMON and potential reuse.
 
Links:

[1] https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@amd.com/
[2] https://lore.kernel.org/linux-mm/20241226012833.rmmbkws4wdhzdht6@ed.ac.uk/T/
[3] https://lore.kernel.org/lkml/Z4XUoWlU-UgRik18@gourry-fedora-PF4VCD3F/T/
[4] https://lore.kernel.org/lkml/20230208073533.715-2-bharata@amd.com/