Re: [RFC PATCH v1 0/2] mm: multi-gen LRU scanning for page promotion

[Bharata B Rao <bharata@amd.com>]

On 25-Mar-25 3:32 AM, Kinsey Ho wrote:
> This patch series introduces a software-based approach to identify
> hot pages for promotion in tiered memory systems, particularly those
> leveraging CXL-attached memory, by utilizing the Multi-Generational
> LRU (MGLRU) framework. This method is designed to complement
> hardware-based hotness detection mechanisms like Intel PMU sampling, AMD
> IBS, or dedicated CXL memory monitoring units, providing a more
> comprehensive view of page access patterns, similar to kmmscand [1].
> 
> We propose to utilize MGLRU's existing infrastructure to provide hot
> page information. A key benefit here is the reuse of the MGLRU page
> table walk code, thus avoiding the overhead and duplication of effort
> involved in implementing a separate page table scanning mechanism. The
> working set reporting proposal [2] also reuses MGLRU's infrastructure,
> but focuses on cold page detection. It provides its own aging daemon,
> which could additionally provide hot page information by integrating
> this proof-of-concept.
> 
> This series relies on kpromoted [3] as the migration engine to implement
> the promotion policies. This is just an early proof-of-concept RFC
> posted now in the context of LSFMM.

Thanks for your patchset. I haven't looked at the patches in detail yet, 
but gave it a quick try with the micro-benchmark that I have been using.

The below numbers can be compared with the base numbers that I have 
posted here 
(https://lore.kernel.org/linux-mm/20250325081832.209140-1-bharata@amd.com/). 
Test 2 in the above link is the one I tried with this patchset.

kernel.numa_balancing = 0
demotion=true
cpufreq governor=performance

Benchmark run configuration:
Compute-node            = 1
Memory-node             = 2
Memory-size             = 206158430208
Hot-region-size         = 1073741824
Nr-hot-regions          = 192
Access pattern          = random
Access granularity      = 4096
Delay b/n accesses      = 0
Load/store ratio        = 50l50s
THP used                = no
Nr accesses             = 25769803776
Nr repetitions          = 512

Benchmark completed in 605983205.0 us

numa_hit 63621437
numa_miss 2721737
numa_foreign 2721737
numa_interleave 0
numa_local 48243292
numa_other 18099882
pgpromote_success 0
pgpromote_candidate 0
pgdemote_kswapd 15409682
pgdemote_direct 0
pgdemote_khugepaged 0
numa_pte_updates 0
numa_huge_pte_updates 0
numa_hint_faults 0
numa_hint_faults_local 0
numa_pages_migrated 19596
pgmigrate_success 15429278
pgmigrate_fail 256

kpromoted_recorded_accesses 27647687
kpromoted_recorded_hwhints 0
kpromoted_recorded_pgtscans 27647687
kpromoted_record_toptier 0
kpromoted_record_added 17184209
kpromoted_record_exists 10463478
kpromoted_mig_right_node 0
kpromoted_mig_non_lru 404308
kpromoted_mig_cold_old 6417567
kpromoted_mig_cold_not_accessed 10342825
kpromoted_mig_promoted 19509
kpromoted_mig_dropped 17164700

When I try to get the same benchmark numbers for kpromoted driven by 
kmmscand, kpromoted gets overwhelmed with the amount of data that 
kmmdscand provides while no such issues with the amount of accesses 
reported by this patchset.

As I have mentioned earlier, the hot page categorization heuristics is 
simplistic in kpromoted and may not have been able to promote more pages 
than what it has for this benchmark.

Regards,
Bharata.