[RFC PATCH v6 0/5] mm: Hot page tracking and promotion infrastructure

Bharata B Rao
2026-03-23T09:51:44Z

Hi,

This is v6 of pghot, a hot-page tracking and promotion subsystem. The
main changes in this version are to retain only hintfault as source
and a lot of cleanups, fixes and restructuring.

This patchset introduces a new subsystem for hot page tracking and
promotion (pghot) with the following goals:

- Unify hot page detection from multiple sources like hint faults,
page table scans, hardware hints (AMD IBS).
- Decouple detection from migration.
- Centralize promotion logic via per-lower-tier-node kmigrated kernel
thread.
- Move promotion rate‑limiting and related logic used by
numa_balancing=2 (NUMAB2, the current NUMA balancing–based promotion)
from the scheduler to pghot for broader reuse.

Currently, multiple kernel subsystems detect page accesses independently.
This patchset consolidates accesses from these mechanisms by providing:

- A common API for reporting page accesses.
- Shared infrastructure for tracking hotness at PFN granularity.
- Per-lower-tier-node kernel threads for promoting pages.

Here is a brief summary of how this subsystem works:

- Tracks frequency and last access time.
- Additionally, the accessing NUMA node ID (NID) for each recorded
access is also tracked in the precision mode.
- These hotness parameters are maintained in a per-PFN hotness record
within the existing mem_section data structure.
- In default mode, one byte (u8) is used for hotness record. 5 bits are
used to store time and bucketing scheme is used to represent a total
access time up to 4s with HZ=1000. Default toptier NID (0) is used as
the target for promotion which can be changed via debugfs tunable.
- In precision mode, 4 bytes (u32) are used for each hotness record.
14 bits are used to store time which can represent around 16s
with HZ=1000.
- Classifies pages as hot based on configurable thresholds.
- Pages classified as hot are marked as ready for migration using the
ready bit. Both modes use MSB of the hotness record as ready bit.
- Per-lower-tier-node kmigrated threads periodically scan the PFNs of
lower-tier nodes, checking for the migration-ready bit to perform
batched migrations. Interval between successive scans and batching
value are configurable via debugfs tunables.

Memory overhead
---------------
Default mode: 1 byte per lower-tier PFN. For a 1TB lower-tier memory
this amounts to 256MB overhead (assuming 4K pages)

Precision mode: 4 bytes per lower-tier PFN. For a 1TB of lower memory
this amounts to 1G overhead.

Bit layout of hotness record
----------------------------
Default mode
- Bits 0-1: Frequency (2bits, 4 access samples)
- Bits 2-6: Bucketed time (5bits, up to 4s with HZ=1000)
- Bit 7: Migration ready bit

Precision mode
- Bits 0-9: Target NID (10 bits)
- Bits 10-12: Frequency (3bits, 8 access samples)
- Bits 13-26: Time (14bits, up to 16s with HZ=1000)
- Bits 27-30: Reserved
- Bit 31: Migration ready bit

Potential hotness sources
-------------------------
1. NUMA Balancing (NUMAB2, Tiering mode)
2. IBS - Instruction Based Sampling, hardware based sampling
mechanism present on AMD CPUs.
3. klruscand - PTE‑A bit scanning built on MGLRU’s walk helpers.
4. folio_mark_accessed() - Page cache access tracking (unmapped
page cache pages)

Changes in v6
=============
- While earlier versions included sample implementation for all
the hotness sources listed above, I have retained only NUMAB2
in this iteration and will include the hardened versions
of them in subsequent iterations.
- Cleaned up NUMAB2 implementation by removing the unused code
(like access time tracking code etc).
- Ensured that NUMAB1 mode works as before. (Earlier versions made
NUMA hint fault handler work only for NUMAB2 version)
- NUMA Balancing tiering mode is moved to its own new config
CONFIG_NUMA_BALANCING_TIERING to make code sharing between
NUMA Balancing and pghot easier.
- A bunch of hot page promotion related stats now depend upon
CONFIG_PGHOT as the promotion engine is part of pghot.
- Fixed kmigrated to take a reference on the folio when walking
the PFNs checking for migrate-ready folios.
- Fixed speculative folio access issue reported by Chris Mason's
review-prompts.
- Added per-memcg NUMA_PAGE_MIGRATE stats accounting for batch
migration API too.
- Added support for initializing hot_maps in the newly added
sections during memory hotplug.
- Default hotness threshold window changed from 4s to 3s as the
maximum time representable in default mode is 3.9s only.
- Lot of cleanups and code restructuring.

Results
=======
Posted as replies to this mail thread.

This v6 patchset applies on top of upstream commit a989fde763f4f and
can be fetched from:

https://github.com/AMDESE/linux-mm/tree/bharata/pghot-rfcv6

v5: https://lore.kernel.org/linux-mm/20260129144043.231636-1-bharata@amd.com/
v4: https://lore.kernel.org/linux-mm/20251206101423.5004-1-bharata@amd.com/
v3: https://lore.kernel.org/linux-mm/20251110052343.208768-1-bharata@amd.com/
v2: https://lore.kernel.org/linux-mm/20250910144653.212066-1-bharata@amd.com/
v1: https://lore.kernel.org/linux-mm/20250814134826.154003-1-bharata@amd.com/
v0: https://lore.kernel.org/linux-mm/20250306054532.221138-1-bharata@amd.com/

Bharata B Rao (4):
mm: migrate: Allow misplaced migration without VMA
mm: Hot page tracking and promotion - pghot
mm: pghot: Precision mode for pghot
mm: sched: move NUMA balancing tiering promotion to pghot

Gregory Price (1):
mm: migrate: Add migrate_misplaced_folios_batch()

Documentation/admin-guide/mm/pghot.txt | 80 ++++
include/linux/migrate.h | 10 +-
include/linux/mm.h | 35 +-
include/linux/mmzone.h | 24 +-
include/linux/pghot.h | 113 +++++
include/linux/vm_event_item.h | 5 +
init/Kconfig | 13 +
kernel/sched/core.c | 7 +
kernel/sched/debug.c | 1 -
kernel/sched/fair.c | 177 +------
kernel/sched/sched.h | 1 -
mm/Kconfig | 25 +
mm/Makefile | 6 +
mm/huge_memory.c | 27 +-
mm/memcontrol.c | 6 +-
mm/memory-tiers.c | 15 +-
mm/memory.c | 36 +-
mm/mempolicy.c | 3 -
mm/migrate.c | 88 +++-
mm/mm_init.c | 10 +
mm/pghot-default.c | 79 ++++
mm/pghot-precise.c | 81 ++++
mm/pghot-tunables.c | 182 ++++++++
mm/pghot.c | 618 +++++++++++++++++++++++++
mm/vmstat.c | 7 +-
25 files changed, 1411 insertions(+), 238 deletions(-)
create mode 100644 Documentation/admin-guide/mm/pghot.txt
create mode 100644 include/linux/pghot.h
create mode 100644 mm/pghot-default.c
create mode 100644 mm/pghot-precise.c
create mode 100644 mm/pghot-tunables.c
create mode 100644 mm/pghot.c

base-commit: a989fde763f4f24209e4702f50a45be572340e68
--
2.34.1