Re: [PATCH v6 00/13] mm/demotion: Memory tiers and demotion

[Hesham Almatary <hesham.almatary@huawei.com>]

On Fri, 10 Jun 2022 19:19:53 +0530
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> wrote:

> The current kernel has the basic memory tiering support: Inactive
> pages on a higher tier NUMA node can be migrated (demoted) to a lower
> tier NUMA node to make room for new allocations on the higher tier
> NUMA node.  Frequently accessed pages on a lower tier NUMA node can be
> migrated (promoted) to a higher tier NUMA node to improve the
> performance.
> 
> In the current kernel, memory tiers are defined implicitly via a
> demotion path relationship between NUMA nodes, which is created during
> the kernel initialization and updated when a NUMA node is hot-added or
> hot-removed.  The current implementation puts all nodes with CPU into
> the top tier, and builds the tier hierarchy tier-by-tier by
> establishing the per-node demotion targets based on the distances
> between nodes.
> 
> This current memory tier kernel interface needs to be improved for
> several important use cases:
> 
> * The current tier initialization code always initializes
>   each memory-only NUMA node into a lower tier.  But a memory-only
>   NUMA node may have a high performance memory device (e.g. a DRAM
>   device attached via CXL.mem or a DRAM-backed memory-only node on
>   a virtual machine) and should be put into a higher tier.
> 
> * The current tier hierarchy always puts CPU nodes into the top
>   tier. But on a system with HBM (e.g. GPU memory) devices, these
>   memory-only HBM NUMA nodes should be in the top tier, and DRAM nodes
>   with CPUs are better to be placed into the next lower tier.
> 
> * Also because the current tier hierarchy always puts CPU nodes
>   into the top tier, when a CPU is hot-added (or hot-removed) and
>   triggers a memory node from CPU-less into a CPU node (or vice
>   versa), the memory tier hierarchy gets changed, even though no
>   memory node is added or removed.  This can make the tier
>   hierarchy unstable and make it difficult to support tier-based
>   memory accounting.
> 
> * A higher tier node can only be demoted to selected nodes on the
>   next lower tier as defined by the demotion path, not any other
>   node from any lower tier.  This strict, hard-coded demotion order
>   does not work in all use cases (e.g. some use cases may want to
>   allow cross-socket demotion to another node in the same demotion
>   tier as a fallback when the preferred demotion node is out of
>   space), and has resulted in the feature request for an interface to
>   override the system-wide, per-node demotion order from the
>   userspace.  This demotion order is also inconsistent with the page
>   allocation fallback order when all the nodes in a higher tier are
>   out of space: The page allocation can fall back to any node from
>   any lower tier, whereas the demotion order doesn't allow that.
> 
> * There are no interfaces for the userspace to learn about the memory
>   tier hierarchy in order to optimize its memory allocations.
> 
> This patch series make the creation of memory tiers explicit under
> the control of userspace or device driver.
> 
> Memory Tier Initialization
> ==========================
> 
> By default, all memory nodes are assigned to the default tier (1).
> The default tier device has a rank value (200).
> 
> A device driver can move up or down its memory nodes from the default
> tier.  For example, PMEM can move down its memory nodes below the
> default tier, whereas GPU can move up its memory nodes above the
> default tier.
> 
> The kernel initialization code makes the decision on which exact tier
> a memory node should be assigned to based on the requests from the
> device drivers as well as the memory device hardware information
> provided by the firmware.
> 
> Hot-adding/removing CPUs doesn't affect memory tier hierarchy.
> 
> Memory Allocation for Demotion
> ==============================
> This patch series keep the demotion target page allocation logic same.
> The demotion page allocation pick the closest NUMA node in the
> next lower tier to the current NUMA node allocating pages from.
> 
> This will be later improved to use the same page allocation strategy
> using fallback list.
> 
> Sysfs Interface:
> =======================
> Listing current list of memory tiers and rank details:
> 
> :/sys/devices/system/memtier$ ls
> default_tier max_tier  memtier1  power  uevent
> :/sys/devices/system/memtier$ cat default_tier
> memtier1
> :/sys/devices/system/memtier$ cat max_tier 
> 3
Should this be renamed to max_tiers or tiers_count? Otherwise one might
confuse max_tier as the highest allowed tier ID.

> :/sys/devices/system/memtier$ 
> 
> Per node memory tier details:
> 
> For a cpu only NUMA node:
> 
> :/sys/devices/system/node# cat node0/memtier 
> :/sys/devices/system/node# echo 1 > node0/memtier 
> :/sys/devices/system/node# cat node0/memtier 
> :/sys/devices/system/node# 
> 
> For a NUMA node with memory:
> :/sys/devices/system/node# cat node1/memtier 
> 1
> :/sys/devices/system/node# ls ../memtier/
> default_tier  max_tier  memtier1  power  uevent
> :/sys/devices/system/node# echo 2 > node1/memtier 
> :/sys/devices/system/node# 
> :/sys/devices/system/node# ls ../memtier/
> default_tier  max_tier  memtier1  memtier2  power  uevent
> :/sys/devices/system/node# cat node1/memtier 
> 2
> :/sys/devices/system/node# 
> :/sys/devices/system/node# cat ../memtier/memtier2/rank 
> 100
> :/sys/devices/system/node# 
> :/sys/devices/system/node# cat ../memtier/memtier1/rank 
> 200
> :/sys/devices/system/node#
> 
> Removing a NUMA node from demotion:
> :/sys/devices/system/node# cat node1/memtier 
> 2
> :/sys/devices/system/node# echo none > node1/memtier 
> :/sys/devices/system/node# 
> :/sys/devices/system/node# cat node1/memtier 
> :/sys/devices/system/node# 
> :/sys/devices/system/node# ls ../memtier/
> default_tier  max_tier  memtier1  power  uevent
> :/sys/devices/system/node# 
> 
> The above also resulted in removal of memtier2 which was created in
> the earlier step.
> 
> 
> Changes from v5:
> * Remove patch supporting N_MEMORY node removal from memory tiers.
> memory tiers are going to be used for features other than demotion.
> Hence keep all N_MEMORY nodes in memory tiers irrespective of whether
> they want to participate in promotion or demotion.
> * Add NODE_DATA->memtier
> * Rearrage patches to add sysfs files later.
> * Add support to create memory tiers from userspace.
> * Address other review feedback.
> 
> 
> Changes from v4:
> * Address review feedback.
> * Reverse the meaning of "rank": higher rank value means higher tier.
> * Add "/sys/devices/system/memtier/default_tier".
> * Add node_is_toptier
> 
> v4:
> Add support for explicit memory tiers and ranks.
> 
> v3:
> - Modify patch 1 subject to make it more specific
> - Remove /sys/kernel/mm/numa/demotion_targets interface, use
>   /sys/devices/system/node/demotion_targets instead and make
>   it writable to override node_states[N_DEMOTION_TARGETS].
> - Add support to view per node demotion targets via sysfs
> 
> v2:
> In v1, only 1st patch of this patch series was sent, which was
> implemented to avoid some of the limitations on the demotion
> target sharing, however for certain numa topology, the demotion
> targets found by that patch was not most optimal, so 1st patch
> in this series is modified according to suggestions from Huang
> and Baolin. Different examples of demotion list comparasion
> between existing implementation and changed implementation can
> be found in the commit message of 1st patch.
> 
> 
> Aneesh Kumar K.V (11):
>   mm/demotion: Add support for explicit memory tiers
>   mm/demotion: Move memory demotion related code
>   mm/demotion: Return error on write to numa_demotion sysfs
>   mm/demotion/dax/kmem: Set node's memory tier to MEMORY_TIER_PMEM
>   mm/demotion: Build demotion targets based on explicit memory tiers
>   mm/demotion: Expose memory tier details via sysfs
>   mm/demotion: Add per node memory tier attribute to sysfs
>   mm/demotion: Add support for memory tier creation from userspace
>   mm/demotion: Add pg_data_t member to track node memory tier details
>   mm/demotion: Update node_is_toptier to work with memory tiers
>   mm/demotion: Add sysfs ABI documentation
> 
> Jagdish Gediya (2):
>   mm/demotion: Demote pages according to allocation fallback order
>   mm/demotion: Add documentation for memory tiering
> 
>  .../ABI/testing/sysfs-kernel-mm-memory-tiers  |  87 ++
>  Documentation/admin-guide/mm/index.rst        |   1 +
>  .../admin-guide/mm/memory-tiering.rst         | 181 ++++
>  drivers/base/node.c                           |  39 +
>  drivers/dax/kmem.c                            |   4 +
>  include/linux/memory-tiers.h                  |  63 ++
>  include/linux/migrate.h                       |  15 -
>  include/linux/mmzone.h                        |   3 +
>  include/linux/node.h                          |   5 -
>  mm/Kconfig                                    |   3 +
>  mm/Makefile                                   |   1 +
>  mm/huge_memory.c                              |   1 +
>  mm/memory-tiers.c                             | 888
> ++++++++++++++++++ mm/migrate.c                                  |
> 453 +-------- mm/mprotect.c                                 |   1 +
>  mm/vmscan.c                                   |  57 +-
>  mm/vmstat.c                                   |   4 -
>  17 files changed, 1316 insertions(+), 490 deletions(-)
>  create mode 100644
> Documentation/ABI/testing/sysfs-kernel-mm-memory-tiers create mode
> 100644 Documentation/admin-guide/mm/memory-tiering.rst create mode
> 100644 include/linux/memory-tiers.h create mode 100644
> mm/memory-tiers.c
>