Re: [RFC] Memory tiering kernel alignment

[Davidlohr Bueso <dave@stgolabs.net>]

On Thu, 25 Jan 2024, David Rientjes wrote:

>Some recent discussions have proven that there is widespread interest in
>some very foundational topics for this technology such as:
>
> - Decoupling CPU balancing from memory balancing (or obsoleting CPU
>   balancing entirely)
>
>   + John Hubbard notes this would be useful for GPUs:
>
>      a) GPUs have their own processors that are invisible to the kernel's
>         NUMA "which tasks are active on which NUMA nodes" calculations,
>         and
>
>      b) Similar to where CXL is generally going, we have already built
>         fully memory-coherent hardware, which include memory-only NUMA
>         nodes.
>
> - In-kernel hot memory abstraction, informed by hardware hinting drivers
>   (incl some architectures like Power10), usable as a NUMA Balancing
>   backend for promotion and other areas of the kernel like transparent
>   hugepage utilization

Regarding the hardware counters, can/will CPU vendors provide something
better for what is currently there for PEBS/IBS - which needs a lot of
stat crunching to make it useful for hot page detection. imo if any sort
of hw assistance is going to be used, it better be *big* win vs anything
only in software - muddy numbers aren't worth the hassle. Power10 accounts
for time decay, and therefore would be better suited. iirc there was some
mention to possibly model after something along these lines.

Thanks,
Davidlohr