Re: [RFC] Mechanism to induce memory reclaim

[Yu Zhao <yuzhao@google.com>]

On Sun, Mar 6, 2022 at 4:11 PM David Rientjes <rientjes@google.com> wrote:
>
> Hi everybody,
>
> We'd like to discuss formalizing a mechanism to induce memory reclaim by
> the kernel.
>
> The current multigenerational LRU proposal introduces a debugfs
> mechanism[1] for this.  The "TMO: Transparent Memory Offloading in
> Datacenters" paper also discusses a per-memcg mechanism[2].  While the
> former can be used for debugging of MGLRU, both can quite powerfully be
> used for proactive reclaim.
>
> Google's datacenters use a similar per-memcg mechanism for the same
> purpose.  Thus, formalizing the mechanism would allow our userspace to use
> an upstream supported interface that will be stable and consistent.
>
> This could be an incremental addition to MGLRU's lru_gen debugfs mechanism
> but, since the concept has no direct dependency on the work, we believe it
> is useful independent of the reclaim mechanism in use (both with and
> without CONFIG_LRU_GEN).
>
> Idea: introduce a per-node sysfs mechanism for inducing memory reclaim
> that can be useful for global (non-memcg constrained) reclaim and possible
> even if memcg is not enabled in the kernel or mounted.  This could
> optionally take a memcg id to induce reclaim for a memcg hierarchy.
>
> IOW, this would be a /sys/devices/system/node/nodeN/reclaim mechanim for
> each NUMA node N on the system.  (It would be similar to the existing
> per-node sysfs "compact" mechanism used to trigger compaction from
> userspace.)
>
> Userspace would write the following to this file:
>  - nr_to_reclaim pages
>  - swappiness factor
>  - memcg_id of the hierarchy to reclaim from, if any[*]
>  - flags to specify context, if any[**]
>
>  [*] if global reclaim or memcg is not enabled/mounted, this is 0 since
>      this is the return value of mem_cgroup_id()
>  [**] this is offered for extensibility to specify the context in which
>       reclaim is being done (clean file pages only, demotion for memory
>       tiering vs eviction, etc), otherwise 0
>
> An alternative may be to introduce a /sys/kernel/mm/reclaim mechanism that
> also takes a nodemask to reclaim from.  The kernel would reclaim memory
> over the set of nodes passed to it.
>
> Some questions to get discussion going:
>
>  - Overall feedback or suggestions for the proposal in general?
>
>  - This proposal uses a value specified in pages to reclaim; this could be
>    a number of bytes instead.  I have no strong opinion, does anybody
>    else?
>
>  - Should this be a per-node mechanism under sysfs like the existing
>    "compact" mechanism or should it be implemented as a single file that
>    can optionally specify a nodemask to reclaim from?
>
> Thanks!
>
> [1] https://lore.kernel.org/linux-mm/20220208081902.3550911-12-yuzhao@google.com
> [2] https://dl.acm.org/doi/10.1145/3503222.3507731 (Section 3.3)

Adding Canonical who also provided additional use cases [3] for this
potential ABI.

[3] https://lore.kernel.org/lkml/20201005081313.732745-1-andrea.righi@canonical.com/