Re: [RFC] Mechanism to induce memory reclaim

[Shakeel Butt <shakeelb@google.com>]

On Mon, Mar 07, 2022 at 03:41:45PM +0100, Michal Hocko wrote:
> On Sun 06-03-22 15:11:23, David Rientjes wrote:
> [...]
> > Some questions to get discussion going:
> >
> >  - Overall feedback or suggestions for the proposal in general?

> Do we really need this interface? What would be usecases which cannot
> use an existing interfaces we have for that? Most notably memcg and
> their high limit?


Let me take a stab at this. The specific reasons why high limit is not a
good interface to implement proactive reclaim:

1) It can cause allocations from the target application to get
throttled.

2) It leaves a state (high limit) in the kernel which needs to be reset
by the userspace part of proactive reclaimer.

If I remember correctly, Facebook actually tried to use high limit to
implement the proactive reclaim but due to exactly these limitations [1]
they went the route [2] aligned with this proposal.

To further explain why the above limitations are pretty bad: The
proactive reclaimers usually use feedback loop to decide how much to
squeeze from the target applications without impacting their performance
or impacting within a tolerable range. The metrics used for the feedback
loop are either refaults or PSI and these metrics becomes messy due to
application getting throttled due to high limit.

For (2), the high limit interface is a very awkward interface to use to
do proactive reclaim. If the userspace proactive reclaimer fails/crashed
due to whatever reason during triggering the reclaim in an application,
it can leave the application in a bad state (memory pressure state and
throttled) for a long time.

[1] https://lore.kernel.org/all/20200928210216.GA378894@cmpxchg.org/
[2] https://dl.acm.org/doi/10.1145/3503222.3507731 (Section 3.3)