Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim

[Yafang Shao <laoar.shao@gmail.com>]

On Mon, Feb 26, 2024 at 10:05 PM Michal Hocko <mhocko@suse.com> wrote:
>
> On Sun 25-02-24 19:42:04, Yafang Shao wrote:
> > In our container environment, we've observed that certain containers may
> > accumulate more than 40GB of slabs, predominantly negative dentries. These
> > negative dentries remain unreclaimed unless there is memory pressure. Even
> > after the containers exit, these negative dentries persist.
>
> Have you considered using memory.high limit to trigger that memory
> reclaim? That is surely not focused on the neg. dentry cache but it
> should keep the overal memory consumption under control.

Upon reaching the high watermark, both pagecache and slabs are
reclaimed. However, it tends to prioritize reclaiming more pagecache
pages over slab pages, leading to persistent memory consumption by
slabs. Setting memory.high too low may negatively impact the working
set.

>
> > To manage disk
> > storage efficiently, we employ an agent that identifies container images
> > eligible for destruction once all instances of that image exit.
> >
> > However, during destruction, dealing with directories containing numerous
> > negative dentries can significantly impact performance.
>
> Performance of what. I have to say I am kind of lost here. We are
> talking about memory or a disk storage?

Removing an empty directory with numerous dentries can significantly
prolong the process of freeing associated dentries, leading to high
system CPU usage that adversely affects overall system performance.

>
> > To mitigate this
> > issue, we aim to proactively reclaim these dentries using a user agent.
> > Extending the memory.reclaim functionality to specifically target slabs
> > aligns with our requirements.
>
> Matthew has already pointed out that this has been proposed several
> times already and rejected.

With that being said, we haven't come up with any superior solutions
compared to the proposals mentioned.

> Dedicated slab shrinking interface is
> especially tricky because you would need a way to tell which shrinkers
> to invoke and that would be very kernel version specific.

The persistence of this issue over several years without any
discernible progress suggests that we might be heading in the wrong
direction. Perhaps we could consider providing a kernel interface to
users, allowing them to tailor the reclamation process based on their
workload requirements.

--
Regards
Yafang