Re: [PATCH] mm, memcg: show memcg min setting in oom messages

[Yafang Shao <laoar.shao@gmail.com>]

On Fri, Nov 22, 2019 at 6:28 PM Michal Hocko <mhocko@kernel.org> wrote:
>
> On Wed 20-11-19 20:23:54, Yafang Shao wrote:
> > On Wed, Nov 20, 2019 at 7:40 PM Michal Hocko <mhocko@kernel.org> wrote:
> > >
> > > On Wed 20-11-19 18:53:44, Yafang Shao wrote:
> > > > On Wed, Nov 20, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> > > > >
> > > > > On Wed 20-11-19 03:53:05, Yafang Shao wrote:
> > > > > > A task running in a memcg may OOM because of the memory.min settings of his
> > > > > > slibing and parent. If this happens, the current oom messages can't show
> > > > > > why file page cache can't be reclaimed.
> > > > >
> > > > > min limit is not the only way to protect memory from being reclaim. The
> > > > > memory might be pinned or unreclaimable for other reasons (e.g. swap
> > > > > quota exceeded for memcg).
> > > >
> > > > Both swap or unreclaimabed (unevicteable) is printed in OOM messages.
> > >
> > > Not really. Consider a memcg which has reached it's swap limit. The
> > > anonymous memory is not really reclaimable even when there is a lot of
> > > swap space available.
> > >
> >
> > The memcg swap limit is already printed in oom messages, see bellow,
> >
> > [  141.721625] memory: usage 1228800kB, limit 1228800kB, failcnt 18337
> > [  141.721958] swap: usage 0kB, limit 9007199254740988kB, failcnt 0
>
> But you do not have any insight on the swap limit down the oom
> hierarchy, do you?
>
> > > > Why not just print the memcgs which are under memory.min protection or
> > > > something like a total number of min protected memory ?
> > >
> > > Yes, this would likely help. But the main question really reamains, is
> > > this really worth it?
> > >
> >
> > If it doesn't cost too much, I think it is worth to do it.
> > As the oom path is not the critical path, so adding some print info
> > should not add much overhead.
>
> Generating a lot of output for the oom reports has been a real problem
> in many deployments.

So why not only print non-zero counters ?
If some counters are 0, we don't print them, that can reduce the oom reports.

Something like "isolated_file:0 unevictable:0 dirty:0 writeback:0
unstable:0" can all be removed,
and we consider them as zero by default.
I mean we can optimze the OOM reports and only print the useful
information to make it not be a problem in many deployments.

> [...]
> > > > I have said in the commit log, that we don't know why the file cache
> > > > can't be reclaimed (when evictable is 0 and dirty is 0 as well.)
> > >
> > > And the counter argument is that this will not help you there much in
> > > many large and much more common cases.
> > >
> > > I argue, and I might be wrong here so feel free to correct me, that the
> > > reclaim protection guarantee (min) is something to be under admins
> > > control. It shouldn't really happen nilly-willy because it has really
> > > large consequences, the OOM including. So if there is a suspicious
> > > amount of memory that could be reclaimed normally then the reclaim
> > > protection is really the first suspect to go after.
> > > --
> >
> > I don't know whether it happens nilly-willy or not.
>
> It is a reclaim protection guarantee (so essentially an mlock like
> thing) so it better have to be properly considered when used.
>
> > But if we all know that it may cause OOMs and it don't take too much
> > effort to show it in the OOM messages,
>
> I do not think we are in agreement here. As mentioned above the oom
> report is quite heavy already. So it should be other way around. There
> should be a strong reason to add something more. A real use case where
> not having that information is making debugging ooms considerably much
> harder.
>
> --
> Michal Hocko
> SUSE Labs