From: Michal Hocko <mhocko@kernel.org>
To: Pavel Machek <pavel@ucw.cz>
Cc: kernel list <linux-kernel@vger.kernel.org>,
Andrew Morton <akpm@osdl.org>,
linux-mm@kvack.org, akpm@linux-foundation.org
Subject: Re: OOM killer not nearly agressive enough?
Date: Thu, 9 Jan 2020 22:25:16 +0100 [thread overview]
Message-ID: <20200109212516.GA23620@dhcp22.suse.cz> (raw)
In-Reply-To: <20200109210307.GA1553@duo.ucw.cz>
On Thu 09-01-20 22:03:07, Pavel Machek wrote:
> On Thu 2020-01-09 12:56:33, Michal Hocko wrote:
> > On Tue 07-01-20 21:44:12, Pavel Machek wrote:
> > > Hi!
> > >
> > > I updated my userspace to x86-64, and now chromium likes to eat all
> > > the memory and bring the system to standstill.
> > >
> > > Unfortunately, OOM killer does not react:
> > >
> > > I'm now running "ps aux", and it prints one line every 20 seconds or
> > > more. Do we agree that is "unusable" system? I attempted to do kill
> > > from other session.
> >
> > Does sysrq+f help?
>
> May try that next time.
>
> > > Do we agree that OOM killer should have reacted way sooner?
> >
> > This is impossible to answer without knowing what was going on at the
> > time. Was the system threshing over page cache/swap? In other words, is
> > the system completely out of memory or refaulting the working set all
> > the time because it doesn't fit into memory?
>
> Swap was full, so "completely out of memory", I guess. Chromium does
> that fairly often :-(.
The oom heuristic is based on the reclaim failure. If the reclaim makes
some progress then the oom killer is not hit. Have a look at
should_reclaim_retry for more details.
> > > Is there something I can tweak to make it behave more reasonably?
> >
> > PSI based early OOM killing might help. See https://github.com/facebookincubator/oomd
>
> Um. Before doing that... is there some knob somewhere saying "hey
> oomkiller, one hour to recover machine is a bit too much, can you
> please react sooner"?
No, there is nothing like that.
> PSI is completely different system, but I guess
> I should attempt to tweak the existing one first...
PSI is measuring the cost of the allocation (among other things) and
that can give you some idea on how much time is spent to get memory.
Userspace can implement a policy based on that and act. The kernel oom
killer is the last resort when there is really no memory to allocate.
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2020-01-09 21:25 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-07 20:44 Pavel Machek
2020-01-09 11:56 ` Michal Hocko
2020-01-09 21:03 ` Pavel Machek
2020-01-09 21:25 ` Michal Hocko [this message]
2020-01-09 22:48 ` Pavel Machek
2020-01-10 1:24 ` Shakeel Butt
2020-01-10 6:31 ` Michal Hocko
2020-01-09 21:46 ` Vito Caputo
2020-01-09 21:58 ` Michal Hocko
2020-01-09 21:05 ` Pavel Machek
2020-01-09 21:28 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200109212516.GA23620@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=akpm@osdl.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=pavel@ucw.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox