Re: Memory overcommit

[KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On Tue, 3 Nov 2009 17:58:04 -0800 (PST)
David Rientjes <rientjes@google.com> wrote:

> On Wed, 4 Nov 2009, KAMEZAWA Hiroyuki wrote:
> 
> > > That's a different point.  Today, we can influence the badness score of 
> > > any user thread to prioritize oom killing from userspace and that can be 
> > > done regardless of whether there's a memory leaker, a fork bomber, etc.  
> > > The priority based oom killing is important to production scenarios and 
> > > cannot be replaced by a heuristic that works everytime if it cannot be 
> > > influenced by userspace.
> > > 
> > I don't removed oom_adj...
> > 
> 
> Right, but we must ensure that we have the same ability to influence a 
> priority based oom killing scheme from userspace as we currently do with a 
> relatively static total_vm.  total_vm may not be the optimal baseline, but 
> it does allow users to tune oom_adj specifically to identify tasks that 
> are using more memory than expected and to be static enough to not depend 
> on rss, for example, that is really hard to predict at the time of oom.
> 
> That's actually my main goal in this discussion: to avoid losing any 
> ability of userspace to influence to priority of tasks being oom killed 
> (if you haven't noticed :).
> 
> > > Tweaking on the heuristic will probably make it more convoluted and 
> > > overall worse, I agree.  But it's a more stable baseline than rss from 
> > > which we can set oom killing priorities from userspace.
> > 
> > - "rss < total_vm_size" always.
> 
> But rss is much more dynamic than total_vm, that's my point.
> 
My point and your point are differnt.

  1. All my concern is "baseline for heuristics"
  2. All your concern is "baseline for knob, as oom_adj"

ok ? For selecting victim by the kernel, dynamic value is much more useful.
Current behavior of "Random kill" and "Kill multiple processes" are too bad.
Considering oom-killer is for what, I think "1" is more important.

But I know what you want, so, I offers new knob which is not affected by RSS
as I wrote in previous mail.

Off-topic:
As memcg is growing better, using OOM-Killer for resource control should be
ended, I think. Maybe Fake-NUMA+cpuset is working well for google system, 
but plz consider to use memcg. 



> > - oom_adj culculation is quite strong.
> > - total_vm of processes which maps hugetlb is very big ....but killing them
> >   is no help for usual oom.
> > 
> > I recommend you to add "stable baseline" knob for user space, as I wrote.
> > My patch 6 adds stable baseline bonus as 50% of vm size if run_time is enough
> > large.
> > 
> 
> There's no clear relationship between VM size and runtime.  The forkbomb 
> heuristic itself could easily return a badness of ULONG_MAX if one is 
> detected using runtime and number of children, as I earlier proposed, but 
> that doesn't seem helpful to factor into the scoring. 
> 

Old processes are important, younger are not. But as I wrote, I'll drop
most of patch "6". So, plz forget about this part.

I'm interested in fork-bomb killer rather than crazy badness calculation, now.

Thanks,
-Kame



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>