Re: Problem with oom-killer in memcg

[KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On Tue, 26 May 2009 19:04:35 +0900
Satoru Moriya <satoru.moriya.br@hitachi.com> wrote:

> Hi all,
> 
> When I tested memcg, I ran into a problem which causes system hang.
> 
> This is what I did.
> - make a cgroup named important for memory
> - add a process named big_memory into it
>   - big_memory uses a lot of memory(allocates memory repeatedly)
>   - big_memory's oom_adj is set to -17
> - after a while, the system will hang
> 
> Judging from syslog and outputs of console, I think we are in the busy
> loop below at that time.
> 1. oom-killer tries to kill big_memory because of memory shortage
> 2. oom-killer fails to kill big_memory because of oom_adj = -17
> 
> I think it's not good thing that troubles in cgroups affect 
> all over the system. 
> 
How many cpus are you using ?

> Further hardware and software details are found below.
> Please let me know if I should provide more information etc.
> 
> Regards,
> 

I think Balbir (and other people) is planning to add "oom handler" for memcg
or oom handler cgroup. But we need more study.

I assume that you use x86. If so, current bahavior is a bit complicated.

do_page_fault()
   -> allocate and charge memory account
         -> memcg's oom kill is called
              -> no progress.
User says "don't use OOM Kill" but no other way than "OOM Kill"

We don't have much choices here..
   - kill in force ?
   - add some sleep ?
   - freeze cgroup under OOM ? (this seems not easy)
   - Ask admin to increase size of memory limit ?

Thank you for reporting, but I can't think of quick fix right now.
I'll remember this in my TO-DO List.

Sorry.
-Kame




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>