From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx132.postini.com [74.125.245.132]) by kanga.kvack.org (Postfix) with SMTP id BE0756B0005 for ; Thu, 7 Feb 2013 06:02:06 -0500 (EST) Received: from m3.gw.fujitsu.co.jp (unknown [10.0.50.73]) by fgwmail6.fujitsu.co.jp (Postfix) with ESMTP id CDCED3EE0B5 for ; Thu, 7 Feb 2013 20:02:04 +0900 (JST) Received: from smail (m3 [127.0.0.1]) by outgoing.m3.gw.fujitsu.co.jp (Postfix) with ESMTP id B4E6945DEBC for ; Thu, 7 Feb 2013 20:02:04 +0900 (JST) Received: from s3.gw.fujitsu.co.jp (s3.gw.fujitsu.co.jp [10.0.50.93]) by m3.gw.fujitsu.co.jp (Postfix) with ESMTP id 8B8C745DEB2 for ; Thu, 7 Feb 2013 20:02:04 +0900 (JST) Received: from s3.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s3.gw.fujitsu.co.jp (Postfix) with ESMTP id 7FCE91DB8041 for ; Thu, 7 Feb 2013 20:02:04 +0900 (JST) Received: from m1001.s.css.fujitsu.com (m1001.s.css.fujitsu.com [10.240.81.139]) by s3.gw.fujitsu.co.jp (Postfix) with ESMTP id 297961DB8038 for ; Thu, 7 Feb 2013 20:02:04 +0900 (JST) Message-ID: <51138999.3090006@jp.fujitsu.com> Date: Thu, 07 Feb 2013 20:01:45 +0900 From: Kamezawa Hiroyuki MIME-Version: 1.0 Subject: Re: [PATCH for 3.2.34] memcg: do not trigger OOM from add_to_page_cache_locked References: <20121224142526.020165D3@pobox.sk> <20121228162209.GA1455@dhcp22.suse.cz> <20121230020947.AA002F34@pobox.sk> <20121230110815.GA12940@dhcp22.suse.cz> <20130125160723.FAE73567@pobox.sk> <20130125163130.GF4721@dhcp22.suse.cz> <20130205134937.GA22804@dhcp22.suse.cz> <20130205154947.CD6411E2@pobox.sk> <20130205160934.GB22804@dhcp22.suse.cz> <20130206021721.1AE9E3C7@pobox.sk> <20130206140119.GD10254@dhcp22.suse.cz> In-Reply-To: <20130206140119.GD10254@dhcp22.suse.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: azurIt , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups mailinglist , Johannes Weiner (2013/02/06 23:01), Michal Hocko wrote: > On Wed 06-02-13 02:17:21, azurIt wrote: >>> 5-memcg-fix-1.patch is not complete. It doesn't contain the folloup I >>> mentioned in a follow up email. Here is the full patch: >> >> >> Here is the log where OOM, again, killed MySQL server [search for "(mysqld)"]: >> http://www.watchdog.sk/lkml/oom_mysqld6 > > [...] > WARNING: at mm/memcontrol.c:2409 T.1149+0x2d9/0x610() > Hardware name: S5000VSA > gfp_mask:4304 nr_pages:1 oom:0 ret:2 > Pid: 3545, comm: apache2 Tainted: G W 3.2.37-grsec #1 > Call Trace: > [] warn_slowpath_common+0x7a/0xb0 > [] warn_slowpath_fmt+0x46/0x50 > [] ? mem_cgroup_margin+0x73/0xa0 > [] T.1149+0x2d9/0x610 > [] ? blk_finish_plug+0x18/0x50 > [] mem_cgroup_cache_charge+0xc4/0xf0 > [] add_to_page_cache_locked+0x4f/0x140 > [] add_to_page_cache_lru+0x22/0x50 > [] filemap_fault+0x252/0x4f0 > [] __do_fault+0x78/0x5a0 > [] handle_pte_fault+0x84/0x940 > [] ? vma_prio_tree_insert+0x30/0x50 > [] ? vma_link+0x88/0xe0 > [] handle_mm_fault+0x138/0x260 > [] do_page_fault+0x13d/0x460 > [] ? do_mmap_pgoff+0x3dc/0x430 > [] page_fault+0x1f/0x30 > ---[ end trace 8817670349022007 ]--- > apache2 invoked oom-killer: gfp_mask=0x0, order=0, oom_adj=0, oom_score_adj=0 > apache2 cpuset=uid mems_allowed=0 > Pid: 3545, comm: apache2 Tainted: G W 3.2.37-grsec #1 > Call Trace: > [] dump_header+0x7e/0x1e0 > [] ? find_lock_task_mm+0x2f/0x70 > [] oom_kill_process+0x85/0x2a0 > [] out_of_memory+0xe5/0x200 > [] pagefault_out_of_memory+0xbd/0x110 > [] mm_fault_error+0xb6/0x1a0 > [] do_page_fault+0x3ee/0x460 > [] ? do_mmap_pgoff+0x3dc/0x430 > [] page_fault+0x1f/0x30 > > The first trace comes from the debugging WARN and it clearly points to > a file fault path. __do_fault pre-charges a page in case we need to > do CoW (copy-on-write) for the returned page. This one falls back to > memcg OOM and never returns ENOMEM as I have mentioned earlier. > However, the fs fault handler (filemap_fault here) can fallback to > page_cache_read if the readahead (do_sync_mmap_readahead) fails > to get page to the page cache. And we can see this happening in > the first trace. page_cache_read then calls add_to_page_cache_lru > and eventually gets to add_to_page_cache_locked which calls > mem_cgroup_cache_charge_no_oom so we will get ENOMEM if oom should > happen. This ENOMEM gets to the fault handler and kaboom. > Hmm. do we need to increase the "limit" virtually at memcg oom until the oom-killed process dies ? It may be doable by increasing stock->cache of each cpu....I think kernel can offer extra virtual charge up to oom-killed process's memory usage..... Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org