From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f71.google.com (mail-it0-f71.google.com [209.85.214.71]) by kanga.kvack.org (Postfix) with ESMTP id 662986B0388 for ; Fri, 10 Feb 2017 03:53:01 -0500 (EST) Received: by mail-it0-f71.google.com with SMTP id s10so43236860itb.7 for ; Fri, 10 Feb 2017 00:53:01 -0800 (PST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com. [58.251.152.64]) by mx.google.com with ESMTPS id e41si1330951ioj.215.2017.02.10.00.52.59 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 10 Feb 2017 00:53:00 -0800 (PST) Subject: Re: [RFC] 3.10 kernel- oom with about 24G free memory References: <9a22aefd-dfb8-2e4c-d280-fc172893bcb4@huawei.com> <20170209132628.GI10257@dhcp22.suse.cz> <20170209134131.GJ10257@dhcp22.suse.cz> <20170210070930.GA9346@dhcp22.suse.cz> From: Yisheng Xie Message-ID: <7d01fea5-66d6-b6ac-918d-19ec8a15dbaf@huawei.com> Date: Fri, 10 Feb 2017 16:48:58 +0800 MIME-Version: 1.0 In-Reply-To: <20170210070930.GA9346@dhcp22.suse.cz> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Tetsuo Handa , Hanjun Guo Hi Michal, Thanks for comment! On 2017/2/10 15:09, Michal Hocko wrote: > On Fri 10-02-17 09:13:58, Yisheng Xie wrote: >> hi Michal, >> Thanks for your comment. >> >> On 2017/2/9 21:41, Michal Hocko wrote: >>> On Thu 09-02-17 14:26:28, Michal Hocko wrote: >>>> On Thu 09-02-17 20:54:49, Yisheng Xie wrote: >>>>> Hi all, >>>>> I get an oom on a linux 3.10 kvm guest OS. when it triggers the oom >>>>> it have about 24G free memory(and host OS have about 10G free memory) >>>>> and watermark is sure ok. >>>>> >>>>> I also check about about memcg limit value, also cannot find the >>>>> root cause. >>>>> >>>>> Is there anybody ever meet similar problem and have any idea about it? >>>>> >>>>> Any comment is more than welcome! >>>>> >>>>> Thanks >>>>> Yisheng Xie >>>>> >>>>> ------------- >>>>> [ 81.234289] DefSch0200 invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0 >>>>> [ 81.234295] DefSch0200 cpuset=/ mems_allowed=0 >>>>> [ 81.234299] CPU: 3 PID: 8284 Comm: DefSch0200 Tainted: G O E ----V------- 3.10.0-229.42.1.105.x86_64 #1 >>>>> [ 81.234301] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20161111_105425-HGH1000008200 04/01/2014 >>>>> [ 81.234303] ffff880ae2900000 000000002b3489d7 ffff880b6cec7c58 ffffffff81608d3d >>>>> [ 81.234307] ffff880b6cec7ce8 ffffffff81603d1c 0000000000000000 ffff880b6cd09000 >>>>> [ 81.234311] ffff880b6cec7cd8 000000002b3489d7 ffff880b6cec7ce0 ffffffff811bdd77 >>>>> [ 81.234314] Call Trace: >>>>> [ 81.234323] [] dump_stack+0x19/0x1b >>>>> [ 81.234327] [] dump_header+0x8e/0x214 >>>>> [ 81.234333] [] ? mem_cgroup_iter+0x177/0x2b0 >>>>> [ 81.234339] [] check_panic_on_oom+0x2e/0x60 >>>>> [ 81.234342] [] mem_cgroup_oom_synchronize+0x34f/0x580 >>>> >>>> OK, so this is a memcg OOM killer which panics because the configuration >>>> says so. The OOM report doesn't say so and that is the bug. dump_header >>>> is memcg aware and mem_cgroup_out_of_memory initializes oom_control >>>> properly. Is this Vanilla kernel? >> >> That means we should raise the limit of that memcg to avoid memcg OOM killer, right? > > Why do you configure the system to panic on memcg OOM in the first > place. This is a wrong thing to do in 99% of cases. For our production think it should use reboot to recovery the system when OOM, instead of killing user's key process. Maybe not the right thing. Thanks Yisheng Xie -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org