From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3549AC3A5A9 for ; Mon, 4 May 2020 15:00:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E59A120721 for ; Mon, 4 May 2020 15:00:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E59A120721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9632D8E0035; Mon, 4 May 2020 11:00:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9132D8E0024; Mon, 4 May 2020 11:00:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 851588E0035; Mon, 4 May 2020 11:00:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id 6DAFF8E0024 for ; Mon, 4 May 2020 11:00:57 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 27CE0824559C for ; Mon, 4 May 2020 15:00:57 +0000 (UTC) X-FDA: 76779348954.12.fight73_42764bd1a260b X-HE-Tag: fight73_42764bd1a260b X-Filterd-Recvd-Size: 6546 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 May 2020 15:00:56 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id u16so9458287wmc.5 for ; Mon, 04 May 2020 08:00:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=tJ5etNEXsxt7E9ZceP21z7vkGhvbdJvXpUDXj5OxOH0=; b=gJicfWnuWGZqlE2zKzb8BiDZD6HhPNqLlHG7J9t6AJT9S8oLw5Zqn4l08sn4AMhknA F2IfHSlkY8sV2Uf05/497Kju8xBhumBCcxD1had6JNee1GKFYDpLSQ/SRnpgDlvmVrvo E91EUOiIxbQ6inTYUiSW9TFqefjObTnWDbakWqlBGhfJW0pOZIEzwL3+0cQQ6HLolSpB Sbb2XZFh3ESExmwcE90hijorFt1CA0jEi/lmEFydAqOlKDws1fukMYT4MdPKThxoZDDD qiK2sG+YubMV+Yaea99QJJiGg1Hi3tlJoT9h2zhqKXPJcMVAdijewLwN0lwHIu1P1aX9 xYXg== X-Gm-Message-State: AGi0PuaOt+ib+lHyJzGPlyg0Yd325E4wEwc2HQVPDSCrGvhLkvFHYuZl lEmzq/WuNx1xMKKHvHLajRk= X-Google-Smtp-Source: APiQypJQHKMQIRZdJ8nNLXQjOA202ammJbzJLV5W2BPzSNYKXJ7sgyYhV0l8zzxSdFgng1kNGOagoA== X-Received: by 2002:a05:600c:220c:: with SMTP id z12mr14797153wml.84.1588604455378; Mon, 04 May 2020 08:00:55 -0700 (PDT) Received: from localhost (ip-37-188-183-9.eurotel.cz. [37.188.183.9]) by smtp.gmail.com with ESMTPSA id 138sm14956344wmb.14.2020.05.04.08.00.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 08:00:54 -0700 (PDT) Date: Mon, 4 May 2020 17:00:52 +0200 From: Michal Hocko To: Shakeel Butt Cc: Johannes Weiner , Roman Gushchin , Greg Thelen , Andrew Morton , Linux MM , Cgroups , LKML Subject: Re: [PATCH] memcg: oom: ignore oom warnings from memory.max Message-ID: <20200504150052.GT22838@dhcp22.suse.cz> References: <20200430182712.237526-1-shakeelb@google.com> <20200504065600.GA22838@dhcp22.suse.cz> <20200504141136.GR22838@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 04-05-20 07:53:01, Shakeel Butt wrote: > On Mon, May 4, 2020 at 7:11 AM Michal Hocko wrote: > > > > On Mon 04-05-20 06:54:40, Shakeel Butt wrote: > > > On Sun, May 3, 2020 at 11:56 PM Michal Hocko wrote: > > > > > > > > On Thu 30-04-20 11:27:12, Shakeel Butt wrote: > > > > > Lowering memory.max can trigger an oom-kill if the reclaim does not > > > > > succeed. However if oom-killer does not find a process for killing, it > > > > > dumps a lot of warnings. > > > > > > > > It shouldn't dump much more than the regular OOM report AFAICS. Sure > > > > there is "Out of memory and no killable processes..." message printed as > > > > well but is that a real problem? > > > > > > > > > Deleting a memcg does not reclaim memory from it and the memory can > > > > > linger till there is a memory pressure. One normal way to proactively > > > > > reclaim such memory is to set memory.max to 0 just before deleting the > > > > > memcg. However if some of the memcg's memory is pinned by others, this > > > > > operation can trigger an oom-kill without any process and thus can log a > > > > > lot un-needed warnings. So, ignore all such warnings from memory.max. > > > > > > > > OK, I can see why you might want to use memory.max for that purpose but > > > > I do not really understand why the oom report is a problem here. > > > > > > It may not be a problem for an individual or small scale deployment > > > but when "sweep before tear down" is the part of the workflow for > > > thousands of machines cycling through hundreds of thousands of cgroups > > > then we can potentially flood the logs with not useful dumps and may > > > hide (or overflow) any useful information in the logs. > > > > If you are doing this in a large scale and the oom report is really a > > problem then you shouldn't be resetting hard limit to 0 in the first > > place. > > > > I think I have pretty clearly described why we want to reset the hard > limit to 0, so, unless there is an alternative I don't see why we > should not be doing this. I am not saying you shouldn't be doing that. I am just saying that if you do then you have to live with oom reports. > > > > memory.max can trigger the oom kill and user should be expecting the oom > > > > report under that condition. Why is "no eligible task" so special? Is it > > > > because you know that there won't be any tasks for your particular case? > > > > What about other use cases where memory.max is not used as a "sweep > > > > before tear down"? > > > > > > What other such use-cases would be? The only use-case I can envision > > > of adjusting limits dynamically of a live cgroup are resource > > > managers. However for cgroup v2, memory.high is the recommended way to > > > limit the usage, so, why would resource managers be changing > > > memory.max instead of memory.high? I am not sure. What do you think? > > > > There are different reasons to use the hard limit. Mostly to contain > > potential runaways. While high limit might be a sufficient measure to > > achieve that as well the hard limit is the last resort. And it clearly > > has the oom killer semantic so I am not really sure why you are > > comparing the two. > > > > I am trying to see if "no eligible task" is really an issue and should > be warned for the "other use cases". The only real use-case I can > think of are resource managers adjusting the limit dynamically. I > don't see "no eligible task" a concerning reason for such use-case. It is very much a concerning reason to notify about like any other OOM situation due to hard limit breach. In this case it is worse in some sense because the limit cannot be trimmed down because there is no directly reclaimable memory at all. Such an oom situation is effectivelly conserved. -- Michal Hocko SUSE Labs