From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DBE1C433DF for ; Thu, 21 May 2020 17:37:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2649920826 for ; Thu, 21 May 2020 17:37:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2649920826 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 80E1480008; Thu, 21 May 2020 13:37:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7981980007; Thu, 21 May 2020 13:37:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65F9F80008; Thu, 21 May 2020 13:37:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id 4B3B780007 for ; Thu, 21 May 2020 13:37:05 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F20375010 for ; Thu, 21 May 2020 17:37:04 +0000 (UTC) X-FDA: 76841431968.06.birds17_5ae6cf7360413 X-HE-Tag: birds17_5ae6cf7360413 X-Filterd-Recvd-Size: 6034 Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 May 2020 17:37:04 +0000 (UTC) Received: by mail-ed1-f68.google.com with SMTP id k19so7214805edv.9 for ; Thu, 21 May 2020 10:37:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=16rYke2VtirNB4gM0beG8w5PFV57NvRuYqspSwtCjSg=; b=WEy+Q3gu9wQL9EOYtUWXkoRW+J3HCJHNqy6RVXDf25uVepWJJp9BpDdTlRXsn+Jtg2 W3Clkazs87976kg/S+UMxeBSamW864WAaTOvayoQQ7s/27OYAoVwiOt7BFcAFEadHNSw HUBWiNy4OCb0JnZ4tVelQCy2qsnmhhY04kxJUg2MVdxg0N+CpWBN6QhiiknBhMD7EssA /7JR2LsuNoa8Qtihd+TxU1jM8r7Y7rercnIGk5vPUWOeT2sOKz4jUWPQ0UmKOYTUFE6g sFh6auq+V7ZJlosYvbF4rKngKxfvlV4e9c7gHAxZ4gH4li6S/z7ZetikhU81+7yfPNnK TK+w== X-Gm-Message-State: AOAM531xYti8dUCihpP11lKcWkajmOH4tHTuw6jFTXOYB0zjOk21I1r5 KIJvf+1xVU8BgIay+DfTaAo= X-Google-Smtp-Source: ABdhPJxIpAKEHjfSswIXBqOwMluDhwqe6aqxhPJc5pVjLhTX7Zqxuv8zlixxjQkThK5S7DqUFJfFFw== X-Received: by 2002:a05:6402:1d2d:: with SMTP id dh13mr8182661edb.169.1590082623404; Thu, 21 May 2020 10:37:03 -0700 (PDT) Received: from localhost (ip-37-188-180-112.eurotel.cz. [37.188.180.112]) by smtp.gmail.com with ESMTPSA id lw27sm5659285ejb.80.2020.05.21.10.37.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 May 2020 10:37:02 -0700 (PDT) Date: Thu, 21 May 2020 19:37:01 +0200 From: Michal Hocko To: Johannes Weiner Cc: Chris Down , Andrew Morton , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling Message-ID: <20200521173701.GX6462@dhcp22.suse.cz> References: <20200520143712.GA749486@chrisdown.name> <20200520160756.GE6462@dhcp22.suse.cz> <20200520165131.GB630613@cmpxchg.org> <20200520170430.GG6462@dhcp22.suse.cz> <20200520175135.GA793901@cmpxchg.org> <20200521073245.GI6462@dhcp22.suse.cz> <20200521135152.GA810429@cmpxchg.org> <20200521143515.GU6462@dhcp22.suse.cz> <20200521163833.GA813446@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200521163833.GA813446@cmpxchg.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 21-05-20 12:38:33, Johannes Weiner wrote: > On Thu, May 21, 2020 at 04:35:15PM +0200, Michal Hocko wrote: > > On Thu 21-05-20 09:51:52, Johannes Weiner wrote: > > > On Thu, May 21, 2020 at 09:32:45AM +0200, Michal Hocko wrote: > > [...] > > > > I am not saying the looping over try_to_free_pages is wrong. I do care > > > > about the final reclaim target. That shouldn't be arbitrary. We have > > > > established a target which is proportional to the requested amount of > > > > memory. And there is a good reason for that. If any task tries to > > > > reclaim down to the high limit then this might lead to a large > > > > unfairness when heavy producers piggy back on the active reclaimer(s). > > > > > > Why is that different than any other form of reclaim? > > > > Because the high limit reclaim is a best effort rather than must to > > either get over reclaim watermarks and continue allocation or meet the > > hard limit requirement to continue. > > It's not best effort. It's a must-meet or get put to sleep. You are > mistaken about what memory.high is. I do not see anything like that being documented. Let me remind you what the documentation says: memory.high A read-write single value file which exists on non-root cgroups. The default is "max". Memory usage throttle limit. This is the main mechanism to control memory usage of a cgroup. If a cgroup's usage goes over the high boundary, the processes of the cgroup are throttled and put under heavy reclaim pressure. Going over the high limit never invokes the OOM killer and under extreme conditions the limit may be breached. My understanding is that breaching the limit is acceptable if the memory is not reclaimable after placing a heavy reclaim pressure. We can discuss what the heavy reclaim means but the underlying fact is that the keeping the consumption under the limit is a best effort. Please also let me remind you that the best effort implementation has been there since the beginning when the memory.high has been introduced. Now you seem to be convinced that the semantic is _obviously_ different. It is not the first time when the high limit behavior has changed. Mostly based on "what is currently happening in your fleet". And can see why it is reasonable to adopt to a real life usage. That is OK most of the time. But I haven't heard why keeping the existing approach and enforcing the reclaim target is not working properly so far. All I can hear is a generic statement that consistency matters much more than all potential problem it might introduce. Anyway, I do see that you are not really willing to have a non-confrontational discussion so I do not bother to reply to the rest and participate in the further discussion. As usual, let me remind you that I haven't nacked the patch. I do not plan to do that because "this is not black&white" as already said. But if your really want to push this through then let's do it properly at least. memcg->memcg_nr_pages_over_high has only very vague meaning if the reclaim target is the high limit. The changelog should be also explicit about a potentially large stalls so that people debugging such a problem have a clue at least. -- Michal Hocko SUSE Labs