From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7039C433B4 for ; Tue, 27 Apr 2021 14:17:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 25E07613DA for ; Tue, 27 Apr 2021 14:17:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25E07613DA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6BEDC6B0036; Tue, 27 Apr 2021 10:17:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6473D6B006E; Tue, 27 Apr 2021 10:17:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C05E6B0070; Tue, 27 Apr 2021 10:17:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id 2C38A6B0036 for ; Tue, 27 Apr 2021 10:17:42 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DF906180AD815 for ; Tue, 27 Apr 2021 14:17:41 +0000 (UTC) X-FDA: 78078350322.31.2209056 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf06.hostedemail.com (Postfix) with ESMTP id 9415DC0007F1 for ; Tue, 27 Apr 2021 14:17:44 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1619533060; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lPnBVd7Xw8yqyfq/ePcZua6Sq04c0XSFOx7Aw4QUk+A=; b=RFrBUn2qDwg7NGZHQeD0zUMFTeNwsyqboz9VhGTQjBlC8pRpDAhcxtAclArMasQjDUm+dJ jKgJ80gR6aJer81ujhwyEpfaHxMy2HjfyzYA2O1x4i6G8K2DOcz+4aMNWUEs2M+2f/i5kZ k1wjtkL1QgsHW9cumO2cNHqHcQFIREE= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 1909CB1B0; Tue, 27 Apr 2021 14:17:40 +0000 (UTC) Date: Tue, 27 Apr 2021 16:17:33 +0200 From: Michal Hocko To: Alexander Sosna Cc: Chris Down , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] Prevent OOM casualties by enforcing memcg limits Message-ID: References: <410a58ba-d746-4ed6-a660-98b5f99258c3@sosna.de> <93fcbc37-8c8c-a752-a191-ff8e3dc02eb1@sosna.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <93fcbc37-8c8c-a752-a191-ff8e3dc02eb1@sosna.de> X-Rspamd-Queue-Id: 9415DC0007F1 X-Stat-Signature: szx656p8gopd3trmjxg1gth7oys44yzs X-Rspamd-Server: rspam02 Received-SPF: none (suse.com>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619533064-159010 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 27-04-21 15:43:25, Alexander Sosna wrote: > > On 27.04.21 14:11, Michal Hocko wrote: [...] > > Well, I am afraid that a reliable and easy solutions would be extremely > > hard to find. A memcg aware overcommit policy is certainly possible but > > as I've said it would require an additional accounting, it would be > > quite unreliable - especially with small limits where the mapped (and > > accounted) address space is not predominant. A lack of background > > reclaim (kswapd in the global case) would result in ENOMEM reported even > > though there is reclaimable memory to satisfy the reserved address space > > etc. > > Thank you very much for this information. Would you share the opinion > that it would be too hacky to define an arbitrary memory threshold here? > One could say that below a used memory of X the memory cgroup limit is > not enforced by denying a malloc(). So that the status quo behavior is > only altered when the memory usage is above X. This would mitigate the > problem with small limits and does not introduce new risks or surprises, > because in this edge case it will behaves identical to the current kernel. It will not. Please read again about the memory reclaim concern. There is no background reclaim so (and I believe Chris has mentioned that in other email) the only way to balance memory consumption (e.g. caches) would be memory allocations which are excluded from the virtual memory accounting. That can lead to a hard to predict behavior. > >> Could > >> you elaborate on where you see "a lot of fallouts"? overcommit_memory 2 > >> is only set when needed for the desired workload. > > > > My above comment was more general to the approach Linux is embracing > > overcommit and relies on oom killer to handle fallouts. This to change > > would lead to lot of fallouts. E.g. many syscalls returning unexpected > > and unhandled ENOMEM etc. > > We are talking about a special use case here. Do you see a problem in > the domain where and how overcommit_memory=2 is used today? yes I do. I believe I have already provided some real challenges. All that being said, a virtual memory overcommit control could be implemented but I am not sure this is worth the additional complexity and overhead introduced by the additional accounting. -- Michal Hocko SUSE Labs