From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6B93C43217 for ; Thu, 1 Dec 2022 09:02:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DD286B0072; Thu, 1 Dec 2022 04:02:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 28CA86B0073; Thu, 1 Dec 2022 04:02:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12D726B0074; Thu, 1 Dec 2022 04:02:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 00F106B0072 for ; Thu, 1 Dec 2022 04:02:08 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C772FC0FC4 for ; Thu, 1 Dec 2022 09:02:08 +0000 (UTC) X-FDA: 80193145536.10.16EA62E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf15.hostedemail.com (Postfix) with ESMTP id 3A248A000B for ; Thu, 1 Dec 2022 09:02:06 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BAD041FD63; Thu, 1 Dec 2022 09:02:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1669885325; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oLNRgcULxl39VgjFJbWNAjDoQ3yDDlm/yFmMG9HhK6M=; b=Ave28KB7tNiE9ZkiI7R4ytVUUUAZgkBbdNR7IQHcXtu89ZpGOoODAgfa8xscLtJtg0hMJJ 6gquAn4cxI2y631J5knFx5hUq5LZS8Udhl8vNAHpi6IWv8yq74McDglEEoEtre9bBwt6TB 9OFPwbsrQMhymA6Fhqf+pMPYOfRnQhE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A3D9E13B4A; Thu, 1 Dec 2022 09:02:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YsXwJ41tiGOJIQAAMHmgww (envelope-from ); Thu, 01 Dec 2022 09:02:05 +0000 Date: Thu, 1 Dec 2022 10:02:05 +0100 From: Michal Hocko To: =?utf-8?B?56iL5Z6y5rab?= Chengkaitao Cheng Cc: "roman.gushchin@linux.dev" , Tao pilgrim , "tj@kernel.org" , "lizefan.x@bytedance.com" , "hannes@cmpxchg.org" , "corbet@lwn.net" , "shakeelb@google.com" , "akpm@linux-foundation.org" , "songmuchun@bytedance.com" , "cgel.zte@gmail.com" , "ran.xiaokai@zte.com.cn" , "viro@zeniv.linux.org.uk" , "zhengqi.arch@bytedance.com" , "ebiederm@xmission.com" , "Liam.Howlett@oracle.com" , "chengzhihao1@huawei.com" , "haolee.swjtu@gmail.com" , "yuzhao@google.com" , "willy@infradead.org" , "vasily.averin@linux.dev" , "vbabka@suse.cz" , "surenb@google.com" , "sfr@canb.auug.org.au" , "mcgrof@kernel.org" , "sujiaxun@uniontech.com" , "feng.tang@intel.com" , "cgroups@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , Bagas Sanjaya , "linux-mm@kvack.org" , Greg Kroah-Hartman Subject: Re: [PATCH] mm: memcontrol: protect the memory in cgroup from being oom killed Message-ID: References: <5019F6D4-D341-4A5E-BAA1-1359A090114A@didiglobal.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5019F6D4-D341-4A5E-BAA1-1359A090114A@didiglobal.com> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669885327; a=rsa-sha256; cv=none; b=XxiFAjaCaKD0wCZ4eONgGMmdbQoDmLQbqBrWMhFf1HFkt3O1vsYtqJlhMJ/CpgHfxjiJGq wJZI/Ii8deX3Thf/OwEZXNHQwJPgKVVRe0kdznRUzs11t3675lX5QHogbEzrFAr3Jms67I tI0sRxtkQd4aYF8T6uBahgOA6yWIXt8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Ave28KB7; spf=pass (imf15.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669885327; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oLNRgcULxl39VgjFJbWNAjDoQ3yDDlm/yFmMG9HhK6M=; b=F6W36+mNkrSM3WBnD/+zlZVEEexJLcou/TDu7exbrnrIFBmcu/GKJRC5J06oQpQmeEE5YQ XYFAXFs4WZaYy/f0Gr7LA04Xud+rmn8KzfJIgwcwiIA3l4aFjya+bJuGCZCw0BB7l+WUrO cg2NGmcLVvQ7aPD5zrEJbKp6L0abjrQ= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3A248A000B X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Ave28KB7; spf=pass (imf15.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-Stat-Signature: 4f16aicedxgbfhi4xdrq696z6rw383jp X-HE-Tag: 1669885326-371168 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 01-12-22 07:49:04, 程垲涛 Chengkaitao Cheng wrote: > At 2022-12-01 07:29:11, "Roman Gushchin" wrote: [...] > >The problem is that the decision which process(es) to kill or preserve > >is individual to a specific workload (and can be even time-dependent > >for a given workload). > > It is correct to kill a process with high workload, but it may not be the > most appropriate. I think the specific process to kill needs to be decided > by the user. I think it is the original intention of score_adj design. I guess what Roman tries to say here is that there is no obviously _correct_ oom victim candidate. Well, except for a very narrow situation when there is a memory leak that consumes most of the memory over time. But that is really hard to identify by the oom selection algorithm in general. > >So it's really hard to come up with an in-kernel > >mechanism which is at the same time flexible enough to work for the majority > >of users and reliable enough to serve as the last oom resort measure (which > >is the basic goal of the kernel oom killer). > > > Our goal is to find a method that is less intrusive to the existing > mechanisms of the kernel, and find a more reasonable supplement > or alternative to the limitations of score_adj. > > >Previously the consensus was to keep the in-kernel oom killer dumb and reliable > >and implement complex policies in userspace (e.g. systemd-oomd etc). > > > >Is there a reason why such approach can't work in your case? > > I think that as kernel developers, we should try our best to provide > users with simpler and more powerful interfaces. It is clear that the > current oom score mechanism has many limitations. Users need to > do a lot of timed loop detection in order to complete work similar > to the oom score mechanism, or develop a new mechanism just to > skip the imperfect oom score mechanism. This is an inefficient and > forced behavior You are right that it makes sense to address typical usecases in the kernel if that is possible. But oom victim selection is really hard without a deeper understanding of the actual workload. The more clever we try to be the more corner cases we can produce. Please note that this has proven to be the case in the long oom development history. We used to sacrifice child processes over a large process to preserve work or prefer younger processes. Both those strategies led to problems. Memcg protection based mechanism sounds like an interesting idea because it mimics a reclaim protection scheme but I am a bit sceptical it will be practically useful. Most for 2 reasons. a) memory reclaim protection can be dynamically tuned because on reclaim/refault/psi metrics. oom events are rare and mostly a failure situation. This limits any feedback based approach IMHO. b) Hierarchical nature of the protection will make it quite hard to configure properly with predictable outcome. -- Michal Hocko SUSE Labs