From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC119C77B7A for ; Wed, 17 May 2023 08:10:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A109900006; Wed, 17 May 2023 04:10:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 55098900003; Wed, 17 May 2023 04:10:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41831900006; Wed, 17 May 2023 04:10:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 35475900003 for ; Wed, 17 May 2023 04:10:31 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 021901204AC for ; Wed, 17 May 2023 08:10:30 +0000 (UTC) X-FDA: 80799025062.11.4A0B314 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf20.hostedemail.com (Postfix) with ESMTP id 1D5F81C0015 for ; Wed, 17 May 2023 08:10:28 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="E/nQr0Tt"; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684311029; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M/JbPTvNpjTaAOSiw7cq2XUFwTTZo3maiLd9kZ3gbPY=; b=zcL9o5IIGYj3VA+wIQaLehJ82cuj439Olqk3tveE9h17xbzzv98U4zZ1FWL7u97/5QE75I ZxMM32UEe6ntHQh9wXEjowD2bSk51ZCRvkoVpb0acJOBS20TnOUIKET7/doUkUFJAmXS03 o4FEAxo94wBKlE1kFg983WBx8bB9Wt0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684311029; a=rsa-sha256; cv=none; b=jd7zS6TjLRQh/dbASRAuDcigA0rSyEYVN0bWeiAgPw5PPomJGSvFBUOZVtFtFXu+0fewI1 Z8iu+gc2FE/9m4bwHXMeRa1oZcc27YlxLoiPoJp1qj+29eM59vVAR+gFJ6L+acs6zLuHs5 xvSGXC+frAhershPcVRaoxpYy8CIlC0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="E/nQr0Tt"; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-96b0235c10bso70436366b.3 for ; Wed, 17 May 2023 01:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684311027; x=1686903027; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=M/JbPTvNpjTaAOSiw7cq2XUFwTTZo3maiLd9kZ3gbPY=; b=E/nQr0TtAbvKOJgpRKAM1JVFuP/MwUX23KxLMaK9E935LDpjE6s9HpTk6zcktHzA3V lVxuSKWxkpaEF/0X7X4fb0A2wQtR+BP46udE3XC2KotT4N3Y20wucgNCN5myvUQPrgYm hf8qXaIk4/mtVqOzBMdebf1uOebU+yZFQwgKC0uSR7A2S8DJaBgTCt2tXTRjmHgu9gTl He1rqLvC6itIyQX551xHdP0ntcFbz4HClZ9Wg7mkLnBAKGlfTSh3/2KEYLgKR0024kd1 FZk9gcyZUQemmV3CyHP8mGDAj9j4MwZkxgKrt5y90zAG/N8tFGr657bHQAGYojKPXnWN MoDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684311027; x=1686903027; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M/JbPTvNpjTaAOSiw7cq2XUFwTTZo3maiLd9kZ3gbPY=; b=FxrUuVBq4AaSwvurYcVWKFNFU915Nh5VbJ16KEc4CZ8+P1rpA69HQq6nuJnmFd02aS kMLtgVXhndAH38+K8sA9863G4wqPgMztKhwEqGZSJ9KoGetG5eoc2MeZuI6QcJ7siKfn aKl3f6XoPxeoxPWyb/xM3l5+qQahQJm+1C+/U/51pYaMNwTxHu5gAiw8Ig3zRqbB0njs FnbT8igJrR0cVmlRe3EbyzEmknw+f8LdOaVoBdz6BAy7QoGuPLgsE/DJxxfCxkQwFfeI GEBpophjNqDB/GHfE8zEfd6zhntipx9S6oGz44h2gVCkHMQJftxqrc3pywwq5NQlTKUQ 3YVg== X-Gm-Message-State: AC+VfDw5SNeeC0x1nXxbAmzaiO4YE75iitxtDT3InGYLYdrUyazIGnWg l+7xKWZJdGPT7XC6sCZoM1Cj3QEspupoRV6tB7oWog== X-Google-Smtp-Source: ACHHUZ45upb/WQWXJ0QlUEgpUZn+CmUofbM6pNnjDG0KW/H53o2SeMyeHn7WakUEa0zZJ6DGueoe8o/J1+MWpbNBXNs= X-Received: by 2002:a17:906:fe04:b0:966:1984:9d21 with SMTP id wy4-20020a170906fe0400b0096619849d21mr30993043ejb.9.1684311027372; Wed, 17 May 2023 01:10:27 -0700 (PDT) MIME-Version: 1.0 References: <09A746CC-E38D-4ECA-B0F4-862EC6229A0F@didiglobal.com> In-Reply-To: <09A746CC-E38D-4ECA-B0F4-862EC6229A0F@didiglobal.com> From: Yosry Ahmed Date: Wed, 17 May 2023 01:09:50 -0700 Message-ID: Subject: Re: [PATCH v4 0/2] memcontrol: support cgroup level OOM protection To: =?UTF-8?B?56iL5Z6y5rabIENoZW5na2FpdGFvIENoZW5n?= Cc: "tj@kernel.org" , "lizefan.x@bytedance.com" , "hannes@cmpxchg.org" , "corbet@lwn.net" , "mhocko@kernel.org" , "roman.gushchin@linux.dev" , "shakeelb@google.com" , "akpm@linux-foundation.org" , "brauner@kernel.org" , "muchun.song@linux.dev" , "viro@zeniv.linux.org.uk" , "zhengqi.arch@bytedance.com" , "ebiederm@xmission.com" , "Liam.Howlett@oracle.com" , "chengzhihao1@huawei.com" , "pilgrimtao@gmail.com" , "haolee.swjtu@gmail.com" , "yuzhao@google.com" , "willy@infradead.org" , "vasily.averin@linux.dev" , "vbabka@suse.cz" , "surenb@google.com" , "sfr@canb.auug.org.au" , "mcgrof@kernel.org" , "feng.tang@intel.com" , "cgroups@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "linux-mm@kvack.org" , David Rientjes Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 8cw83keqpt4nhtys4dzyxtndfrpnxbih X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1D5F81C0015 X-Rspam-User: X-HE-Tag: 1684311028-919155 X-HE-Meta: U2FsdGVkX1+yzEoBpKHAgrXzfSor3yUJE8Cd/maoNPxZiT7HuXAZKn7A9yWlXpV3M9/fxUSntZ5DiJx4dGyg9/I/0VWF8nY6PmE4UM/is75ydctJ6BlsbhYiiKmThrG8qtUdUI08sk9BKJjaigT+x5mwL0eMAqYeuBN+dPNwpDm3EqbVd+Bbt29F5MLYVS1JiRINFrCJ/KmhD/v8LrztjgtWAQbvYnj1WAW2yu24jRXzJ+ZZyu9gsmyUbsc0LiulDv76KGy/9G3rnSd2iDqukf9sQUfsWAAM1MSvEvVmC80fN5jEF7pE6qFldE2At9N0XdeQMdofqFQM5aRsF+y4hI61IFPmCsAaF1jriB7ISYJG8diOSW7nZU52/glR02P4uHCaim3X35F5KF3VTWgKeGwXQEPzCZL/GSNqCo0csgY9dv3gy/w9vpq4Lapvpp1HTF3u5ScxtUK0qjW+Pqowd0wH8X+ZIZt6M7S2B+E+htHLKvOin+X3enWCxDtl+M4gflkLdWusEWHPE8eSo8lbEwkgbTcRQN87gllHYIMpf7FvW002N5Y84ymoF+1Podwd7QhdPbbJiRsvnVG5f1+v9z/IdcLGzMKIlH5kRNlvN2Nojmr06jxLWNLpWTRfbEUYCxYBsSjtpsgPkwLWqFlFZJrG+JItTMxR8PkCwsSD/oS6QX8TGm90/rCRpVFrL3FJgE+tnem5dUgFAXM+JkH337iKk59zKeD3zpgClGUvZ1eGuwR85N7j0hbAA+bSuYZxan8xjbX7R2JdCzko00/riKk7KlhNz0h1s2YFqetlYHxKBUJStArw8PhuaWya0QkohfbFJvSxx4CQHwjZet2doeXUsLUYMGUdaSJvhCLF26ze3aSNUSdu5oOf/knvY+1+rQkSiEaTi0208JuBv7w60hN6TZQ2aHPSnfJw9M0uh2LGmC+nWA+umWfQyB/fgiW6TcfTAA1zcjFko5ZsuLH e29QB751 pyGxBH35yTa35gD6dGr4lQYtTC6DU7+ULdddVfdxmaicTba1GcRz0f8TDk0JCzy9SY16RwMGJlw/kJnB3nzJYjpLzjoyClgRc0RS2eEY/4wwHZF3reItAgUUtrW3f21RULoENmtzRLqmzh8ysDTEu7zZ/003koeQi5ChFZkCvw8/J19Y0WrtqyAfs1+O8hcsqYDWQYZGx7i827saO6vfBWujDxENztCAm56GOPWdLsDBn0jyaoQd0/6MDrihb8bpWgl+p8fCPPT7ADp5379qyGtZJBRhOOVtwntV0R10QQsCALAULmeII3Do5nLzAeRqVViA5/u6qM/hM3kgr49bOuv6l6CcorV021z/AsTCrMN7SY5MdP2IgPQOs0Ara5Wi3CwjO4ZfaxKgn5nQvX6XqLnZie+xWTuMqu1Bv1CBwhwRBKiOqxLBPwwTdnNuEPPK0y/kdavAj8X4GMrwJJlJEBpr48oCVQkkv91d/CGd+gXQQdnrhJXOw/2RsrQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 17, 2023 at 1:01=E2=80=AFAM =E7=A8=8B=E5=9E=B2=E6=B6=9B Chengka= itao Cheng wrote: > > At 2023-05-17 14:59:06, "Yosry Ahmed" wrote: > >+David Rientjes > > > >On Tue, May 16, 2023 at 8:20=E2=80=AFPM chengkaitao wrote: > >> > >> Establish a new OOM score algorithm, supports the cgroup level OOM > >> protection mechanism. When an global/memcg oom event occurs, we treat > >> all processes in the cgroup as a whole, and OOM killers need to select > >> the process to kill based on the protection quota of the cgroup. > >> > > > >Perhaps this is only slightly relevant, but at Google we do have a > >different per-memcg approach to protect from OOM kills, or more > >specifically tell the kernel how we would like the OOM killer to > >behave. > > > >We define an interface called memory.oom_score_badness, and we also > >allow it to be specified per-process through a procfs interface, > >similar to oom_score_adj. > > > >These scores essentially tell the OOM killer the order in which we > >prefer memcgs to be OOM'd, and the order in which we want processes in > >the memcg to be OOM'd. By default, all processes and memcgs start with > >the same score. Ties are broken based on the rss of the process or the > >usage of the memcg (prefer to kill the process/memcg that will free > >more memory) -- similar to the current OOM killer. > > Thank you for providing a new application scenario. You have described a > new per-memcg approach, but a simple introduction cannot explain the > details of your approach clearly. If you could compare and analyze my > patches for possible defects, or if your new approach has advantages > that my patches do not have, I would greatly appreciate it. Sorry if I was not clear, I am not implying in any way that the approach I am describing is better than your patches. I am guilty of not conducting the proper analysis you are requesting. I just saw the thread and thought it might be interesting to you or others to know the approach that we have been using for years in our production. I guess the target is the same, be able to tell the OOM killer which memcgs/processes are more important to protect. The fundamental difference is that instead of tuning this based on the memory usage of the memcg (your approach), we essentially give the OOM killer the ordering in which we want memcgs/processes to be OOM killed. This maps to jobs priorities essentially. If this approach works for you (or any other audience), that's great, I can share more details and perhaps we can reach something that we can both use :) > > >This has been brought up before in other discussions without much > >interest [1], but just thought it may be relevant here. > > > >[1]https://lore.kernel.org/lkml/CAHS8izN3ej1mqUpnNQ8c-1Bx5EeO7q5NOkh0qrY= _4PLqc8rkHA@mail.gmail.com/#t > > -- > Thanks for your comment! > chengkaitao >