From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E22FC433F5 for ; Mon, 4 Apr 2022 11:35:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FEA76B0072; Mon, 4 Apr 2022 07:35:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9AE016B0073; Mon, 4 Apr 2022 07:35:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89CB96B0074; Mon, 4 Apr 2022 07:35:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 795E46B0072 for ; Mon, 4 Apr 2022 07:35:28 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 29EA1183D71EF for ; Mon, 4 Apr 2022 11:35:18 +0000 (UTC) X-FDA: 79318990716.22.E14F396 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf29.hostedemail.com (Postfix) with ESMTP id 508B5120028 for ; Mon, 4 Apr 2022 11:35:17 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id r127so7279869qke.13 for ; Mon, 04 Apr 2022 04:35:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qm48wnLSa24puHCxxJ0e1Qe5gtn6QPwMgY9+Vycw7ig=; b=Ki+gCYwbS0XnQ5UhlqPCCi5C/H/hPmbA8WTAV8iHSC/oPkMxbbfr5ufUIJQvh9nj1i XCN/Wi+j7sh1PrPInyPGbJu5zCIY80Gfriv7q+IdRApbUkvSPn3aMNravOLC5GIKRUwl OfNTazXCkloY2UyAgYCq8ZwIAPfaWXToSKqA/3S7pjhQWgbTkqX3HSyVTknG9lMnugFJ HRpB4d4IwoyNiYzqewo/s4PVA6Ort+cs8Oj2H7qox2hHXoC9rVGnaFu5bXYDUzitT99z 3DrHOw0r8yWErgJQ/g+1Mb6g86bvmWYMS6GC7TYvlvgWEQG17Z2FeElDO8q6TZ0HTvgE hH0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qm48wnLSa24puHCxxJ0e1Qe5gtn6QPwMgY9+Vycw7ig=; b=BTWH1IIIO+ufg+SCDilw6umac2I0NfwOvM1F3+6P/+8ON64ahp/iVqxumli8oBhtJT kAZVXmcdrRGpjMZR2PU0H1bfklRRQoStjH+5QGRNeOlKaxmvR3dTAZ2chCALuxkyClmE 2MvTHt/QyhDnHAB5nH28rmXONGZWn7MYj7e09CmZjC4xIzc4ImtE/u+Oeu9+95lAAroq 1PY/hoswmDCeC/OsmpDbLjsrmdkSVd8Ep/7pRud7kOSqHSsUTauY/odhf/KLbZ/AzZYY euwKdiUyCbJPUagdZTGtr3S9vcKir+5yupkzphIECWMRxGsHS6GJtnUTy8aKOAqmIcE/ fcVQ== X-Gm-Message-State: AOAM530V4UfQIPOAO39ogOTcvUIl2myDq5AI9cTX6obNiRWiRHwIYA8N aw2KGgUSQP7O0Ft+811Mu4WoZzpIh4fsoFzXbbMfssA8BHz28w== X-Google-Smtp-Source: ABdhPJxfRY3HDCYw3miLPJVu8yAYK0vpS0RnLxCLR7sWI87lweGn7P71lMLm1l9RE/O8PZiw2PigCOYrbAcIPCxjy1g= X-Received: by 2002:a05:620a:4586:b0:67e:d240:ed1d with SMTP id bp6-20020a05620a458600b0067ed240ed1dmr13123222qkb.630.1649072116559; Mon, 04 Apr 2022 04:35:16 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Zhaoyang Huang Date: Mon, 4 Apr 2022 19:35:05 +0800 Message-ID: Subject: Re: [RFC PATCH] cgroup: introduce dynamic protection for memcg To: Michal Hocko Cc: Suren Baghdasaryan , "zhaoyang.huang" , Andrew Morton , Johannes Weiner , Vladimir Davydov , "open list:MEMORY MANAGEMENT" , LKML , cgroups mailinglist , Ke Wang Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: sokbp48z5m3rcq1up4ktpoaxr8dc6w57 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Ki+gCYwb; spf=pass (imf29.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 508B5120028 X-HE-Tag: 1649072117-842746 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 4, 2022 at 5:36 PM Michal Hocko wrote: > > On Mon 04-04-22 11:32:28, Michal Hocko wrote: > > On Mon 04-04-22 17:23:43, Zhaoyang Huang wrote: > > > On Mon, Apr 4, 2022 at 5:07 PM Zhaoyang Huang wrote: > > > > > > > > On Mon, Apr 4, 2022 at 4:51 PM Michal Hocko wrote: > > > > > > > > > > On Mon 04-04-22 10:33:58, Zhaoyang Huang wrote: > > > > > [...] > > > > > > > One thing that I don't understand in this approach is: why memory.low > > > > > > > should depend on the system's memory pressure. It seems you want to > > > > > > > allow a process to allocate more when memory pressure is high. That is > > > > > > > very counter-intuitive to me. Could you please explain the underlying > > > > > > > logic of why this is the right thing to do, without going into > > > > > > > technical details? > > > > > > What I want to achieve is make memory.low be positive correlation with > > > > > > timing and negative to memory pressure, which means the protected > > > > > > memcg should lower its protection(via lower memcg.low) for helping > > > > > > system's memory pressure when it's high. > > > > > > > > > > I have to say this is still very confusing to me. The low limit is a > > > > > protection against external (e.g. global) memory pressure. Decreasing > > > > > the protection based on the external pressure sounds like it goes right > > > > > against the purpose of the knob. I can see reasons to update protection > > > > > based on refaults or other metrics from the userspace but I still do not > > > > > see how this is a good auto-magic tuning done by the kernel. > > > > > > > > > > > The concept behind is memcg's > > > > > > fault back of dropped memory is less important than system's latency > > > > > > on high memory pressure. > > > > > > > > > > Can you give some specific examples? > > > > For both of the above two comments, please refer to the latest test > > > > result in Patchv2 I have sent. I prefer to name my change as focus > > > > transfer under pressure as protected memcg is the focus when system's > > > > memory pressure is low which will reclaim from root, this is not > > > > against current design. However, when global memory pressure is high, > > > > then the focus has to be changed to the whole system, because it > > > > doesn't make sense to let the protected memcg out of everybody, it > > > > can't > > > > do anything when the system is trapped in the kernel with reclaiming work. > > > Does it make more sense if I describe the change as memcg will be > > > protect long as system pressure is under the threshold(partially > > > coherent with current design) and will sacrifice the memcg if pressure > > > is over the threshold(added change) > > > > No, not really. For one it is still really unclear why there should be any > > difference in the semantic between global and external memory pressure > > in general. The low limit is always a protection from the external > > pressure. And what should be the actual threshold? Amount of the reclaim > > performed, effectivness of the reclaim or what? > > Btw. you might want to have a look at http://lkml.kernel.org/r/20220331084151.2600229-1-yosryahmed@google.com > where a new interface to allow pro-active memory reclaim is discussed. > I think that this might turn out to be a better fit then an automagic > kernel manipulation with a low limit. It will require a user agent to > drive the reclaim though. Ok. But AFAIK, there are some of this kinds of method working as out of tree code now. such as PPR in android etc. As I have replied to Suren, there is always latency issue on this scheme as the agent should poll the event/read current status/write to launch the action. This patch is aiming at solve part of these issues. > -- > Michal Hocko > SUSE Labs