From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E19FC32793 for ; Wed, 24 Aug 2022 02:23:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF5E6940008; Tue, 23 Aug 2022 22:23:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA5A0940007; Tue, 23 Aug 2022 22:23:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96CD9940008; Tue, 23 Aug 2022 22:23:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 81F3F940007 for ; Tue, 23 Aug 2022 22:23:44 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5327E1209D7 for ; Wed, 24 Aug 2022 02:23:44 +0000 (UTC) X-FDA: 79832890368.21.B4D4C1B Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf21.hostedemail.com (Postfix) with ESMTP id 0C94E1C004A for ; Wed, 24 Aug 2022 02:23:43 +0000 (UTC) Received: by mail-lf1-f51.google.com with SMTP id d8so9745066lfq.0 for ; Tue, 23 Aug 2022 19:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=zuPw7dTqyiFGf+aANdlA2zpaqeNoNbr+dKLXTuVUbxM=; b=YE9TTztfCeLOMQPUNlSlZw4VXHYmG6hYDIYDMAGrUMVCu8/6Dj8VY6NB0OxLXJ64yf xOeKgtqWo+k8y9K/QdIbLW02sxThwCpeD9Cz/dxmyqM8BCwbLLzqjGsT0RIy4nQxGz9g K8HQKd1lko4Qhd9ZDPYVlU+yqqEdjfvPAZf6D/FnL7JIktcKWpaXUMTMyzGxE9qh9GH6 Z1Rzok9wOm4LrCwQ5eOR6TdhtVCf8u3KHeoFW71c6JEFAOOATw+fl0imQQBsqxGpg3DT I7BoK1rzgVFLhzPRMOkds9yBPdwoqFoMLWnzbPCZGrLeVxTd1VK+vbstm/Gm/bt4CpRx i4rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=zuPw7dTqyiFGf+aANdlA2zpaqeNoNbr+dKLXTuVUbxM=; b=MoBoZP2HHOSPuPmPV/VpQFuHj+mUZoHb5KrPt43b8aAmxk/Aghw4yQziMkg4RGNw51 sLTazJaLMT0sRiySFdxUbrXlMhcsaD7vszb0hZSgfATj5SlhXF/2UEYYTCcOTA8ExiAO Gs8tYVQQh3dZeIZ1BkLAVzXWGDKkvCYaTVuByyHdhaVtRDo51vrAmpD94VNYK0QYK3Ka Lojgy2Kc2oehHFpiF0s22KFoNZfZjM7RabRD2rI7As1Rr7bM+19xSjkKF25ShqqiQb4P fJ2EC1GQdACGhDY7QfPtnyDs27Ky5j4Y2ymRWzv8yFwA9iCwPHHoJBX1Bf3A09Z5pLhG UQlg== X-Gm-Message-State: ACgBeo0TsleFelNTN/jjKiXjVyla7a2Yt6Z+r2jgajkpvDIvivuJWc6k WDr0s1wIlOmga5gI6qefYjZis2Nm4C4S1drsVWo= X-Google-Smtp-Source: AA6agR4HTb0hBhxDqdY2t6DaDtqy8EYwp9PJeyFO7bHO7hUUYgxDXGt5ugPao5GpHcjthjAbEg0/VqZOJ9I1ETy5f9k= X-Received: by 2002:a05:6512:3049:b0:492:f394:11cd with SMTP id b9-20020a056512304900b00492f39411cdmr2178884lfb.165.1661307822271; Tue, 23 Aug 2022 19:23:42 -0700 (PDT) MIME-Version: 1.0 References: <1660908562-17409-1-git-send-email-zhaoyang.huang@unisoc.com> In-Reply-To: From: Zhaoyang Huang Date: Wed, 24 Aug 2022 10:23:14 +0800 Message-ID: Subject: Re: [RFC PATCH] memcg: use root_mem_cgroup when css is inherited To: Michal Hocko Cc: Suren Baghdasaryan , Tejun Heo , Shakeel Butt , "zhaoyang.huang" , Johannes Weiner , Linux MM , LKML , Cgroups , Ke Wang , Zefan Li , Roman Gushchin , Muchun Song Content-Type: text/plain; charset="UTF-8" ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YE9TTztf; spf=pass (imf21.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661307824; a=rsa-sha256; cv=none; b=Z5+xNe2HFikAqBadUSPLLRVR2SB/Ncvc9+6AimVZjfKsjPwWHlCzKntw8HUSgOYWRerlnk mlsUqwTOqza1yIlDYuMxSi+3gfbTE9CaSTSE/zwaKKQ8be5xNx29h5ToieSvZsk5N3oa7F ghXbqou3AWrasMGLF41LFWGQwCwYNsw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661307824; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zuPw7dTqyiFGf+aANdlA2zpaqeNoNbr+dKLXTuVUbxM=; b=V5aCPQ6yV4zmed7HDHtcfMtyeoyHjxvcX27wfy+BVHIAaYfNn3skaQpWmRpxp/x9jsFx+Z uwrkKBJqaeb/IyvP7WH5NER5Bg4iEWMRgBSH/KsU39/5G5Qw9PMcD43Hb+o0taef3gp+aM wDH7+nGPY8AvUtiUmH13uY4iq+/R0+o= Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YE9TTztf; spf=pass (imf21.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 0C94E1C004A X-Stat-Signature: ffmuw1mbfsdddcf89cqqjr4hi6y7s8oc X-HE-Tag: 1661307823-968800 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 23, 2022 at 7:51 PM Michal Hocko wrote: > > On Tue 23-08-22 17:20:59, Zhaoyang Huang wrote: > > On Tue, Aug 23, 2022 at 4:33 PM Michal Hocko wrote: > > > > > > On Tue 23-08-22 14:03:04, Zhaoyang Huang wrote: > > > > On Tue, Aug 23, 2022 at 1:21 PM Michal Hocko wrote: > > > > > > > > > > On Tue 23-08-22 10:31:57, Zhaoyang Huang wrote: > > > [...] > > > > > > I would like to quote the comments from google side for more details > > > > > > which can also be observed from different vendors. > > > > > > "Also be advised that when you enable memcg v2 you will be using > > > > > > per-app memcg configuration which implies noticeable overhead because > > > > > > every app will have its own group. For example pagefault path will > > > > > > regress by about 15%. And obviously there will be some memory overhead > > > > > > as well. That's the reason we don't enable them in Android by > > > > > > default." > > > > > > > > > > This should be reported and investigated. Because per-application memcg > > > > > vs. memcg in general shouldn't make much of a difference from the > > > > > performance side. I can see a potential performance impact for no-memcg > > > > > vs. memcg case but even then 15% is quite a lot. > > > > Less efficiency on memory reclaim caused by multi-LRU should be one of > > > > the reason, which has been proved by comparing per-app memcg on/off. > > > > Besides, theoretically workingset could also broken as LRU is too > > > > short to compose workingset. > > > > > > Do you have any data to back these claims? Is this something that could > > > be handled on the configuration level? E.g. by applying low limit > > > protection to keep the workingset in the memory? > > I don't think so. IMO, workingset works when there are pages evicted > > from LRU and then refault which provide refault distance for pages. > > Applying memcg's protection will have all LRU out of evicted which > > make the mechanism fail. > > It is really hard to help you out without any actual data. The idea was > though to use the low limit protection to adaptively configure > respective memcgs to reduce refaults. You already have data about > refaults ready so increasing the limit for often refaulting memcgs would > reduce the trashing. > > [...] > > > A.cgroup.controllers = memory > > > A.cgroup.subtree_control = memory > > > > > > A/B.cgroup.controllers = memory > > > A/B.cgroup.subtree_control = memory > > > A/B/B1.cgroup.controllers = memory > > > > > > A/C.cgroup.controllers = memory > > > A/C.cgroup.subtree_control = "" > > > A/C/C1.cgroup.controllers = "" > > Yes for above hierarchy and configuration. > > > > > > Is your concern that C1 is charged to A/C or that you cannot actually make > > > A/C.cgroup.controllers = "" because you want to maintain memory in A? > > > Because that would be breaking the internal node constrain rule AFAICS. > > No. I just want to keep memory on B. > > That would require A to be without controllers which is not possible due > to hierarchical constrain. > > > > Or maybe you just really want a different hierarchy where > > > A == root_cgroup and want the memory acocunted in B > > > (root/B.cgroup.controllers = memory) but not in C (root/C.cgroup.controllers = "")? > > Yes. > > > > > > That would mean that C memory would be maintained on the global (root > > > memcg) LRUs which is the only internal node which is allowed to have > > > resources because it is special. > > Exactly. I would like to have all groups like C which have no parent's > > subtree_control = memory charge memory to root. Under this > > implementation, memory under enabled group will be protected by > > min/low while other groups' memory share the same LRU to have > > workingset things take effect. > > One way to achieve that would be shaping the hierarchy the following way > root > / \ > no_memcg[1] memcg[2] > |||||||| ||||| > app_cgroups app_cgroups > > with > no_memcg.subtree_control = "" > memcg.subtree_control = memory > > no? According to my understanding, No as there will be no no_memcg. All children groups under root would have its cgroup.controllers = memory as long as root has memory enabled. Under this circumstance, all descendants group under 'no_memcg' will charge memory to its parent group. This is caused by e_css policy when apply subsys control which have child group use its first level ancestors css. > > You haven't really described why you need per application freezer cgroup > but I suspect you want to selectively freeze applications. Is there > any obstacle to have a dedicated frozen cgroup and migrate tasks to be > frozen there? > -- > Michal Hocko > SUSE Labs