From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D92FDC433FE for ; Sat, 2 Apr 2022 03:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 398C26B0071; Fri, 1 Apr 2022 23:21:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 348A46B0072; Fri, 1 Apr 2022 23:21:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 210EE8D0001; Fri, 1 Apr 2022 23:21:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 125956B0071 for ; Fri, 1 Apr 2022 23:21:57 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id C1EDF60C15 for ; Sat, 2 Apr 2022 03:21:46 +0000 (UTC) X-FDA: 79310489412.10.448B740 Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by imf28.hostedemail.com (Postfix) with ESMTP id 4D166C0024 for ; Sat, 2 Apr 2022 03:21:46 +0000 (UTC) Received: by mail-qt1-f171.google.com with SMTP id b18so3665270qtk.13 for ; Fri, 01 Apr 2022 20:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jmwCIqyPXnCB8aA5w+kz04qMXqDkkG33h9HD7X6PkyE=; b=Es64RWPEOMaeUpsTBsp9QH8SB0bTj+XPwPQVv+sjmZQ1aJBaPnYGsq8c+Uy7XOMGtP LEsk1MmN8KAZ9r1EtKVWw7XQ54XW3BEEQGGAAN/xVyMstrKhZ03d6MBD7GhxYMiE2hwV my4S7GUbgUaEQ4ieOpwX7MorMufPrHPiQaoi3YOr/QUSdwLLCLkPu1iMoTtqmflRd441 jQoboPhRpY28knGoYz++639zH09EBeFHXFLHv8uQ69pW83wM3Y5PNyqN9w8dRKAaMEH8 GymvHjC6LcuOsiBTSZvemZIa3crKu9ddu09nOPbIu7IQe3Dw4xL45KBG1+5720KDS3tI AlPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jmwCIqyPXnCB8aA5w+kz04qMXqDkkG33h9HD7X6PkyE=; b=sSfD15bcdGoJWDPEpQ5H1wv4esudMaR45QynZSss8Ezprr17vVzVYUlczqTlPWaneK sl3R5b0QOHm4897TiV4EkyzEW5FFy8VLbns6ofePvOz+/nhUhYGPgM7HJV/+bZvchLyr pD03lbvwBhNarjwz1AC+KJkdN2mhOGYecYrd3pIaWygkcXXwVH6chNQ1WHoHpBaGPYAR aL1zY4e7zwxUUq8LgguHz17tVyQmRMjIcywfVFJ6kfKtZoQZXspvlAA1PXg4INP++h6s tncPb5RAuXlSd5wdLxRxJf6XCeBNHvEyGbo0L3nNUacB3iqIymP+l2Yrl+8iaFEaIYBU sYSg== X-Gm-Message-State: AOAM532SR9hRFQDSfdIBjMDdoKipJ8j+megSa51tB4pl2KlWJqDLhgN1 pOhFyD7AEYEeP/9PAscq/jBdjBysNCg5AS+OfBg= X-Google-Smtp-Source: ABdhPJzZUw7eKxiuUyP4w4PLhuAmsTpE+B+QsKzIKOPkf8rtTiuaQHiNwk70ehTZClxFoowef6RGMtugN/J3WmCs7ZI= X-Received: by 2002:a05:622a:1013:b0:2e1:c9c0:9831 with SMTP id d19-20020a05622a101300b002e1c9c09831mr10709011qte.245.1648869705560; Fri, 01 Apr 2022 20:21:45 -0700 (PDT) MIME-Version: 1.0 References: <1648713656-24254-1-git-send-email-zhaoyang.huang@unisoc.com> In-Reply-To: From: Zhaoyang Huang Date: Sat, 2 Apr 2022 11:21:18 +0800 Message-ID: Subject: Re: [RFC PATCH] cgroup: introduce dynamic protection for memcg To: Suren Baghdasaryan Cc: Michal Hocko , "zhaoyang.huang" , Andrew Morton , Johannes Weiner , Vladimir Davydov , "open list:MEMORY MANAGEMENT" , LKML , cgroups mailinglist , Ke Wang Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: 8asac39t516jpgejkhzp8377458cfuqg Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Es64RWPE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4D166C0024 X-HE-Tag: 1648869706-502525 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 1, 2022 at 12:46 PM Suren Baghdasaryan wrote: > > On Thu, Mar 31, 2022 at 6:51 PM Zhaoyang Huang wrote: > > > > On Fri, Apr 1, 2022 at 3:26 AM Suren Baghdasaryan wrote: > > > > > > On Thu, Mar 31, 2022 at 4:35 AM Michal Hocko wrote: > > > > > > > > On Thu 31-03-22 19:18:58, Zhaoyang Huang wrote: > > > > > On Thu, Mar 31, 2022 at 5:01 PM Michal Hocko wrote: > > > > > > > > > > > > On Thu 31-03-22 16:00:56, zhaoyang.huang wrote: > > > > > > > From: Zhaoyang Huang > > > > > > > > > > > > > > For some kind of memcg, the usage is varies greatly from scenarios. Such as > > > > > > > multimedia app could have the usage range from 50MB to 500MB, which generated > > > > > > > by loading an special algorithm into its virtual address space and make it hard > > > > > > > to protect the expanded usage without userspace's interaction. > > > > > > > > > > > > Do I get it correctly that the concern you have is that you do not know > > > > > > how much memory your workload will need because that depends on some > > > > > > parameters? > > > > > right. such as a camera APP will expand the usage from 50MB to 500MB > > > > > because of launching a special function(face beauty etc need special > > > > > algorithm) > > > > > > > > > > > > > Furthermore, fixed > > > > > > > memory.low is a little bit against its role of soft protection as it will response > > > > > > > any system's memory pressure in same way. > > > > > > > > > > > > Could you be more specific about this as well? > > > > > As the camera case above, if we set memory.low as 200MB to keep the > > > > > APP run smoothly, the system will experience high memory pressure when > > > > > another high load APP launched simultaneously. I would like to have > > > > > camera be reclaimed under this scenario. > > > > > > > > OK, so you effectivelly want to keep the memory protection when there is > > > > a "normal" memory pressure but want to relax the protection on other > > > > high memory utilization situations? > > > > > > > > How do you exactly tell a difference between a steady memory pressure > > > > (say stream IO on the page cache) from "high load APP launched"? Should > > > > you reduce the protection on the stram IO situation as well? > > > > > > IIUC what you are implementing here is a "memory allowance boost" > > > feature and it seems you are implementing it entirely inside the > > > kernel, while only userspace knows when to apply this boost (say at > > > app launch time). This does not make sense to me. > > I am wondering if it could be more helpful to apply this patch on the > > background services(system_server etc) than APP, while the latter ones > > are persistent to the system. > > > > > > > > > > > [...] > > > > > > One very important thing that I am missing here is the overall objective of this > > > > > > tuning. From the above it seems that you want to (ab)use memory->low to > > > > > > protect some portion of the charged memory and that the protection > > > > > > shrinks over time depending on the the global PSI metrict and time. > > > > > > But why this is a good thing? > > > > > 'Good' means it meets my original goal of keeping the usage during a > > > > > period of time and responding to the system's memory pressure. For an > > > > > android like system, memory is almost forever being in a tight status > > > > > no matter how many RAM it has. What we need from memcg is more than > > > > > control and grouping, we need it to be more responsive to the system's > > > > > load and could sacrifice its usage under certain criteria. > > > > > > > > Why existing tools/APIs are insufficient for that? You can watch for > > > > both global and memcg memory pressure including PSI metrics and update > > > > limits dynamically. Why is it necessary to put such a logic into the > > > > kernel? > > > > > > I had exactly the same thought while reading through this. > > > In Android you would probably need to implement a userspace service > > > which would temporarily relax the memcg limits when required, monitor > > > PSI levels and adjust the limits accordingly. > > As my response to Michal's comment. Userspace monitors introduce > > latency. Take LMKD as an example, it is actually driven by the > > PSI_POLL_PERIOD_XXX_MS after first wakeup, which means > > PSI_WINDOW_SIZE_MS could be too big to rely on. IMHO, with regards to > > the responding time, LMKD is less efficient than lmk driver but more > > strong in strategy things. I would like to test this patch in real > > android's work load and feedback in next version. > > LMKD is a reactive mechanism which does not know when memory pressure > might rise, therefore its response latency matters. > The usecases you mentioned seemed to imply that userspace was aware of > increased memory demands of the process (app startup time, maybe the > moment the app becomes foreground, etc.). Therefore the userspace > could relax memory allowance before that memory is requested. Was my > understanding incorrect? In terms of Android specific scenarios, I think it is not the best choice to have AMS relax memory allowance directly when activity arised, as: 1. AMS also has to face latency issues like LMKD does when polling systems PSI value 2. AMS needs to launch syscalls to act. A read for either memcg.usage/watermark and a write on memcg.min/low. 3. Processes in CACHED adj are not capable of shrinking themselves except waiting to be killed by LMKD. IMO, this patch is alike an Pressure Aware Per Process Reclaim which does NOT launch real reclaiming. > > > > > > > > > > > > -- > > > > Michal Hocko > > > > SUSE Labs