From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A4A3C433F5 for ; Fri, 6 May 2022 15:58:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A090A6B0072; Fri, 6 May 2022 11:58:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B8E26B0073; Fri, 6 May 2022 11:58:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 880F06B0074; Fri, 6 May 2022 11:58:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 794546B0072 for ; Fri, 6 May 2022 11:58:22 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 5BFB812187A for ; Fri, 6 May 2022 15:58:22 +0000 (UTC) X-FDA: 79435775244.20.7D322B5 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf13.hostedemail.com (Postfix) with ESMTP id E804120009 for ; Fri, 6 May 2022 15:58:07 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id n18so7845064plg.5 for ; Fri, 06 May 2022 08:58:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VHy+wusyPADBOcjPYMIU3GWI4e85gM3jZeodGyJVQhA=; b=YGRbRScUA8LR419d9cSOfjxmPI30qASgatv9LYfKxT1s7ZO5i8WQkqAVMm0P3oJ4nk pBy3k6/8jLxgJSVy7MuL4DUxNA0WmvQ8gnfUfHudyju0I6slP1Mp0HLBuo7jjUPXvoc1 +2ppp9aeS4CY2viyRJUfAC2jdWa9UL/LWVB3lmNlPkBvpy+G+NMd9SazdehqRdbxwrXH edI8bgPLYjrN3G/mKqxCdQimCdKP4M6VVivakQcXQ6nYR6xoLOwI1tp+SrxtmQ4MRnQT ApUDsqAx5WIGKy18Lb7Oir3iBhYA4ONUJGRhDTeizI26JCjF/olW7Vs2VyIXz2JNtXnY rwDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VHy+wusyPADBOcjPYMIU3GWI4e85gM3jZeodGyJVQhA=; b=OgJHvbnSlPbUj6WoktxuVika5KaOdr+g9FPZZnuBcdTfvTFv7u/bsuKplqwdtatF1G bh1ffmUGh5exuAtUGrtxCDKLyNn5P5ySxZVDV4I9EpFJTRhr/qdN1/RVDAtev++ShsTn 4Q6Qtczw86yl/V0MBqk6Wx5aRDiifDRJu+Wqp+7HiDZ6ZWwxMN35LD8wAzMjykjfozR8 h8ypHretiiwk+vUqmGGh8Fkkyt1psn5qrT+/JOqNhFae7h6cCHMfX/m/eyTh4lFIxzGn fm1DaxDtYCySh1Ty3l547rAPkirvwP7LsnI1WchydzfwfrLO103rnc2yOwU2Jhwior7J LDBA== X-Gm-Message-State: AOAM533nnpD09jra11HrdNKA0bQ4SWd7gUQObjZSLyYMKPOGrMF/m8ir gZ7LTTpxURNuodLLDyN8PFibMKHdO/7E0PF2uwsyNQ== X-Google-Smtp-Source: ABdhPJyp2Q4nMMCgRmMgg2kKkOudiFFK7W93k8BUew9TnmnymZ67RQIeu7iOUYVz9L1gYPVH47+ppGfqXEhc9fEUOWs= X-Received: by 2002:a17:90a:ea18:b0:1da:4630:513d with SMTP id w24-20020a17090aea1800b001da4630513dmr4840715pjy.237.1651852700201; Fri, 06 May 2022 08:58:20 -0700 (PDT) MIME-Version: 1.0 References: <20220505121329.GA32827@us192.sjc.aristanetworks.com> In-Reply-To: From: Shakeel Butt Date: Fri, 6 May 2022 08:58:09 -0700 Message-ID: Subject: Re: [PATCH] mm/memcontrol: Export memcg->watermark via sysfs for v2 memcg To: Ganesan Rajagopal Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Cgroups , Linux MM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: E804120009 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=YGRbRScU; spf=pass (imf13.hostedemail.com: domain of shakeelb@google.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Stat-Signature: 7gjxzyuqch9y9qe7xdy1hf8gwbhbkpnt X-HE-Tag: 1651852687-180995 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000872, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 5, 2022 at 10:27 AM Ganesan Rajagopal wrote: > > On Thu, May 5, 2022 at 9:42 PM Shakeel Butt wrote: > > > > On Thu, May 5, 2022 at 5:13 AM Ganesan Rajagopal wrote: > > > > > > v1 memcg exports memcg->watermark as "memory.mem_usage_in_bytes" in > > > > *max_usage_in_bytes > > Oops, thanks for the correction. > > > > sysfs. This is missing for v2 memcg though "memory.current" is exported. > > > There is no other easy way of getting this information in Linux. > > > getrsuage() returns ru_maxrss but that's the max RSS of a single process > > > instead of the aggregated max RSS of all the processes. Hence, expose > > > memcg->watermark as "memory.watermark" for v2 memcg. > > > > > > Signed-off-by: Ganesan Rajagopal > > > > Can you please explain the use-case for which you need this metric? > > Also note that this is not really an aggregated RSS of all the > > processes in the cgroup. So, do you want max RSS or max charge and for > > what use-case? > > We run a lot of automated tests when building our software and used to > run into OOM scenarios when the tests run unbounded. We use this metric > to heuristically limit how many tests can run in parallel using per test > historical data. > > I understand this isn't really aggregated RSS, max charge works. We just > need some metric to account for the peak memory usage. We don't need > it to be super accurate because there's significant variance between test > runs anyway. We conservatively use the historical max to limit parallelism. > > Since this metric is not exposed in v2 memcg, the only alternative is to > poll "memory.current" which would be quite inefficient and grossly > inaccurate. > Oh also include the details you explained here in your commit message.