From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 160A9C11F68 for ; Fri, 2 Jul 2021 10:40:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 716A861263 for ; Fri, 2 Jul 2021 10:40:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 716A861263 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=canonical.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 01A0E6B0011; Fri, 2 Jul 2021 06:40:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE53D6B0036; Fri, 2 Jul 2021 06:40:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D374F6B005D; Fri, 2 Jul 2021 06:40:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id A90D36B0011 for ; Fri, 2 Jul 2021 06:40:34 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CBB9A18041E94 for ; Fri, 2 Jul 2021 10:40:32 +0000 (UTC) X-FDA: 78317303904.11.41210AE Received: from youngberry.canonical.com (youngberry.canonical.com [91.189.89.112]) by imf12.hostedemail.com (Postfix) with ESMTP id 5F14E10000A2 for ; Fri, 2 Jul 2021 10:40:32 +0000 (UTC) Received: from mail-ed1-f70.google.com ([209.85.208.70]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1lzGb0-0000up-MN for linux-mm@kvack.org; Fri, 02 Jul 2021 10:40:30 +0000 Received: by mail-ed1-f70.google.com with SMTP id d5-20020a0564020785b02903958939248aso4869659edy.15 for ; Fri, 02 Jul 2021 03:40:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=OR6WjwPbbFIFhDonpm4AQaAwK32w3oXxzxHcnLwLZew=; b=NIwoattjKWpbLyMZ2fE+jfHxrXeiD14QxiUzE9K2DkJVtkiZtW+Igr5HnW+q65owli dlMqFRkRYw1KBb/VQaD98DR9qV4YcG/677czcfDkyb1I33FltXchDdRfMkBZOpFhvLGm sESj08qCCKZfOpZZoDqe+wghfcPHJDnJgEJlQj+qQMIOOugydQRCpjw7YSyTNphHVY3p AcLGEc18p6lFCHiGXoc/ZOQ8pFyvYB2zyKEgcC0nkO/hNGSPHeq1DkrahivNh2OwsH+h ZlgrpFgg9eiHN/pYkZpD6UM7DPTwxRQAP3Hc8vsLlBsDPe/wYIsK4mW6AG/AyB7SXHcD dewg== X-Gm-Message-State: AOAM533b05pXyrBR7fxcS8HRN+PtCERwYCH3BIpztoee3Jj+9EbMJfVd 4pqjuqnT4dfcp2ytW+26WLWV3OGqFqIyPiiUhsQtGmuZ+J6VTxmlh5AXRaTDF0bZC2jHQOLwOrV ZLw0rtirgoIloyJ3npIACpMc5bURj X-Received: by 2002:aa7:d413:: with SMTP id z19mr5907620edq.37.1625222430440; Fri, 02 Jul 2021 03:40:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy0IUtpnbrLKHbt0KupmEL3fvtTCZOa0qAkasNQJvQvbEvFFgPaoPxxOpALo+M8FkUcL26Ikw== X-Received: by 2002:aa7:d413:: with SMTP id z19mr5907601edq.37.1625222430186; Fri, 02 Jul 2021 03:40:30 -0700 (PDT) Received: from [192.168.1.115] (xdsl-188-155-177-222.adslplus.ch. [188.155.177.222]) by smtp.gmail.com with ESMTPSA id x2sm1142545edv.61.2021.07.02.03.40.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 Jul 2021 03:40:29 -0700 (PDT) Subject: Re: Process memory accounting (cgroups) accuracy To: Michal Hocko Cc: Andrew Morton , Johannes Weiner , Vladimir Davydov , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org References: <69ffd3a0-2cb7-8baa-17d0-ae45a52595af@canonical.com> From: Krzysztof Kozlowski Message-ID: <85b8a4f9-b9e9-a6ca-5d0c-c1ecb8c11ef3@canonical.com> Date: Fri, 2 Jul 2021 12:40:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 5F14E10000A2 Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=canonical.com (policy=none); spf=none (imf12.hostedemail.com: domain of krzysztof.kozlowski@canonical.com has no SPF policy when checking 91.189.89.112) smtp.mailfrom=krzysztof.kozlowski@canonical.com X-Stat-Signature: u8yn873dg8bhxaptr33gysqz3o5ywpz3 X-HE-Tag: 1625222432-97319 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 02/07/2021 11:08, Michal Hocko wrote: > On Fri 02-07-21 09:50:11, Krzysztof Kozlowski wrote: > [...] >> The questions: How accurate are now the cgroup counters? > > The precision depends on the number of CPUs the workload is running on > as we do a per-cpu charge caching to optimize the accounting. This is > MEMCG_CHARGE_BATCH (32) pages currently. You can learn more by checking > try_charge function (mm/memcontrol.c). This explains the 32 pages, thanks! > >> I understood they should charge only pages allocated by the process, so >> why mmap(4 kB) causes max_usage_in_bytes=132 kB? > > Please note that kernel allocations (marked by __GFP_ACCOUNT) are > accounted as well so this is not only about mmaped memory. > >> Why mmap(4 MB) causes max_usage_in_bytes=4 MB + 34 pages? > > The specific number will depend on the executing - e.g. use up all but 3 > pages from CPU0 batch and have 31 pages on another cpu. > >> What is being accounted there (stack guards?)? >> >> Or maybe the entire LTP test checking so carefully memcg limits is useless? > > Well, I haven't really checked details of those tests and their > objective but aiming for an absolute precision is not really something > that is very useful IMHO. We are very likely to do optimizations like > the one mentioned above as the runtime tends to be much more important > than to-the-page precision. > > Hope this clarifies this a bit. Yes, thanks! Best regards, Krzysztof