From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6C02EB64DD for ; Mon, 14 Aug 2023 17:56:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAF7D6B0075; Mon, 14 Aug 2023 13:56:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B60068E0002; Mon, 14 Aug 2023 13:56:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4E518E0001; Mon, 14 Aug 2023 13:56:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 927DB6B0075 for ; Mon, 14 Aug 2023 13:56:38 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5EF8816096B for ; Mon, 14 Aug 2023 17:56:38 +0000 (UTC) X-FDA: 81123465276.23.1E34B79 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf14.hostedemail.com (Postfix) with ESMTP id 66C38100002 for ; Mon, 14 Aug 2023 17:56:36 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=cloudflare.com header.s=google header.b=eEoTsU2y; dmarc=pass (policy=reject) header.from=cloudflare.com; spf=pass (imf14.hostedemail.com: domain of ivan@cloudflare.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=ivan@cloudflare.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692035796; a=rsa-sha256; cv=none; b=YU+ghxKwK640NlcJALJuZyQ+X0zWrF7E1Fz2xza/w++4tjuoZksNREcInFgeTZUTzLd5b3 v4ZWqhzeqbEaV8mCB0oKPSUccocsIiEled1HPzu0gW5PJYwlxfX3YHWk/JAevFTc4sCJPZ Ql1uEcLPsuo5NgiAHgFGCdeUkqnhroc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=cloudflare.com header.s=google header.b=eEoTsU2y; dmarc=pass (policy=reject) header.from=cloudflare.com; spf=pass (imf14.hostedemail.com: domain of ivan@cloudflare.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=ivan@cloudflare.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692035796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jeN/HGuUgCuQPe5kPvEPkK0pdFU40c32aDEDmB1vFRU=; b=3LA4laY2dwogNj4X7wAIJKLUt0uotBVsAgJHqs8mSXUMrDlGtq1A63KtkJdcAUMQtq2fIt dOHrqMhJDryWOOjASQgI3Y18oPqTiNXErYGl7bv60bU60/sAGT7CNcjmlTfwrsH7Ew9sZK vwS3vXtF6EQ1M+Zc0p5iDMD3HNRivVo= Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-3fe8a158fcbso28206015e9.2 for ; Mon, 14 Aug 2023 10:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; t=1692035795; x=1692640595; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=jeN/HGuUgCuQPe5kPvEPkK0pdFU40c32aDEDmB1vFRU=; b=eEoTsU2yEih/aF37TTSzmNf2+4o3WWF1X80OF2loX8zGBdBwW1B6vzzBdK1y20zH/r Sn4ahzedMLMHlay9fr3/eue4Hp1AwOn310a632QCVrN47BPnt+Ui5eBFBTUf6vVlWfFy LM6cmv43Q3SIto1Jjd8qhi9w9cYWGb0eYddco= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692035795; x=1692640595; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jeN/HGuUgCuQPe5kPvEPkK0pdFU40c32aDEDmB1vFRU=; b=gsy2qbDp3eZe+AEIcPVH0ymZm5yx2FCvRayNNv00O3zn1VS9hl/PRKHB1R0L+zk3Ym wOgLwuS3nnPGwndjc3D33xDHpYHmx7DZXzxwjHN/eBEGp3CrqLBRsweYGEs3AQ7ngZwW 2LT6Ns8EKNiUkULgfwBmUGJp6k+XmFseTpr2YoN7e2XQFNpsFwYgU1L6UiFH74+aTk8h 8bDSKFyucOzhSO3azyf10U5F+FNS92sp9vNgdVqTHo84Uy1AsLT+JrMNmuR1Q2cTziF0 cod005+oo87tI5nkXgGWrPQ+txs8g9P6SnCsJSydpYwkJdgB4/Cc6Ni2DF6xwFSnqE+Z 1N2w== X-Gm-Message-State: AOJu0YyPZIRsI8xmDMQgDjfITyYbo0AniSK869ipyhkC8agBM/Ybd2Bx L++8iz6jD5MlUgk4hANC+a5S6waMy0iKVJwvQmrREQ== X-Google-Smtp-Source: AGHT+IEtr8YhK9rwdhdGnQqyqtbe4YECBg/+Z36CICXvXvvNP5p+ftYBInLJlzQ0Abpx2QVpHp4v6za0CEi0bsvOQJ4= X-Received: by 2002:a7b:c3cd:0:b0:3fb:d68d:4c6f with SMTP id t13-20020a7bc3cd000000b003fbd68d4c6fmr7747060wmj.14.1692035794784; Mon, 14 Aug 2023 10:56:34 -0700 (PDT) MIME-Version: 1.0 References: <20230706062045.xwmwns7cm4fxd7iu@google.com> In-Reply-To: From: Ivan Babrou Date: Mon, 14 Aug 2023 10:56:23 -0700 Message-ID: Subject: Re: Expensive memory.stat + cpu.stat reads To: Shakeel Butt Cc: Waiman Long , cgroups@vger.kernel.org, Linux MM , kernel-team , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , linux-kernel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 66C38100002 X-Stat-Signature: ppd73pjjnqrfh3ibod3m3rx88jqaasjk X-HE-Tag: 1692035796-999760 X-HE-Meta: U2FsdGVkX1+8vEfW8CfiWVFSBK+MrJxPH2XQrFvRXb1GB4qTzthA3L0mklLEdjhTs1h64ZFT5brUVL9tWMbumxzTEDgtQm8NqDlz5WKkihIlLuutCO9z5mv2jJXgG/+fi1feyir6cNLO2wRkvmADc480j6ar2YUQW1RMisWSJOfh4tvJglCOEgad0qlVDGvqIt6oNJAGdyyx4BLH4HhwS5mLiau4+CvSkv7UK+ABYV/BcnHafbz5OjJschVzQBZUwGxks6W1tFC0xe9S+vpitJsfpG/3cRpqBNsrFBfhxQpvM73xp3/enZw5Ukg3gVY7J3hPxrcHGFRlmdVv0k9ZYEF57spwkPxDTtSvLSF0pNsgAmwkjH11QoaLBnkagqKsQ5jc2IFGw+gEuFcr4oDLDIE+a1l0jazy8aogw7P75iSHTDXp7WRMPOeaJVR7Wybs83epOpCHK6v62fV2rACHmWRa1sVC9xQCl5711Cruhj24r0FJY+CZE8mKzNHVq/E+wj5Lr4U+0qVpM92ErGuxsOA7iRwpfRbubDFeU4AunD37s+lyU7d3YJ4CPRPcAiwkYGYl6voU4JxGPqHhes5kkpOlWPc/X8cbyr1f3Y11x7K4AlZTHUUUV+d4VhFtaHaHFRmRY/wg3DOSlmuD6yJt+cCiYL1EYfhmKZ5ZmVRc1MfwsrxnCPyeBxAVJYlTmj2vLQWUv3XFqxGmkANHPqfFKEBlHucDSjmjmbxWzz8FUH5pZXdWafOGt3FwjmASTGhmN/IrIhPx1V5k4olhijpER59crAx20exTcdArE+3wtEP1SG+eUABG3j20wOncisyXdnlUTxa9yScJ7RjPOn/AFMUeG8FluRAk/v/1KuJnksDIIYr7MBJZiW7P5mJQB4AjrGo+EmwXIOlYkdcbV49Bq7aaQkTBKsTXMrk+6DM0PK2eWAAGi1qHVRETcidqhmCaI8w2SFHp2tMAo8li0Eq LyOE6XRE D75MfGjx1zAnBYw/9PU7ZnqxMlT+PD6/agqpzRFBW853wv9jqz1Ul80FrIPty6sTVXv3exoe4maQoMxfPjjXRZL9k2art9mAxqlNm8ymQ8F4ocoVsjOnNfPJiBv8Iyp6Iu6MNfIuCnffzgmv6e0tphD1YFTi04o1oHAqRUfQOAc/Zt75pCusPwFxPyQa+pSNb/H6N966hzCL4f+yMPOCjuds2RmvoQkHEyTZr1+0uTNK8CfINPEok/IuSyhoFsfvi4QnP4Lqhros3CZuwfC1zl1gCb+YR2URhB0eDqYYud1PVEdi2uddaeAZlQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 11, 2023 at 7:34=E2=80=AFPM Shakeel Butt = wrote: > > Hi Ivan, > > (sorry for late response as I was away) > > On Fri, Aug 11, 2023 at 3:35=E2=80=AFPM Ivan Babrou = wrote: > [...] > > > > I spent some time looking into this and I think I landed on a fix: > > > > > > > > * https://github.com/bobrik/linux/commit/50b627811d54 > > > > > > > > I'm not 100% sure if it's the right fix for the issue, but it reduc= es > > > > the runtime significantly. > > In your patch, can you try to replace mem_cgroup_flush_stats() with > mem_cgroup_flush_stats_ratelimited() instead of cgroup_rstat_flush(). > I wanted to see if you observe any stale stats issues. We scrape cgroup metrics every 53s and at that scale I doubt we'd notice any staleness. With 2s FLUSH_TIME it would be in the noise. We can even remove any sort of flushing from memory.stat as long as cpu.stat is read right before it as it does flushing for both. I agree with Yosry that there's probably no reason to flush both cpu and memory stats at the same time. It doesn't seem right to use delayed flush for memory and direct flush for cpu. Perhaps it doesn't matter as much to warrant a bigger rework of how it's done, but maybe then my patch to use the same mechanism between cpu and memory flushing makes sense?