From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3167C00528 for ; Tue, 1 Aug 2023 17:01:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E38C94002B; Tue, 1 Aug 2023 13:01:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 293AA940010; Tue, 1 Aug 2023 13:01:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1821794002B; Tue, 1 Aug 2023 13:01:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 072AF940010 for ; Tue, 1 Aug 2023 13:01:03 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 30883B206D for ; Tue, 1 Aug 2023 17:01:01 +0000 (UTC) X-FDA: 81076150722.24.066C610 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf26.hostedemail.com (Postfix) with ESMTP id 05BEC142405 for ; Tue, 1 Aug 2023 16:40:14 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=p6fiXJVO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690908016; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4qjFUw6+lEzkCCKqkbY3T91pufwJOps9NprX6UVn+h8=; b=viJnFQaRM8mZ4BSd4nGwAP6bVb5Tn5tOuPrdENqt7PqvuVbFQnd2yr2MDlz2Dr/Q1jb1nO NVeiJU9xEI+p8jOPpcz1sg18hToN3VUUwzNx3fgcsbQg4cqpNWOe0rZ4Y13I8eYukgrOtg s9vVRq9Gj5GbkIynAloIB9Fn7FxEfHA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=p6fiXJVO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690908016; a=rsa-sha256; cv=none; b=uQf9bDEtLgE5CZJcLbuvs8mrANlBnSRjPL+H3KUCQ+4jQYwSb5sCoI8xDTtpzP+BHYTBnx 6oGTlYVYa/ghCg6+f6jLkGt6uOxtvt/FktdN/YkIQoBKHvbrwpgdAzjjVU2n5d6pK4J53a njhFXY/dxKku36C50tAPYyZGeLN+dEU= Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-99c3c8adb27so34954866b.1 for ; Tue, 01 Aug 2023 09:40:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690908013; x=1691512813; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4qjFUw6+lEzkCCKqkbY3T91pufwJOps9NprX6UVn+h8=; b=p6fiXJVO6VAT7ddDlyeNeKo0u9g75XBshHTDYQkWpxGEUOGoOVAa7lHgsGvQZ5v/+W OL8o5krg+sbCk3k/ILPv5kJcGtc7Hl7e/Ptk68GNbUYDFS1ZLOXSdMFbtMgISIwiBvXM xA5PCRA4fUyBY1hNJB6x57RpX4pHwvLO70MzF7vtOTGutOLMa2G02e6PwMjrWoV32oEH b6YlMDU2PYjQiFSB63POIJeR3bgDccie71bMuCrG1M0DF1oPPZLzqfczUjMBmkdzgXbl zrQDN3zGYffW49VbgVfFPFSdFMhkF99msYQ0hnralZcPwwvzI8CmOTzVDqdh+UFOFJjq Lm5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690908013; x=1691512813; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4qjFUw6+lEzkCCKqkbY3T91pufwJOps9NprX6UVn+h8=; b=NCpLsjtpMI9ALg9vfuK3D79O1Ysqa8ogpm+fOuyWFzDXDFCr+mdK4h7iKF/gj0GsoC a6XUSAVrOsuj/7d1tQ7JLu5vlffyC/mMrbSsDs1a+ZrgTjk+jblhaHQwDFCVgWEtUahl bFhQWxiwwGBWJqxpIdYRC4GdFFNdQTancOoPt25sVOoXmTR7vUPQGNCHS8txeZ9Cxapt iqceb7maYXuzeqDVTq/nIdWcU+dGo5Tu7BznoV5WUHsZLNFRrm3FvgG5IONO+X2+w5r5 Kro3FlILxi/DlFmYpCRtBoa39uN/RpcayPm7MQ6YJozISYC/d+QwRGYVzCGyYFz/EqTm wIoA== X-Gm-Message-State: ABy/qLZolR0dXYPlr8xSEOXwDQGpgXOMsIoOhbJmZHbf5fV88gEZojQl UxUpQl/MG60gLO9/uvOJ3AwQJZhuG9Itxc5hZQ9+ug== X-Google-Smtp-Source: APBJJlEOr1teCw40TvLuDNBikvF5ICCk1wj7SeHDxgnRGozU1Gwr56AHW9TbZUOhBSqnj1vnRQImF4blOlMh+ybaaJ4= X-Received: by 2002:a17:907:75d0:b0:989:450:e567 with SMTP id jl16-20020a17090775d000b009890450e567mr2977498ejc.65.1690908012991; Tue, 01 Aug 2023 09:40:12 -0700 (PDT) MIME-Version: 1.0 References: <20230726153223.821757-1-yosryahmed@google.com> <20230726153223.821757-2-yosryahmed@google.com> In-Reply-To: From: Yosry Ahmed Date: Tue, 1 Aug 2023 09:39:36 -0700 Message-ID: Subject: Re: [PATCH v3] mm: memcg: use rstat for non-hierarchical stats To: Michal Hocko Cc: Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: hzy6gtn767tix3bfe1squisb9cixs747 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 05BEC142405 X-HE-Tag: 1690908014-675081 X-HE-Meta: U2FsdGVkX1/2YgY/u7Nz+NK0Ad9X8MurtvLHLzVgzI7pISavNo1jlB6zxsJ+lqUWAmrDhZ9MkYm3zOD4zNdiTF2refI/OoYC9RfyN2Rg1o170LuS4rc5pSAf0Hzn9k5J3oz1dAkwIhcgw8NeAdCDAbCDvblOwcUwP4NftAMyxNNNs9DQnK/5CgQucm2ih8v7yX7lC02MIwYRgHNRWI0Fcx0nUilhB5BQ/Ug0IaMBqHO3fApyuwcdDdPGvZlRa8Azgrm1xtB3W1JSJL7Cr2DFIHHuMIqgYcnr2uZeiki+0vxUg/RetnNeA+dtzSZ8SaOq7kq0lEeRTQpQbSY/M87e0QyLWxV1R8Ni+lklENEQWP5img3BX9126lMVFVPnuxTm3ppAwgGaZ/rWVym7dvamK2PFARIhs0s3dCW3pO/vrK3Tec5PawRGlImtVS8qIjT1eT0l1EG5U7Pxf2ctiEzbq5A6fZ/96XLYcmTF+uhGFhdPmQkb9kZfaTXBahfKj0nIcxaDF6wOEGA14xLO2st0jIQtFul5SpQpW82SqGgjAMDYmozZybnRAmycPF8B+51adiWdRyMxFv8MgHpqnXdCiHxMpojTrtH7j8yScs6/a4wAE4r87aJyZlJmQn8kigO+ZymaxLGsDdQJQNRYT9R82qqalnqC9ZfJDlL0k1lpZljVF5+BDLT/vshPEDrgLFVfjQSSfshw7P9I9PlEriy2XAEMOH2iCT4rydv4zJ3JwFbxdIR9qY7R5OwT5x1xSh7vINAJ2Tk6WL9SFWhxhUVbs0HV+fxpojwXTsaimBPwmO9rzNVOfgUliSqCKWbEiIZgCiLd+w4vMyYav/Hl5ZakVFQMbvfJhkZX/2cuHR0PFP8rWJ5S8HaqdgWZAuerBdA+t9Ej3uGS6+mcD8QWUKVCtozs/A53HpGZDZWagUjmXh3QN14lV0QFywNDfNNojocGmeXP5GeWPb3NatOTbGr jSap95qD KyBt+1tyJCGXKBH/rfWHVknsSzAayMMxnKoeaR22VYFjaByJf/7vQHMZChVnO3JDLsYWHftgLQQcYtY8INwjNrEqujgJ9MM80iyWV1LIQOKemfuorZ80LeyjV/xmOt2FWxHiSJsoqiJqanGsC5wjNg4VCuJ2E5PXX4epryti2DgGfJnIXZp3q9/BMo6POdVi/ED6si6JcR1pTmbU7cy2N8mxQoSyvZY615pNNos45oSaVvTBfkF4nGJ/l+2dV1ZHj8GRJ/W/kHvSgt3pxcoxHo/Y3Cy8FqjB22T/O4gIE7xJcq915lmqxvUXtox4ltERlEJvF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 1, 2023 at 7:30=E2=80=AFAM Michal Hocko wrote= : > > On Wed 26-07-23 15:32:23, Yosry Ahmed wrote: > > Currently, memcg uses rstat to maintain aggregated hierarchical stats. > > Counters are maintained for hierarchical stats at each memcg. Rstat > > tracks which cgroups have updates on which cpus to keep those counters > > fresh on the read-side. > > > > Non-hierarchical stats are currently not covered by rstat. Their > > per-cpu counters are summed up on every read, which is expensive. > > The original implementation did the same. At some point before rstat, > > non-hierarchical aggregated counters were introduced by > > commit a983b5ebee57 ("mm: memcontrol: fix excessive complexity in > > memory.stat reporting"). However, those counters were updated on the > > performance critical write-side, which caused regressions, so they were > > later removed by commit 815744d75152 ("mm: memcontrol: don't batch > > updates of local VM stats and events"). See [1] for more detailed > > history. > > > > Kernel versions in between a983b5ebee57 & 815744d75152 (a year and a > > half) enjoyed cheap reads of non-hierarchical stats, specifically on > > cgroup v1. When moving to more recent kernels, a performance regression > > for reading non-hierarchical stats is observed. > > > > Now that we have rstat, we know exactly which percpu counters have > > updates for each stat. We can maintain non-hierarchical counters again, > > making reads much more efficient, without affecting the performance > > critical write-side. Hence, add non-hierarchical (i.e local) counters > > for the stats, and extend rstat flushing to keep those up-to-date. > > > > A caveat is that we now need a stats flush before reading > > local/non-hierarchical stats through {memcg/lruvec}_page_state_local() > > or memcg_events_local(), where we previously only needed a flush to > > read hierarchical stats. Most contexts reading non-hierarchical stats > > are already doing a flush, add a flush to the only missing context in > > count_shadow_nodes(). > > > > With this patch, reading memory.stat from 1000 memcgs is 3x faster on a > > machine with 256 cpus on cgroup v1: > > # for i in $(seq 1000); do mkdir /sys/fs/cgroup/memory/cg$i; done > > # time cat /dev/cgroup/memory/cg*/memory.stat > /dev/null > > real 0m0.125s > > user 0m0.005s > > sys 0m0.120s > > > > After: > > real 0m0.032s > > user 0m0.005s > > sys 0m0.027s > > Have you measured any potential regression for cgroup v2 which collects > all this data without ever using it (AFAICS)? I did not. I did not expect noticeable regressions given that all the extra work is done during flushing, which should mostly be done by the asynchronous worker, but can also happen in the stats reading context. Let me run the same script on cgroup v2 just in case and report back. > -- > Michal Hocko > SUSE Labs