From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3297FC001DF for ; Tue, 1 Aug 2023 17:44:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2D9C940041; Tue, 1 Aug 2023 13:44:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BB734940010; Tue, 1 Aug 2023 13:44:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7EE1940041; Tue, 1 Aug 2023 13:44:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 97C17940010 for ; Tue, 1 Aug 2023 13:44:53 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4DDE5B232C for ; Tue, 1 Aug 2023 17:44:53 +0000 (UTC) X-FDA: 81076261266.26.573C3E1 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf10.hostedemail.com (Postfix) with ESMTP id 0E1AFC0995 for ; Tue, 1 Aug 2023 17:30:17 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=3YMahqT1; spf=pass (imf10.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690911018; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QWyPkbdgs7yx55emQXRqaXaZzIyQirdAz8BzArZ40eU=; b=SyOj3/yOS4shS3Eu2ZivM+mZs1M9wqvUoDXN2ABzEUz5aa5yPWxGAUd/T55bdwDx1ZRwdI /l8SWw/cJaWx2+nreY9jnbEzPgTKzED90KKfV8FRD3L5PpcG2ziCuh4glkfiLAjNtLqdih KMfyfcbknzgEw4MMFzDdoqWRpDkhpPA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690911018; a=rsa-sha256; cv=none; b=0h59maJcFfjM3n0pl78DgyBwvtJSK+mwjW3A26H2WOGscNaeGkIMXbsJKf4TOjbsF9lwvm d3IcbBmfDO0bRTTl7X+jBjn73wIx8SOgvZLkrSvqpI9CgQqR4oPXOvNAwcF2kMzZKbOCUF SrE9jMUYPdZ7S9aBrBeGq7AynBXEAlM= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=3YMahqT1; spf=pass (imf10.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-99bccc9ec02so914035766b.2 for ; Tue, 01 Aug 2023 10:30:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690911016; x=1691515816; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=QWyPkbdgs7yx55emQXRqaXaZzIyQirdAz8BzArZ40eU=; b=3YMahqT1T68lBAKLGC6klWlHs6/yoQcVKMihs8hlR5o+bqg0M8L54pskZkt06Cdy/6 peUqd8125n6i4m5Ix0JkAVgL2Rl/GZMCGJ92U3RaWPEgl6QroNJ9Zb/UT25pvXvk6yVH EDEKpsRJSX91B50bjz3WQnRgEkZbHP+/LXhrCz90OWxSXXWtIxecGf2Xz+bAqo3EbQGF V/g+rrJv4LGWlojIM4vxR6lEiEupKHUdVqqiawBYFt51n1NovxS0llWPtb+8KKREyNCk wummIomryE/q6enBx1sCWQ6NZhLB3umXzBvp3no92lVinNnDKuii9Z9EcHMa+vIptRrb XU2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690911016; x=1691515816; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QWyPkbdgs7yx55emQXRqaXaZzIyQirdAz8BzArZ40eU=; b=geN/2YfuG5Yrhs4cfcdSnesNVVdUUXs3Ri0rIGmYr23w/hqDMgx9k4fosk6sbqtBGp mREgv8S66mLY5BF9v9zwc9UDGyhnySQXkpSrIe/4MNHDZB8KWp0CH9EqFMFE9h7puM16 lhMPYaU2NT7wuiis6PO5idtlRTm2NXozXFO2Vp/RhJ22OXfYVO6E0GAaTPmm7QjPCyGQ N8LAsWcEpJ69KG0yaz0iR4jaVhMIrT7+L7h6g/TB+w36g3kvzLd2Kt0P36+7plaDtn+c 4R3zlj78tNqUlz7h8fbsHBzGnYbAego6LURW2dS5ep3FHvGAQqyo7Dk0urJ+GOmuYPKm 30NQ== X-Gm-Message-State: ABy/qLZm4iILVwBWRos6vBnktxBeK5oWGuZNJ1X082TZy5YIyyoj/hEF XD8iIMqJBzDXiN97BwbtfX901tXbbKs+Ia1ZMoLsdw== X-Google-Smtp-Source: APBJJlGJPVqDqQPC7nXfVN/Cb0vCjHGDfw0oAOT5O0W+9Lp/LUzmV3bVYAJWlWHC7S8wuT2dHXGRWIqsfQc4Kfhtbhg= X-Received: by 2002:a17:907:7751:b0:99c:281:9987 with SMTP id kx17-20020a170907775100b0099c02819987mr3341587ejc.36.1690911015665; Tue, 01 Aug 2023 10:30:15 -0700 (PDT) MIME-Version: 1.0 References: <20230726153223.821757-1-yosryahmed@google.com> <20230726153223.821757-2-yosryahmed@google.com> In-Reply-To: From: Yosry Ahmed Date: Tue, 1 Aug 2023 10:29:39 -0700 Message-ID: Subject: Re: [PATCH v3] mm: memcg: use rstat for non-hierarchical stats To: Michal Hocko Cc: Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 0E1AFC0995 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: npd4e79nqujfgiitpo59fnm95augw5sb X-HE-Tag: 1690911017-605033 X-HE-Meta: U2FsdGVkX18wfXZmmsDtY87RJhzDY6TFUwJNlbUs9rsD6GsEqlZA852SiNZvNSBQKcH62F076jwrC/8lP7RHQrbRWKFIv7TE1fM7kCmqCpbwr/7RbALcU9zLDlgK91r68EfdAfmPIAIkUW+DxLTp8wPyci7Rlt3X++rWXWnDptAqCgzH1AW9nNZQBgiY9eRMBChfX1WStvQTT+AnlMHWDOveG28z5Pvb24EpWo5CUMRglY7buS/H1dsqROxo2JDIPR45VB+Ve7N1shR6NzLi6dziwWRfyKqkcKo5C6U8Uho8dPTDfnhAr/Dq7mKo1wLbib6wO+uG2vwZDiY7RzA3zTVIICLdwtcx5Bw0vm7mz4tciXkpKNgDwtwTY7SepoY5BeflJl3xVccatyeJfC+EQxOwbNpGHcYw6I30GqCDHdxl8E126hA1J2FTmxc5Hbl1bhXkXUpobWZkYWMCrsTy64hfat6LKUm9U1uJUQA8K0sAaBef9H1QmdLoORn8TBuwrs4vTc+owzomXFWVvwr4aEJ5SVzAK8Z+gk+uDnFwFveuVmDQxqFk2yEH6fAz+iCJjZSF0x/97lcZRTsmhgDfd22TV4RYGe2Uf5KhQmO0OOrJq+LgKJQ2HX6nAIH8TDkEeKm+jLyGhlUPSL5jxp2DrQQbk3pIkq1QlI/r+UGmoaIPNFej7KC0LnshuU02as8oWYgQoEuyMBYXUPKjP1NpFfV+uc09QGrm9iAmGDsfD0B5HQRfudCj/0ZEjea9A2kAYzkg1+GjV3kfcN5p4r2Om0vpM4J5Am33SuyXh5tBInQkIpjNj5EuCNDtHVgFBbv1YdnVNdTf6b8cP6uDZBiVh0wP4Rjb7N7/c1U+NFhJk/l8PudjouDaHsvy20P2PgSYEBFWT+CNkm8wq9vND9Ofl1eRJA3wrJfx0rzeqB2S9poMV5RI47/nOU+qEdhfo6jHgTF+Z1wDKHTd3hLvncQ 4c0NaV/N wjcZO4E4ALSma56YUJMo0PW68jj4wn3K+L1oXr30fu6MUoZPZn2/Uay7MPMGAQawUPQhUmtyNot6tLYncockUrAPHip6VmqWxLS9VxL84Asz6O3yWcdO9OXbjy5m7FYaGYp0u4XT4Q/tS/ofYcHrqi+Y8pWp8hQtlUqYQBiIyX1c3B9+8pEBHlkSDZtcvGXKNCn346QtDVMgnwdM1wicCWoz6XUehgxDEjk66+zHePLF5B8MD/Z8sqyTJiWqrJRz8OtxTnbAfQw78wJCHOAcAxzM/jIWAY87RmRtOezCgVUyBsGM8SldxdMCr9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 1, 2023 at 9:39=E2=80=AFAM Yosry Ahmed = wrote: > > On Tue, Aug 1, 2023 at 7:30=E2=80=AFAM Michal Hocko wro= te: > > > > On Wed 26-07-23 15:32:23, Yosry Ahmed wrote: > > > Currently, memcg uses rstat to maintain aggregated hierarchical stats= . > > > Counters are maintained for hierarchical stats at each memcg. Rstat > > > tracks which cgroups have updates on which cpus to keep those counter= s > > > fresh on the read-side. > > > > > > Non-hierarchical stats are currently not covered by rstat. Their > > > per-cpu counters are summed up on every read, which is expensive. > > > The original implementation did the same. At some point before rstat, > > > non-hierarchical aggregated counters were introduced by > > > commit a983b5ebee57 ("mm: memcontrol: fix excessive complexity in > > > memory.stat reporting"). However, those counters were updated on the > > > performance critical write-side, which caused regressions, so they we= re > > > later removed by commit 815744d75152 ("mm: memcontrol: don't batch > > > updates of local VM stats and events"). See [1] for more detailed > > > history. > > > > > > Kernel versions in between a983b5ebee57 & 815744d75152 (a year and a > > > half) enjoyed cheap reads of non-hierarchical stats, specifically on > > > cgroup v1. When moving to more recent kernels, a performance regressi= on > > > for reading non-hierarchical stats is observed. > > > > > > Now that we have rstat, we know exactly which percpu counters have > > > updates for each stat. We can maintain non-hierarchical counters agai= n, > > > making reads much more efficient, without affecting the performance > > > critical write-side. Hence, add non-hierarchical (i.e local) counters > > > for the stats, and extend rstat flushing to keep those up-to-date. > > > > > > A caveat is that we now need a stats flush before reading > > > local/non-hierarchical stats through {memcg/lruvec}_page_state_local(= ) > > > or memcg_events_local(), where we previously only needed a flush to > > > read hierarchical stats. Most contexts reading non-hierarchical stats > > > are already doing a flush, add a flush to the only missing context in > > > count_shadow_nodes(). > > > > > > With this patch, reading memory.stat from 1000 memcgs is 3x faster on= a > > > machine with 256 cpus on cgroup v1: > > > # for i in $(seq 1000); do mkdir /sys/fs/cgroup/memory/cg$i; done > > > # time cat /dev/cgroup/memory/cg*/memory.stat > /dev/null > > > real 0m0.125s > > > user 0m0.005s > > > sys 0m0.120s > > > > > > After: > > > real 0m0.032s > > > user 0m0.005s > > > sys 0m0.027s > > > > Have you measured any potential regression for cgroup v2 which collects > > all this data without ever using it (AFAICS)? > > I did not. I did not expect noticeable regressions given that all the > extra work is done during flushing, which should mostly be done by the > asynchronous worker, but can also happen in the stats reading context. > Let me run the same script on cgroup v2 just in case and report back. A few runs on mm-unstable with this patch: # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.020s user 0m0.005s sys 0m0.015s # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.017s user 0m0.005s sys 0m0.012s # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.016s user 0m0.004s sys 0m0.012s A few runs on mm-unstable with the patch reverted: # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.020s user 0m0.005s sys 0m0.015s # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.016s user 0m0.004s sys 0m0.012s # time cat /sys/fs/cgroup/cg*/memory.stat > /dev/null real 0m0.017s user 0m0.005s sys 0m0.012s It looks like there are no regressions on cgroup v2 when reading the stats. Please let me know if you want me to send a new version with the cgroup v2 results as well in the commit log -- or I can just send a new commit log. Whatever is easier for Andrew. > > > -- > > Michal Hocko > > SUSE Labs