From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A88AEC4345F for ; Thu, 18 Apr 2024 21:23:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C93376B009D; Thu, 18 Apr 2024 17:23:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C43BC6B009F; Thu, 18 Apr 2024 17:23:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0AF46B00A0; Thu, 18 Apr 2024 17:23:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 92B806B009D for ; Thu, 18 Apr 2024 17:23:41 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EEB2C414AC for ; Thu, 18 Apr 2024 21:23:40 +0000 (UTC) X-FDA: 82023929400.30.E60C434 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by imf12.hostedemail.com (Postfix) with ESMTP id 225D940018 for ; Thu, 18 Apr 2024 21:23:38 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=k6Z2VKGU; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713475419; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wYalYvd4bYEay973j3KTZyV29Ho5YwrE1KyGZ1KnlgE=; b=loa6A/GpjeoYB2F7DvWQB2gIJ/oJKahgmCqVibAB+RtFAbzzoJEHlYFq7g14bUosb41kvr vJE1kzV5l+oc74Gj2uabrdKUyOfv2l5BuXlKazj+5hNN18zXCotogmlvkFkLn7Ims8U8E1 JoT0B/nBgHS4ZtDwXTewkSzXwDYHkB0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713475419; a=rsa-sha256; cv=none; b=nacV1o5i+C7jIhJkvSxu/bPIL1TZfp9Q22Hp2a+8RBWk8YgfQaBrnq8vUMBnRt3wz2/vFY ffJC7wZ/79dtG23aownban2AD6F8h/T6jfQmAWEz7mGbjY28yhjZfWRdwR8ec0wDroawUb 0yRylyxI5WDuEeIKIqF1GZuZzb5/2rE= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=k6Z2VKGU; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a5269c3f9c7so145076266b.2 for ; Thu, 18 Apr 2024 14:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713475417; x=1714080217; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wYalYvd4bYEay973j3KTZyV29Ho5YwrE1KyGZ1KnlgE=; b=k6Z2VKGUobSFQXWVHSIcfa/1o1CvetF5qRxo2DY3DiRtv5ZxVowQ5m/MJp2W7YKvdE tlexQfrK0KuRKwPoE9JLRM9P0io/DHX+/cw6T443pQLOPcurAXLkuFKud0kvTB9rpvvj BasU8LGgleT85NJz2szt9dyqF+kVKO+b32WQBUMPFFUPvtaU30urxIhFyGCk9Dn5Fmx4 5zB2Rj7VtD7ZqElbkpsV7gfn/qDIf/5T4Q/654Cxki8z80YPisUcU6Vxs4/bK/KmqsYZ ZAOeagSlbN9Dz/+3hrRd610vueK0CDCYUc1JFSEYu0ohEcmta54u4ZQOrSu8O/JdWgla lnnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713475417; x=1714080217; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wYalYvd4bYEay973j3KTZyV29Ho5YwrE1KyGZ1KnlgE=; b=uzcj4biaOLlQaCeXnV2aQMOxSZfqg+jbbfADoR5/vqqL0MFl4ZzFYWHvG8qzEsWxs8 E2RRKlw7PYGcfaIvUBQvNbNozRgq7xnIWMvvv2tjEkyY2EyqyYPX2cjMKQ0wOdu85UPN GTeCAiMLfAJnLx8hukWeO12pQrA+mGXwiPfKU31QkJ9rIrKfmM1rZdLopBSL1UBf6Vt3 059BstOP941RAWmL45RXeOiZtTpofORdpSrJx1fS/g1t3va+UQVI7+/mPqr/RgLHJZeS ktAql/foB8K7UVPsMPOJhUEgB8bSpaq7sbHEez5N6zmkl6NfsO8HBfNe5s50XzcF9bQR X7TQ== X-Forwarded-Encrypted: i=1; AJvYcCXpFO6MB9L9ApuQip9h5f3B+IpalroFzmaqDYcqwFvlun3/MsL2gOy51+hVdrYuHR6bU/r2YwMRlarng69IqZdbM/Q= X-Gm-Message-State: AOJu0YwAFE4HX61oEvh+BNibLC4Mn2CGdsMVZsrQvjSs5oDesd24wAtV oJGUricVc0evYrlbtSQxftMdD+Qw5l+5eRESgAtWuDejPY6etw0vYZVc0hM4AV1ssoUZt3XBTGW KRkbhxBahipV8jqqExayZkcEtHjCbaivmOgfU X-Google-Smtp-Source: AGHT+IE+jIlHU4WcaqMzFdFWnRNrWh1I/I1LmnfLavnHBCPlaJu7GYVzwrGYnuyYgovrzEFHVjqFlQyyrOWhsCBSVS4= X-Received: by 2002:a17:906:f1d6:b0:a55:5c04:89a4 with SMTP id gx22-20020a170906f1d600b00a555c0489a4mr182160ejb.21.1713475417327; Thu, 18 Apr 2024 14:23:37 -0700 (PDT) MIME-Version: 1.0 References: <171328983017.3930751.9484082608778623495.stgit@firesoul> <171328990014.3930751.10674097155895405137.stgit@firesoul> <72e4a55e-a246-4e28-9d2e-d4f1ef5637c2@kernel.org> In-Reply-To: From: Yosry Ahmed Date: Thu, 18 Apr 2024 14:22:58 -0700 Message-ID: Subject: Re: [PATCH v1 3/3] cgroup/rstat: introduce ratelimited rstat flushing To: Tejun Heo Cc: Jesper Dangaard Brouer , hannes@cmpxchg.org, lizefan.x@bytedance.com, cgroups@vger.kernel.org, longman@redhat.com, netdev@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, shakeel.butt@linux.dev, kernel-team@cloudflare.com, Arnaldo Carvalho de Melo , Sebastian Andrzej Siewior , mhocko@kernel.org, Wei Xu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 225D940018 X-Stat-Signature: dymdztboo8bb9ihu4og9jbbf8cux5mk5 X-HE-Tag: 1713475418-202188 X-HE-Meta: U2FsdGVkX19xSH1xQN92Svd5tsDmEXOQtxjr/xHcUlLB8Mw98SFKfwOiQ0VxsMek+Ksn74g4v1nTs5WneWCb/XLVMR8kIRAVEBs9Ij03aj9CzUXpTgLm0+aWVrwddshiDSKt7Ocg33sH0QdNwZuANu1bCdBolhJnDBONEwOA5YqHoNgTAXx1+5WJNad/d0HNrQyofhTiBRVGEbTbm6Bz6zfmxDY7BX/38+GkP5B4j5r7zaeS6ql5EbT+fCZaguLBIzI2Aq5gl0zoa8Gyp4Y7q6XsqwrNnuIQiKoiI+RQP49bzn9pIbkFSv0Ix9qfljvFX+zmIeSyLdDaVSJNkkfmJCjgktbLxBXmsArzYeMG7ovksfuMBw7XFc+c4o7nkzSBzYpCt8ZGL0Hjs9zVzSoVeR9498E4/mLaoxr50lza3B2QoftBh9CUZy3Qd8rcROU36tqFlewOja6LaBArakWU4GzcjPIx3hzlN8FY94vEPwxx1EEeypsFRvVd1r6+XP/QwsjkpVJwyg1qZFgbWn3NzRr8WcVq8n7/BaVkRCEETuHb+7XrB07zlv11oUJ34R1odK2C7foJ90FpWHvxruKnJji+aTYzLzipWc1et6GWA3DyGP7ZGu3iTzyopKEvCFTIvHr4gK6rAL0ckf68xc010XLs0UBnNm7wxlPohXdN97sTwsuYicq+mTWA3cN4YMFZ2VCqlluYruFbVerdq4YpypeQypZq4An6TQwBzEO2y0dmwxPmVKITuQ6IipXBBo20BZn8jHvarCK6pyTv9xckp/rUcaRpUAZg0Nr2n0nSTbOBddwxLeqbEKHURgi3LXdUZGkncf52rSvsIaW862qZdeb0mNsx2uG/usqMn5RtZOcoM+EdbWt0ix77tTmhS0M20z2EGM5EzBRYAQn3yBSLUNIMn3itlN2ddLmLReyH0uB/y25AHqdYVosc9Yxx+tsqzXd5RO//VHNQcjA5fQB 8OcXCHeV zx6hX1z0KoXr/z2GiWjg7DKiD70C9OC775JhtLAt31gSllo1D2VT/Uhu+Q5GMaHAbK+NeRgKxmaKF3WBjY0BYQFYNUEvlYCH3GT7gQaFWbqQ9r7rPxnFzSffZOEyJ6UZHPn8g3Zv50P2bCylamO9lP4YhMM9HtIjPvYh3w/t9GtpJspzBMEz/v/LvJKFxGKPEEIcLkjAHqsLhD6u7XK/bpRvCDnRiboBk4IMf9A4iLJf2RtwG17UPrfc1Z1zB/3MKcfeOzLyGfJInjqZX/09/rk8cqUUkrW+VGkKL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000273, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Apr 18, 2024 at 2:15=E2=80=AFPM Tejun Heo wrote: > > Hello, Yosry. > > On Thu, Apr 18, 2024 at 02:00:28PM -0700, Yosry Ahmed wrote: > ... > > I think this is an artifact of different subsystems sharing the same > > rstat tree for no specific reason. I think almost all flushing calls > > really need the stats from one subsystem after all. > > > > If we have separate trees, lock contention gets slightly better as > > different subsystems do not compete. We can also have different > > subsystems "customize" their trees, for e.g. by setting different > > time-based or magnitude-based rate-limiting thresholds. > > > > I know this is a bigger lift, just thinking out loud :) > > I have no objection to separating out rstat trees so that it has > per-controller tracking. However, the high frequency source of updates ar= e > cpu and memory, which tend to fire together, and the only really high > frequency consumer seems to be memory, so I'm not too sure how much benef= it > separating the trees out would bring. Well, we could split the global lock into multiple ones, which isn't really a solution, but it would help other controllers not to be affected by the high frequency of flushing from the memory controller (which has its own thresholding). For updates, cpu and memory would use separate percpu locks as well, which may help slightly. Outside of this, I think it helps us add controller-specific optimizations. For example, I tried to generalize the thresholding code in the memory controller and put it in the rstat code, but I couldn't really have a single value representing the "pending stats" from all controllers. It's impossible to compare memory stats (mostly in pages or bytes) to cpu time stats for instance. Similarly, with this proposal from Jesper (which I am not saying I am agreeing with :P), instead of having time-based ratelimiting in both the rstat code and the memcg code to support different thresholds, we could have the memory controller set a different threshold for itself. So perhaps the lock breakdowns are not enough motivation, but if we start generalizing optimizations in some controllers, we may want to split the tree for flexibility.