From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7F2BEB64D9 for ; Mon, 10 Jul 2023 23:24:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 45A9D8E0005; Mon, 10 Jul 2023 19:24:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40AF38D0001; Mon, 10 Jul 2023 19:24:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D3008E0005; Mon, 10 Jul 2023 19:24:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1AD858D0001 for ; Mon, 10 Jul 2023 19:24:00 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D13AB16018B for ; Mon, 10 Jul 2023 23:23:59 +0000 (UTC) X-FDA: 80997282198.10.B450EC8 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by imf13.hostedemail.com (Postfix) with ESMTP id AD49620005 for ; Mon, 10 Jul 2023 23:23:57 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=cloudflare.com header.s=google header.b=BLXTGXAR; dmarc=pass (policy=reject) header.from=cloudflare.com; spf=pass (imf13.hostedemail.com: domain of ivan@cloudflare.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=ivan@cloudflare.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689031437; a=rsa-sha256; cv=none; b=fk0wccM9EJIxlQw/wdVZ/8tsbPvBAPhea0NfFUrHakZgidU42quqRZUHKFdfkujauESs4m MT9iC1mfnyhkOhrXnuEN49NDlthLskLzBRv08MfIeYSJxC/PCJ59maYke1m9RWzKh53d2r kiyq38/HBgLnreDSns0Xyxmip5abtBw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=cloudflare.com header.s=google header.b=BLXTGXAR; dmarc=pass (policy=reject) header.from=cloudflare.com; spf=pass (imf13.hostedemail.com: domain of ivan@cloudflare.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=ivan@cloudflare.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689031437; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ax+jVrYfNmxpClqi51wVQRM9DJaaLI9Ll9U+M+qNnIE=; b=BTNl48482ftmEbfxFIQVxZQSTqOMAEimSQO9ctsVYvU+dwkc12k9SUquxQPjtsLti36JMe OM9s6wRkl6AC9Oj8YNUd6c1pX8rl+2fXusC2buKhFD0wJG2upNJCA4cKveWHoDamwdv/CI nge43f5hm3e5wrObpK3lfQBGIKe3k+E= Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-3144bf65ce9so5230399f8f.3 for ; Mon, 10 Jul 2023 16:23:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; t=1689031436; x=1691623436; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Ax+jVrYfNmxpClqi51wVQRM9DJaaLI9Ll9U+M+qNnIE=; b=BLXTGXARwPCt0Z3QY6Hkyqe8P+UvKJuwOcDpcUhFrh7oauHoH7ek9S8FnzkxNKruxX h3Fkat+06xrYJVtdH+Ksjd099ymifYhPIVXtCSUuBFupjA4JzaCHCo03hWSvfs1PRNB+ k5eZnIfD3YabAza1cBlawD8pI6YWJibUkXyDA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689031436; x=1691623436; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ax+jVrYfNmxpClqi51wVQRM9DJaaLI9Ll9U+M+qNnIE=; b=VQiqPvC4KDZcmgtJ8PHYZX3egDEjxKd+dHl/mY0XxsNAOYb+tXJB7AHM0nTUBrPnHb QuzKsvmBtlzv6W1M3KhFlUkrpy0odL9S2fUKjyIcTZH3j0VRPqpJlDXq4zNCj+oMMALn TRS4KSQBDl1mnEWAHZTq8P2c1aWtEfpS6cqe9XKmhoyd31w7FFl7oyGpXKP5DkUK+q36 M28D/jHSarIK0InlL5zzvlPVUEBM5y7YTcCW2DrWROElSb6GQf65Sdo8QPj/6Czty5eo U4x+wmg51/VxbU4QZPp9rXKuzHWt3rlNeXYVDbO/bbI4Giq1eUA79EKkvNOfysy9A4ZT TWtA== X-Gm-Message-State: ABy/qLZ+GTtuIMzhuNx0rqsyOmcy70oV/lS/uGxhNr2UaEQFvmE4z+b7 LItO78oeudMGCrPazDCLWHVsz47D4UkVx3O9joUzVQ== X-Google-Smtp-Source: APBJJlGFm5WtVXhSQ1rr5NrU0fVlV2Q2DdNuwoBjwqZ+pOWiiG+QXxeIsX5OhCIZtxaZ8KfCz9A622VrkOCXlZb7k1M= X-Received: by 2002:a5d:6546:0:b0:314:420c:5ef7 with SMTP id z6-20020a5d6546000000b00314420c5ef7mr11980548wrv.11.1689031436427; Mon, 10 Jul 2023 16:23:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ivan Babrou Date: Mon, 10 Jul 2023 16:23:45 -0700 Message-ID: Subject: Re: Expensive memory.stat + cpu.stat reads To: =?UTF-8?Q?Michal_Koutn=C3=BD?= Cc: cgroups@vger.kernel.org, Linux MM , kernel-team , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , linux-kernel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AD49620005 X-Stat-Signature: 3qroiw4powb34zgjgtkhq5equons3hr9 X-HE-Tag: 1689031437-867658 X-HE-Meta: U2FsdGVkX196J8abr+91nDB2jk6hFO/mS6TmvBJK+Fik0Mm/WoOA+YE0UAdet+O4LEWiy5ZWFgnmFefsMdIzVfqbLnUAB2hSc/y8Juzm2TKeGPiVLb1gegoDGdCWgPNnyzLuvi63x+sBjoIjdViUpRjoRx0i448IGKMdtc+EpS4yl2jO/msiiqlpJsiV51d4whXJGOU693uzA2TWVFXs2XhEfiKbLuLrbTWH3nJUH8wzKjAJDhVA/SZXLqAdlKVjNMIkOpGZMgrvsYasCes/JdaKk4jT5dDAXrQIiQ9kc4AUHv+9LSFvPBgHwkldsIFcvyedU6AAAe6Jqm/y4UMGH1xT5v04h0pWMXvyyHodk7BFk2WpM7irHEKa1ONLehhL7LHx8DuBx2nyFHLOA/oKljWIHbT4V4aUKhidlGXoxg+muTgkujxKyptek3c2v7hnQHYvktSLFWMhKWApQAzZIBMHNvx7CyCECoGSQlHtBmSem11rayHfRVRquyFG6MEsSO1Bx0X3XEazErNUziHuVb2LE7Pop1F9hwDNauaqTQXFf8PXqSKJFkrEktomwG0agJqKAi6KbUjnnIdOgsZLiaWMiTtL1XmHkX6YM96Z5aaq90fIIHMPj8NlTRMEjbrgcEkh9qikF53RPBmnwValy4ZBtcdXf1TtK8mVAi0cS/ixNZjd7MlvamvM70r2YRSdol/Woy2BDaGdhceLlGDXQqclqIfhaBMB8F8X/BzbEP2gUKL9qUAbgPGw8PbbHabwYTQ6ny9jarhoURUMKIk5jjn2YpQbnNRgn6PzoeuBPgdmuK/n6bObT1k+80XWxn2IcsLlSSiO1/Ia43bDC5xx/hpUYBR8NoOzoTOO/JS225xbzKGIR7vfV0tY2sReskSsEhw5AmVaefSgpfA0xexwqBaVMOyZ83J/Z5vAejkOqz7DQG8HBO7EN2LFn4z2RXGH7yaGxJlCOv6WGrRn/6i PCfdW0w9 4cFMtTAhiaqI+QEq5ZnuLhYmYdVsfg29RePvhKKJtnFizQURCjBdyHmmOVvnq8TMPtrOCHDWmwxTJOpfMRAsW3lrNQ7vKko8xwZTEsm/xmdqHqaT8a4KxGyC2TFydwnzVcLFUO2dGo+1eDgI7V3/Ry6sqIncOEfNK1H8c87AeMuWabOZgEMgNTo6/ZSCkA3OCDdk8vuSVWvv7foR+5xxecD9l/viwph1ODexEechGrNq7iSvJHignH8suD9rnHFyKmFyL7L1SszDXlm0zbCV3pSHXH1Dcy/JcpivUbsdO4xshYaqUxWs4L9qGmuc3/U7kpr++ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 10, 2023 at 7:44=E2=80=AFAM Michal Koutn=C3=BD wrote: > > Hello. > > On Fri, Jun 30, 2023 at 04:22:28PM -0700, Ivan Babrou wrote: > > As you might've noticed from the output, splitting the loop into two > > makes the code run 10x faster. > > That is curious. > > > We're running Linux v6.1 (the output is from v6.1.25) with no patches > > that touch the cgroup or mm subsystems, so you can assume vanilla > > kernel. > > Have you watched for this on older kernels too? We've been on v6.1 for quite a while now, but it's possible that we weren't paying enough attention before to notice. > > I am happy to try out patches or to do some tracing to help understand > > this better. > > I see in your reproducer you tried swapping order of controllers > flushed. > Have you also tried flushing same controller twice (in the inner loop)? > (Despite the expectation is that it shouldn't be different from half the > scenario where ran two loops.) Same controller twice is fast (whether it's mem + mem or cpu + cpu): warm-up completed: 17.24s [manual / cpu-stat + mem-stat] completed: 1.02s [manual / mem-stat+mem-stat] completed: 0.59s [manual / cpu-stat+cpu-stat] completed: 0.44s [manual / mem-stat] completed: 0.16s [manual / cpu-stat] running completed: 14.32s [manual / cpu-stat + mem-stat] completed: 1.25s [manual / mem-stat+mem-stat] completed: 0.42s [manual / cpu-stat+cpu-stat] completed: 0.12s [manual / mem-stat] completed: 0.50s [manual / cpu-stat]