From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D2DCC41513 for ; Thu, 12 Oct 2023 03:14:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E56E28D0104; Wed, 11 Oct 2023 23:14:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE02B8D0002; Wed, 11 Oct 2023 23:14:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C81038D0104; Wed, 11 Oct 2023 23:14:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B63BD8D0002 for ; Wed, 11 Oct 2023 23:14:37 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 85AB7804D0 for ; Thu, 12 Oct 2023 03:14:37 +0000 (UTC) X-FDA: 81335341794.25.507EFDF Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by imf20.hostedemail.com (Postfix) with ESMTP id AEA071C0005 for ; Thu, 12 Oct 2023 03:14:34 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LJkskgVD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697080474; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z4TVrvq8uoH1eycNvhhxc6PoR8dTnuXdQn6h8TXV/H0=; b=dGdmGLc9E5SfwIr+J3s0tHrLtC9xeraDG5GGTat+W2ycqGJxmSfQd74HU1GmnBJ9Z0WtsP 50WNDkGpDv405ZaaEJuFmdphleGg2M4tviFFCopLlAEJz2TmqqENumaDjOBM/ZiimQPvpS Qw39WKDhNHrfX3oxKB4qNFjW/So5tk8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LJkskgVD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697080474; a=rsa-sha256; cv=none; b=AXusEYg7KnHePO7y1ExvFPWfUs0sgJ8nEN/DfHnZECROz/2miW1vOTOB/3bJRhYy6eXGU5 g9ZfYAIOImj4V19I7KJHzvsokS3RegHi2ROjSpmEfO9HjOJDP6tqpfxj89z8FPU5kxpYH+ dDtIijnakgu8C29OJadjPr+mkbF+4rc= Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-9936b3d0286so78658066b.0 for ; Wed, 11 Oct 2023 20:14:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1697080473; x=1697685273; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Z4TVrvq8uoH1eycNvhhxc6PoR8dTnuXdQn6h8TXV/H0=; b=LJkskgVDjDRlcf+UXdhOwdFSdBgf/C0lQnIvN6v0nze52RA52dMEYjdT0nj+x/aawG sjBgHWRcO7wGdxdfjubiFs8TS4C9gaBPS9smULO7DZ+HXXDmEPF6kxG0R84sJZFJDOVL O/FFUiA5xKtjJMJeSdU5fimvwDp8+Huex/wSaBw+hq/bPUymN/u13rXNzIyHJ3aImgJX obSJmN75JovBaCPVIyEDy8+swooNig6G8RWi7Zx0qznT9oSYaJBxgLIyphRKt9Axw/Tl s+2d8NNT6JUM+wa82kAGnC+zBg7q+qt6o5yp1uhC7xzAxXQbrwbXDrx4rbgKYVKJgVbf NcFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697080473; x=1697685273; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z4TVrvq8uoH1eycNvhhxc6PoR8dTnuXdQn6h8TXV/H0=; b=A3SwKZabQov02EyFXBcWMhGBu6EpQJtTyOlX4HQe1QPi98MBgvkj228Fu8ha5LoXKj aVpE+P/VmzXZ27KFZM9niyV4cEGujnsZ/o7/ZZBQSCKXXdaQmL6FUQhlDUa458hTArnN dNcaCF+HhppYx3XAuJm7xnjK1QQEpym5U8WDupMKOXs/w0uL7mWYw2f79hrIY40FIYMp Oz0jS/H8FXllFZlzQh7HOT18ABkRV7MsyGVkdiza7re08t1Tf8O7iTnIpNAEepKtt6jy Zxd8RR9A/LHzlL+tlTvIg0C/w0wlRpVgnpcxJEMO6fPVdPICFY4ZqtwfH3aU8FUJJ5ZW buXw== X-Gm-Message-State: AOJu0Yx1QwoC9+yqsSqWa+3xykeM0MEsAw9usmJIrI9oGPmN8KxKHvoJ PEXLdq8bBOPRkOEyxBD0zuzGjkKmXeIw/a421FcTZw== X-Google-Smtp-Source: AGHT+IErCQ3tOY754uhLEcdp3aTeunkuxdqdmWI575ba8qvAsvXa0b1TTgbRuhl+wIWMyNjlZ+f7ry2dzvLEnYLpplc= X-Received: by 2002:a17:906:104e:b0:9b2:ba65:db21 with SMTP id j14-20020a170906104e00b009b2ba65db21mr21469636ejj.45.1697080472968; Wed, 11 Oct 2023 20:14:32 -0700 (PDT) MIME-Version: 1.0 References: <20231010032117.1577496-1-yosryahmed@google.com> <20231010032117.1577496-4-yosryahmed@google.com> <20231011003646.dt5rlqmnq6ybrlnd@google.com> In-Reply-To: From: Yosry Ahmed Date: Wed, 11 Oct 2023 20:13:55 -0700 Message-ID: Subject: Re: [PATCH v2 3/5] mm: memcg: make stats flushing threshold per-memcg To: Shakeel Butt Cc: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Ivan Babrou , Tejun Heo , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: AEA071C0005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: cxu6korxw3c8cjxr84fopqemhkc4huyt X-HE-Tag: 1697080474-589687 X-HE-Meta: U2FsdGVkX196LNMwc3SJfNTJd8Ym9XJ7Z7GS/NVUfuGtGO3atJVc4yoFxAkX+zPKFnqUeZ/olKbXqy4+ZNxLt24l1Kx+Uw3zjj+YqpRNNN2nhfwKHnKBluYwtkiZ7lA/MzpBhhlVyOFRqYDGNGMAAKXYzobKqB6K38bCoqAE4PdRrLfRf0AqByHgTUFa5CJAHfUaCH+UYIr9mZUl5heo2dy60nYsEIUUdmSUYhbzOPkIhOJhlLWtBPmkKeklLO4hG8Q0bLva0rNaJPFtNVg16Je4udhRupGcpCX+7WY8mwvJnioQSqngyktyPJyOeH/LlCRcbT8KltRckfZ1+z7s5whQeu2lDL7izK3DjMEGK8SvWKShFl1QfIZyQV6mpRt6jPj7O8stxm9iQa24iEtbqtbcSpOLiE5ptw44fDI9HhQzT0XBJd+A98sxtupEcp2lG9cQB8+E0lmOS97hxt5HBKFdF3zjMk3uXVUO8Ma2rkQERdRSGF36LBZboOjMtY3pI+xx5BddOVFNDiDucC8BNEccHbONSYeKSh3mc7byfT06GbVfej5Qi2jJ2+DYNj2BMZ4EfkM8oDJeXaV1WuHyydENxkNjHCxOf/LECsnBVucBY0ASX9MeP3LDbh9sSV1kjqcFtzRlV6LeQnSQ2Vxrtm2O3HawHFsDd5V/mbLSR38FAHcSbl1k9E2lJt4IUySQgpHLxPsE2QbRNNuIfYss8Ldv89DUZbquYrAbaeH8wsFstNGcU7jxDSB0FoxrBcqGUeVNoGxGXTTn1Sz4uJwaIGK4vLBLO0i6lBu+cU+zTa11jgUCS6UcTXWafDeiHQ0297fohC/eN62G1Ieo7OukHeNECtiQ7Tw/3FiqZIAg1Wy2yFrXvgPbtAa9AcdnYKRkhzoncNssPmkaDW+pTODPHUHvKU8oaCRXTmQ/LGxvziYLUfZy1ylUEgXwZGpMZzqSkSxNheDYNEVZsCj0Qv9 CwadhRXa Gl0e6SITC+9+xRxd3xN7kCcr8Vht1VXMt6P0+ZJ2diJ3XkxvcqSvH5lg16qUzwTIcVlTfT4fr3yFADlfGbic4bDwMw+jv5m7yrEL9EBCHIcOzBkk02XO12KQc9BX3w1d1OxFF+uoXVbcbELT9MGT7Mg8G5oGIdqY2p/CW4thuvtBmLrcC7L0DylIC0/TwbxOGQ1X0NLm9LfXFGzt6CgGut98x56VpobuXENAkogjxGgF/2IXeQxcusR1qHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.034760, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 11, 2023 at 5:46=E2=80=AFAM Shakeel Butt = wrote: > > On Tue, Oct 10, 2023 at 6:48=E2=80=AFPM Yosry Ahmed wrote: > > > > On Tue, Oct 10, 2023 at 5:36=E2=80=AFPM Shakeel Butt wrote: > > > > > > On Tue, Oct 10, 2023 at 03:21:47PM -0700, Yosry Ahmed wrote: > > > [...] > > > > > > > > I tried this on a machine with 72 cpus (also ixion), running both > > > > netserver and netperf in /sys/fs/cgroup/a/b/c/d as follows: > > > > # echo "+memory" > /sys/fs/cgroup/cgroup.subtree_control > > > > # mkdir /sys/fs/cgroup/a > > > > # echo "+memory" > /sys/fs/cgroup/a/cgroup.subtree_control > > > > # mkdir /sys/fs/cgroup/a/b > > > > # echo "+memory" > /sys/fs/cgroup/a/b/cgroup.subtree_control > > > > # mkdir /sys/fs/cgroup/a/b/c > > > > # echo "+memory" > /sys/fs/cgroup/a/b/c/cgroup.subtree_control > > > > # mkdir /sys/fs/cgroup/a/b/c/d > > > > # echo 0 > /sys/fs/cgroup/a/b/c/d/cgroup.procs > > > > # ./netserver -6 > > > > > > > > # echo 0 > /sys/fs/cgroup/a/b/c/d/cgroup.procs > > > > # for i in $(seq 10); do ./netperf -6 -H ::1 -l 60 -t TCP_SENDFILE = -- > > > > -m 10K; done > > > > > > You are missing '&' at the end. Use something like below: > > > > > > #!/bin/bash > > > for i in {1..22} > > > do > > > /data/tmp/netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K & > > > done > > > wait > > > > > > > Oh sorry I missed the fact that you are running instances in parallel, = my bad. > > > > So I ran 36 instances on a machine with 72 cpus. I did this 10 times > > and got an average from all instances for all runs to reduce noise: > > > > #!/bin/bash > > > > ITER=3D10 > > NR_INSTANCES=3D36 > > > > for i in $(seq $ITER); do > > echo "iteration $i" > > for j in $(seq $NR_INSTANCES); do > > echo "iteration $i" >> "out$j" > > ./netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K >> "out$j" & > > done > > wait > > done > > > > cat out* | grep 540000 | awk '{sum +=3D $5} END {print sum/NR}' > > > > Base: 22169 mbps > > Patched: 21331.9 mbps > > > > The difference is ~3.7% in my runs. I am not sure what's different. > > Perhaps it's the number of runs? > > My base kernel is next-20231009 and I am running experiments with > hyperthreading disabled. Using next-20231009 and a similar 44 core machine with hyperthreading disabled, I ran 22 instances of netperf in parallel and got the following numbers from averaging 20 runs: Base: 33076.5 mbps Patched: 31410.1 mbps That's about 5% diff. I guess the number of iterations helps reduce the noise? I am not sure. Please also keep in mind that in this case all netperf instances are in the same cgroup and at a 4-level depth. I imagine in a practical setup processes would be a little more spread out, which means less common ancestors, so less contended atomic operations.