From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD768C433EF for ; Fri, 24 Jun 2022 14:44:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3EAC78E022A; Fri, 24 Jun 2022 10:44:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 399048E020E; Fri, 24 Jun 2022 10:44:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 262668E022A; Fri, 24 Jun 2022 10:44:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 164B38E020E for ; Fri, 24 Jun 2022 10:44:05 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DF87B35444 for ; Fri, 24 Jun 2022 14:44:04 +0000 (UTC) X-FDA: 79613399208.05.FD31B8D Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf01.hostedemail.com (Postfix) with ESMTP id 3008C40030 for ; Fri, 24 Jun 2022 14:44:02 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-3178d560c16so23393397b3.8 for ; Fri, 24 Jun 2022 07:44:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4626xia5d5LkF7af7H2/cvkuUYPcK6Kx8h2FacVJFEk=; b=th1qe6nCen6zmaCB190GBZfnRnMr8qckPf7xXdSq0EVD8gYjLt6WzhfTitRavGwMgd vXilcNgCDBJRRUwP86/NE2cFk2yt4LhrpvKbk+OSVaOhZ9QX7RoXNrnjse0RW8wrov0A e/AjVrKc+b2APqiS3MmTV8TXpGYJnfibLdZbRt8tPJAQ4VejcBVaHjm3xiHfRthqt3hO 14SO9lJx7tqftr6orGS/SfyVF/xr5P71GTXTgmaE3fA6eyIJ4dLNQMv3/dfvge1guWtE 73rzZBtWVKfBk7sNXfxQEVTU5QpWxtJpBQ75xhUnQaYU9jnnZSOcZn/PcgtTEVHgJAUL xvTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4626xia5d5LkF7af7H2/cvkuUYPcK6Kx8h2FacVJFEk=; b=kKKCK+fP+oILgwHuqln26GBNiirnt0mJTU2YZlIX/1l6WCjzf3cZLJzEbER1Z/iaUB GwWWUaTU6zTH9DoTVPgXXyZrRrZ4/YsCkjYO+NXtKe9ZAfBnnoD7j7rEoHiMakC+sxSi zrODxb6S0fPAAcY75o5YG68EpNoeGECFBXpNZTKAT9otYWXQS+V5K/gUSpFSPI6YjIMB lI/xTtFE8WJMOVyodrB1rBvggy21xmkajBmIADifWzpssBdqEdxetbAO/2sSwXde1ikN 007xsriSZGyrykpjGBgPhYp/aM+8REfiu/I5MEOPWN1UkwHCei08WsxLcYAya5AvXdzY TzVA== X-Gm-Message-State: AJIora/EugBSmGywipoOqGE2MmUB/BlW4Ho90fDiqBA1fYBwHWriJj8e i9NIJgTOZrVp6Rrr2FFCS2PgK/VeZtr3jA== X-Google-Smtp-Source: AGRyM1u8PEp1pM9/KxJIfa3nFsPEcGkFKuipmMhOWQIlB56Ud7icllIM9M0fY7r3dfrgjjZsEQIIA/WAW9Z2GQ== X-Received: from shakeelb.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:28b]) (user=shakeelb job=sendgmr) by 2002:a25:5bc3:0:b0:669:b722:beb8 with SMTP id p186-20020a255bc3000000b00669b722beb8mr5658217ybb.447.1656081841097; Fri, 24 Jun 2022 07:44:01 -0700 (PDT) Date: Fri, 24 Jun 2022 14:43:58 +0000 In-Reply-To: <20220624070656.GE79500@shbuild999.sh.intel.com> Message-Id: <20220624144358.lqt2ffjdry6p5u4d@google.com> Mime-Version: 1.0 References: <20220619150456.GB34471@xsang-OptiPlex-9020> <20220622172857.37db0d29@kernel.org> <20220623185730.25b88096@kernel.org> <20220624070656.GE79500@shbuild999.sh.intel.com> Subject: Re: [net] 4890b686f4: netperf.Throughput_Mbps -69.4% regression From: Shakeel Butt To: Feng Tang Cc: Eric Dumazet , Linux MM , Andrew Morton , Roman Gushchin , Michal Hocko , Johannes Weiner , Muchun Song , Jakub Kicinski , Xin Long , Marcelo Ricardo Leitner , kernel test robot , Soheil Hassas Yeganeh , LKML , network dev , linux-s390@vger.kernel.org, MPTCP Upstream , "linux-sctp @ vger . kernel . org" , lkp@lists.01.org, kbuild test robot , Huang Ying , Xing Zhengjun , Yin Fengwei , Ying Xu Content-Type: text/plain; charset="us-ascii" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656081844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4626xia5d5LkF7af7H2/cvkuUYPcK6Kx8h2FacVJFEk=; b=UQt/EJJrkm9pjBEGoCdJ8AusNlZ0N0Oq3Ena8ZLDG1KrELBSzi5U2rV2rcxiGZ7rorRhDV DywINPLih9D4oQVnXb/lOE/ddNHBkVrp42wdx5uXXkXQXIYJECZPMn3dGps3eeV+Pc8Mr3 92ymoqDK1XBTYSbyOdOPv/lCGTtEO00= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=th1qe6nC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3sc21YggKCEAujcmggndiqqing.eqonkpwz-oomxcem.qti@flex--shakeelb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3sc21YggKCEAujcmggndiqqing.eqonkpwz-oomxcem.qti@flex--shakeelb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656081844; a=rsa-sha256; cv=none; b=68GTZ69k3sO1axPD9gM97o2qazcioNv88btdO6HhgZBUWIf4p7F1jsD34NqzvLYurZgue7 DVHI+c/1nSAlkLESBBx6w/KgyAQqM2tmAm8yZ15hESFtpN5xv7UW0DQyRfYqQd0ztZD/l1 /7SE/7nXz+RcLP8D5y+/hZRCw6mZTuA= X-Rspamd-Queue-Id: 3008C40030 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=th1qe6nC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3sc21YggKCEAujcmggndiqqing.eqonkpwz-oomxcem.qti@flex--shakeelb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3sc21YggKCEAujcmggndiqqing.eqonkpwz-oomxcem.qti@flex--shakeelb.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: h9ecycskxqejsyhfcgj4hfzdsf6g3nk4 X-HE-Tag: 1656081842-381622 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 24, 2022 at 03:06:56PM +0800, Feng Tang wrote: > On Thu, Jun 23, 2022 at 11:34:15PM -0700, Shakeel Butt wrote: [...] > > > > Feng, can you please explain the memcg setup on these test machines > > and if the tests are run in root or non-root memcg? > > I don't know the exact setup, Philip/Oliver from 0Day can correct me. > > I logged into a test box which runs netperf test, and it seems to be > cgoup v1 and non-root memcg. The netperf tasks all sit in dir: > '/sys/fs/cgroup/memory/system.slice/lkp-bootstrap.service' > Thanks Feng. Can you check the value of memory.kmem.tcp.max_usage_in_bytes in /sys/fs/cgroup/memory/system.slice/lkp-bootstrap.service after making sure that the netperf test has already run? If this is non-zero then network memory accounting is enabled and the slowdown is expected. > And the rootfs is a debian based rootfs > > Thanks, > Feng > > > > thanks, > > Shakeel