From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 319D1C32774 for ; Thu, 25 Aug 2022 15:25:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C12436B0074; Thu, 25 Aug 2022 11:25:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC183940008; Thu, 25 Aug 2022 11:25:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A89D3940007; Thu, 25 Aug 2022 11:25:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9A3C06B0074 for ; Thu, 25 Aug 2022 11:25:52 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 67A5D1A05D6 for ; Thu, 25 Aug 2022 15:25:52 +0000 (UTC) X-FDA: 79838490144.05.5FCAEFD Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf02.hostedemail.com (Postfix) with ESMTP id 0DCCE80002 for ; Thu, 25 Aug 2022 15:25:51 +0000 (UTC) Received: by mail-pl1-f175.google.com with SMTP id y4so18821057plb.2 for ; Thu, 25 Aug 2022 08:25:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=Zsqv9AykgJ+WV9feY4vZLdyL0HO37C5wC/zUOZBB/Rs=; b=HWnl7UrUYlzNgRl6uEFPplyGVawiIDXFH7m8PBsKuwLAyY4wVOH5/4W+KHE3cm4dmy 4VKpGXbIOjUVTNJ5qTAEfFLY5j3Yf3cnEjwgexlgBZzwA/NZqy+OA837il6nxkXV5MZ8 bWeOe7RYnCeBZeCaEDApmwAvBPMIcLaY9ZVBN4TcWiNWpqvDtnBHm4rNi4l+iIhXSBbt Vb5TpW9i3LVxD0fenWNvYo2kmlhk6MMD/fpNzQjOSsWwzJLxUQdlnEY1aV6QOjHMKsIC o6Aj0pItiOXOcalFXaXlzd4qp/y4sLZ4X8K4ywWFjTAOSTa/Uzz4A9Ct6uhXYJn4n24B gkOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=Zsqv9AykgJ+WV9feY4vZLdyL0HO37C5wC/zUOZBB/Rs=; b=PyhF4X+/wF1BJQUHqLZF7GfmBgzBtVsjZD9QF+5flTzisQPZlD5HeUucUNINj5Ltxy yaX8VZYJ/rC/q637nsbtLOuWG/MzAIpoQhSU8ceXTnj7oQ7dYoP2FeE4B8pvy5MhO+3Y ZAj4oo2wuDAcCut3ARcj9DKFfGLhrh0TdIRluMIO40PQjne5ueX1Wcn2Wp4cBA5p8fXP EI2tqLirqNmnhW1iAu0uD8NYbgzYp3aoVBxjJwWrkUAQj7pIu46evof+ZVUgqpOi5Z6C ox2cFRVkjo8SO0JSIJcygTUaA6rGpEGwooHxYG1Hy67u72szVgRFxNaC+JSlHLZAxHVy 3DTQ== X-Gm-Message-State: ACgBeo1sJDLA7UZ2Rkl2cohK3TdXJSwVAbJAAkuhf4RTnkp/UJ5yMRu0 KpgY8xbazJp1YVvkWbO8NIlWGw862PdkgL5hAhrkfg== X-Google-Smtp-Source: AA6agR7yKFRvtMiJWUoOhnBb6dODuZY59WJG0S4sLUXExTnTCnnQec25vcU6SQQtE/2OZn6xCtg2SViS0OjZjhqHnPQ= X-Received: by 2002:a17:902:d58f:b0:173:75b:6ad with SMTP id k15-20020a170902d58f00b00173075b06admr4194683plh.172.1661441150947; Thu, 25 Aug 2022 08:25:50 -0700 (PDT) MIME-Version: 1.0 References: <20220825000506.239406-1-shakeelb@google.com> <20220825000506.239406-3-shakeelb@google.com> In-Reply-To: From: Shakeel Butt Date: Thu, 25 Aug 2022 08:25:38 -0700 Message-ID: Subject: Re: [PATCH v2 2/3] mm: page_counter: rearrange struct page_counter fields To: Michal Hocko Cc: Johannes Weiner , Roman Gushchin , Muchun Song , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Eric Dumazet , Soheil Hassas Yeganeh , Feng Tang , Oliver Sang , Andrew Morton , lkp@lists.01.org, Cgroups , Linux MM , netdev , LKML Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661441152; a=rsa-sha256; cv=none; b=cuai8eWJJa1mf7vmHQgp/V1Ha6LN2O99hh01Qw4NAHD3yxKpAaqV7wjXC7W8PgUeQ1LVKB q0G56sGV/kmngKWI12BxDcfa0D3EYqr+YCi5iLNJS/lTY+NprNHh6YgpO0xmCCwmp45ptN AIQfe3QpCZYPGRh0K0nPgT1azwFwJ/A= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HWnl7UrU; spf=pass (imf02.hostedemail.com: domain of shakeelb@google.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661441152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zsqv9AykgJ+WV9feY4vZLdyL0HO37C5wC/zUOZBB/Rs=; b=qIoADgzS1UUB7nDUSVZD9oQuR+MuuVAj2HSb2Xg5Wxev5E7fDeITwBON/L/MLN/NnU9ykU 3f4A88jCFja3slbeG7Pdk9M/zLyqQ/h6PVIqk3+NaTeOJ9Cj6tbWLUF0UlTSu3gKq6HS7i PXwXgDH0yCLSYOZSXcShJx9mf/CdE0g= X-Stat-Signature: n7mij3eu5e4sj6eftkmjj5yk8ht3bkok X-Rspamd-Queue-Id: 0DCCE80002 X-Rspam-User: X-Rspamd-Server: rspam06 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HWnl7UrU; spf=pass (imf02.hostedemail.com: domain of shakeelb@google.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1661441151-951316 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 24, 2022 at 11:47 PM Michal Hocko wrote: > > On Thu 25-08-22 00:05:05, Shakeel Butt wrote: > > With memcg v2 enabled, memcg->memory.usage is a very hot member for > > the workloads doing memcg charging on multiple CPUs concurrently. > > Particularly the network intensive workloads. In addition, there is a > > false cache sharing between memory.usage and memory.high on the charge > > path. This patch moves the usage into a separate cacheline and move all > > the read most fields into separate cacheline. > > > > To evaluate the impact of this optimization, on a 72 CPUs machine, we > > ran the following workload in a three level of cgroup hierarchy. > > > > $ netserver -6 > > # 36 instances of netperf with following params > > $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K > > > > Results (average throughput of netperf): > > Without (6.0-rc1) 10482.7 Mbps > > With patch 12413.7 Mbps (18.4% improvement) > > > > With the patch, the throughput improved by 18.4%. > > > > One side-effect of this patch is the increase in the size of struct > > mem_cgroup. For example with this patch on 64 bit build, the size of > > struct mem_cgroup increased from 4032 bytes to 4416 bytes. However for > > the performance improvement, this additional size is worth it. In > > addition there are opportunities to reduce the size of struct > > mem_cgroup like deprecation of kmem and tcpmem page counters and > > better packing. > > > > Signed-off-by: Shakeel Butt > > Reported-by: kernel test robot > > Reviewed-by: Feng Tang > > Acked-by: Soheil Hassas Yeganeh > > Acked-by: Roman Gushchin > > Acked-by: Michal Hocko > Thanks. > One nit below > > > --- > > Changes since v1: > > - Updated the commit message > > - Make struct page_counter cache align. > > > > include/linux/page_counter.h | 35 +++++++++++++++++++++++------------ > > 1 file changed, 23 insertions(+), 12 deletions(-) > > > > diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h > > index 679591301994..78a1c934e416 100644 > > --- a/include/linux/page_counter.h > > +++ b/include/linux/page_counter.h > > @@ -3,15 +3,26 @@ > > #define _LINUX_PAGE_COUNTER_H > > > > #include > > +#include > > #include > > #include > > > > +#if defined(CONFIG_SMP) > > +struct pc_padding { > > + char x[0]; > > +} ____cacheline_internodealigned_in_smp; > > +#define PC_PADDING(name) struct pc_padding name > > +#else > > +#define PC_PADDING(name) > > +#endif > > + > > struct page_counter { > > + /* > > + * Make sure 'usage' does not share cacheline with any other field. The > > + * memcg->memory.usage is a hot member of struct mem_cgroup. > > + */ > > atomic_long_t usage; > > - unsigned long min; > > - unsigned long low; > > - unsigned long high; > > - unsigned long max; > > + PC_PADDING(_pad1_); > > > > /* effective memory.min and memory.min usage tracking */ > > unsigned long emin; > > @@ -23,18 +34,18 @@ struct page_counter { > > atomic_long_t low_usage; > > atomic_long_t children_low_usage; > > > > - /* legacy */ > > unsigned long watermark; > > unsigned long failcnt; > > These two are also touched in the charging path so we could squeeze them > into the same cache line as usage. > > 0-day machinery was quite good at hitting noticeable regression anytime > we have changed layout so let's see what they come up with after this > patch ;) I will try this locally first (after some cleanups) to see if there is any positive or negative impact and report here. > -- > Michal Hocko > SUSE Labs