From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F10AC00140 for ; Mon, 22 Aug 2022 02:13:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC0DF940007; Sun, 21 Aug 2022 22:13:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B70A66B0074; Sun, 21 Aug 2022 22:13:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C2F7940007; Sun, 21 Aug 2022 22:13:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 849E76B0073 for ; Sun, 21 Aug 2022 22:13:09 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 605F4A0874 for ; Mon, 22 Aug 2022 02:13:09 +0000 (UTC) X-FDA: 79825606098.20.BE0E786 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf22.hostedemail.com (Postfix) with ESMTP id 70D97C0094 for ; Mon, 22 Aug 2022 02:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661134316; x=1692670316; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=78YdFVWcZoZhkv1NVUGW31kEaECXwO+E+G/qPSQW6CE=; b=W7lDZ+Ye1DHBSylmf3+4F4nhYQrg0uEfiS4s5En3XpWQel8VBQqjK+C4 qOfqr0Sa72g4StpiJS+9bDlWk008x+DloGpRGeCLuY2y8yfC+AvMY5PdC oeH+DtZUFmQR/lLZDTcGUNnoMr9IJB6RamPW6UA2C6Js0w+knP1aYSRuW AuoAbZCAAeV2Z+zPos/t3oIeF8WCILplAnZ1u3fj1zTOS+v/olydrJvdp vmJ4E1/8e3XyGPKA0Lde6IHa3hiCulLxgxiU5meD96ReNdP2yxnlX3j1+ Sx7vzPLMgecEB9NgRQH3ySnRGcDTo920ScO4lAedtrL7NnkB4JxV7e0BV Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10446"; a="294584027" X-IronPort-AV: E=Sophos;i="5.93,254,1654585200"; d="scan'208";a="294584027" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Aug 2022 19:11:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,254,1654585200"; d="scan'208";a="585320276" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orsmga006.jf.intel.com with ESMTP; 21 Aug 2022 19:11:54 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 21 Aug 2022 19:11:54 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 21 Aug 2022 19:11:53 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Sun, 21 Aug 2022 19:11:53 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.108) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.28; Sun, 21 Aug 2022 19:11:53 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hSPfGxouI/uVT8DD2X3g5vpiDx7hi3ey9gDOyIlUAlRH/N7e7WqxTZ6VvBnqlqDMMSq7BoJYREDPqrH8fYr7dL6y3jqoYoNv8/UJrHRzJmjjiMUcJGxBlqPfZ+ttkmQ+wu86gYtbcbFvhuw/Cegm4tdUkC5otzgdHetOs9PkEf1jTNAjlfxs1nLai7Hou2YIepSR/86T/Zn2b5wu2k50zejgQGZP+6hE50G3r7qbUKOAY1aLvfPSufQPlQQNSTtgZbe5qoFZlZH4q3NY/fZULVk9/EdUScsmApD17P2PWzSdSYrm/05XE2evviYU1uFC7ENI1A+mVDq+1d/0OKfnlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4QS46K2SjSk7QYsRrN9eqQROkEiFpdvrl4ySRGovL7E=; b=GxT1cvx44ULZkyYW4VDEFFodvpamZ17fa7PDNdwnh/OnBtxbaMqhXlvTHy7cnyGntRRolySEAvbKj8EjeBiQ4AjkoiJFA3gggSGp6yAz6eLJwScnqyKpfMq4SFfgnkEkIR/TRM1IYS/iJ1aMFLBjnVcsWZpWzyXO4SYMZQ4vnsPNQLOLDPnWcvf44nAI1jPpB6Co/SWhFkGscpkY6/V0Y5E8acezd060Qv0fqjv0AQufxHaqLVQJkEsJEcNAsU2aWlvU6XOmq5ItUW2cBn1EMVmx1dkzl++j1CAUJCFn98Gq33/gOOy9tIyAnT0fh7+8Ujr6yHRMH5wVg0fG2yn9EQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MN0PR11MB6304.namprd11.prod.outlook.com (2603:10b6:208:3c0::7) by MWHPR11MB1886.namprd11.prod.outlook.com (2603:10b6:300:110::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.18; Mon, 22 Aug 2022 02:11:46 +0000 Received: from MN0PR11MB6304.namprd11.prod.outlook.com ([fe80::d446:1cb8:3071:f4e8]) by MN0PR11MB6304.namprd11.prod.outlook.com ([fe80::d446:1cb8:3071:f4e8%6]) with mapi id 15.20.5546.022; Mon, 22 Aug 2022 02:11:46 +0000 Date: Mon, 22 Aug 2022 10:10:58 +0800 From: Feng Tang To: Shakeel Butt CC: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Michal Koutn?? , Eric Dumazet , Soheil Hassas Yeganeh , "Sang, Oliver" , Andrew Morton , "lkp@lists.01.org" , "cgroups@vger.kernel.org" , "linux-mm@kvack.org" , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH 2/3] mm: page_counter: rearrange struct page_counter fields Message-ID: References: <20220822001737.4120417-1-shakeelb@google.com> <20220822001737.4120417-3-shakeelb@google.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20220822001737.4120417-3-shakeelb@google.com> X-ClientProxiedBy: SG2PR02CA0034.apcprd02.prod.outlook.com (2603:1096:3:18::22) To MN0PR11MB6304.namprd11.prod.outlook.com (2603:10b6:208:3c0::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6355a0f8-9e87-4525-6e94-08da83e3a9dc X-MS-TrafficTypeDiagnostic: MWHPR11MB1886:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kDTuGsoA2UrOQU2vNAJUHmZep+OSBorVFbsozBxAOUF39TOID5ISxQm0A270GcXePKZw1KimfYyGaEK/uiAKlT1hazQN0MPdYM1xWc3sI5vyNWm9PUGOFslSnKWyFAgeRHoglTLbIuLsC5C3/DHgg7tCm2QL7EJfpj/3EfjGAZhF0ii/mlwIVWmnxAynLUUr9D15+aOFrH/LPHnVZNgYhxTs1Shx49/+FJlkgCGUiQw5eSyAISkkCtTK5hBGVhV3NQa6nWbBFv6FKocDnOlIcB+w+WMlpjY/SzPe9vO5+WHCck3Tjc9Pq/w+z2PxHE+QtD2WOnh1/JvZxz6v4um2KNfcLUwLrVO4j9wL6sXZoODZ22ncBhRp64Eji8oIvxyzIwiLsVqwspgkDi5AQg6l1B2p+BewMtVfHPFbgBeUdzT3TeQwdMUEyMUOev7d16mzdSA4Ckbag/om4QjiG5l3HvRUzC7G6iAFNLGT90rTdnZXrHtBzgQ2DLESIEDTlZpnoxbbSInkgiEoy70JgQEcepkYNqjVGhLBKV0lvuk7TmXBd5Z0DRZPvZKvhOjjMEZTCh5/IfGpLC2+EKlO2ABx3FAanOFqmVlTa4bAvK90yyXYmRMxchsiWtS0mpQW3N22k7ib7IpP4QVZUjSPYPZuaGC5LWIa6Zh2EolJonLhtzP2JH5Wp3b0pS6TBrcjU6oI0oVQf5cPPfdIz75gHpicbA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN0PR11MB6304.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(7916004)(376002)(39860400002)(396003)(346002)(136003)(366004)(186003)(8936002)(66946007)(66556008)(66476007)(8676002)(7416002)(316002)(38100700002)(5660300002)(6916009)(54906003)(4326008)(26005)(2906002)(478600001)(6486002)(9686003)(6512007)(82960400001)(86362001)(83380400001)(33716001)(41300700001)(6506007)(44832011)(6666004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2Zh8NzrPLTbsQX5cKrejTA+nIDFMBgb6ddZ/wFV8IFrc7sGDV+AeSYM8k7T7?= =?us-ascii?Q?jmpCIPP+qf7gVzUycm1uTlbOoDV8cxQaZ9hBa/0TkS3ifY3bGzIciqrfjp4O?= =?us-ascii?Q?n4T7NgxZ0B1fRPdGPynthXym30nv2ECDhAyyvT80zHl2A5Z2MiDoPksOHno9?= =?us-ascii?Q?7j0iBbC5voOgG3ntu5FYXYBwE/msOCEInOij5ukyJsrwiqlQG6o6h7632GzO?= =?us-ascii?Q?1kJCH2hzbqtX6gx6yJc1PR+VGQXjl7BqM1zncx6deU/F0ktRDr1CifPHi0dy?= =?us-ascii?Q?wkCbHHwdt7+mkVichVwB3CsqWQeFsE4pf0s6Djbqn2r8c8FlR80R3fz5dxe7?= =?us-ascii?Q?LeaYogZRpWJ5956YWjPdg+V3pyJvyadioghJ0bC5YageDcaSMQkr12SwpK6f?= =?us-ascii?Q?2wpVoj4s9NGsQd3BUkOdVkAfXAvGqTbDuxUgdY1plgaO5vrT2bfp5ab40SGy?= =?us-ascii?Q?k3aE1+foxTZE8xgv/cnD+RX8nwVPf1YvMyMv7UlKteffxZbMIhLZhifV9HgP?= =?us-ascii?Q?kQSzy0q5GX5OVF6U9Y/l1gFWPhwjwba/yZt3Rl/DDzGfSxTjYRErAxBTjRjl?= =?us-ascii?Q?0PUxOPePDvgz3wBBASMH5HIEkZt1zWQskNxN2/E9OyTiZn1VHAHqok6R3Znh?= =?us-ascii?Q?laKm9eYHCjF867ojxQO2GvkE+A39L2W7i2SxLM004H9jkewr6+9E/SY/x9A2?= =?us-ascii?Q?Vn3RGRCej6GumzWWCRTrn6OKT41JyWnluuYW0se2Afi3NnS1OUAnV0z/qKlt?= =?us-ascii?Q?NpLEhG4UQT6g6j4I0jPqQxVvD+5VCRxHzsc1roMUGYhlT5MAWnmY7geI0aVd?= =?us-ascii?Q?8fwIZn9bIE4rFjl7a9oI79bPvLdFxmMEijrAz4FueYBK2p2iqb/ak6t010YF?= =?us-ascii?Q?VZZR9OnR75ZbZkw/wRW61QtHeS1/rH73kZe5wuWDjh1KjG4Xgc0xGaSD1QtR?= =?us-ascii?Q?nhnhXnXkkEQbSrZAm4gnrg+SpSYrimoIV5zFPrX16gCSgLhv2S6T5l3sHi6i?= =?us-ascii?Q?H8nSe3d3dxkJ6fAajYHVdetCwRT3gQ7ajJ2OgjYEcUGWEcdhmM08Wkcj6bD7?= =?us-ascii?Q?/BoRAMmUcSV4RA1dvlt1ujgyJVX81NNpEVu0q1eQ8LcictlIX/E/W6KMNvDN?= =?us-ascii?Q?t4/SeemwgEyZRq6mpKrPejCKkDOAMhlG+kV3KZcmQe1qpw6z135QvOr/V+yV?= =?us-ascii?Q?jndBkmDEPsv3T9FpZ14MmIp0DJHdCdMgFDiA/KDKrEXJ8xRHI9qA9l2CPczd?= =?us-ascii?Q?6VHqe7/Xk99FLgn12AC//l6LUxcIZCYOuxzSAFkXkNzD7whloB3UiIuPJQbP?= =?us-ascii?Q?V32UNoixv5QaB1POqmE7S5gsLl4j5W6NuOxTKIc87G89hqxA/EmvBiTCe+ZG?= =?us-ascii?Q?FsAbVZNqlxmKZsAFpEhosJEYVQ4T/y3+F3foXMlTZ2WGkxZNRgvDqXsUgBaA?= =?us-ascii?Q?Kn5VZP/d9fdz6gm6WCpKiOMymgwSWZztaDP1+lNRDDsBPQKB8YCjManctVQm?= =?us-ascii?Q?HseUrOgyjzUhFaEzd+CMv5dGyan/DeDDqGGgCjeSRCFZ3E5Nrn1/GYjaJWEW?= =?us-ascii?Q?IYOMsgC1NPEGjk4XrAel89cnHtIbHp+rvBa4g0r3?= X-MS-Exchange-CrossTenant-Network-Message-Id: 6355a0f8-9e87-4525-6e94-08da83e3a9dc X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Aug 2022 02:11:45.8519 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pIlH2IyQRt6EOclIXk5563xG60kLW7uO2GGsEVzuIfeBr8xXflf0brXIMoz/V0KIpt1L+/mKDEstwP5/LtaUxQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1886 X-OriginatorOrg: intel.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1661134316; a=rsa-sha256; cv=fail; b=q2P3ZbU+XZwnCRHqAoy1U8O8Dx1fB4MpWkWRnrFdV7a1udve7aO3KJb29R4sx+Cw6vjdfA 8aYWlRWYOMJmTdV2m9N+omB56buFewwxG1Yx+FgEKo2FaZVsmnSsoGTSgeyFCY2YwN9GX0 71b8HIuv7BxjtmABXSgYJUKzT6YzIck= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=W7lDZ+Ye; spf=pass (imf22.hostedemail.com: domain of feng.tang@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=feng.tang@intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661134316; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4QS46K2SjSk7QYsRrN9eqQROkEiFpdvrl4ySRGovL7E=; b=d1Iisqu6YwDFNuj4qnP8ZnzG9tzITYgNTAbNlIEDbt18LdbUO3ZBKjdUwfl9hfKOwnjPJD u6uBUEeQkIzj0ly4XBGAEhCH0a0cI1sOKj9HcScYvUX0F8Cdw1MxoCfJWb1yJXHvEbXK7Z FXK8Gp4bqAFEuuC88rLlupmY7z9xD6Q= X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 70D97C0094 X-Rspam-User: X-Stat-Signature: emac1kymeoykmow6u6ymxdt3ixb9z7gg Authentication-Results: imf22.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=W7lDZ+Ye; spf=pass (imf22.hostedemail.com: domain of feng.tang@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=feng.tang@intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=pass (policy=none) header.from=intel.com X-HE-Tag: 1661134316-17173 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Aug 22, 2022 at 08:17:36AM +0800, Shakeel Butt wrote: > With memcg v2 enabled, memcg->memory.usage is a very hot member for > the workloads doing memcg charging on multiple CPUs concurrently. > Particularly the network intensive workloads. In addition, there is a > false cache sharing between memory.usage and memory.high on the charge > path. This patch moves the usage into a separate cacheline and move all > the read most fields into separate cacheline. > > To evaluate the impact of this optimization, on a 72 CPUs machine, we > ran the following workload in a three level of cgroup hierarchy with top > level having min and low setup appropriately. More specifically > memory.min equal to size of netperf binary and memory.low double of > that. > > $ netserver -6 > # 36 instances of netperf with following params > $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K > > Results (average throughput of netperf): > Without (6.0-rc1) 10482.7 Mbps > With patch 12413.7 Mbps (18.4% improvement) > > With the patch, the throughput improved by 18.4%. > > One side-effect of this patch is the increase in the size of struct > mem_cgroup. However for the performance improvement, this additional > size is worth it. In addition there are opportunities to reduce the size > of struct mem_cgroup like deprecation of kmem and tcpmem page counters > and better packing. > > Signed-off-by: Shakeel Butt > Reported-by: kernel test robot Looks good to me, with one nit below. Reviewed-by: Feng Tang > --- > include/linux/page_counter.h | 34 +++++++++++++++++++++++----------- > 1 file changed, 23 insertions(+), 11 deletions(-) > > diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h > index 679591301994..8ce99bde645f 100644 > --- a/include/linux/page_counter.h > +++ b/include/linux/page_counter.h > @@ -3,15 +3,27 @@ > #define _LINUX_PAGE_COUNTER_H > > #include > +#include > #include > #include > > +#if defined(CONFIG_SMP) > +struct pc_padding { > + char x[0]; > +} ____cacheline_internodealigned_in_smp; > +#define PC_PADDING(name) struct pc_padding name > +#else > +#define PC_PADDING(name) > +#endif There are 2 similar padding definitions in mmzone.h and memcontrol.h: struct memcg_padding { char x[0]; } ____cacheline_internodealigned_in_smp; #define MEMCG_PADDING(name) struct memcg_padding name struct zone_padding { char x[0]; } ____cacheline_internodealigned_in_smp; #define ZONE_PADDING(name) struct zone_padding name; Maybe we can generalize them, and lift it into include/cache.h? so that more places can reuse it in future. Thanks, Feng