From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C554DC3DA63 for ; Tue, 23 Jul 2024 15:44:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 380D46B009E; Tue, 23 Jul 2024 11:44:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3093F6B009F; Tue, 23 Jul 2024 11:44:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1833B6B00A0; Tue, 23 Jul 2024 11:44:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E7BF06B009E for ; Tue, 23 Jul 2024 11:44:16 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 97A181401B0 for ; Tue, 23 Jul 2024 15:44:16 +0000 (UTC) X-FDA: 82371438912.14.B200453 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) by imf13.hostedemail.com (Postfix) with ESMTP id 101D520005 for ; Tue, 23 Jul 2024 15:44:12 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=AFWubk18; spf=pass (imf13.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.184 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721749406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BzOavXBxTEmctX7/gPCPhp7QPQ125ryV696udEGlprI=; b=whSzn2TZqE5H1xEH7o98j2IsVoNCzmN7D/W4PRbF8ZSfdiNy5q9fz4YERZEJJU0UC0CYcr GGHPgnIaFXnWvR7PgEIwrMz7SUyUAUhqz7avtT10MmxZEhDVry7xWQYvUW1T6e/kCeoQ20 hC1jWkMcCB/zkznlfRWATO3AjH5aYfk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721749406; a=rsa-sha256; cv=none; b=qmPxPU4NEcCYOUJXazFNcjg1BetAvKcpXQDi2+zqdZTizqT7fs8bWOH9zeiJP4OzygVD54 txpyZQabLzxZKZFrX9I+3SBed/V2zhqlgJfnyya5rTcAAiPYqXfhgQN2q/Jy86i3ke5RSH vF4TmTeBCP2M5REoU2YwgZzHECo39lw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=AFWubk18; spf=pass (imf13.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.184 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Envelope-To: oliver.sang@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1721749450; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=BzOavXBxTEmctX7/gPCPhp7QPQ125ryV696udEGlprI=; b=AFWubk18jH2pDOFUO30PGFuVdPj4sx6AweKRiBHY7NFRU0y1BETM/e/NPRMcQ37/GHjjFL aoCH96eoFsL4amxaJhQ6miIubhmZxgYWqK7z/HRkx8A7657cOnVuD8YWcs+h0Xe/5pZLx2 e+6pykHoWS103NFurddF+DHX2Lr5aj0= X-Envelope-To: roman.gushchin@linux.dev X-Envelope-To: oe-lkp@lists.linux.dev X-Envelope-To: lkp@intel.com X-Envelope-To: linux-mm@kvack.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: mhocko@kernel.org X-Envelope-To: muchun.song@linux.dev X-Envelope-To: cgroups@vger.kernel.org X-Envelope-To: ying.huang@intel.com X-Envelope-To: feng.tang@intel.com X-Envelope-To: fengwei.yin@intel.com Date: Tue, 23 Jul 2024 08:44:01 -0700 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: Oliver Sang Cc: Roman Gushchin , "oe-lkp@lists.linux.dev" , lkp , Linux Memory Management List , Andrew Morton , Johannes Weiner , Michal Hocko , Muchun Song , "cgroups@vger.kernel.org" , "Huang, Ying" , "Tang, Feng" , "Yin, Fengwei" Subject: Re: [linux-next:master] [mm] : aim7.jobs-per-min -29.4% regression Message-ID: References: <202407121335.31a10cb6-oliver.sang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 101D520005 X-Stat-Signature: 9a4emuxzfxcutxkqs8ecpdgckrr7x41z X-HE-Tag: 1721749452-413263 X-HE-Meta: U2FsdGVkX18T0ba8dKRZX1u9ms/vBsjnN+YMolgmB59MFNT8Ab+2zS7YluiVyIy2fubhTko0Vmz6WWrmEmpRgiNZPVUOmNvzPrit7fQnsI4bRtWqFAVDDfdVDZAkyLp8MhHE/bK1/thVUn3pimjkHzwkOCpLF3ezqK2oto/czWBplv5kOZZlJaw6n0LSsBJeardau9kUuSvcxaRjLmHRLDvfq5V0C4+rvKW8fwUMgOXMCSoNwHgHDICTsQaqRd0JeMSCW2tKVItpcH4OePNMtp4lEHxIsMaa/w743MpTWJBcIssRved/1TGWQi9VagQKFN6gUw10nvW3DJkNw+aZkFE2mxfFuudvQmIX6pui4Qfh/gIrSpubC0DYU3qxGhqIyVngFLXgCd0/DussC3R+k7ZBEBisTKdQ1QQ+LUz5EItBQgwowKyIs8nHy0vOEEz1SOQyMqR/JAM9H69FlsRvFDaNijO77M8LZOE9fHZyXbFFTYl6w0JgxZJ+M/K/N+nD9f3pYb5VROGvD29pKR1CosYvDCcV2HWIZ/mAOdwj94259+grHNwnd/8OPTf4YPRwo+1vSiu+BD0MCBqQEKk+AAbacSxRG6Ls7CuEbR5RO6JgRX47sjfai64QoBvYI4xDQAjrmbHMNMEfhiHdgix6fRjNNTDx5+7E2ZhHvRr3bYNMGFh4lydfxayhbR+/GLQlHBvlBSNyMEukZ9AhJe2Rc9zELyf+1iRNNFCiiLMC/IY2x5aPwG4YIXdNC/hc46L8bBd9U/HptV83dT5gezaNt9l/A4AnaxLQEwwFhT+7tY/i4Ih23rUrT/zthe8jv1iXOMwCRnCLXcmmeuEIAPLgETCjK/qlHjF+DHu5OXeYj58FjjmatB0/2fejIdqM5LNaFAhFVQl1KyL9n7qZ6he+uJPbpddZgXoKIV+t0OwYDvGdJvRpLAi5cxXf73h/kj87sGWzWIFSl2icMmJjHtz JcB+mOiu +yyCMlpKRe+GnI4crkXfTeLg5sdf8TsUPKvldIL+Ug8U6HiefCWWxQKzdheamJP1/jcfW+xocaKxP3vDYeEgd0aa/UNHc0kO/O7FGvZgu8NvOHbPTwzUQnG7nCQgznk1QYphpulqbI/t0iAcc/boukXWjI3yagxpql83BPYLe8mSdTGYQk3aupSQsMcQYtKfzNPnS7InAyuwuooKQPGESPFN8ty58iGaW7vzB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 23, 2024 at 11:42:33AM GMT, Oliver Sang wrote: > hi, Roman, > > On Sat, Jul 20, 2024 at 01:14:16AM +0800, Roman Gushchin wrote: > > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > > hi, Roman, > > > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > > hi, Roman Gushchin, > > > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > > > Hello, > > > > > > > > > > > > thank you for the report! > > > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > > > Thank you! > > > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > > > Thank you for providing the detailed information! > > > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > > > no. in our tests, the following patch has little impact. > > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > > let me know, thanks) > > > > Hm, interesting. And thank you for the confirmation, you did everything correct. > > Because the only thing the original patch did was a removal of few fields from > > the mem_cgroup_per_node struct, there are not many options left here. > > Would you mind to try the following patch? > > > > Thank you and really appreciate your help! > > you are welcome! > > though we saw there are further discussions, we still share our test results to > you. > > in our tests, by your new version patch, the regression is entirely resoloved. > [...] This is very interesting as this shows (possibly) there was false sharing between lruvec and lru_zone_size. I will check if these two fields were accidentally on different cacheline before the series. > > > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 7e2eb091049a..0e5bf25d324f 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > > > /* Fields which get updated often at the end. */ > > struct lruvec lruvec; > > + CACHELINE_PADDING(_pad2_); > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > struct mem_cgroup_reclaim_iter iter; > > }; > > > > > >