From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2B64C25B7E for ; Tue, 28 May 2024 06:30:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7C786B0083; Tue, 28 May 2024 02:30:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E2C3F6B0088; Tue, 28 May 2024 02:30:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF41C6B0089; Tue, 28 May 2024 02:30:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B18456B0083 for ; Tue, 28 May 2024 02:30:49 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2821F8161E for ; Tue, 28 May 2024 06:30:49 +0000 (UTC) X-FDA: 82166831418.12.AE16DC1 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) by imf11.hostedemail.com (Postfix) with ESMTP id 8CB6440018 for ; Tue, 28 May 2024 06:30:46 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hzyKk+rF; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf11.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716877846; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MbLS+hldK79Ntn839dxW3kttgjPsUGk4+ECEhPAEkeY=; b=iE+QCKv1RK2oSlCbE+R86zvtY28ZYbqRueJX1lFwUi3gYWAQiE5c4Eh5ioQeR/pxiX+Osy WcybzNxKUFKOV/YAwKxi6vtm+fbNqi/O44QDihjvvBd5h3iAJuHKXXK5ScdgZ1lWFgXqhE A27wt1q7501a5Xw4C8pG2YKdFMMsmec= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hzyKk+rF; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf11.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716877846; a=rsa-sha256; cv=none; b=prk+DbVJ7+cAZv4/pA1cabDDYITKUEZuOwM7GfG1eLsckgoFTvYc5Pthy/vIaar//eJsH2 LA9Xni5cmXWDce2M3p98hcnz0C9k6p+c5CcCFBzQjFCQo/T2wRmEbMLyXvRXDJPrqi6iyc wMu2a2OtrP+pydOpY7EStajLteFzrPg= X-Envelope-To: oliver.sang@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716877844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MbLS+hldK79Ntn839dxW3kttgjPsUGk4+ECEhPAEkeY=; b=hzyKk+rF3/zR5yFq3NVthNkP/0ysnZGdyis/A9KfUVFaJ981RBccwGUJcVpsy2MErLg5F3 t1zNnua1YEqvKpayuzNqDbWlwicG0Ia1zfdlnQMgyh2gQYV++SJ83FAy2/fyAoVM+VBWk9 W6rEhw4ym5c5/MZaMVz4knO+QvgyJYY= X-Envelope-To: oe-lkp@lists.linux.dev X-Envelope-To: lkp@intel.com X-Envelope-To: linux-mm@kvack.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: yosryahmed@google.com X-Envelope-To: tjmercier@google.com X-Envelope-To: roman.gushchin@linux.dev X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: mhocko@kernel.org X-Envelope-To: muchun.song@linux.dev X-Envelope-To: cgroups@vger.kernel.org X-Envelope-To: ying.huang@intel.com X-Envelope-To: feng.tang@intel.com X-Envelope-To: fengwei.yin@intel.com Date: Mon, 27 May 2024 23:30:38 -0700 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: Oliver Sang Cc: oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , Andrew Morton , Yosry Ahmed , "T.J. Mercier" , Roman Gushchin , Johannes Weiner , Michal Hocko , Muchun Song , cgroups@vger.kernel.org, ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com Subject: Re: [linux-next:master] [memcg] 70a64b7919: will-it-scale.per_process_ops -11.9% regression Message-ID: <6gtp47pv4h27txb7lxvxjavrsamxvzamsclrsbpsl62i3nn2bp@yhamzxapsmsf> References: <20240520034933.wei3dffiuhq7uxhv@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 8CB6440018 X-Stat-Signature: sf5d16kq8txyygod9ntkewjdyh83ui6g X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1716877846-504986 X-HE-Meta: U2FsdGVkX19XcMjuvt5FasG0ZYeZQeeNdugfZKdKKStCx+d+4viV3ezKL6ZqcEItvk55j2pxK9BdwoQnr7ERT8sCbnch0MuLBKNlaZ07VgoJ2FbbmG0aMUOOo4u8tgJqIT9qlaj631Ecc33xvnTtXKMasACwxEW4EgGAEuFmznIpOAuSb/50gHwgEhPlq2lVChWUB1OocTWPvwkYz+EY7aEA/7ZpjCtvgBZtQ9uMg0NKvURYyUtWtkOd4OrPz+qcmFBAwEW1WrNOSdApdYtslxfTJLs7oiySLQVsd71JI9aKDZhwLPFLml0c65Ck755y2onnq19K6Lk3n/stRj6ULXGakIcx93kcORWgDVX0Irpv6KNT/LIrXXNMAYtIQG2kJfHdYa+jKAd0GB9TvMLdv2j5w0JX0Fd6IEPIrWF1OMJCdlMzGGmxqZYXxY+BIJsZgwnoK831DmuN+lJIOU//QGjgxvycQZSUBycdGawnNRRZVPUIqwZDJK3aD44MD1kc7CSUJbZL9yCoVre4TDft6vwtROaazlyV8WG5d4hwPuTdtiHb4j/wjsHI5hH/ztKXFQZb97GnMPch75+Zlhq+HypEMQmjl1SCjpQvAQrgjsCIfL87gqLodNQvXBroJbiNbynkAS22Ld9MnDSHzejBhlz2qKrf+TIsnjyfc9C9FJNMKpudw4IJuIHSg/WEnm1lM4k5UVp7DbuccioGYg8T0wOk+hCijE/H8q7W155dkl33UgD4dKibDsbbCq0rBIqTNW94jgo9Bl7Cz6z9uaFwR5QYyQjj50pNOcajNbb3mlIb8CqtDSH19VDeEyAtM5xi8GReLq25ncYtHf5ZrMl0iScOfsxW9Ie31qPrLf4uRqmF2Hui5RkhDxbRsexO/A3JNM+l1eYEbZdn0tgiSdMmEXqOYWcgYAZbmwIoIBkUIRBeNZk2tAxEqtBLhrYXOTg8CWc8Sp4mAW4dmMafD0b mJxeTw9C BF2/qJV9O/kKKRrfjor71RKAciFSx7u8P1sZLk8cLSMCGJvyFpYXr1iFWFHfjL2c0os8mbJmcTWvSn2ST19vL0E4OaATNTHMSf3SsiCp0BeztyorCdsY8e5AGvdFkgVbYFVRohLLoLPKGxUc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 24, 2024 at 11:06:54AM GMT, Shakeel Butt wrote: > On Fri, May 24, 2024 at 03:45:54PM +0800, Oliver Sang wrote: [...] > I will re-run my experiments on linus tree and report back. I am not able to reproduce the regression with the fix I have proposed, at least on my 1 node 52 CPUs (Cooper Lake) and 2 node 80 CPUs (Skylake) machines. Let me give more details below: Setup instructions: ------------------- mount -t tmpfs tmpfs /tmp mkdir -p /sys/fs/cgroup/A mkdir -p /sys/fs/cgroup/A/B mkdir -p /sys/fs/cgroup/A/B/C echo +memory > /sys/fs/cgroup/A/cgroup.subtree_control echo +memory > /sys/fs/cgroup/A/B/cgroup.subtree_control echo $$ > /sys/fs/cgroup/A/B/C/cgroup.procs The base case (commit a4c43b8a0980): ------------------------------------ $ python3 ./runtest.py page_fault2 295 process 0 0 52 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 52,2796769,0.03,0,0.00,0 $ python3 ./runtest.py page_fault2 295 process 0 0 80 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 80,6755010,0.04,0,0.00,0 The regressing series (last commit a94032b35e5f) ------------------------------------------------ $ python3 ./runtest.py page_fault2 295 process 0 0 52 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 52,2684859,0.03,0,0.00,0 $ python3 ./runtest.py page_fault2 295 process 0 0 80 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 80,6010438,0.13,0,0.00,0 The fix on top of regressing series: ------------------------------------ $ python3 ./runtest.py page_fault2 295 process 0 0 52 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 52,3812133,0.02,0,0.00,0 $ python3 ./runtest.py page_fault2 295 process 0 0 80 tasks,processes,processes_idle,threads,threads_idle,linear 0,0,100,0,100,0 80,7979893,0.15,0,0.00,0 As you can see, the fix is improving the performance over the base, at least for me. I can only speculate that either the difference of hardware is giving us different results (you have newer CPUs) or there is still disparity of experiment setup/environment between us. Are you disabling hyperthreading? Is the prefetching heuristics different on your systems? Regarding test environment, can you check my setup instructions above and see if I am doing something wrong or different? At the moment, I am inclined towards asking Andrew to include my fix in following 6.10-rc* but keep this report open, so we continue to improve. Let me know if you have concerns. thanks, Shakeel