From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27899C25B74 for ; Fri, 24 May 2024 18:07:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87D106B0085; Fri, 24 May 2024 14:07:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 82DCA6B008A; Fri, 24 May 2024 14:07:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F5356B008C; Fri, 24 May 2024 14:07:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5114B6B0085 for ; Fri, 24 May 2024 14:07:07 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 093EC1C04D3 for ; Fri, 24 May 2024 18:07:07 +0000 (UTC) X-FDA: 82154070894.06.6DE0D9F Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by imf25.hostedemail.com (Postfix) with ESMTP id C3805A001F for ; Fri, 24 May 2024 18:07:02 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=CzCYRIqk; spf=pass (imf25.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716574023; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IDgNlxcmbX6SMIrBizByyCwxnmHxeA7yY8zL5+NMTsw=; b=Y3K4acR4nNxLuMnMZbSD9vqqfQIvqufgz8b7pH5vSAWGCxfGd5FrZ/DLajvPoJe5KYDzvl eIXxee7nhuhtktS7YOjjcnwwgdhSHQTUcYJ19WfnxHSiJypbQFJWVgPjjCBNEwwi+VrGCN 4ipYyZWvt4tp2ImtUcFpsmD+NYj4bIg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716574023; a=rsa-sha256; cv=none; b=2dx8PjnABhkRr0J/MK2MbceserHKFlH9tTZWW2dM+xi72HrjdeOd5JGiwHWdDydeJwiUwc NPIlnmq+fYtxv3EHyJLMTRZLskzMpbnn4oWqgUyYJEopUjiBb1ohudtv2sGQG8tsZ3yGO6 PYBG68Dz5wkjrhoIO/Czu9Z0G6Sfp5s= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=CzCYRIqk; spf=pass (imf25.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Envelope-To: oliver.sang@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716574020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=IDgNlxcmbX6SMIrBizByyCwxnmHxeA7yY8zL5+NMTsw=; b=CzCYRIqkeylIyXvDionKYbCNCGay0hGXS99GHgIWwd5C1jk5WaZbGpZtxmm3noUHV9XG2k mgDWUyTmEjb329zjGJQLfbvwnEOC1eETy7qHmDXsHb1RaRUfQ6gJdEqM+tI3KCRe4qWLVu ddJny1VrghI8tNtIKYq1S192KtbQpqU= X-Envelope-To: oe-lkp@lists.linux.dev X-Envelope-To: lkp@intel.com X-Envelope-To: linux-mm@kvack.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: yosryahmed@google.com X-Envelope-To: tjmercier@google.com X-Envelope-To: roman.gushchin@linux.dev X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: mhocko@kernel.org X-Envelope-To: muchun.song@linux.dev X-Envelope-To: cgroups@vger.kernel.org X-Envelope-To: ying.huang@intel.com X-Envelope-To: feng.tang@intel.com X-Envelope-To: fengwei.yin@intel.com Date: Fri, 24 May 2024 11:06:54 -0700 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: Oliver Sang Cc: oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , Andrew Morton , Yosry Ahmed , "T.J. Mercier" , Roman Gushchin , Johannes Weiner , Michal Hocko , Muchun Song , cgroups@vger.kernel.org, ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com Subject: Re: [linux-next:master] [memcg] 70a64b7919: will-it-scale.per_process_ops -11.9% regression Message-ID: References: <20240520034933.wei3dffiuhq7uxhv@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: 69a4f3wnuuu5ef8qr9bt3zsr7nk8eppu X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C3805A001F X-HE-Tag: 1716574022-649532 X-HE-Meta: U2FsdGVkX1+LWu0H+bWODauB9qFFyIXZ5/c1EZEOxqYk7nXwpk0c8OF/qQE7JIq2dQEHpOQBcX5cYz1KNppeHgDTdkdsm0u1q5VJjz0RDW/EY6h2Dpu/f2jQS0tKVG3DD8rrVJabC9A6LoJU8quDXUbu8ymDPjDee0cT4qjm1Pzxjox+XRiMmyIwydQdcYfeLyYaCWhlRPPBBp44mVV8uq2rfB/IiaAZ7tskE41GpCrp01vaeOuL1wUD1hE8MfsdiP2BNFPKq0LwnuWAIsxM0ojW6KJtH/3Fzm0APrYiQrXIk+8ajyCRrDvJtEKiKPreT6s9ieKK8HpLnkY1ICmlzXfN+lMDw0m+JKEb+b61tgjIMG2/iBKEO67CO6s9+StkRbZMuIiyI/XFU9VEnFKmcegVSu4hTKQVWHoVsf8nf2gqnPeVd/kpIquusmxA4V7TZj8QYewHgHROyQVh8NwYL8In4WbpU2pvDyJYZGTpooZBVsFTja2P4i8t/fcXN7y4RXUPQYNXGK3vFvCPl5eukI8VtKimUva4w4iWAsR6L1prWV91gPQvKh31MdrJlEzvqc5Noh4mqvolLWyIEHITwjjPmRYDbGB1a3DP5LpBjCYxev1GJvnJCx7KjPWJkBBApHZOUw5RA7F9pvA9o9QtCJwxT2B5C/l6lJr0OJtfdcUtCMizUcvIYicR9yD4ou7ps5FSVe8YL+A921xeb8gFk3xvPuAND/j1kZZPO24VqelMcTIKUTLa03+Ksp2LRpWCh93N10P3ByJEdehyRBuF5iFaUMxkCmheeUS1OVIbaXKRVb3RMf7nfIDqtnD4JnIdFyEA4/5Fu8vbwqF5KJx+0UDudUeKYuQnBEY2vu7soR27ZUsfzP5AseOrU4Wby0RTg4+vHlHDKcREt0sxE0SeGo9jiDB6Tn0PAOMWXecya37hWvQRTvvVkzUeRdkiEuLEYIW5FoIB/Ha7gqg0Iik nz2gYtoD cMdsEQRK7FllrFZf+qomOfqXd3wpBatvqor6KWrie11W+tNlLFakLv/zhIugYTZO92vKiT0rVxaYZAHQaEjDX4a2ChCEbLx/PwvUjPnVOJ3glc4kOOxyLVKg5+DxpYPPRboVYyQu0awccsUs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 24, 2024 at 03:45:54PM +0800, Oliver Sang wrote: > hi, Shakeel, > [...] > > > > > 1. What is the baseline kernel you are using? Is it linux-next or linus? > > If linux-next, which one specifically? > > base is just 59142d87ab03b, which is in current linux-next/master, > and is already merged into linus/master now. > > linux$ git rev-list linux-next/master | grep 59142d87ab03b > 59142d87ab03b8ff969074348f65730d465f42ee > > linux$ git rev-list linus/master | grep 59142d87ab03b > 59142d87ab03b8ff969074348f65730d465f42ee > > > the data for it is the first column in the tables we supplied. > > I just applied your patch upon a94032b35e5f9, so: > > linux$ git log --oneline --graph fd2296741e2686ed6ecd05187e4 > * fd2296741e268 fix for 70a64b7919 from Shakeel <----- your fix patch > * a94032b35e5f9 memcg: use proper type for mod_memcg_state <--- patch-set tip, I believe > * acb5fe2f1aff0 memcg: warn for unexpected events and stats > * 4715c6a753dcc mm: cleanup WORKINGSET_NODES in workingset > * 0667c7870a186 memcg: cleanup __mod_memcg_lruvec_state > * ff48c71c26aae memcg: reduce memory for the lruvec and memcg stats > * aab6103b97f1c mm: memcg: account memory used for memcg vmstats and lruvec stats > * 70a64b7919cbd memcg: dynamically allocate lruvec_stats <--- we reported this as 'fbc' in original report > * 59142d87ab03b memcg: reduce memory size of mem_cgroup_events_index <--- base > Cool, let's stick to the linus tree. I was actually taking next-20240521 and reverting all the patches in the series to treat as the base. One request I have would be to make the base the patch previous to the 59142d87ab03b i.e. not 59142d87ab03b. > > > > > 2. What is the cgroup hierarchy where the workload is running? Is it > > running in the root cgroup? > > Our test system uses systemd from the distribution (debian-12). The workload is > automatically assigned to a specific cgroup by systemd which is in the > sub-hierarchy of root, so it is not directly running in the root cgroup. > > > > > 3. For the followup experiments when needed, can you please remove the > > whole series (including 59142d87ab03b8ff) for the base numbers. > > I cannot understand this very well, if the patch is to fix the regression > cause by this series, seems to me the best way is to apply this patch on top > of the series. anything I misunderstood here? > Sorry I just meant to make the 'base' case to compare against the commit previous to 59142d87ab03b as I said above. I will re-run my experiments on linus tree and report back.