From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E987EEB64D9 for ; Wed, 14 Jun 2023 08:38:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58E436B0074; Wed, 14 Jun 2023 04:38:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 53E376B0075; Wed, 14 Jun 2023 04:38:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4065C8E0002; Wed, 14 Jun 2023 04:38:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2CC9C6B0074 for ; Wed, 14 Jun 2023 04:38:05 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E9823C08A4 for ; Wed, 14 Jun 2023 08:38:04 +0000 (UTC) X-FDA: 80900700888.08.2911857 Received: from wp530.webpack.hosteurope.de (wp530.webpack.hosteurope.de [80.237.130.52]) by imf23.hostedemail.com (Postfix) with ESMTP id D147E140022 for ; Wed, 14 Jun 2023 08:38:02 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of regressions@leemhuis.info designates 80.237.130.52 as permitted sender) smtp.mailfrom=regressions@leemhuis.info; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686731883; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dXvBRl+nZmOHHCkhAZICknwEZdiwGeAQThhEhTEo5Tw=; b=GPVtlMAwn0/JLaeJ7kq9xWWKv/1Lt57fmKK+Mx0MQ8zOujrRZz5UjDdkpL2GE8R5Tz9mY4 nj2O46jjYIIroLxfjw9/y+JnlJLwT1aHHKr/DheUk8x59odg/ysfgv6vs77w5XxSOo8jxe QvYOv8t46VJjbOsPbPXuX/WVbqFqoek= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686731883; a=rsa-sha256; cv=none; b=eHfCqoc8hJ5moiWX6GmESDmM1axbqOkac/hywbmBVNCwceUoTRgYw68IXhZtoZUJR1K8M9 UGvmiTsbD5M+ZVdt6co7NjrqF8eS1mf6s1TTpXcMd7m4q+wLdUaRpGadmvxIHK/Y8GtwF0 2KtWvNe2rGniX6NmUkH6CFY2ps+grbg= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of regressions@leemhuis.info designates 80.237.130.52 as permitted sender) smtp.mailfrom=regressions@leemhuis.info; dmarc=none Received: from [2a02:8108:8980:2478:8cde:aa2c:f324:937e]; authenticated by wp530.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) id 1q9M0t-0000Xr-Sh; Wed, 14 Jun 2023 10:37:59 +0200 Message-ID: <2cf7bd0e-39b8-b1ac-5b60-2d67144d43ad@leemhuis.info> Date: Wed, 14 Jun 2023 10:37:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH] mm: convert mm's rss stats into percpu_counter Content-Language: en-US, de-DE To: Jan Kara , Shakeel Butt Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, mhocko@suse.cz, vbabka@suse.cz, regressions@lists.linux.dev, Yu Ma , Linux kernel regressions list References: <20221024052841.3291983-1-shakeelb@google.com> <20230608111408.s2minsenlcjow7q3@quack3> From: "Linux regression tracking #adding (Thorsten Leemhuis)" Reply-To: Linux regressions mailing list In-Reply-To: <20230608111408.s2minsenlcjow7q3@quack3> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-bounce-key: webpack.hosteurope.de;regressions@leemhuis.info;1686731883;c2f650a7; X-HE-SMSGID: 1q9M0t-0000Xr-Sh X-Stat-Signature: d54axg3jwbnassehttoaataweps799a6 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D147E140022 X-Rspam-User: X-HE-Tag: 1686731882-444524 X-HE-Meta: U2FsdGVkX18yr2Ftf9p+Im/61lbA8dv6RHg9LaFTY9IRxelMpJTNwZ9yQApw+5Qd3c5zKdKRRH8C7aBNWOmdh9+Xi/pKt+96TL0TsGCvUw4MvCQFZUYHqgi5ZLdtyKdEU+dG+PrKVa9YDWBgRt2G0bCQB/CCEBkrtVGVWp3nQ2H60hUuunv17gY7gWi0qBKrquOT2jAf//jRR0EHw36n5YdUnuaFNBnChH2vQ+6DnoFLFnIRy3/LA1kXiumaWBWOqVDhSkis5BJC0PCr9HSych/qoIHgLKJR4r7z0URSzSaFXSq/xZ5f1NcGU/X3SIsbHXElzcZKxf3MWw62YrnD5NReYOIIwkUZwqOPxGu0EH84JQekzr7CsUoM/ztCqbuawFtP6z4Zx5sNRqrySazjtXVzop+He9yBBJ7OzLakGjlBUT5v0wf1Yo37sXSKWp8D65iEmcDu6JyaG2FF699g7/h85K6+cJDCji3jzTy8HbRAwuUVQwiqhtz8fjke2Aoiiz6LSH7VVst/VEmtAsqv0d2a3izfeLYql+GFoeQiU5Ni3CFmWRrB8+tctuPjKJXPl44f2hqr6laTDD9XBNyjs4SBHIVfAzqtAXR9Bdw41kwfWM6Gw/gjeaXyeaMg0n1vQRT7kWllkYIoNmiMagZFFwryv4bUs9ScD9aTmOyElfMmD6PSEDoLI8AZteDBOK6wWI9sMc7rTlECWDlXDXkIdNnwyAW1pBxm96E4/a4DoB+tKrbSxoROn7sXgvGKeVDaV7PBqDTH40pcf3Ir2hkkye3D7lEOIP+CnGI6ErAsfSGQtLu3xN7byKklQ3rcBlipqD9NJZG1GUfq6QOxZjiQmc4NUahrjCj3pXVykV7ghBcSrzOJm/NnaHKxgmyLOuX2pnTZsxUkExOHZad1UObTSH0iFeuFiz0yG6z01ohq7CiY6xyCQ6DQxcGmOem2PBfM6WrM6mhNXBLl09KI8YM LAK2/tBT tUooY4hdqdfrb//mooCHkqJrhwObzNwVhCbciBcKjmanSX7fASKrEEsK0whNRLUrXmGlo0nlqYLVrDaPXFCqunWVSM16SbCVK+f98sKYy3mRL0Ka0/M3hzn4gAIKSN6YLsuFgxcFvM2bT5ft8j/AcaO9TErkL5d1PsUANtMVD1+6Ch+8J0uuSVDuA0NkjjSQTETzZzPeUGkICnujX8dJ7iIgTgrCKR+p96/t4wTRPE3dQ+ykLXHBKwUnHiL4fUEG2s5zjI6GwGUXl9SDnBWVBkt4zSf+1TwUZO6a5bYE4ZhvChz0+F3Lntfk0ugUv/EJILUD3G6hvr92GxcxvgHfPNFT4us7D+WaQ7bI78xq+J3poLnrkNdPMAt+0E38Rgq4+q7XGx7ODT7x0YDKHVciaYHI+2jDzEwk1YeUFV39eOyJkhCaKUAjuTMv/PYEUpG6tH554wK5Pv/E6KjMzLwE73AjeXt5VSqMbvq4/T6BHRBPF+GOHHNzTMjczwAl86Zw7Fg4V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: [CCing the regression list, as it should be in the loop for regressions: https://docs.kernel.org/admin-guide/reporting-regressions.html] [TLDR: I'm adding this report to the list of tracked Linux kernel regressions; the text you find below is based on a few templates paragraphs you might have encountered already in similar form. See link in footer if these mails annoy you.] On 08.06.23 13:14, Jan Kara wrote: > On Mon 24-10-22 05:28:41, Shakeel Butt wrote: >> Currently mm_struct maintains rss_stats which are updated on page fault >> and the unmapping codepaths. For page fault codepath the updates are >> cached per thread with the batch of TASK_RSS_EVENTS_THRESH which is 64. >> The reason for caching is performance for multithreaded applications >> otherwise the rss_stats updates may become hotspot for such >> applications. >> >> However this optimization comes with the cost of error margin in the rss >> stats. The rss_stats for applications with large number of threads can >> be very skewed. At worst the error margin is (nr_threads * 64) and we >> have a lot of applications with 100s of threads, so the error margin can >> be very high. Internally we had to reduce TASK_RSS_EVENTS_THRESH to 32. >> >> Recently we started seeing the unbounded errors for rss_stats for >> specific applications which use TCP rx0cp. It seems like >> vm_insert_pages() codepath does not sync rss_stats at all. >> >> This patch converts the rss_stats into percpu_counter to convert the >> error margin from (nr_threads * 64) to approximately (nr_cpus ^ 2). >> However this conversion enable us to get the accurate stats for >> situations where accuracy is more important than the cpu cost. Though >> this patch does not make such tradeoffs. >> >> Signed-off-by: Shakeel Butt > > Somewhat late to the game but our performance testing grid has noticed this > commit causes a performance regression on shell-heavy workloads. For > example running 'make test' in git sources on our test machine with 192 > CPUs takes about 4% longer, system time is increased by about 9%: Thanks for the report. I noticed this is nothing urgent. Nevertheless to be sure the issue doesn't fall through the cracks unnoticed, I'm adding it to regzbot, the Linux kernel regression tracking bot: #regzbot ^introduced f1a7941243c #regzbot title mm: performance regression on shell-heavy workloads #regzbot backburner: not urgent according to reporter #regzbot ignore-activity This isn't a regression? This issue or a fix for it are already discussed somewhere else? It was fixed already? You want to clarify when the regression started to happen? Or point out I got the title or something else totally wrong? Then just reply and tell me -- ideally while also telling regzbot about it, as explained by the page listed in the footer of this mail. Developers: When fixing the issue, remember to add 'Link:' tags pointing to the report (the parent of this mail). See page linked in footer for details. Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) -- Everything you wanna know about Linux kernel regression tracking: https://linux-regtracking.leemhuis.info/about/#tldr That page also explains what to do if mails like this annoy you.