From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6E7C27C53 for ; Wed, 12 Jun 2024 13:38:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56C2F6B0095; Wed, 12 Jun 2024 09:38:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51C3D6B0096; Wed, 12 Jun 2024 09:38:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40AD66B0098; Wed, 12 Jun 2024 09:38:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 22A356B0095 for ; Wed, 12 Jun 2024 09:38:33 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A5AA1141775 for ; Wed, 12 Jun 2024 13:38:32 +0000 (UTC) X-FDA: 82222341264.29.1136B37 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf29.hostedemail.com (Postfix) with ESMTP id 812CB120013 for ; Wed, 12 Jun 2024 13:38:29 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linuxfoundation.org header.s=korg header.b=ePJun8sf; spf=pass (imf29.hostedemail.com: domain of gregkh@linuxfoundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org; dmarc=pass (policy=none) header.from=linuxfoundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718199510; a=rsa-sha256; cv=none; b=sOlloWF9GWffPWZDp6GhZn3DorEiRhfIa/uYRfyv0bGKCtN7MVgZ3Z5msR1rPP/m8wCiV3 atxzRbzX575FYvHife5a2YPJ68hsWMRIBVYAGD3mOBUFoi3KjBtmRyN+QBM+kg2Z/NFmuS k/wz3pCpPi04klLqTlzljkynNJn8qy8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linuxfoundation.org header.s=korg header.b=ePJun8sf; spf=pass (imf29.hostedemail.com: domain of gregkh@linuxfoundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org; dmarc=pass (policy=none) header.from=linuxfoundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718199510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SBAseAMNvlYPGvp76B9qAt4Ol2Gnc1K5ldj6yVEw4bk=; b=ZSnUB+EAnwDtkSQHHhDIAcfHpHRvzTcprsNC1H2qD/xTgvQdSt+KmmRbA3yAwoULEEJwkv CtwQ0N6XdAvkJ+cTNgvRGpeZ+XNrK5+PhCAjTpNMB7oxf9EMgOMEk9sfyPRQ0H6XTJvbfw UFGhqscBK0kQiOWdH1f3fezYLh2DXp0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B189ECE1CAF; Wed, 12 Jun 2024 13:38:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00988C3277B; Wed, 12 Jun 2024 13:38:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1718199502; bh=19BoleewHlYCHFKQnLXkBsG1yHfDLMIbAb+2JgXiqXk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ePJun8sfQIIWJDpB2iUOO1/g5/U2Q3YqARvQ2f+Ux2BMyElq0gi3ofJKm4yW8jAHf H2bdBmNezKQZCJ22WpsZiLHrOPF+9rmTxIEbXZdkzDlNI22f/8ehxT4QrB37NMR1Bw LEPUe1ylawVfZgALSajtoNwzX77j6yRk1KJy5wtQ= Date: Wed, 12 Jun 2024 15:38:19 +0200 From: Greg KH To: Jesper Dangaard Brouer Cc: stable@vger.kernel.org, yosryahmed@google.com, shakeel.butt@linux.dev, tj@kernel.org, hannes@cmpxchg.org, lizefan.x@bytedance.com, cgroups@vger.kernel.org, longman@redhat.com, linux-mm@kvack.org, kernel-team@cloudflare.com Subject: Re: [PATCH 6.6.y] mm: ratelimit stat flush from workingset shrinker Message-ID: <2024061209-tubular-saline-5719@gregkh> References: <171776806121.384105.7980809581420394573.stgit@firesoul> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <171776806121.384105.7980809581420394573.stgit@firesoul> X-Stat-Signature: xejmmrtqmoygau488t4ux8eqoxzb3ixz X-Rspamd-Queue-Id: 812CB120013 X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1718199509-85096 X-HE-Meta: U2FsdGVkX1/E8EWcLg+x9WckUSIfbTn8MW0qyU0EUvyR/3d0oKrs2DcIgjLhbxA54uJk1zph0QS3FDWH9AQ8FdcqLwAA/hdkmbSNWe7OmIugvxYwrpuE0amXccIgI5TD7H9E2vPi2OMv9Z6/7xzjABabcunafS5B+HESn2TA50VsGKa3FnW9VZFaw7dbHn8NR2VdVows6fDdrBYa55svn6CqAOuF/9w+0lnm+BvYOtSpbfS2osht8IKDdFcHXYFmebUoYZ9w286FPulKkjouc9eDMTtvdYCE88Nu0U4/R+SsFGHEYgJz53dLb8qqtKg4J+9KTQhLWCfxySGdVLfDbflSlDv4KKJ6Tw56XqVFIfKyBgfyiRm9alffO+GAK9fyrxsy/m5Dpx+wCYTdo+oePLXJnYn2q+C8abZ2AryWfWkWfMMJ2JGpOeBqfaaalvFjqq8j6ZsaQkMvXR+mpBuIuLCLot3wNjb/ygtIyDjclsh56fn6/nzBGSMBAyu59o+YOVulG0pIkrZHubI1eUWS72EZ9eps/PLQBb0RPzIrsVVdvwAvZ3knH7vgrcsww9tbsKLZG7in+0g3v4JVFyT7TQBCof03ouTEL6NiyKitzfAwgX9mntkKZZrrc9FXyLlqh5ftTLbos4XIN2oV9MuHC0SNOnIwy2ROYGB1aYwgMoV8rcUP2KJzv047Ylj1gDmylAxO9u7mFxSgnpZrstPGNOIcp8D565RPe7PJpVrRLeruMtuICq3RcLQBW2bfEVUd0feS95yK8pt1SK12o03/wf+hnZ4bnhgFkBrDs+KWJ+YRfx3zb2trWqR73XngavKwlRq50zVlPoCSLXnotN4PZ416q7ksdYOOC1Hih+V1oJByXX37qtqLdBF+k6oYW3uSnzpl5EHKwXOaiS8o4ltviRnUmiO7RN3vEEdH9/+EcQcffyX14317WzgOZPdd1wu7rOgG1WoA4NlplujJ0rA nINqKI2s ECqz6vHl5J0dr/jlVsgQZpU81E74t83CHd2TGENdkvnb5KeiuAmhBVsG5h++cqBefNyLjbCuW0EZE55MSuBE/RSHqWSVFVgs2bU2hrnHVqLCcNWy/BnbwL4AmyQNMhT7bT+wdG7snpWp99sPLXmv/dnF1zNUC/PhLDXqGxY7vToP7TmPxeg2M2UWT0s7+CxSfsqwuso9r5a8/NhZ/1D4PvSzuAlzCj0wmBoz9iQJ1r/xo39JQO5v/UdDejk577A3wRCIJG+VHfPutWNkY4JrgWCaLIuQztaNFb+ted9xNr0FTXoCntIbbpEzuNu3O6gP+sTQeyhIO58ECsTiEdYzMuqe7jf1L93XnJh06Uvr0/S+RUwbxiuHULq7E0UHvihQWWzQi7ctFSOW7wwIBW8GC+QUuzbxGcyDGK9+gPHHIgFJeB33fu9g+u1i92ra4lf2qo3byuq6+W6XgAe1QzRSxyq5sltWKip0KavH05XKAhp3khJebbaCpnPyf5WxghUE6LAQsVwzQsnMw6hKFASfiqHwTzQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jun 07, 2024 at 03:48:06PM +0200, Jesper Dangaard Brouer wrote: > From: Shakeel Butt > > commit d4a5b369ad6d8aae552752ff438dddde653a72ec upstream. > > One of our workloads (Postgres 14 + sysbench OLTP) regressed on newer > upstream kernel and on further investigation, it seems like the cause is > the always synchronous rstat flush in the count_shadow_nodes() added by > the commit f82e6bf9bb9b ("mm: memcg: use rstat for non-hierarchical > stats"). On further inspection it seems like we don't really need > accurate stats in this function as it was already approximating the amount > of appropriate shadow entries to keep for maintaining the refault > information. Since there is already 2 sec periodic rstat flush, we don't > need exact stats here. Let's ratelimit the rstat flush in this code path. > > Link: https://lkml.kernel.org/r/20231228073055.4046430-1-shakeelb@google.com > Fixes: f82e6bf9bb9b ("mm: memcg: use rstat for non-hierarchical stats") > Signed-off-by: Shakeel Butt > Cc: Johannes Weiner > Cc: Yosry Ahmed > Cc: Yu Zhao > Cc: Michal Hocko > Cc: Roman Gushchin > Cc: Muchun Song > Signed-off-by: Andrew Morton > Signed-off-by: Jesper Dangaard Brouer > > --- > On production with kernel v6.6 we are observing issues with excessive > cgroup rstat flushing due to the extra call to mem_cgroup_flush_stats() > in count_shadow_nodes() introduced in commit f82e6bf9bb9b ("mm: memcg: > use rstat for non-hierarchical stats") that commit is part of v6.6. > We request backport of commit d4a5b369ad6d ("mm: ratelimit stat flush > from workingset shrinker") as it have a fixes tag for this commit. > > IMHO it is worth explaining call path that makes count_shadow_nodes() > cause excessive cgroup rstat flushing calls. Function shrink_node() > calls mem_cgroup_flush_stats() on its own first, and then invokes > shrink_node_memcgs(). Function shrink_node_memcgs() iterates over > cgroups via mem_cgroup_iter() for each calling shrink_slab(). The > shrink_slab() calls do_shrink_slab() that via shrinker->count_objects() > invoke count_shadow_nodes(), and count_shadow_nodes() does > a mem_cgroup_flush_stats() call, that seems unnecessary. > > Backport differs slightly due to v6.6.32 doesn't contain commit > 7d7ef0a4686a ("mm: memcg: restore subtree stats flushing") from v6.8. Now queued up, thanks. greg k-h