From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 087A9C433F5 for ; Fri, 25 Feb 2022 17:22:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDF2F8D0002; Fri, 25 Feb 2022 12:22:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8E388D0001; Fri, 25 Feb 2022 12:22:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7BF78D0002; Fri, 25 Feb 2022 12:22:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id C88288D0001 for ; Fri, 25 Feb 2022 12:22:55 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 916F3621DE for ; Fri, 25 Feb 2022 17:22:50 +0000 (UTC) X-FDA: 79181972100.08.7012455 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf27.hostedemail.com (Postfix) with ESMTP id A8E8C40011 for ; Fri, 25 Feb 2022 17:22:48 +0000 (UTC) Received: by mail-pl1-f180.google.com with SMTP id b22so5370577pls.7 for ; Fri, 25 Feb 2022 09:22:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NmsBcjZVKJ1TlrseNtG+KOWpB0xHMpSPQm1c1EfUCX8=; b=YiRbNBljpgWH/7bh6OvxiEYTgkYDhUdYgoTtSPe/vxyF/6mmAqkpu4mT57n90ZYVEH AYC+7u5/K1kHY5AXEz0MmCdMROf1JFpmFEJLD650kcGp/Uy76645WO48gi+J9tyfHT11 hliJDjxDSW9qHBAwljHk2OCcqA+70ebxU4U/MNzSefsbJM0+mglYqSzPMqblIoJF9TyU 5ad1Zu5w2GCgOH6rhgZ09QseRBKXSUGJF3BkPuBokXKVIhr1KaHXHALZ9nacCd3g04Md LKkTO4Sx4u/+zbiTywek+LKNEvUwsBBHmJWRUSs9LjpnulIB0wV8c0g5XhhfitfjsPWq gYQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NmsBcjZVKJ1TlrseNtG+KOWpB0xHMpSPQm1c1EfUCX8=; b=xefQp8kYn4tfW6zZj8Q75xeM6MesDy9vpJxtUwXnbIXklGPqOo5HvNTB4mWomnqwp2 b8LNVTC/cSjKCYi4dkyGcsFHLMUW45TusAdQT07LWb0RDx/Z/rXdJizC6iGG1mqbZhDs uWA1ufLaK9z/GQISm8WYMHBiGzrGnDkeSy1Yh/CsBLv8xA5QzFGFceKGgvkk1FlFM1BT 4aIYCHQGDIE2IXhAEW6tWLWMhIFu+8UxUSuimMOnaQZq4vnwqWw2LKaRIhZ3n6zKyvAn lZ+XJhGk1WWPDb/wlWQldhoo4QDxPza/uG7k6Sp529YR4/WgGaudGx/qIvwtx9RAeOUT OtPg== X-Gm-Message-State: AOAM533BGmwDQvtGxB+Hc0HsaTNk/vImgUVjHPUM2a/rL8RUmeUc6Pci r5keDBqAXbRu2mqKSfpZnNTnZW76y4lpFSlgPj5E5w== X-Google-Smtp-Source: ABdhPJyrOj0bme1/fBligbyhad9ZoWH2FCh/913mdAdTZddACAm92sfa5cuRF+CJWebuq0TYfhQ1jxW2xfDXMsoV30E= X-Received: by 2002:a17:903:2cb:b0:14f:4fb6:2fb0 with SMTP id s11-20020a17090302cb00b0014f4fb62fb0mr8253884plk.172.1645809751372; Fri, 25 Feb 2022 09:22:31 -0800 (PST) MIME-Version: 1.0 References: <20220224165838.oir5clpkkqpstpx3@google.com> <20220224185236.qgzm3jpoz2orjfcw@google.com> In-Reply-To: From: Shakeel Butt Date: Fri, 25 Feb 2022 09:22:20 -0800 Message-ID: Subject: Re: Regression in workingset_refault latency on 5.15 To: Ivan Babrou Cc: Daniel Dao , kernel-team , Linux MM , Johannes Weiner , Roman Gushchin , Feng Tang , Michal Hocko , Hillf Danton , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Andrew Morton , Linus Torvalds Content-Type: text/plain; charset="UTF-8" X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=YiRbNBlj; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of shakeelb@google.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=shakeelb@google.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A8E8C40011 X-Stat-Signature: osuymb9kj5trr4ygtef36tdhgdkey9sk X-HE-Tag: 1645809768-958082 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000639, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 25, 2022 at 9:09 AM Ivan Babrou wrote: > > On Fri, Feb 25, 2022 at 2:23 AM Daniel Dao wrote: > > I think this looks good so far. > > I compared a flamegraph before to a flamegraph after (10s @ 99Hz on > 96-core CPU evenly loaded to ~75% in both cases). > > Before: 1.4% spent in workingset_refault. > After: 0.5% spent in flush_memcg_stats_dwork. > > The latter is all in kworkers (as expected), while the former is > spread across IO active tasks. > > This seems like a great first step that should be merged on its own. > It would be good to also do something to improve the CPU time spent in > delayed work, if possible, as 0.5% of on-CPU time is not a negligible > amount. I will send the official async patch soon and I agree that we need to improve rstat flush infrastructure as well but that would need more thought and time.