From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 957EEC433F5 for ; Tue, 1 Mar 2022 17:21:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D84618D0002; Tue, 1 Mar 2022 12:21:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D5AF48D0001; Tue, 1 Mar 2022 12:21:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4AEB8D0002; Tue, 1 Mar 2022 12:21:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id B7F348D0001 for ; Tue, 1 Mar 2022 12:21:26 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 5146F1219E4 for ; Tue, 1 Mar 2022 17:21:26 +0000 (UTC) X-FDA: 79196483772.13.0EAFB54 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf01.hostedemail.com (Postfix) with ESMTP id BC1A140010 for ; Tue, 1 Mar 2022 17:21:25 +0000 (UTC) Received: by mail-pl1-f182.google.com with SMTP id n15so11746162plf.4 for ; Tue, 01 Mar 2022 09:21:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bDu+y+f8JQYU3RGpZYFrasVkU7NMBDyxJALCa/zmka4=; b=fZDK1hgTtMcHNo/1P6qNEVs6yPXFW/deYvEker6YOPN/13zmSjk1YYcJXGviYMEHVE 8E5wYMvxqyk9r+yvmbYSX6r1jorlcBKFdppXneLtzmArqDVTPbn4BjCzfdnEoQYkvwpC +1KPBbrEQXutzxvFKYDHODHzf6BNyzYGQyini1yAIT1q5rKocES28Miw5n8nRC4DsrUW 95jHg8JdltfmO7jbciMVzAMSOll3yN75VkTePIEZC/jJNJdH+LM5DcXvo1Lu0SyJaJgb LvnaHbrHPb/kASBRMt331ZL1U+jq0zO09UlbhoWsibz9JmgWc/nM7dCtnLg2Pfsa+8HM kCQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bDu+y+f8JQYU3RGpZYFrasVkU7NMBDyxJALCa/zmka4=; b=kJNFgxVruHjuGDK3nOwlUiJ5p94jlrXEsOWhTaCrbW3MFFXJ8jjbBLzcyD4sjJAVDU AD+H8RC8BDuvzpul95EpcjEyJa1E93nJx2irnefvjiVIAI1Au2n0N4sw5ypDmC+sKBtD LX0pkZYyncReO+C1kCzZ/7oi1Q0zhOUkOihCNIhrUKwaZ+myfsopRe30ieMaapGPPWWz LtkBsV8Yh1ztRkwdNHBpbjQ4pMdBGCuX6LCNk82UerSY7SyCTNr7EY0HndXsq4Sca6ke SzcoVcc3CMC3N5XB1UYL6HRhtkYmJHcVNjBvybp3Z7Dh55hgSjP/AAJHWg4evyJ9Ghgz Klhg== X-Gm-Message-State: AOAM533xKRgu3HOgA3ferMWrGEyPofy5435HHLy5zWI+xB6R8VKjgWqj rejc6PzKsfy27vNjn9nktB+sRrPjl6Kfc+lA1GNjDw== X-Google-Smtp-Source: ABdhPJyt2R86MfvJOKl18sRXeH5vqN/UgE3OOP6++3snjuxTLntHdzYRzxuBoBm6IARu9orWbMDqL72UVWnlthx7f2w= X-Received: by 2002:a17:90a:eb0b:b0:1be:ddea:29ef with SMTP id j11-20020a17090aeb0b00b001beddea29efmr3604540pjz.126.1646155284124; Tue, 01 Mar 2022 09:21:24 -0800 (PST) MIME-Version: 1.0 References: <20220226002412.113819-1-shakeelb@google.com> In-Reply-To: From: Shakeel Butt Date: Tue, 1 Mar 2022 09:21:12 -0800 Message-ID: Subject: Re: [PATCH] memcg: async flush memcg stats from perf sensitive codepaths To: Michal Hocko Cc: =?UTF-8?Q?Michal_Koutn=C3=BD?= , Johannes Weiner , Roman Gushchin , Ivan Babrou , Andrew Morton , Cgroups , Linux MM , LKML , Daniel Dao , stable Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: BC1A140010 X-Stat-Signature: ho8jtabs4nysjby56ib4wyx6qu71obx7 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=fZDK1hgT; spf=pass (imf01.hostedemail.com: domain of shakeelb@google.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-HE-Tag: 1646155285-800223 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000018, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 1, 2022 at 1:05 AM Michal Hocko wrote: > > On Fri 25-02-22 16:24:12, Shakeel Butt wrote: > > Daniel Dao has reported [1] a regression on workloads that may trigger > > a lot of refaults (anon and file). The underlying issue is that flushing > > rstat is expensive. Although rstat flush are batched with (nr_cpus * > > MEMCG_BATCH) stat updates, it seems like there are workloads which > > genuinely do stat updates larger than batch value within short amount of > > time. Since the rstat flush can happen in the performance critical > > codepaths like page faults, such workload can suffer greatly. > > > > The easiest fix for now is for performance critical codepaths trigger > > the rstat flush asynchronously. This patch converts the refault codepath > > to use async rstat flush. In addition, this patch has premptively > > converted mem_cgroup_wb_stats and shrink_node to also use the async > > rstat flush as they may also similar performance regressions. > > Why do we need to trigger flushing in the first place from those paths. > Later in the thread you are saying there is a regular flushing done > every 2 seconds. What would happen if these paths didn't flush at all? > Also please note that WQ context can be overwhelmed by other work so > these flushes can happen much much later. > > So in other words why does async work (that can happen at any time > without any control) make more sense than no flushing? > -- Without flushing the worst that can happen in the refault path is false (or missed) activations of the refaulted page. For reclaim code, some heuristics (like deactivating active LRU or cache-trim) may act on old information. However I don't think these are too much concerning as the kernel can already missed or do false activations on refault. For the reclaim code, the kernel does force deactivation if it has skipped it in the initial iterations, so, not much to worry. Now, coming to your question, yes, we can remove the flushing from these performance critical codepaths as the stats at most will be 2 second old due to periodic flush. Now for the worst case scenario where that periodic flush (WQ) is not getting CPU, I think it is reasonable to put a sync flush if periodic flush has not happened for, let's say, 10 seconds.