From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8139C7EE43 for ; Tue, 6 Jun 2023 22:30:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 336BB6B0071; Tue, 6 Jun 2023 18:30:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BFC48E0002; Tue, 6 Jun 2023 18:30:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 138C58E0001; Tue, 6 Jun 2023 18:30:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F10A56B0071 for ; Tue, 6 Jun 2023 18:30:19 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B815EAF441 for ; Tue, 6 Jun 2023 22:30:19 +0000 (UTC) X-FDA: 80873767758.23.0D56EC6 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf07.hostedemail.com (Postfix) with ESMTP id AEADB4000F for ; Tue, 6 Jun 2023 22:30:17 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=fromorbit-com.20221208.gappssmtp.com header.s=20221208 header.b=tRbk3F0e; spf=pass (imf07.hostedemail.com: domain of david@fromorbit.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686090617; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iSnLlqw5JsJTFwRAjaS9oa9cX/ConG9itjHh20/fwu8=; b=J7qDrUGzccgpwsICYsdLeRwki0RRBQtPx0pDvns7v8KK9Exb3IR5QW1WBzMn4dUjoIBPyi DA6wOOu07cLrwi2wjLifxqQpdzI66szLaDAEgSLkQBSLrsrI/SYc7W1S5ExZBDfh1T3QJF cqK+PH+KfhCjx3opCvW7aNLC8TeG9DI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686090617; a=rsa-sha256; cv=none; b=L3aeMonYGEPnY1aj0VRH3GVO7rHWZkEhTYZat0mBdpzQLz/JkYo4gLIiqv4x3Ia0V9JDxs 1a5gVNvST3XlaptbrQdFwiFGn4+qm04SNFKuOlDmAPg96opeVl1QSyTxVdmSziX6MpLQXR HY8pQzZcvRMmQ+C9v4DSqhxFObcLcJk= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=fromorbit-com.20221208.gappssmtp.com header.s=20221208 header.b=tRbk3F0e; spf=pass (imf07.hostedemail.com: domain of david@fromorbit.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-256932cea7aso3187872a91.3 for ; Tue, 06 Jun 2023 15:30:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20221208.gappssmtp.com; s=20221208; t=1686090616; x=1688682616; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=iSnLlqw5JsJTFwRAjaS9oa9cX/ConG9itjHh20/fwu8=; b=tRbk3F0e2NEfHzwsFKlsj7+W8XnJxSTq0qUj/yTV4f0BBC+EQul40bTO1JmyH5XOpO 2MwHPZmHnAV3TEvoSSrLkpYy/Br7CIQ9IrmMtQgr64AfUtdAM+NV5BW98YDb2EnPwFVy +YRPfrCjeEJIJMyCOOgtFUVbkY+Fp+2eyLH2Ee2WZoL5fqy6xuy6hwdW9gGV0j8RTNiJ M0qvAwyg0FP5CZyxhVS/hwJDCN8FYTyjFJfEl6eGPQopEG/Cdz0TtoJYJmrM91ncyloD hvIemNkR84xMOQB6BVUx3MWfvoCUqBByNNcr47OaIlR+a4sufD3Vw1u98Y99egFiBVG7 rgiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686090616; x=1688682616; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=iSnLlqw5JsJTFwRAjaS9oa9cX/ConG9itjHh20/fwu8=; b=enC/Kaz3Klye+Yt2bNI1jAyEXi7dnGhtP8d07jk5WUzG1iG2cfpLlmI9OSufpWP+0e xYQFL/gtQLak9IP1i4uGJ0uhbjg5Ovb7B2o7KQzJ5AWqRfOKuApDCPJJfdj5xOABJmUw xU8BDvvlIkwDaeDSbwo78F/Qkp6KnyEu7EIsj4VoBqO2EWg5ZLg4/dC5vT9vRxKDyf7C ZkWzNK1XIwCgllP7nM2BCWr16MQJ98/gb71D853CoeRtRH53dDUn02yCTRqV6StJNxLd CLuFX7jtXv9+Q99FCNWl2Bqcm2t3/gxptIJ8qCAExY6Xy+8LtRQvVMJVhyk8E3y3m4cb i0Eg== X-Gm-Message-State: AC+VfDz588uTOmF83MTmnR/a6Db0/06Ic5njfhYHX+9IF319PDOyuGFn nrEFgmqfG10qzshIe/gzM8aKXw== X-Google-Smtp-Source: ACHHUZ4rURXFwCxvuvdkNt2OeJTIkA+TKnI42M3OuN0Lh/vTVsVNzkPvYww20WmLoUsPE1K/XOiANQ== X-Received: by 2002:a17:90b:68c:b0:259:3cc4:f978 with SMTP id m12-20020a17090b068c00b002593cc4f978mr2167413pjz.19.1686090616399; Tue, 06 Jun 2023 15:30:16 -0700 (PDT) Received: from dread.disaster.area (pa49-179-79-151.pa.nsw.optusnet.com.au. [49.179.79.151]) by smtp.gmail.com with ESMTPSA id p11-20020a17090a284b00b0025621aa4c6bsm48148pjf.25.2023.06.06.15.30.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 15:30:15 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1q6fBt-008eth-0l; Wed, 07 Jun 2023 08:30:13 +1000 Date: Wed, 7 Jun 2023 08:30:13 +1000 From: Dave Chinner To: Kirill Tkhai Cc: Roman Gushchin , akpm@linux-foundation.org, vbabka@suse.cz, viro@zeniv.linux.org.uk, brauner@kernel.org, djwong@kernel.org, hughd@google.com, paulmck@kernel.org, muchun.song@linux.dev, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, zhengqi.arch@bytedance.com Subject: Re: [PATCH v2 3/3] fs: Use delayed shrinker unregistration Message-ID: References: <168599103578.70911.9402374667983518835.stgit@pro.pro> <168599180526.70911.14606767590861123431.stgit@pro.pro> <65785745-1fd3-e0d7-26e8-dd74b1074d37@ya.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <65785745-1fd3-e0d7-26e8-dd74b1074d37@ya.ru> X-Rspamd-Queue-Id: AEADB4000F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jeij61s7rieeqw8kr5wwbtg8dfy869g7 X-HE-Tag: 1686090617-856832 X-HE-Meta: U2FsdGVkX1/HqbpHwlIfIkzuLQPJ+bqolUElPkmsUe7sr9cm5nf1qMpl0BuaHBBtQdVyvGhAgLU1dpAVti/ACfSgYBGz3N0vTS3uArzhBPJiMstcMVv5tg3IZa5Vr1DsfOGLh6OjOPey7YFVhoHz2DZlBwMn74uhc4Bl/BO6PHVcGd/icBu/r3oimtyai+g8dN0phGxH4Ee+apYTdUtzRqaDMFURdPIvCmJP8SNsymIKHdoyjZuUAR0SjOpApOe6tidI7vDBJ6qp/6h3IRiPK6tiM/tm0iqVCXDkpzA8R4tnZVEl1y3EVIqzh7jZBHusIsqF+pAdM7K67JaeCFLOY5pfZKpLF1KCCs4EJiTBYWBIJyj4GlgAF4jNdiijPEFWBCf903GB0aW2yyL+twMzk3D3K0VeHVOzZUmvzqCt4k0VcNJr5kJAiUxPz4wVz1E8brYuzapTtW9FGE98KlXvisn3nWf4jJ7/r32f6xnXlavo5LnlyqqCbmjE+Duy8JWFMHsJzj/W9XWPrrxPdpfq+VpGY63jeVRKJp1REGKHuxUlzvZkm43BQ/p+tGSdiv1uFsx4qmxrlbU+f44FxtMtPyypryVNmJvCIvy4+mq9UcT7zYSyB1EunQ4mXF4bf73M73ttaYwV9BNxn494Xynfxu9Ur+Nh3LbvsK5BTa/au85W8/liU3TzLMfFNl65R/fjLH3TxqayUAAfx+lMbquKlMQhO2guAHHkiqCqdToP/TihVjiLKOOOroCv4VyjbqUT3GBtQihtRalmQC26yr3gc4/pJ6lo/lM/Y5ewYlP8qlVNJvNAaF2l1BTjvIW8TIiy8rbYYTjEClHtzSIRzxzIsYBNt8VGmbRCQILNXKY8AetS2iXNFEfHbo3JxUe3oH/Zvp2pJRH/EaH3dDd/G23y7eb+1FktLXyOKqKgp3HWPDrQnoTNdsO73TWpVYcrkBMqifJugqY4hQAd562I7lc TmPYu4cQ /Z+1cRuM67OUMWtkxJ7J/jeqYRAhnjDpQeVhgn2shtF9bm6AFFgPlYlJRMA4GrtNcxT+3rcsXGpp0nidX0kygzK0Rfz+KBqMTfd4OqzpLI+kQA9WGpzu8MFyCZTf23d56MahKc/A1A686uHAIsUJMBf4+eYNqIvRSZ5G3fT5WiaVxaMm85pYNZQkGBLn73+wrQQN6Q0d4LpidHA6MJ4CjhANdjoprj2BbpIOmW40qsWE9aDKyPJRA8cdCKJCTgDMCh15F8j/0w3lp93jbiP0xpN4v97lm8KkR81QlPTw3dbvEo+wBBtq4KbcWffV+GpncPyL7Z+ks48Ggbd4Qg8vuv7hbhWg6/COczZoxM3lrnutzAx7WtD3iJSdRIIevyStw4+MkIZ8mArKeSoTvcmQv+nb1CgWHpEoQRAMT4lJdpGJ2e6LTvd405Lumq7Fxzh1Ei521oX1oBLScf324HeO6JPbWlVFuMXYXiMzPMR/gGYssOc4mBN+IHxlUGIjYJXcMuJA4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jun 07, 2023 at 12:21:42AM +0300, Kirill Tkhai wrote: > On 06.06.2023 04:24, Dave Chinner wrote: > > On Mon, Jun 05, 2023 at 05:38:27PM -0700, Roman Gushchin wrote: > >> On Mon, Jun 05, 2023 at 10:03:25PM +0300, Kirill Tkhai wrote: > >>> Kernel test robot reports -88.8% regression in stress-ng.ramfs.ops_per_sec > >>> test case caused by commit: f95bdb700bc6 ("mm: vmscan: make global slab > >>> shrink lockless"). Qi Zheng investigated that the reason is in long SRCU's > >>> synchronize_srcu() occuring in unregister_shrinker(). > >>> > >>> This patch fixes the problem by using new unregistration interfaces, > >>> which split unregister_shrinker() in two parts. First part actually only > >>> notifies shrinker subsystem about the fact of unregistration and it prevents > >>> future shrinker methods calls. The second part completes the unregistration > >>> and it insures, that struct shrinker is not used during shrinker chain > >>> iteration anymore, so shrinker memory may be freed. Since the long second > >>> part is called from delayed work asynchronously, it hides synchronize_srcu() > >>> delay from a user. > >>> > >>> Signed-off-by: Kirill Tkhai > >>> --- > >>> fs/super.c | 3 ++- > >>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>> > >>> diff --git a/fs/super.c b/fs/super.c > >>> index 8d8d68799b34..f3e4f205ec79 100644 > >>> --- a/fs/super.c > >>> +++ b/fs/super.c > >>> @@ -159,6 +159,7 @@ static void destroy_super_work(struct work_struct *work) > >>> destroy_work); > >>> int i; > >>> > >>> + unregister_shrinker_delayed_finalize(&s->s_shrink); > >>> for (i = 0; i < SB_FREEZE_LEVELS; i++) > >>> percpu_free_rwsem(&s->s_writers.rw_sem[i]); > >>> kfree(s); > >>> @@ -327,7 +328,7 @@ void deactivate_locked_super(struct super_block *s) > >>> { > >>> struct file_system_type *fs = s->s_type; > >>> if (atomic_dec_and_test(&s->s_active)) { > >>> - unregister_shrinker(&s->s_shrink); > >>> + unregister_shrinker_delayed_initiate(&s->s_shrink); > >> > >> Hm, it makes the API more complex and easier to mess with. Like what will happen > >> if the second part is never called? Or it's called without the first part being > >> called first? > > > > Bad things. > > > > Also, it doesn't fix the three other unregister_shrinker() calls in > > the XFS unmount path, nor the three in the ext4/mbcache/jbd2 unmount > > path. > > > > Those are just some of the unregister_shrinker() calls that have > > dynamic contexts that would also need this same fix; I haven't > > audited the 3 dozen other unregister_shrinker() calls around the > > kernel to determine if any of them need similar treatment, too. > > > > IOWs, this patchset is purely a band-aid to fix the reported > > regression, not an actual fix for the underlying problems caused by > > moving the shrinker infrastructure to SRCU protection. This is why > > I really want the SRCU changeover reverted. > > > > Not only are the significant changes the API being necessary, it's > > put the entire shrinker paths under a SRCU critical section. AIUI, > > this means while the shrinkers are running the RCU grace period > > cannot expire and no RCU freed memory will actually get freed until > > the srcu read lock is dropped by the shrinker. > > Why so? Doesn't SRCU and RCU have different grace period and they does not prolong > each other? No idea - Documentation/RCU/whatisRCU.rst doesn't describe any differences between SRCU and RCU except for "use SRCU if you need to sleep in the read side" and there's no discussion of how they interact, either. maybe there's some discussion in other RCU documentation, but there's nothing in the "how to use RCU" documentation that tells me they use different grace period definitions... > Also, it looks like every SRCU has it's own namespace like shrinker_srcu for shrinker. > Don't different SRCU namespaces never prolong each other?! RIght, SRCU vs SRCU is well defined. What is not clear from anything I've read is SRCU vs RCU interactions, so I can only assuming from the shared "RCU" in the name there are shared implementation details and interactions... Cheers, Dave. -- Dave Chinner david@fromorbit.com