From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDC6AC433EF for ; Fri, 4 Mar 2022 03:26:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7A428D0002; Thu, 3 Mar 2022 22:26:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D285D8D0001; Thu, 3 Mar 2022 22:26:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF03F8D0002; Thu, 3 Mar 2022 22:26:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id B0D268D0001 for ; Thu, 3 Mar 2022 22:26:49 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3CE23181C5573 for ; Fri, 4 Mar 2022 03:26:49 +0000 (UTC) X-FDA: 79205266938.22.FDB9E0E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf12.hostedemail.com (Postfix) with ESMTP id E468640110 for ; Fri, 4 Mar 2022 03:26:31 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B8EE1218D9; Fri, 4 Mar 2022 02:47:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1646362056; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bRC9lR391e01DA8hmQQ11oCmTpgoNtai0DhkKiv/gqA=; b=DPBLhpCy8tSefU0o/0cGFUNHFB2ENiUdz7DL+/QuL66B+hanK6/YVNBgksSwMuDLV2K7do gLidl4GftwWDduPicJm1Lt/WdYmoepv57PKKCdtFCC0ehwriXODLDFTWOGNFVdmoGAplJO DhmYXWeTCsa6asBh2ztOs4cXy4qTCUw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1646362056; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bRC9lR391e01DA8hmQQ11oCmTpgoNtai0DhkKiv/gqA=; b=OC3LPp6wdT68pzKK+cC/1gJqegadfEfURfONkeOYiYNj+tndz1PDCoyP7hDil8izMoBJdD 3yON2S4St8a2wPDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AA36313AF7; Fri, 4 Mar 2022 02:47:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id dPl5GcF9IWJxAgAAMHmgww (envelope-from ); Fri, 04 Mar 2022 02:47:29 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 From: "NeilBrown" To: "Jeff Layton" Cc: "Andrew Morton" , "Jan Kara" , "Wu Fengguang" , "Jaegeuk Kim" , "Chao Yu" , "Ilya Dryomov" , "Miklos Szeredi" , "Trond Myklebust" , "Anna Schumaker" , "Ryusuke Konishi" , "Darrick J. Wong" , "Philipp Reisner" , "Lars Ellenberg" , "Paolo Valente" , "Jens Axboe" , linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-nilfs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, ceph-devel@vger.kernel.org, drbd-dev@lists.linbit.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH 06/11] ceph: remove reliance on bdi congestion In-reply-to: References: <164549971112.9187.16871723439770288255.stgit@noble.brown>, <164549983739.9187.14895675781408171186.stgit@noble.brown>, , <164568131640.25116.884631856219777713@noble.neil.brown.name>, Date: Fri, 04 Mar 2022 13:47:26 +1100 Message-id: <164636204663.29369.1845040729675190216@noble.neil.brown.name> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E468640110 X-Stat-Signature: twtyu836p7rcw4jfds15jzc1u93kkq48 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=DPBLhpCy; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=OC3LPp6w; spf=pass (imf12.hostedemail.com: domain of neilb@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=neilb@suse.de; dmarc=pass (policy=none) header.from=suse.de X-Rspam-User: X-HE-Tag: 1646364391-419833 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 24 Feb 2022, Jeff Layton wrote: > On Thu, 2022-02-24 at 16:41 +1100, NeilBrown wrote: > > On Thu, 24 Feb 2022, Jeff Layton wrote: > > > On Tue, 2022-02-22 at 14:17 +1100, NeilBrown wrote: > > > > The bdi congestion tracking in not widely used and will be removed. > > > >=20 > > > > CEPHfs is one of a small number of filesystems that uses it, setting > > > > just the async (write) congestion flags at what it determines are > > > > appropriate times. > > > >=20 > > > > The only remaining effect of the async flag is to cause (some) > > > > WB_SYNC_NONE writes to be skipped. > > > >=20 > > > > So instead of setting the flag, set an internal flag and change: > > > > - .writepages to do nothing if WB_SYNC_NONE and the flag is set > > > > - .writepage to return AOP_WRITEPAGE_ACTIVATE if WB_SYNC_NONE > > > > and the flag is set. > > > >=20 > > > > The writepages change causes a behavioural change in that pageout() c= an > > > > now return PAGE_ACTIVATE instead of PAGE_KEEP, so SetPageActive() will > > > > be called on the page which (I think) wil further delay the next atte= mpt > > > > at writeout. This might be a good thing. > > > >=20 > > > > Signed-off-by: NeilBrown > > >=20 > > > Maybe. I have to wonder whether all of this is really useful. > > >=20 > > > When things are congested we'll avoid trying to issue new writeback > > > requests. Note that we don't prevent new pages from being dirtied here - > > > - only their being written back. > > >=20 > > > This also doesn't do anything in the DIO or sync_write cases, so if we > > > lose caps or are doing DIO, we'll just keep churning out "unlimited" > > > writes in those cases anyway. > >=20 > > I think the point of congestion tracking is to differentiate between > > sync and async IO. Or maybe "required" and "optional". > > Eventually the "optional" IO will become required, but if we can delay > > it until a time when there is less "required" io, then maybe we can > > improve perceived latency. > >=20 > > "optional" IO here is write-back and read-ahead. If the load of > > "required" IO is bursty, and if we can shuffle that optional stuff into > > the quiet periods, we might win. > >=20 >=20 > In that case, maybe we should be counting in-flight reads too and deny > readahead when the count crosses some threshold? It seems a bit silly to > only look at writes when it comes to "congestion". I agree that seems a bit silly. >=20 > > Whether this is a real need is an important question that I don't have an > > answer for. And whether it is better to leave delayed requests in the > > page cache, or in the low-level queue with sync requests able to > > over-take them - I don't know. If you have multiple low-level queue as > > you say you can with ceph, then lower might be better. > >=20 > > The block layer has REQ_RAHEAD .. maybe those request get should get a > > lower priority ... though I don't think they do. > > NFS has a 3 level priority queue, with write-back going at a lower > > priority ... I think... for NFSv3 at least. > >=20 > > Sometimes I suspect that as all our transports have become faster, we > > have been able to ignore the extra latency caused by poor scheduling of > > optional requests. But at other times when my recently upgraded desktop > > is struggling to view a web page while compiling a kernel ... I wonder > > if maybe we don't have the balance right any more. > >=20 > > So maybe you are right - maybe we can rip all this stuff out. > >=20 >=20 > I lean more toward just removing it. The existing implementation seems a > bit half-baked with the gaps in what's being counted. Granted, the > default congestion threshold is pretty high with modern memory sizes, so > it probably doesn't come into play much in practice, but removing it > would reduce some complexity in the client. I'd love to have some test that could reliably generate congestion and measure latencies for other IO. Without that, it is mostly guess work. So I cannot argue against your proposal, and do agree that removing the code would reduce complexity. I have no idea what the costs might be - if any. Hence my focus was on not changing behaviour. Thanks, NeilBrown