From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D958C7EE29 for ; Tue, 23 May 2023 10:38:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B477900002; Tue, 23 May 2023 06:38:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 764B76B0075; Tue, 23 May 2023 06:38:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62C7D900002; Tue, 23 May 2023 06:38:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 504CF6B0074 for ; Tue, 23 May 2023 06:38:24 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1DAB81407AD for ; Tue, 23 May 2023 10:38:24 +0000 (UTC) X-FDA: 80821170528.07.35D6F5A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 51BFD1C0004 for ; Tue, 23 May 2023 10:38:22 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HkbPfWqj; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of brauner@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=brauner@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684838302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BmWYf0l1xX4KUypDdxO3IQSBKvoNjB0GpgI35P1puCE=; b=EU3MbeRniLJLOfGpw/brPEQjc2oVFbONMVvr24SDDWyJBGV43eoPbXAs4NcijY8cP+Ll1/ cynAguuvRBnK38DXQiMIJdYaI1HXmIQLkvc4eSraNKYds9QNvDHQTXhjFqY1CEee1zyL/t bp9y9jO8ZJgFDW/GyoQn97GMlsPRv6Y= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HkbPfWqj; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of brauner@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=brauner@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684838302; a=rsa-sha256; cv=none; b=XUZmt1ViEuRVgHUSkobMY/C9OIDhsE4N8GaCQVksO5LIPsWwYmhBql6ycrAKOx8jRm0HXE RIO2qjxl8bCixL6ccZtM7otSluwflMivKcwkGCxMtcFAwd/uw8OfuPLbv3XFsam58FOXwS DP/++y0YSbHVdBy7nhykleFj7LW6Py4= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 369BB62EC4; Tue, 23 May 2023 10:38:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 494EBC433D2; Tue, 23 May 2023 10:38:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684838300; bh=iccCfOHrskC0H4R3YMclbxF6J/FZ+5JAYqkJc7wrxX0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=HkbPfWqjMihgGRsu8YoEmd77NAWKV8nH70aM7Bd21uX6286VtZDB6cVtmOaHHtFNu 4wBSvOCc5KAfCQKZZn6M5T4YYMKfdDE72HkPYwwfVhRxchw2y158osOL/GlsyxPK/u Ij5K+u9pT1Lu/cwQf0Qyz2NMGOaPoHgufrvXrbSADcKafQJwkoAcIwy5zwnbsjg2Ph /M0cBnzrV4NeVcQ2Dsaybc2KXIlzl1N4g8doRumHeW0rV5iZ/kE5eH+sa7phuDUg5q Rhd1WMuZim5ovKWxHDwCf+3M5e81vP4rq2OLmfOyhQSgOHqbq/Fb8iBomvshVEMBtr pvWYXXjvWfxFA== Date: Tue, 23 May 2023 12:38:10 +0200 From: Christian Brauner To: Jan Kara Cc: Jeff Layton , Alexander Viro , "Darrick J. Wong" , Hugh Dickins , Andrew Morton , Dave Chinner , Chuck Lever , Amir Goldstein , David Howells , Neil Brown , Matthew Wilcox , Andreas Dilger , Theodore T'so , Chris Mason , Josef Bacik , David Sterba , Namjae Jeon , Steve French , Sergey Senozhatsky , Tom Talpey , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org Subject: Re: [PATCH v4 2/9] fs: add infrastructure for multigrain inode i_m/ctime Message-ID: <20230523-bekochen-makaber-6edbaa685390@brauner> References: <20230518114742.128950-1-jlayton@kernel.org> <20230518114742.128950-3-jlayton@kernel.org> <20230523100240.mgeu4y46friv7hau@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230523100240.mgeu4y46friv7hau@quack3> X-Rspamd-Queue-Id: 51BFD1C0004 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 1mi5x158tadkc8n5nhfif9bianrhusca X-HE-Tag: 1684838302-188062 X-HE-Meta: U2FsdGVkX19uKg36MvgKR6jKF2ZgZ4PhcOZkWJvJj390RlqJpjYZZpUE3Ix9do7yEuWttsQd/xcY4EEM2uC/zASIvo48uCOYu1t8A/i7EOerq1PevsAGsJZDUtMwIedGGC5UEmQjfC7bIZ17urpYIpYREvx95DpZMKbX1Or/1rbwWnIfMk3SoHEkjpeQ/x+r1TUrHuTrKvPg8YU1kH6AUj4UmZnhAyowfBH27cjVv1WaGuUCUP+ZV7SML7rQxa0fFkyu2zTv0Ix6heZqESTfC4gZl25jKGcyxIdBiJtnwnsj58Fjq9++DOBiyYbRjsbZGLwSYJNJYDjyql2P5lIhAbro8tzO0nTy5LlQ/FBuSlPDd57HAZEeLYIx4rSDb2NnLY8nhofNBZz/sSdd93+pzqSI4AwTRKS2Xv+Sb+fTuSRzaUVYj9ZlN7xLQBzI6zd8pyY8yFHAx4+dZTnrVeKnDG++zzpI645WO/1DYwBRbxqTLaX8JyM407YG07cET96DzHqksmjVxd7rE3Pt3rYvrKiRq06fgLvMtxHSw0hzT4cKzsux1FS04tVMvaer6dqbfmKLwZuEtaw1QTh2Gzb2DwB8WiCtVoPZVMlhy4KNBPCxoO9zsSd4pDtLoA0WZU0IH3Et6jAVi+ONTFQVk+ObHUgtmDyjQdI046CRtpa7HCWVMTMPGAINKfStdPbGRE7ZoIWs2b+NcGKjxovxQTYc0eo1vfUpGCDD/HzoE0upmyAvRKd/sEhEJjy2/0JYQkompbNWmaVyrttE/P8G38lIjAJR2ybzr5Avk/k3hA4qqxrC6vDi0A49MqYE+E08lWGPNJL4LNFtWaurKzT++lPN+XjNI3QFdw3PhuUvtkLHNAJR3RvWk0lQ9QAZ8zLI1bQKvVUOc4w0ucWRjIohy9h1FLtZ+pYNbsgwYD+IulVsuYOtYkoYWGLqaxC2wgTYpk457W+UAoFb2GJb6p49oMd nEYkt7+U xurLB/wXa9ENDf7YU1DkOIIJSoyKEd9eNaSJjielmahzPyRBXaHU9xMkNUMzyTi14bEVem0NFp4w/51xgjXsNwDhzM0Q7KrLEmvdHkCuP1TVljPb+TegAJIJwHioqZs8e9dcx9RoKTcGopxPFR1dobbiqa1GtCF/m3QJx6tcq/af7mbu+JBmEBtsPyWdRvNlqzJIO9dPt5CKCIKCVIScTYTlWB6pskwH98pnUu42Lw45H3L4M5bplxlNY0QW2/VvnHvKSwOaFSDpIhKXiW8qe6Z1Yr/MR0KRHomfQjzN6ZgfJQk1uKT4Ohc1GUXVxNg4NcDE6roDeY4bxI5D4Cq++t2UAh8rs6rmRYkQZH4qPAxPW4iFO7kdsG7PNws1dEO7eCUJtXGbW9a0K2/s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 23, 2023 at 12:02:40PM +0200, Jan Kara wrote: > On Thu 18-05-23 07:47:35, Jeff Layton wrote: > > The VFS always uses coarse-grained timestamp updates for filling out the > > ctime and mtime after a change. This has the benefit of allowing > > filesystems to optimize away a lot metadata updates, down to around 1 > > per jiffy, even when a file is under heavy writes. > > > > Unfortunately, this has always been an issue when we're exporting via > > NFSv3, which relies on timestamps to validate caches. Even with NFSv4, a > > lot of exported filesystems don't properly support a change attribute > > and are subject to the same problems with timestamp granularity. Other > > applications have similar issues (e.g backup applications). > > > > Switching to always using fine-grained timestamps would improve the > > situation, but that becomes rather expensive, as the underlying > > filesystem will have to log a lot more metadata updates. > > > > What we need is a way to only use fine-grained timestamps when they are > > being actively queried. > > > > The kernel always stores normalized ctime values, so only the first 30 > > bits of the tv_nsec field are ever used. Whenever the mtime changes, the > > ctime must also change. > > > > Use the 31st bit of the ctime tv_nsec field to indicate that something > > has queried the inode for the i_mtime or i_ctime. When this flag is set, > > on the next timestamp update, the kernel can fetch a fine-grained > > timestamp instead of the usual coarse-grained one. > > > > This patch adds the infrastructure this scheme. Filesytems can opt > > into it by setting the FS_MULTIGRAIN_TS flag in the fstype. > > > > Later patches will convert individual filesystems over to use it. > > > > Signed-off-by: Jeff Layton > > So there are two things I dislike about this series because I think they > are fragile: > > 1) If we have a filesystem supporting multigrain ts and someone > accidentally directly uses the value of inode->i_ctime, he can get bogus > value (with QUERIED flag). This mistake is very easy to do. So I think we > should rename i_ctime to something like __i_ctime and always use accessor > function for it. > > 2) As I already commented in a previous version of the series, the scheme > with just one flag for both ctime and mtime and flag getting cleared in > current_time() relies on the fact that filesystems always do an equivalent > of: > > inode->i_mtime = inode->i_ctime = current_time(); > > Otherwise we can do coarse grained update where we should have done a fine > grained one. Filesystems often update timestamps like this but not > universally. Grepping shows some instances where only inode->i_mtime is set > from current_time() e.g. in autofs or bfs. Again a mistake that is rather > easy to make and results in subtle issues. I think this would be also > nicely solved by renaming i_ctime to __i_ctime and using a function to set > ctime. Mtime could then be updated with inode->i_mtime = ctime_peek(). > > I understand this is quite some churn but a very mechanical one that could > be just done with Coccinelle and a few manual fixups. So IMHO it is worth > the more robust result. Yeah, these are all good points. > > Some more nits below. > > > +/** > > + * current_mg_time - Return FS time (possibly fine-grained) > > + * @inode: inode. > > + * > > + * Return the current time truncated to the time granularity supported by > > + * the fs, as suitable for a ctime/mtime change. If the ctime is flagged > > + * as having been QUERIED, get a fine-grained timestamp. > > + */ > > The comment should also mention that QUERIED flag is cleared from the ctime. > > > +static struct timespec64 current_mg_time(struct inode *inode) > > +{ > > + struct timespec64 now; > > + atomic_long_t *pnsec = (atomic_long_t *)&inode->i_ctime.tv_nsec; > > + long nsec = atomic_long_fetch_andnot(I_CTIME_QUERIED, pnsec); > > + > > + if (nsec & I_CTIME_QUERIED) { > > + ktime_get_real_ts64(&now); > > + } else { > > + struct timespec64 ctime; > > + > > + ktime_get_coarse_real_ts64(&now); > > + > > + /* > > + * If we've recently fetched a fine-grained timestamp > > + * then the coarse-grained one may still be earlier than the > > + * existing one. Just keep the existing ctime if so. > > + */ > > + ctime = ctime_peek(inode); > > + if (timespec64_compare(&ctime, &now) > 0) > > + now = ctime; > > + } > > + > > + return now; > > +} > > + > > ... > > > +/** > > + * ctime_nsec_peek - peek at (but don't query) the ctime tv_nsec field > > + * @inode: inode to fetch the ctime from > > + * > > + * Grab the current ctime tv_nsec field from the inode, mask off the > > + * I_CTIME_QUERIED flag and return it. This is mostly intended for use by > > + * internal consumers of the ctime that aren't concerned with ensuring a > > + * fine-grained update on the next change (e.g. when preparing to store > > + * the value in the backing store for later retrieval). > > + * > > + * This is safe to call regardless of whether the underlying filesystem > > + * is using multigrain timestamps. > > + */ > > +static inline long ctime_nsec_peek(const struct inode *inode) > > +{ > > + return inode->i_ctime.tv_nsec &~ I_CTIME_QUERIED; > > This is somewhat unusual spacing. I'd use: > > inode->i_ctime.tv_nsec & ~I_CTIME_QUERIED > > > +} > > + > > +/** > > + * ctime_peek - peek at (but don't query) the ctime > > + * @inode: inode to fetch the ctime from > > + * > > + * Grab the current ctime from the inode, sans I_CTIME_QUERIED flag. For > > + * use by internal consumers that don't require a fine-grained update on > > + * the next change. > > + * > > + * This is safe to call regardless of whether the underlying filesystem > > + * is using multigrain timestamps. > > + */ > > +static inline struct timespec64 ctime_peek(const struct inode *inode) > > +{ > > + struct timespec64 ctime; > > + > > + ctime.tv_sec = inode->i_ctime.tv_sec; > > + ctime.tv_nsec = ctime_nsec_peek(inode); > > + > > + return ctime; > > +} > > Given this is in a header that gets included in a lot of places, maybe we > should call it like inode_ctime_peek() or inode_ctime_get() to reduce > chances of a name clash? I think I mentioned this in an earlier comment. Independent of this series, it would be kinda nice if we could start moving stuff out of fs.h so we end up with a finer grained split of fs.h.