From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A290DC77B75 for ; Tue, 23 May 2023 10:17:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01BCD900002; Tue, 23 May 2023 06:17:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0DBA6B0075; Tue, 23 May 2023 06:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD6D6900002; Tue, 23 May 2023 06:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CB2786B0074 for ; Tue, 23 May 2023 06:17:27 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A4A4EADD97 for ; Tue, 23 May 2023 10:17:27 +0000 (UTC) X-FDA: 80821117734.23.37E89D5 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf11.hostedemail.com (Postfix) with ESMTP id 6D05E4000D for ; Tue, 23 May 2023 10:17:25 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yJZJT1Og; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=smBRmOb0; dmarc=none; spf=pass (imf11.hostedemail.com: domain of jack@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684837045; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pJearGp7XvoKQH8r+JXuFUaSaDT86GZEmHPZFyporzw=; b=INSaDHvqZBXUkQDjxFkvTZ6Pbffqob1XeBGIpyVHkNGFiv6yupgojeybIGBUvOo7e6ayAI GTf60ht5qIPtJBsgfIEqQbt1BskhKUqFX78J1oiZUAzTNO2VyRZz1OE6rFX3+U5pPEZJdW NVrsbushurr7JND+8LoPL8BQctwYcxM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yJZJT1Og; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=smBRmOb0; dmarc=none; spf=pass (imf11.hostedemail.com: domain of jack@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684837045; a=rsa-sha256; cv=none; b=r1btmIiBvd2fdeFLs0FGGUj7N4yihwxtiKgzizajieNjS0xUDV5PjIMO9MHIfW5v1L5s7z qk6zc/gnnOahD9yuqNSQspQHHl8DbUGzglVzcrxSVN7a4UhUGyW3Olj97Hn2HtxeEXmF7b /LizFn1ds7artQhw0TItWE5RyiBKyA0= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DBA97204B2; Tue, 23 May 2023 10:17:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1684837043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pJearGp7XvoKQH8r+JXuFUaSaDT86GZEmHPZFyporzw=; b=yJZJT1OgORi5RtyIixB/rOLuR+h4YOf/eNgFYXfGwJwrUNG0IuaqlOt1xJwUh2rdfYfa3z TRq3yl9IzwjdiXabcxYBb7c/VkGt/gWrkXIOV6zWQqw/TlF8q9NzxPsA6vuoHaKhRfDVsx 7dDUe7LfbfgHwItCuI47Lu50P+j1AYg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1684837043; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pJearGp7XvoKQH8r+JXuFUaSaDT86GZEmHPZFyporzw=; b=smBRmOb08zACD6WGFlZMUpHyGypof+tbliDQZkhMAqleKrtZ7ouMKaKN/V4ftcEBe+q1Hl HJ7kYRewaUgDqADg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C61E113588; Tue, 23 May 2023 10:17:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EVhVMLOSbGQlbAAAMHmgww (envelope-from ); Tue, 23 May 2023 10:17:23 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 4F79CA075D; Tue, 23 May 2023 12:17:23 +0200 (CEST) Date: Tue, 23 May 2023 12:17:23 +0200 From: Jan Kara To: Jeff Layton Cc: Alexander Viro , Christian Brauner , "Darrick J. Wong" , Hugh Dickins , Andrew Morton , Dave Chinner , Chuck Lever , Jan Kara , Amir Goldstein , David Howells , Neil Brown , Matthew Wilcox , Andreas Dilger , Theodore T'so , Chris Mason , Josef Bacik , David Sterba , Namjae Jeon , Steve French , Sergey Senozhatsky , Tom Talpey , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org Subject: Re: [PATCH v4 2/9] fs: add infrastructure for multigrain inode i_m/ctime Message-ID: <20230523101723.xmy7mylbczhki6aa@quack3> References: <20230518114742.128950-1-jlayton@kernel.org> <20230518114742.128950-3-jlayton@kernel.org> <20230523100240.mgeu4y46friv7hau@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230523100240.mgeu4y46friv7hau@quack3> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6D05E4000D X-Stat-Signature: fxky6ihykzgg6nnpw8y9g3tcuwmfw8up X-HE-Tag: 1684837045-140067 X-HE-Meta: U2FsdGVkX19dQhMrkEnxLCfrvf6/uKwsaB6wkooJ2X7BPHQifY9deXGaKlRlepjuwzgLxKlpjPeJnoRIP18LAEeNSlz4x7MWeoiTYDRl0QfgR23gfIYplz77SZmauqK3sjBKTTiEVu9k5H4sJfwyJOzaKK8VjPAxWrzPatxeECjCP87YQ83aK8jW0675EEYTC2lhKRf4w0YxBNZY38QjWeRnYOky/w7+1d/xQ45x/SdJy16AUeFDImNK05+0S0FmRBbT5Jprv5ViuLVJKxgGQ11PUcTb84JbN5AAViQGxd8E+5V9mU51k+k0BszXdrM/EPCCiI6dLrxAoCK6WGwi8X9yAzMt13lnRoBD/jakrsdfR7idlqJt8Nf8EoVGdKWkmvCglDUefJJiM9if9vXaHd1ZwHAKf1Ig7K7BfXj4CqLOQk7FODL8i6BXae99jnkpjqpaHJIdN1CUd5yGi4TRmVjB1H6BPbm80JHOqSAEsvLH23a4hYl6dcvRJV2NZQl/3Vti6uK1Z06Gj0hnanA3A3rpLAEU4obMj8wkcnyr+a842FAHw7YH15lDeTfb1wyz6HBrsvgi5NcXoxEwnH9JuhsPagMb7Mp8Fl/2US2bwosEkVzRRax1Feu5cOlWpkmDrhcSGqx9FdU8zmuS28StQ6KYtgjDyX7eQdv55/FBT5Mo4nizaCV6t0pdrfHWUCmccwwMGtEnxmixMyoPDoyBJ62MD6KKq50HjjmJPHmzsO2PQnZqSqq6MfITXGzbcVgwRdAhW7Z1r3HKizlZJHRdpfesBMP9oZYG6YEoqDWguPkElO0DCgJ1lNhviCt44CUHSz1VwTa2sZiUZO14xjDd2c2+1RsoCxXgYFcXvqbA46CkFKoYv5a8AliBxlOBDlzrfpXyyZtR72Xv2e3msK2U4UV3iRqzJq0egdB1swYvki1IJEAtqwPtayxoeMyA1HHCYnUnw1Bu8CSd/gyDP+F 9beclnOh 0f4sfSPBaTa+nJ4tKgnGYh8EqCaZDz76S+51SpwmSn5SAbUrSQ3jEpzMxj15lG2OX0FfUWh4lvxCUBloE6zjwJsJ7eTFdrZkipgoC38weiD1TTaYYprnDVEpio3lHOMrNTYfkSaHgfLPrL0E09XFtDfaJA4NTOPiZedxUxmyUWDthp3JZZTj5sSXX7QtjD2+1eu1GN2qRBXKmokBjVQWyu23Oard8hwhLx3Hlm3qnGJJGAnNdwGrCgiy3C5c/zCUhRCCIe1xGnlt4BxtzerVPYeTcQcfgvJ2EsEVcnaSyCZECmq/5S/utTgUqeuj2yUYeoj3oL/y0ylXbq7mm6m/ycOLwHpPRtZhjVi73oD6LBBwnJZw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 23-05-23 12:02:40, Jan Kara wrote: > On Thu 18-05-23 07:47:35, Jeff Layton wrote: > > The VFS always uses coarse-grained timestamp updates for filling out the > > ctime and mtime after a change. This has the benefit of allowing > > filesystems to optimize away a lot metadata updates, down to around 1 > > per jiffy, even when a file is under heavy writes. > > > > Unfortunately, this has always been an issue when we're exporting via > > NFSv3, which relies on timestamps to validate caches. Even with NFSv4, a > > lot of exported filesystems don't properly support a change attribute > > and are subject to the same problems with timestamp granularity. Other > > applications have similar issues (e.g backup applications). > > > > Switching to always using fine-grained timestamps would improve the > > situation, but that becomes rather expensive, as the underlying > > filesystem will have to log a lot more metadata updates. > > > > What we need is a way to only use fine-grained timestamps when they are > > being actively queried. > > > > The kernel always stores normalized ctime values, so only the first 30 > > bits of the tv_nsec field are ever used. Whenever the mtime changes, the > > ctime must also change. > > > > Use the 31st bit of the ctime tv_nsec field to indicate that something > > has queried the inode for the i_mtime or i_ctime. When this flag is set, > > on the next timestamp update, the kernel can fetch a fine-grained > > timestamp instead of the usual coarse-grained one. > > > > This patch adds the infrastructure this scheme. Filesytems can opt > > into it by setting the FS_MULTIGRAIN_TS flag in the fstype. > > > > Later patches will convert individual filesystems over to use it. > > > > Signed-off-by: Jeff Layton > > So there are two things I dislike about this series because I think they > are fragile: > > 1) If we have a filesystem supporting multigrain ts and someone > accidentally directly uses the value of inode->i_ctime, he can get bogus > value (with QUERIED flag). This mistake is very easy to do. So I think we > should rename i_ctime to something like __i_ctime and always use accessor > function for it. > > 2) As I already commented in a previous version of the series, the scheme > with just one flag for both ctime and mtime and flag getting cleared in > current_time() relies on the fact that filesystems always do an equivalent > of: > > inode->i_mtime = inode->i_ctime = current_time(); > > Otherwise we can do coarse grained update where we should have done a fine > grained one. Filesystems often update timestamps like this but not > universally. Grepping shows some instances where only inode->i_mtime is set > from current_time() e.g. in autofs or bfs. Again a mistake that is rather > easy to make and results in subtle issues. I think this would be also > nicely solved by renaming i_ctime to __i_ctime and using a function to set > ctime. Mtime could then be updated with inode->i_mtime = ctime_peek(). > > I understand this is quite some churn but a very mechanical one that could > be just done with Coccinelle and a few manual fixups. So IMHO it is worth > the more robust result. Also as I'm thinking about it your current scheme is slightly racy. Suppose the filesystem does: CPU1 CPU2 statx() inode->i_ctime = current_time() current_mg_time() nsec = atomic_long_fetch_andnot(QUERIED, &inode->i_ctime.tv_nsec) nsec = atomic_long_fetch_or(QUERIED, &inode->i_ctime.tv_nsec) if (nsec & QUERIED) - not set ktime_get_coarse_real_ts64(&now) return timestamp_truncate(now, inode); - QUERIED flag in the inode->i_ctime gets overwritten by the assignment => we need not update ctime due to granularity although it was queried One more reason to use explicit function to update inode->i_ctime ;) Honza -- Jan Kara SUSE Labs, CR