From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 195E5C4345F for ; Wed, 1 May 2024 09:27:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A831F6B008A; Wed, 1 May 2024 05:27:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0C7D6B0098; Wed, 1 May 2024 05:27:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 886B46B00B9; Wed, 1 May 2024 05:27:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 664AD6B008A for ; Wed, 1 May 2024 05:27:43 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 089338050C for ; Wed, 1 May 2024 09:27:43 +0000 (UTC) X-FDA: 82069299606.24.1E0F28A Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf28.hostedemail.com (Postfix) with ESMTP id 05FE7C0027 for ; Wed, 1 May 2024 09:27:40 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=IgeIOn+C; spf=pass (imf28.hostedemail.com: domain of david@fromorbit.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714555661; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jA6/54rvNPCCZDDxvJHTB+TObNeo0PCUCsJwkWJa9LE=; b=GEdWEV+g40S27pN0TZRrClrCZUy4yj2UxOD3yiks5luqE5oaw09dDIq0iTmErfGhZzuaYB 7Nl00kiU6vC9wgaCS4xx5gbn0NmRSpaCeWIq7ncvvrLSWw4VqjHNTtbrvEIA3EGu2/no8s a3HCE03vVnxWMMOSf/zl3KkAYrVAFTo= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=IgeIOn+C; spf=pass (imf28.hostedemail.com: domain of david@fromorbit.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714555661; a=rsa-sha256; cv=none; b=rnDFVr78kATZq4kc7vACpHmEWWsY23b/2Dgn0ripwedRC55eVRfgF9UzrZRnop195/9oYQ qnTqRQnjW0IDQsiRNPwTWAXPYxiZSfBD5xNt2QEzXzuMkrV112Yjy2vYj934qFO9mM0JkF LyOKN5lF6SqZ/i+YtgnbJyWY/0bt8mg= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1e86d56b3bcso60467375ad.1 for ; Wed, 01 May 2024 02:27:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1714555660; x=1715160460; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jA6/54rvNPCCZDDxvJHTB+TObNeo0PCUCsJwkWJa9LE=; b=IgeIOn+CMjoov01xBHpHZbiUSe0ayw9FC5WEp0trPlwyIpUPfug8i2onIIoLppVdjq b6AcMYs/IHV9okBIrWryvSnPxVgsU6zhnc6ZgzraKnUVZVIZL29cywQCHO9O4vBZbEtW k20aB1Ij5zoNuqxDfjU345/CWtUAqrY/y0HoGsmGLHMkHRwaUgSpGhetmWZL7PxnS/5v +rEKJ2hpskpIL69kvS+vuzBej9A4CKLxTmoytvYsi3nF8mSh4L7CmbrVqKM2j1bip7ZJ BYOL45dickd14okeMxKfULxz7V7ra/g+iS3QmRN6F6o6zQJL+CF1N3rj3IPGU6vTEC96 dTsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714555660; x=1715160460; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jA6/54rvNPCCZDDxvJHTB+TObNeo0PCUCsJwkWJa9LE=; b=HDPIxSEE84yRoQ0Nuh75SvARV+rhtvRL5bs5zUZ0Q76qZJMvWUqpexj/kiM7hq03LH ixTd0o6eWW4EkyMlLjYigJJBDRCEPKWW7T6/nzZf1Od2O8Lr5mDNjzSMVcS6qrwPtQP1 Aoce2Eao99sr9TTSOZuH6zDPK/EFQsqKTrhUgxNMAVsyOTveqG8fuidMLl6o3MByOc/i 3XdSw/uyTQWi9C7ku92LPP2BCoLddU1uD+AB1hBW62tVcQln+BMklgsmkbCac+nAL8iJ kNWmoH32mxlEeoC4MoFUk63z0NYu3B0/wVMmpcNLGg1/UyLo3jYDz+VoTe60nVW4XNEb nwQQ== X-Forwarded-Encrypted: i=1; AJvYcCVk8ndCtBsmznkj2IyINAJPZVURMRpAaqOWOS+cPp6/8AbxP0x+qsDhqYGbCc7NpvH2cFWnMs1Xwurn3Z0voxGIz28= X-Gm-Message-State: AOJu0Ywvwnh/16XQALCQ+5kZXt+hN7mEpM97ypBOKuyW2wWn307u3vNB Kiv7KzE5qaPIZzig9dmWzEo7Q28YcYgIQizOAUCyX/uLr5z8zsHviYLvAAd8oJk= X-Google-Smtp-Source: AGHT+IFu0jN+KODiUpxx3eJcKXu/sUXR0fU20jgfQUgpRPXKLy/xtaPE3yAAScbeG4/pjwQ1tWY9yw== X-Received: by 2002:a17:903:41cd:b0:1ea:c52d:e03e with SMTP id u13-20020a17090341cd00b001eac52de03emr2187607ple.13.1714555659445; Wed, 01 May 2024 02:27:39 -0700 (PDT) Received: from dread.disaster.area (pa49-179-32-121.pa.nsw.optusnet.com.au. [49.179.32.121]) by smtp.gmail.com with ESMTPSA id a8-20020a170902ecc800b001e944fc9248sm19566438plh.194.2024.05.01.02.27.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 02:27:39 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1s26FU-00HD6J-2B; Wed, 01 May 2024 19:27:36 +1000 Date: Wed, 1 May 2024 19:27:36 +1000 From: Dave Chinner To: Zhang Yi Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, tytso@mit.edu, adilger.kernel@dilger.ca, jack@suse.cz, ritesh.list@gmail.com, hch@infradead.org, djwong@kernel.org, willy@infradead.org, zokeefe@google.com, yi.zhang@huawei.com, chengzhihao1@huawei.com, yukuai3@huawei.com, wangkefeng.wang@huawei.com Subject: Re: [RFC PATCH v4 33/34] ext4: don't mark IOMAP_F_DIRTY for buffer write Message-ID: References: <20240410142948.2817554-1-yi.zhang@huaweicloud.com> <20240410150313.2820364-5-yi.zhang@huaweicloud.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240410150313.2820364-5-yi.zhang@huaweicloud.com> X-Stat-Signature: zahurzmasowa9zxzkqp93z3dx6rujg3a X-Rspamd-Queue-Id: 05FE7C0027 X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1714555660-315045 X-HE-Meta: U2FsdGVkX183uKu0Z8NXQFkFYfuwoTv89UD/d+gtxQStDpVVv8jUihq+TBgbc5YbeE+r113tIRoZVZIYSQM0Fof+3cD07N4h+7XD5zPbq9kBbSjm1sOP1pls8PYyvcS3MYGmdZBsa9L9nT/zsEqUIOvPGQjWj5lBLeridRA3amc5X9svZOlZFz8BqCsIRHYE+TvQ4oInGugUgbAA/kLtzz8b7CLAWcKW116gS2Yl0tgEVVoY0OX0Qd9qf59QAkv56UPV6tQRcpEXh8PnMVfK2FznzE9xyBzsMeXNAsPT26Ilu5Ww3+6RDVlr2VGEa4wo2MkwoykFrU51IoSVx5om6h7Pir5ZOAS8Vua1aBDLyVhUlk/FS+FPVPqea3twTKPoc2HEF1YwKv4fVs2wMcWMtg9AWzO9OGbx6No/aB7dRPHOfQhUFdjJTSD7P9EL4RpF7Su+wbGrAcEz2T++10O7k6t5v/eZt1qpkF+lGdlnQQQ6pfE8KEcieTTKuY6mU+h1uCfCB9290cZmPr9MzUKgy9qGjAC/APAFSp42N7e8Oj0vPx9WUoxBBrY3nNbhQ9BJMtzJ7mQlECdFRdZGGC12RmUNJdNHkYIIYeuYGDHMYPe0f6mzY4SYpt2MS0afs3bVnPde+sTh/pulB8PSJ2mj50aBKrLk9WWlzz1keAYncjlfIFBXtCyBqsklxacJT1s1shFWIgakkgIjxqQlU46ALqjAcVVHkrEX1rTWAAfdFYeeBwsAZLga7hoFq0pZErPVkoWlJybgnXZn+mdPOzCJFmLdN1WbgdXpbRhiMjIsoNhAubSo+McsA+SgxY/3wQmL08p+vJnPqwAMHaVRThEnZmYU5b4oJXZi9j4j7rfs0irwj15Gf5pWpYweJveVHYKO1vf6aI9VNhUhmjr33RQ7+PTe45OABfSW2xqDj4w9szKQG+zCU6w50LPbDvJn4YUi1mEwn4EEPmLUUzm6Rtd OXB75rMd Wl3GEAUUkOxOTMbCK9z7Ydma1FhmpN3bFsFViOYO1PQLGerEgrTJpx2e2rhBC33NzSDBMRpwrZzR/WFfejAirYxHatIwgc+Wdzz0dwbythSJ3mieNHILsdLiII0rdb8nXRJMjSMCdpqOcSvY/d7vMk93H8+MuQih+Yj7KpPfN5gTTLWx+sjkAnj1iO0SazuTpTaBaZXP+2QG7NquUHVjRCiuw+wqYRk8QbUld0gY78JtN6bvziM7UusoXDRaI+BgADnkCbTJCN3C2Ixe7J6y23TfcdXMgP9TdHDMJyyveEkPz3vN/GMFoDzvRf6oXO/EvT3Vf56TONTL7g40t2bZrK6D66BdnAt8B1HNYL6+0Clm97nOaqCjGq1fgXgUsP0z/Y9IkuEZzx48hp2XoBBwx0LPRMjinCmclQBGpEIUMMur+SlTnmGT6T6MZXx6bOWjYcAfkRiWEZkmZCWamP109Je+128FYDplGA/Xpr/WgaHykWubiPAX7ilLfjOX5V0od2TsMo4c2LN43XHJHZeVcOt5aHJN9uZrUe8vpUhvfsNmIi8i9lO9NyU5GmqWnxzL91zQJfgEo4qXnO+fK8IBuKoz6ug== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 10, 2024 at 11:03:12PM +0800, Zhang Yi wrote: > From: Zhang Yi > > The data sync dirty check in ext4_inode_datasync_dirty() is expansive > since jbd2_transaction_committed() holds journal->j_state lock when > journal is enabled, it costs a lot in high-concurrency iomap buffered > read/write paths, but we never check IOMAP_F_DIRTY in these cases, so > let's check it only in swap file, dax and direct IO cases. Tested by > Unixbench on 100GB ramdisk: > > ./Run -c 128 -i 10 fstime fsbuffer fsdisk > > == without this patch == > 128 CPUs in system; running 128 parallel copies of tests > > File Copy 1024 bufsize 2000 maxblocks 6332521.0 KBps > File Copy 256 bufsize 500 maxblocks 1639726.0 KBps > File Copy 4096 bufsize 8000 maxblocks 24018572.0 KBps > > == with this patch == > 128 CPUs in system; running 128 parallel copies of tests > > File Copy 1024 bufsize 2000 maxblocks 49229257.0 KBps > File Copy 256 bufsize 500 maxblocks 24057510.0 KBps > File Copy 4096 bufsize 8000 maxblocks 75704437.0 KBps > > Signed-off-by: Zhang Yi > --- > fs/ext4/inode.c | 10 +++++++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c > index 1cb219d347af..269503749ef5 100644 > --- a/fs/ext4/inode.c > +++ b/fs/ext4/inode.c > @@ -3281,9 +3281,13 @@ static void ext4_set_iomap(struct inode *inode, struct iomap *iomap, > * there is no other metadata changes being made or are pending. > */ > iomap->flags = 0; > - if (ext4_inode_datasync_dirty(inode) || > - offset + length > i_size_read(inode)) > - iomap->flags |= IOMAP_F_DIRTY; > + if ((flags & (IOMAP_DAX | IOMAP_REPORT)) || > + ((flags & (IOMAP_WRITE | IOMAP_DIRECT)) == > + (IOMAP_WRITE | IOMAP_DIRECT))) { > + if (offset + length > i_size_read(inode) || > + ext4_inode_datasync_dirty(inode)) > + iomap->flags |= IOMAP_F_DIRTY; > + } NACK. This just adds a nasty landmine that anyone working on the iomap infrastructure can step on. i.e. any time we add a new check for IOMAP_F_DIRTY in the generic infrastructure, ext4 is going to break because it won't set the IOMAP_F_DIRTY flag correctly. If checking an inode is dirty is expensive on ext4, then make it less expensive and everyone will benefit. /me goes and looks at jbd2_transaction_committed() Oh, it it's just a sequence number comparison, and it needs a lock because it has to dereference the running/committed transactions structures to get the current sequence numbers. Why not just store the commiting/running transaction tids in the journal_t, and then you can sample them without needing any locking and the whole ext4_inode_datasync_dirty() scalability problem goes away... -Dave. -- Dave Chinner david@fromorbit.com