From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 500A5C369CB for ; Wed, 23 Apr 2025 17:09:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B73DA6B002C; Wed, 23 Apr 2025 13:09:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFA126B002D; Wed, 23 Apr 2025 13:09:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94D216B002F; Wed, 23 Apr 2025 13:09:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 73B106B002C for ; Wed, 23 Apr 2025 13:09:32 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0989C1A1639 for ; Wed, 23 Apr 2025 17:09:33 +0000 (UTC) X-FDA: 83365945026.03.190A865 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf09.hostedemail.com (Postfix) with ESMTP id A467114000B for ; Wed, 23 Apr 2025 17:09:30 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=CO06bkUk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aGH2myP6; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=CO06bkUk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aGH2myP6; dmarc=none; spf=pass (imf09.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745428171; a=rsa-sha256; cv=none; b=rda0HSYLLArpWJpCXLRl945r1kmFfyxxc2yti/VR46G7TRi138W+FL5he1QhlyI9cW7Os6 pxGYlT5QCUSBWvT5IcadtsmVI1aeJyiCA4alZv5Zow1Gv7QUIHl6GOyTM17vjymD/D5vgw FjFXu3MQCuVMuOqFcc4U/s4BSlIOwHI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=CO06bkUk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aGH2myP6; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=CO06bkUk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aGH2myP6; dmarc=none; spf=pass (imf09.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745428171; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UokPV57Q89xRAxxA7GjXgjX2aUYQnotNsdxc/ZHHrKE=; b=RS53+4lmYk/RIzMi4VE7WuJ9RVLaF6qUW8LhLN8U37ltNTjNBxqBMOPp2MfzUviAvA+rrX oAYHJVCa4afH360z2HqmLHk9QPMOLA5Nmlkjv1unxtFqv2qwTr0wHJ1DDr2gRQabqpRSob ItBvvl/r5AZzr1irf38aU1jfVLTuwD4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 8C5881F449; Wed, 23 Apr 2025 17:09:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1745428168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UokPV57Q89xRAxxA7GjXgjX2aUYQnotNsdxc/ZHHrKE=; b=CO06bkUkXHVAbGl8DvWmpRJkYgUd7fNvWa2jMCogBG7bw17GinK+YCJAdm+zcohqOJRqFF EE6mxXHk0sEfQf6PTpgoPjaKdupiTyXBnEX0vYGv/r4+OvyMjEJIEyAO0RTfkeZpALem3p Ii6S5iq8BZI0VJkQQw/ojYXhSJ0akwM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1745428168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UokPV57Q89xRAxxA7GjXgjX2aUYQnotNsdxc/ZHHrKE=; b=aGH2myP62Ln+0PYOVqR721kCl5prNt9Qm9wBuNSj9dINbOA9W702N6sHejb6kwHWXLXb42 nghq/W56hrSci+CQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1745428168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UokPV57Q89xRAxxA7GjXgjX2aUYQnotNsdxc/ZHHrKE=; b=CO06bkUkXHVAbGl8DvWmpRJkYgUd7fNvWa2jMCogBG7bw17GinK+YCJAdm+zcohqOJRqFF EE6mxXHk0sEfQf6PTpgoPjaKdupiTyXBnEX0vYGv/r4+OvyMjEJIEyAO0RTfkeZpALem3p Ii6S5iq8BZI0VJkQQw/ojYXhSJ0akwM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1745428168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UokPV57Q89xRAxxA7GjXgjX2aUYQnotNsdxc/ZHHrKE=; b=aGH2myP62Ln+0PYOVqR721kCl5prNt9Qm9wBuNSj9dINbOA9W702N6sHejb6kwHWXLXb42 nghq/W56hrSci+CQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7BBE513A3D; Wed, 23 Apr 2025 17:09:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qPrxHcgeCWgXHQAAD6G6ig (envelope-from ); Wed, 23 Apr 2025 17:09:28 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 0F268A0949; Wed, 23 Apr 2025 19:09:28 +0200 (CEST) Date: Wed, 23 Apr 2025 19:09:28 +0200 From: Jan Kara To: Luis Chamberlain Cc: Jan Kara , dave@stgolabs.net, brauner@kernel.org, tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, riel@surriel.com, willy@infradead.org, hannes@cmpxchg.org, oliver.sang@intel.com, david@redhat.com, axboe@kernel.dk, hare@suse.de, david@fromorbit.com, djwong@kernel.org, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, syzbot+f3c6fda1297c748a7076@syzkaller.appspotmail.com Subject: Re: [PATCH v2 1/8] migrate: fix skipping metadata buffer heads on migration Message-ID: References: <20250410014945.2140781-1-mcgrof@kernel.org> <20250410014945.2140781-2-mcgrof@kernel.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="4v4cuvrwajlncncp" Content-Disposition: inline In-Reply-To: X-Rspamd-Action: no action X-Rspamd-Queue-Id: A467114000B X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 8qzy4qt7tye8p1s1kq8fcao94bhta3ad X-HE-Tag: 1745428170-690495 X-HE-Meta: U2FsdGVkX1+/k1ULm4RkQve/pjRYcH2tLsNGLdio36jNuYCaAAzjSsyW6I/GswjnRvrz7Q5FNHTYdF5eSHfJ8o0HCttlD9gidvqYoJg5k5k08EZeNEFts41MH8GHqWp4N6Ve6YGQedgiP2uHC0WqOvVYvfKYKc+ZHxQAUsrbe98l6jdb6oIX5NIdIOfMxJn7qQysWDJbokR2Xd+/E0TPGJGaDf6+Z/eoeRnzOBoaCoZ346yT4HQyLpyE1hsI4SNHVpkQcCoYi7JjRRukQqH/uzkovEmlBFtNu3xrTCDbnaZ7bObd8fFoHr4zqtLpK6Dp89iBfOEOs06RACO4Stg0nK5/eiDw23aC0IChV4iWdQghb2H7m/1PGNUgiUwXeZh/o4D+55sy490Me5OX+0u5wcOHElF2MEDaEybvqIg8iik1fRWPT6d2FgnhVprfn3GILcBiw+Fk9QZJ3Vj94mY9zfaHvp5MYdDGvr3dmf4y0Bln9R9wOGrGsTu/icXkclNcc+Cs/fzjf0s9B4vaKD2rSnYN/HIjb3RsNof4hWKZ5rAHs+0EIIlVvikOcnxeNiB7bD0EtGk7IBEbelYvwXqM8pQhKkD/wr60pdjVlanu0rxXJiWXmdu7Ub1+1IDpwnPZEO1jjyvA0koTL/OzZ9iegMZ7THoJyqlirSk23PwO90KjRGCDhH29/wTmc8M+K8QyokoGVULUs2VxGEB4ASk09djFFWKxGO0rOSrX2Fow8HfTgM7riGz+VS5nafoiO5ugVJcEHISuCNRtz6a2ZXQXNvuq/7kZDXfUd1Z2adIcCpq4hA87MrI3hScWQyTomB/NTuR+bc6bOgGyvUhlD7XjSYqlSsY2C5l9dn6v/lfqEn2S+0IZC0ZUwqCQb3Fvv28V2zFZZzn19i3QCVE1jlHGrQj76OWAtXAVG3bgD8VUXJc1fiu+ZEKrxc5cqJNykfb5ho4OEFxMP8V8VPJMHqh pZfkA9yw Tj/w54Bgtes2ray0W7peHx0LSJnSFBsoDMotkLjkDwte8Nh1l8MYlNZVbd7t04S+X04hslrzIdVNeDec4QlCnECWWSsaUUDNXIKXffH2F+fgh+TaicHVhVr6ywjth20aQYVdDc9Am9KyViMwrpVmK1ItqQXHM5vvvQ1Kfx5s9Gypbrx+IMRMhlQlC2bV9rKvsbVtju7T/lo+6ah5hJvP1EFwurE/iVJwJ3vRzbMYcAmMchiifTA4MTa3bb1cHfbg2mKmU8CqRJiq4vbIuViwbUkl+2QxBWzHe0VUIjAaTMMpI5tLtbCkI1BGXYjUIV1Y06TqtVar/vS+px1O4wo6UCXMIRlixiOq8iDBbKL8sIKHMzRV3NwtZr/NH2Rk2ymUAhZC98s4ZAchsB7n7tI/guOq0v7IcTwND9MkK/dLM5Mh0QgyZXILRXyvEQ2VWoPblBTwS3pwgplS1s7UzEOclMOP0m79Qt12oDagLm7o4w93ujcJAtM89GOooJCDqU5y7f+BGtmuhjXEg/NX3azwTYTUTWeJtV45bBTKhbmVYEIlC2yOtN64U3yhr25a/LnsxfAbUzIyvBmyCVwrRwXsffaUDqcb1fsgQ0NUMPL+nTYlkLFqXMjXCNUf1CaTl/kdiHwY7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --4v4cuvrwajlncncp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed 16-04-25 09:58:30, Luis Chamberlain wrote: > On Tue, Apr 15, 2025 at 06:28:55PM +0200, Jan Kara wrote: > > > So I tried: > > > > > > root@e1-ext4-2k /var/lib/xfstests # fsck /dev/loop5 -y 2>&1 > log > > > e2fsck 1.47.2 (1-Jan-2025) > > > root@e1-ext4-2k /var/lib/xfstests # wc -l log > > > 16411 log > > > > Can you share the log please? > > Sure, here you go: > > https://github.com/linux-kdevops/20250416-ext4-jbd2-bh-migrate-corruption > > The last trace-0004.txt is a fresh one with Davidlohr's patches > applied. It has trace-0004-fsck.txt. Thanks for the data! I was staring at them for some time and at this point I'm leaning towards a conclusion that this is actually not a case of metadata corruption but rather a bug in ext4 transaction credit computation that is completely independent of page migration. Based on the e2fsck log you've provided the only damage in the filesystem is from the aborted transaction handle in the middle of extent tree growth. So nothing points to a lost metadata write or anything like that. And the credit reservation for page writeback is indeed somewhat racy - we reserve number of transaction credits based on current tree depth. However by the time we get to ext4_ext_map_blocks() another process could have modified the extent tree so we may need to modify more blocks than we originally expected and reserved credits for. Can you give attached patch a try please? Honza -- Jan Kara SUSE Labs, CR --4v4cuvrwajlncncp Content-Type: text/x-patch; charset=us-ascii Content-Disposition: attachment; filename="0001-ext4-Fix-calculation-of-credits-for-extent-tree-modi.patch" >From 4c53fb9f4b9b3eb4a579f69b7adcb6524d55629c Mon Sep 17 00:00:00 2001 From: Jan Kara Date: Wed, 23 Apr 2025 18:10:54 +0200 Subject: [PATCH] ext4: Fix calculation of credits for extent tree modification Luis and David are reporting that after running generic/750 test for 90+ hours on 2k ext4 filesystem, they are able to trigger a warning in jbd2_journal_dirty_metadata() complaining that there are not enough credits in the running transaction started in ext4_do_writepages(). Indeed the code in ext4_do_writepages() is racy and the extent tree can change between the time we compute credits necessary for extent tree computation and the time we actually modify the extent tree. Thus it may happen that the number of credits actually needed is higher. Modify ext4_ext_index_trans_blocks() to count with the worst case of maximum tree depth. Link: https://lore.kernel.org/all/20250415013641.f2ppw6wov4kn4wq2@offworld Reported-by: Davidlohr Bueso Reported-by: Luis Chamberlain Signed-off-by: Jan Kara --- fs/ext4/extents.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index c616a16a9f36..43286632e650 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -2396,18 +2396,19 @@ int ext4_ext_calc_credits_for_single_extent(struct inode *inode, int nrblocks, int ext4_ext_index_trans_blocks(struct inode *inode, int extents) { int index; - int depth; /* If we are converting the inline data, only one is needed here. */ if (ext4_has_inline_data(inode)) return 1; - depth = ext_depth(inode); - + /* + * Extent tree can change between the time we estimate credits and + * the time we actually modify the tree. Assume the worst case. + */ if (extents <= 1) - index = depth * 2; + index = EXT4_MAX_EXTENT_DEPTH * 2; else - index = depth * 3; + index = EXT4_MAX_EXTENT_DEPTH * 3; return index; } -- 2.43.0 --4v4cuvrwajlncncp--