From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BECCDC369AB for ; Tue, 15 Apr 2025 11:25:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 601D12801D5; Tue, 15 Apr 2025 07:25:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58A332800BD; Tue, 15 Apr 2025 07:25:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42B712801D5; Tue, 15 Apr 2025 07:25:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 23CDA2800BD for ; Tue, 15 Apr 2025 07:25:47 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 57587C1234 for ; Tue, 15 Apr 2025 11:25:48 +0000 (UTC) X-FDA: 83336048376.22.01233F4 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf26.hostedemail.com (Postfix) with ESMTP id 1747E14000B for ; Tue, 15 Apr 2025 11:25:45 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=xRdJy+XJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=GP4bwSBm; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=xRdJy+XJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=GP4bwSBm; dmarc=none; spf=pass (imf26.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744716346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RUvz3mNVXZTDsjj2f6GMq8aLSKexmcTgxLqp8L/MMy0=; b=554/yPYxS769gklm+OUocJyuCab9Nt8yrEjub4Hq4s5sozsmZoy53gZ3pJ8DD8ln6cSpJz BInaodlc3fLgwVQdC9nf3W1rrG9Y+RhiwF5YUiurt6xrxHK6Qah+W2tnxAKRcRoddweYIM k2WeVtSgvRiYuvfENI2VPuHjorTiUIc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=xRdJy+XJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=GP4bwSBm; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=xRdJy+XJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=GP4bwSBm; dmarc=none; spf=pass (imf26.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744716346; a=rsa-sha256; cv=none; b=cLGRCsvu7rPfzE5GTSHnSBbOV3jM/XWPpPLypj4PTNNmbaq1pvCM+YlXyfrL0pHW2L4N8d xOq77odvWvvMoj5XKi5GF+Z2KPdyKcpeUVu996gOwcj4e2tSTcIjazXHF5pUFlF/X+7lf1 1o2y7r8eauwq5X23ruG6gVbNxbD1Sso= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 643BF1F385; Tue, 15 Apr 2025 11:25:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1744716344; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RUvz3mNVXZTDsjj2f6GMq8aLSKexmcTgxLqp8L/MMy0=; b=xRdJy+XJ29Awx/kROqOxvU4q6tic7kS4iYfO9xfn35gvEkqRUaY13MwytQIse64+tsXz3e fvbPUs6mWwW8gbfwpf3yKjFrg2E3FqnTNczigumOM9u5L8M+UTcb3QT9I5S9sACusKYRJE 4utjoG9TEOVahZtkHLsHCoNRAXqOVTM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1744716344; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RUvz3mNVXZTDsjj2f6GMq8aLSKexmcTgxLqp8L/MMy0=; b=GP4bwSBmoBaymtsVFhoQxU+aHL7na/qw6+h2MQSb4zXNNXodEL0kiKHeKu/RSv/YeIZ/y5 ea3Ldo5R5d3JRKAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1744716344; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RUvz3mNVXZTDsjj2f6GMq8aLSKexmcTgxLqp8L/MMy0=; b=xRdJy+XJ29Awx/kROqOxvU4q6tic7kS4iYfO9xfn35gvEkqRUaY13MwytQIse64+tsXz3e fvbPUs6mWwW8gbfwpf3yKjFrg2E3FqnTNczigumOM9u5L8M+UTcb3QT9I5S9sACusKYRJE 4utjoG9TEOVahZtkHLsHCoNRAXqOVTM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1744716344; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RUvz3mNVXZTDsjj2f6GMq8aLSKexmcTgxLqp8L/MMy0=; b=GP4bwSBmoBaymtsVFhoQxU+aHL7na/qw6+h2MQSb4zXNNXodEL0kiKHeKu/RSv/YeIZ/y5 ea3Ldo5R5d3JRKAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5437A137A5; Tue, 15 Apr 2025 11:25:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +2KOFDhC/mdyfAAAD6G6ig (envelope-from ); Tue, 15 Apr 2025 11:25:44 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 12498A0947; Tue, 15 Apr 2025 13:25:44 +0200 (CEST) Date: Tue, 15 Apr 2025 13:25:44 +0200 From: Jan Kara To: Davidlohr Bueso Cc: Luis Chamberlain , brauner@kernel.org, jack@suse.cz, tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, riel@surriel.com, willy@infradead.org, hannes@cmpxchg.org, oliver.sang@intel.com, david@redhat.com, axboe@kernel.dk, hare@suse.de, david@fromorbit.com, djwong@kernel.org, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, syzbot+f3c6fda1297c748a7076@syzkaller.appspotmail.com Subject: Re: [PATCH v2 1/8] migrate: fix skipping metadata buffer heads on migration Message-ID: References: <20250410014945.2140781-1-mcgrof@kernel.org> <20250410014945.2140781-2-mcgrof@kernel.org> <20250415013641.f2ppw6wov4kn4wq2@offworld> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250415013641.f2ppw6wov4kn4wq2@offworld> X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 1747E14000B X-Stat-Signature: ee77urc9b8b1udqtdzsg1eg8dmpo31d7 X-Rspam-User: X-HE-Tag: 1744716345-648410 X-HE-Meta: U2FsdGVkX181Qz2P/gKzoWQU8OSaZZa0Ghjuizgq7hylzuDxfUQ/EqqZY+z0YMgMHcWCP+tPtwa5XXUcSyT/ekoTWvbThFh9wzW8/dCfsnsqhL+TS7odRryki6SpJPPaEZc2r9KwQvTgVuBLpVGJHyWLaVaEz0tv2oIrlnKs2YElr08rujkGIE4DrKhpXfbIm7qJUnOePeTDPWVyaVb7QLPvk5tJmYeQq5ZUtj7c6CHdR6DIoc7Iy6qI5CFpnnK2K7YcCzCjVz5f7Gx9Tn44v8Nq9Z1RdRm8KYxh+FjihPtiTU3T3N7kxr0kp3R0BjJVJzCxljpv+ajwNAtaQq3weAjoOeIZA8v9kP51gQ2paVj3VNU4Zzl/nf0f7o8rx/QbN+XIz4XaeqkuKBAfcHwsZyeAy8y9oararoJ+QI7L89/j2mHcq2IBk+gI4lHd9THE1jGWu3zW4NLK+n3jJRLZjhVp0T152poX4RPbOllwCtEyce1bP8uQO2WAQD+nd6WhROCsnsD24euWx8PnJTI3X1/iGzw9nzhl5Wa5JpNOSsv+gLUhKtwpQ9hT0biQFxGFQ8zBZUw8JoVbjSgLQS8v2lXTToErM2cTTPZ/xW69Bsu5/5leZBZXicJBwp7gaW1pO2PxR1U9WPgU5g08n/4fwW6GJGnPB6/Uxoj3bTp5maH3dKwt72bzyi2S0k6fie5SrCyg8m8lue4mu3G3sPH+OZCuiqrINJD904lzf73rOTuVkan4E80KjfwwyBOc0sgDI3+w9gWuYqtcSOpalCAG2WSlF+BunhER8N3m5GlCrc4FqH1iJk8flBVssWOLj2fspCf+SIy6oQ2tZaLccJ5aGyYnOSeeiS4yrZkYtUIljKsy7a2ibKCkX6zpColBeq/YyWJZJh4Z7JD0Y2kY5S7ZucQZTk9n4bghd/98JKwxFGr+dIcxxvj6C5eHPlzTlUGfYSrpoT4AwLEYbrLr1mC mt7SvoSw POsvFGnXM9Ix9HzDu1r3RzmgsO20+Mjz4eG8Xp1MwBSUIGuKJyWwKePa7sW0qBNKXkeVWR3uRFR3fynx/ByrIxx85W78aQOo+nQNwv/HyBjQ+lhp7p+QvQfIfFS40U5n8YSM3HPc49da6JBWk3xeb1MdfTvaLMO/6I9jdUpO+AiLlRwmPQs8UKPEBbUF8BSGbqj/Ks+UIJCJ4HmRi7JcXPfjN6DqL50FYMgE/Nm7ULLMFj6riePyj339je61aqb8x5SdEoFcxTxIHk6seFEUI7GO/jz3dLt3+OYGBvEaUHzJ1hU5qLm016MsJo3iAueCi+oxR7h0SpgCL/cgCRPTCqgj8DQ2sQOtWoTTxwCWyRzi2whTWL1Q6e1puLhJGI7TiGkV7y2yfyJQNJb1xpKvwm67gZmid5thlGVUhD+lCLON/Q5FyyyqeH39zmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon 14-04-25 18:36:41, Davidlohr Bueso wrote: > On Wed, 09 Apr 2025, Luis Chamberlain wrote: > > > corruption can still happen even with the spin lock held. A test was > > done using vanilla Linux and adding a udelay(2000) right before we > > spin_lock(&bd_mapping->i_private_lock) on __find_get_block_slow() and > > we can reproduce the same exact filesystem corruption issues as observed > > without the spinlock with generic/750 [1]. > > fyi I was actually able to trigger this on a vanilla 6.15-rc1 kernel, > not even having to add the artificial delay. OK, so this is using generic/750, isn't it? How long did you have to run it to trigger this? Because I've never seen this tripping... Honza > > [336534.157119] ------------[ cut here ]------------ > [336534.158911] WARNING: CPU: 3 PID: 87221 at fs/jbd2/transaction.c:1552 jbd2_journal_dirty_metadata+0x21c/0x230 [jbd2] > [336534.160771] Modules linked in: loop sunrpc 9p kvm_intel nls_iso8859_1 nls_cp437 vfat fat crc32c_generic kvm ghash_clmulni_intel sha512_ssse3 sha256_ssse3 sha1_ssse3 aesni_intel gf128mul crypto_simd 9pnet_virtio cryptd virtio_balloon virtio_console evdev joydev button nvme_fabrics nvme_core dm_mod drm nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vsock autofs4 ext4 crc16 mbcache jbd2 btrfs blake2b_generic efivarfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 md_mod virtio_net net_failover virtio_blk failover psmouse serio_raw virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev virtio virtio_ring > [336534.173218] CPU: 3 UID: 0 PID: 87221 Comm: kworker/u36:8 Not tainted 6.15.0-rc1 #2 PREEMPT(full) > [336534.175146] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 2025.02-5 03/28/2025 > [336534.176947] Workqueue: writeback wb_workfn (flush-7:5) > [336534.178183] RIP: 0010:jbd2_journal_dirty_metadata+0x21c/0x230 [jbd2] > [336534.179626] Code: 30 0f 84 5b fe ff ff 0f 0b 41 bc 8b ff ff ff e9 69 fe ff ff 48 8b 04 24 4c 8b 48 70 4d 39 cf 0f 84 53 ff ff ff e9 32 c3 00 00 <0f> 0b 41 bc e4 ff ff ff e9 41 ff ff ff 0f 0b 90 0f 1f 40 00 90 90 > [336534.183983] RSP: 0018:ffff9f168d38f548 EFLAGS: 00010246 > [336534.185194] RAX: 0000000000000001 RBX: ffff8c0ae8244e10 RCX: 00000000000000fd > [336534.186810] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 > [336534.188399] RBP: ffff8c09db0d8618 R08: ffff8c09db0d8618 R09: 0000000000000000 > [336534.189977] R10: ffff8c0b2671a83c R11: 0000000000006989 R12: 0000000000000000 > [336534.191243] R13: ffff8c09cc3b33f0 R14: ffff8c0ae8244e18 R15: ffff8c0ad5e0ef00 > [336534.192469] FS: 0000000000000000(0000) GS:ffff8c0b95b8d000(0000) knlGS:0000000000000000 > [336534.193840] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [336534.194831] CR2: 00007f0ebab4f000 CR3: 000000011e616005 CR4: 0000000000772ef0 > [336534.196044] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > [336534.197274] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > [336534.198473] PKRU: 55555554 > [336534.198927] Call Trace: > [336534.199350] > [336534.199701] __ext4_handle_dirty_metadata+0x5c/0x190 [ext4] > [336534.200626] ext4_ext_insert_extent+0x575/0x1440 [ext4] > [336534.201465] ? ext4_cache_extents+0x5a/0xd0 [ext4] > [336534.202243] ? ext4_find_extent+0x37c/0x3a0 [ext4] > [336534.203024] ext4_ext_map_blocks+0x50e/0x18d0 [ext4] > [336534.203803] ? mpage_map_and_submit_buffers+0x23f/0x270 [ext4] > [336534.204723] ext4_map_blocks+0x11a/0x4d0 [ext4] > [336534.205442] ? ext4_alloc_io_end_vec+0x1f/0x70 [ext4] > [336534.206239] ? kmem_cache_alloc_noprof+0x310/0x3d0 > [336534.206982] ext4_do_writepages+0x762/0xd40 [ext4] > [336534.207706] ? __pfx_block_write_full_folio+0x10/0x10 > [336534.208451] ? ext4_writepages+0xc6/0x1a0 [ext4] > [336534.209161] ext4_writepages+0xc6/0x1a0 [ext4] > [336534.209834] do_writepages+0xdd/0x250 > [336534.210378] ? filemap_get_read_batch+0x170/0x310 > [336534.211069] __writeback_single_inode+0x41/0x330 > [336534.211738] writeback_sb_inodes+0x21b/0x4d0 > [336534.212375] __writeback_inodes_wb+0x4c/0xe0 > [336534.212998] wb_writeback+0x19c/0x320 > [336534.213546] wb_workfn+0x30e/0x440 > [336534.214039] process_one_work+0x188/0x340 > [336534.214650] worker_thread+0x246/0x390 > [336534.215196] ? _raw_spin_lock_irqsave+0x23/0x50 > [336534.215879] ? __pfx_worker_thread+0x10/0x10 > [336534.216522] kthread+0x104/0x250 > [336534.217004] ? __pfx_kthread+0x10/0x10 > [336534.217554] ? _raw_spin_unlock+0x15/0x30 > [336534.218140] ? finish_task_switch.isra.0+0x94/0x290 > [336534.218979] ? __pfx_kthread+0x10/0x10 > [336534.220347] ret_from_fork+0x2d/0x50 > [336534.221086] ? __pfx_kthread+0x10/0x10 > [336534.221703] ret_from_fork_asm+0x1a/0x30 > [336534.222415] > [336534.222775] ---[ end trace 0000000000000000 ]--- -- Jan Kara SUSE Labs, CR