From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3F98C3DA7A for ; Mon, 2 Jan 2023 09:41:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB4C48E0002; Mon, 2 Jan 2023 04:41:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E64FA8E0001; Mon, 2 Jan 2023 04:41:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2D138E0002; Mon, 2 Jan 2023 04:41:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BF8DF8E0001 for ; Mon, 2 Jan 2023 04:41:43 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 94667C07EC for ; Mon, 2 Jan 2023 09:41:43 +0000 (UTC) X-FDA: 80309366886.28.AD57851 Received: from r3-25.sinamail.sina.com.cn (r3-25.sinamail.sina.com.cn [202.108.3.25]) by imf21.hostedemail.com (Postfix) with ESMTP id A424E1C000E for ; Mon, 2 Jan 2023 09:41:40 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf21.hostedemail.com: domain of hdanton@sina.com designates 202.108.3.25 as permitted sender) smtp.mailfrom=hdanton@sina.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672652502; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I8Ctut8hIz+5gpU9PD43PycjFAFLlxfw7dTTSjUoH50=; b=pSwSx0JV2ocarbVN4ExhrYtOyG5PwS5OaIGVwu2HIDBl3PnZM1Ip0Wu3RfW8qtliKIq8B+ rJr7G2kR91nKtJjaoHcbcQl3j7gcq7mcroXFwbaSHMEILxBC1lW2GMdo2uLzTRQQAje7mj gjaruRsZGwvPy9FBK/H485rsXPmOyCI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf21.hostedemail.com: domain of hdanton@sina.com designates 202.108.3.25 as permitted sender) smtp.mailfrom=hdanton@sina.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672652502; a=rsa-sha256; cv=none; b=Xh/5J9tBG87fFeYgNEqnO6Zv6TKiuwDIjfXKAsB9BOH6awPw4TCoYnYAM7sEsk0OL8IK5T O116GMCdVxEcbpXEFVXIdS5ntI0k0ZrZNv2mYHoaoql+lvwel5Y7BOYxFq+AGMGAWGVgWt zO+y2oY0+6HaWmiJ6zzPPoR9qrQcpJw= Received: from unknown (HELO localhost.localdomain)([114.249.57.238]) by sina.com (172.16.97.23) with ESMTP id 63B2A6210000C946; Mon, 2 Jan 2023 17:38:43 +0800 (CST) X-Sender: hdanton@sina.com X-Auth-ID: hdanton@sina.com X-SMAIL-MID: 37921754920226 From: Hillf Danton To: Tetsuo Handa Cc: syzbot , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Waiman Long , Matthew Wilcox , Linus Torvalds , syzkaller-bugs@googlegroups.com Subject: Re: [syzbot] [ntfs3?] INFO: task hung in do_user_addr_fault (3) Date: Mon, 2 Jan 2023 17:41:25 +0800 Message-Id: <20230102094125.3552-1-hdanton@sina.com> In-Reply-To: <6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SAKURA.ne.jp> References: <00000000000060d41f05f139aa44@google.com> <20230102005409.3474-1-hdanton@sina.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A424E1C000E X-Rspam-User: X-Stat-Signature: m9yef9jjwdghymqz14tjochbc3iozmco X-HE-Tag: 1672652500-993181 X-HE-Meta: U2FsdGVkX1+pqjGhIDKXFgr1n57Dul++wMDz0CxsX/boZ1QATpUKxbioVjOlVTcDvOPwToMBYhvn94pmoE7tqWLME+3jx/q38zaGc+/WCKpBfsoQc6ROZmmm+9/kWx+TSN6H/TYPeWRccWpFoSe0Csyj9GLKEXwVTeveiA/NYFwKUYkVLzt/pwUthJHCoHWuaoEI2+grdX6q+ehGHcKKfQ20AxHK9Hgi12TZtuycdQDcUNZ1jx2bDUl2BEYigOPzcHiyahJeRmOmRpe/g83pw/3hF1CypJIDMKBHnj0F6e2iGg3RDHgmQxYzYhVi6yATql2r26n4rkb2i5m0whPTuu4QzNBaYKZhSnD7XS2FpKt4J4KhTrPvjDeE8CZLhpMxeQdTS1F2q5igltEDAFzwqDvcU9rQ/DzcLkrSvjUo3mhw8uTr94tfYeDAgd8jWJVfRh6ts56U536D4zD3jU4oRu6IsbS4gZiFQWz/KD3M8VE/Gl3z/8lpr8bL/NT6EolFKMb8a34VsjFm+plyIB00jkI6b+eRvHBvYGNNYmmztj8mUGtt4pun7D+E1TN2rxt28LXoZL4zN3fgVTw8ySyfg06kj89yTOR48pmPXDCu22AiAl1CX5qV7YVBeappbjawMRmvRHVvnWB5l/383dkiNf9E0H76nO5psBUboZPvSlq3Dn+Wq73VmiWx+8znBqdMjWHX5h/fi9L1fSkplolrOKodUKJnHGGSanzqwVfBAgDXaz6+xt7BnsOQBkONyElz8z7c6fy9iPTFguC7nASQRc6nhes6UeiU68EI7cjC0brWhjqjdzPOyd6D0Wnqo/cd+EljI3kM0gt+WuSWbeKHa5673XxbkkCTSYv4bidkZmYyNidTimm4kdeM0bDYAWx5TPOPXnRI6R8WWqyUGk7u+v6687N0GFeAgLRPRXPhC41yCRQy/iwptI65tEi8jKjPikJ1al6OV3qfo9FnvDW 6enVa5wR HXfxCFwJgEWgpEyM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2 Jan 2023 17:24:24 +0900 Tetsuo Handa > On 2023/01/02 10:40, Linus Torvalds wrote: > > So I think that we have: > > > > - ntfs_truncate() gets the ni_lock (fs/ntfs3/file.c:393) > > > > - it then - while holding that lock - calls (on line 395): > > > > truncate_setsize -> > > truncate_pagecache -> > > truncate_inode_pages -> > > truncate_inode_pages_range -> > > folio_lock > > > > but that deadlocks on another process that wants to read that page, > > and that needs ni_lock to do so. > > > > So yes, it does look like a ntfs3 deadlock involving ni_lock. > > Yes, I think you are right. My patch confirmed that other threads are not > holding ni_lock lock, which means that this is a deadlock between > PG_locked bit and ni_lock lock. > > filemap_update_page() calls filemap_read_folio() after calling > folio_trylock(). Since folio_trylock() sets PG_locked bit, > > mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799 > ni_lock fs/ntfs3/ntfs_fs.h:1122 [inline] > attr_data_get_block+0x4a6/0x2e40 fs/ntfs3/attrib.c:919 > ntfs_get_block_vbo+0x374/0xd20 fs/ntfs3/inode.c:573 > do_mpage_readpage+0x98b/0x1bb0 fs/mpage.c:208 > mpage_read_folio+0x103/0x1d0 fs/mpage.c:379 > filemap_read_folio+0x1ba/0x7f0 mm/filemap.c:2426 > filemap_update_page+0x3ca/0x550 mm/filemap.c:2511 > filemap_get_pages+0x8d8/0x1110 mm/filemap.c:2624 > filemap_read+0x3e7/0xee0 mm/filemap.c:2694 > > is trying to take ni_lock after setting PG_locked bit. > > On the other hand, folio_lock() waits until PG_locked bit is cleared, > but unfortunately ntfs3_setattr() already took ni_lock before calling > folio_lock(). > > io_schedule+0x83/0x100 kernel/sched/core.c:8811 > folio_wait_bit_common+0x8ca/0x1390 mm/filemap.c:1297 > folio_lock include/linux/pagemap.h:938 [inline] > truncate_inode_pages_range+0xc8d/0x1650 mm/truncate.c:421 > truncate_inode_pages mm/truncate.c:448 [inline] > truncate_pagecache mm/truncate.c:743 [inline] > truncate_setsize+0xcb/0xf0 mm/truncate.c:768 > ntfs_truncate fs/ntfs3/file.c:395 [inline] > ntfs3_setattr+0x5a5/0xca0 fs/ntfs3/file.c:696 > > Since no lockdep annotation is used for e.g. PG_locked bit, this deadlock > cannot be detected by lockdep... Good work, Tetsuo, thanks. Given inode_lock before ni_lock in the reported lock chain, > > 3 locks held by syz-executor394/5222: > > #0: ffff88801ee04460 (sb_writers#9){.+.+}-{0:0}, at: mnt_want_write+0x3b/0x80 fs/namespace.c:508 > > #1: ffff888073930b00 (&sb->s_type->i_mutex_key#14){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline] > > #1: ffff888073930b00 (&sb->s_type->i_mutex_key#14){+.+.}-{3:3}, at: do_truncate+0x205/0x300 fs/open.c:63 > > #2: ffff888073930860 (&ni->ni_lock/4){+.+.}-{3:3}, at: ni_lock fs/ntfs3/ntfs_fs.h:1122 [inline] > > #2: ffff888073930860 (&ni->ni_lock/4){+.+.}-{3:3}, at: ntfs_truncate fs/ntfs3/file.c:393 [inline] > > #2: ffff888073930860 (&ni->ni_lock/4){+.+.}-{3:3}, at: ntfs3_setattr+0x596/0xca0 fs/ntfs3/file.c:696 a workaround is take ni_lock after truncate_setsize(). Lets see the echo from syzbot. #syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master --- x/fs/ntfs3/file.c +++ y/fs/ntfs3/file.c @@ -390,10 +390,10 @@ static int ntfs_truncate(struct inode *i new_valid = ntfs_up_block(sb, min_t(u64, ni->i_valid, new_size)); - ni_lock(ni); - truncate_setsize(inode, new_size); + ni_lock(ni); + down_write(&ni->file.run_lock); err = attr_set_size(ni, ATTR_DATA, NULL, 0, &ni->file.run, new_size, &new_valid, ni->mi.sbi->options->prealloc, NULL); --