From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E85D6C433EF for ; Thu, 17 Feb 2022 15:52:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01EB66B0074; Thu, 17 Feb 2022 10:52:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F10906B0075; Thu, 17 Feb 2022 10:52:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD7AD6B0078; Thu, 17 Feb 2022 10:52:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id CDFB76B0074 for ; Thu, 17 Feb 2022 10:52:12 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6F67B181AC9C6 for ; Thu, 17 Feb 2022 15:52:12 +0000 (UTC) X-FDA: 79152713304.08.22DDC80 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by imf25.hostedemail.com (Postfix) with ESMTP id F352DA0005 for ; Thu, 17 Feb 2022 15:52:11 +0000 (UTC) Received: from cwcc.thunk.org (pool-108-7-220-252.bstnma.fios.verizon.net [108.7.220.252]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 21HFp9Kg004053 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 17 Feb 2022 10:51:11 -0500 Received: by cwcc.thunk.org (Postfix, from userid 15806) id 4815815C34C8; Thu, 17 Feb 2022 10:51:09 -0500 (EST) Date: Thu, 17 Feb 2022 10:51:09 -0500 From: "Theodore Ts'o" To: Byungchul Park Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, chris@chris-wilson.co.uk, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, bfields@fieldses.org, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, axboe@kernel.dk, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jack@suse.com, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, airlied@linux.ie, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com Subject: Re: [PATCH 00/16] DEPT(Dependency Tracker) Message-ID: References: <1645095472-26530-1-git-send-email-byungchul.park@lge.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1645095472-26530-1-git-send-email-byungchul.park@lge.com> X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F352DA0005 X-Stat-Signature: 8jns9nzprm1qo89hsikq5pizswsdciwr X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=mit.edu (policy=none); spf=none (imf25.hostedemail.com: domain of tytso@mit.edu has no SPF policy when checking 18.9.28.11) smtp.mailfrom=tytso@mit.edu X-HE-Tag: 1645113131-664785 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 17, 2022 at 07:57:36PM +0900, Byungchul Park wrote: > > I've got several reports from the tool. Some of them look like false > alarms and some others look like real deadlock possibility. Because of > my unfamiliarity of the domain, it's hard to confirm if it's a real one. > Let me add the reports on this email thread. The problem is we have so many potentially invalid, or so-rare-as-to-be-not-worth-the-time-to-investigate-in-the- grand-scheme-of-all-of-the-fires-burning-on-maintainers laps that it's really not reasonable to ask maintainers to determine whether something is a false alarm or not. If I want more of these unreliable potential bug reports to investigate, there is a huge backlog in Syzkaller. :-) Looking at the second ext4 report, it doesn't make any sense. Context A is the kjournald thread. We don't do a commit until (a) the timeout expires, or (b) someone explicitly requests that a commit happen waking up j_wait_commit. I'm guessing that complaint here is that DEPT thinks nothing is explicitly requesting a wake up. But note that after 5 seconds (or whatever journal->j_commit_interval) is configured to be we *will* always start a commit. So ergo, there can't be a deadlock. At a higher level of discussion, it's an unfair tax on maintainer's times to ask maintainers to help you debug DEPT for you. Tools like Syzkaller and DEPT are useful insofar as they save us time in making our subsystems better. But until you can prove that it's not going to be a massive denial of service attack on maintainer's time, at the *very* least keep an RFC on the patch, or add massive warnings that more often than not DEPT is going to be sending maintainers on a wild goose chase. If you know there there "appear to be false positives", you need to make sure you've tracked them all down before trying to ask that this be merged. You may also want to add some documentation about why we should trust this; in particular for wait channels, when a process calls schedule() there may be multiple reasons why the thread will wake up --- in the worst case, such as in the select(2) or epoll(2) system call, there may be literally thousands of reasons (one for every file desriptor the select is waiting on) --- why the process will wake up and thus resolve the potential "deadlock" that DEPT is worrying about. How is DEPT going to handle those cases? If the answer is that things need to be tagged, then at least disclose potential reasons why DEPT might be untrustworthy to save your reviewers time. I know that you're trying to help us, but this tool needs to be far better than Lockdep before we should think about merging it. Even if it finds 5% more potential deadlocks, if it creates 95% more false positive reports --- and the ones it finds are crazy things that rarely actually happen in practice, are the costs worth the benefits? And who is bearing the costs, and who is receiving the benefits? Regards, - Ted