From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 711E5C38141 for ; Sat, 21 Jan 2023 04:48:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 747976B0072; Fri, 20 Jan 2023 23:48:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F68B6B0073; Fri, 20 Jan 2023 23:48:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BE3E6B0074; Fri, 20 Jan 2023 23:48:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4EF286B0072 for ; Fri, 20 Jan 2023 23:48:10 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1DBE2140875 for ; Sat, 21 Jan 2023 04:48:10 +0000 (UTC) X-FDA: 80377574340.01.3C34364 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf21.hostedemail.com (Postfix) with ESMTP id 3F2A41C0003 for ; Sat, 21 Jan 2023 04:48:06 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf21.hostedemail.com: domain of byungchul.park@lge.com designates 156.147.23.53 as permitted sender) smtp.mailfrom=byungchul.park@lge.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674276488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=ul1RvK1wtFWv/B+lYL4YwInmCUbnSryRyE9INTmsVcw=; b=ciYD9Rryr8dCBe1tztXjuq8CgqnOt5azPfZMeRrU0LhEgulpJCbj/Jlnp6X2wynLP2gojp jg+EGm04WUXKiRiH657kzoeM9GPwhFQO51fyquj/oNcqU2e0kYYfGDj9mW6w0EDpDgjF+F FPKnY2tc58xIKNeAEWDvu/rOszUo0Ws= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf21.hostedemail.com: domain of byungchul.park@lge.com designates 156.147.23.53 as permitted sender) smtp.mailfrom=byungchul.park@lge.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674276488; a=rsa-sha256; cv=none; b=vWiSqbI2S3WWXb33gdBRAisJNB/yH9yA5hFJgWmDAzQ+ovnIu2VJpOee54O++xl0vRb9+F Sr9tJXBUQ9zmOUEJRfa368PSm5BZlBjyPuT4m410VqgWZSkbcVmdpmHNbVJjXr07OXHCPp cu8Dm3mFeEwu55to4/kzjictw66kVsM= Received: from unknown (HELO lgemrelse7q.lge.com) (156.147.1.151) by 156.147.23.53 with ESMTP; 21 Jan 2023 13:48:04 +0900 X-Original-SENDERIP: 156.147.1.151 X-Original-MAILFROM: byungchul.park@lge.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.151 with ESMTP; 21 Jan 2023 13:48:04 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: byungchul.park@lge.com From: Byungchul Park To: boqun.feng@gmail.com Cc: linux-kernel@vger.kernel.org, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, longman@redhat.com Subject: Re: [PATCH RFC v7 00/23] DEPT(Dependency Tracker) Date: Sat, 21 Jan 2023 13:47:49 +0900 Message-Id: <1674276469-31793-1-git-send-email-byungchul.park@lge.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: References: X-Rspamd-Queue-Id: 3F2A41C0003 X-Stat-Signature: em79ar5fiuc349rnuts968og57out8r7 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1674276486-909818 X-HE-Meta: U2FsdGVkX1+IkvcXOwfTRAOMQ4ks3JYF8ZLHbh31W5g7A03nD73uBpaIjnbczMLxg2adqkcjbwu//5i/SqQR3kVpK6mooETHKSo/UGVOxabsZWbcJ1/tFS/aDesDRVEsdMNSZRd3ifjq+Afw9Hen1QOHQzhHswJRWNK+gJdk+1ePbc6YaNPP5iJOpLONHasmQU/nK1E4ELJXLMoGCEccFyNfucVkL8lcI+H8LLa2z7Wxy/QsHgaLlPA6q/NFld+Yl9mOTEXiPCPhikGsoyLq1h9lQqyUVcUbP4WlXKNFCB2nVhVI4u6Q4hJPqwSIir3QCZmifQZ2NIEmIzhqPyW2ncoLIYNj7i+DXEMRjCHRGYBnQW00I/yeDJpdO/KYdxxMDWeuttDLicxYlTmj5H6Pb6fsiMD7vxouzjLEtxK7iqw6aVfl6r+BjRv8s9Ka28O13AwFD98i+EuB0zY7QGt7izaBs/1PGX8xTFmhbqA8FsAptlxxMK1qi37A3w1CokKJynQ+xGLOFh9zUEShgz37KFgZaRQfC+UY/GK4+kfKjsXLa8yxZ9RdFrnZTn1TzVLCN7OrU0ttPlbbV2Qb1WlvNW5vvyex+1/8kG4PrAg3IIQ08rqmV0gVWdF76m2flWvPOawDz6nPDZ7aY5CxByAu7BX5va3tVs1lELc/Nrl9DWtmwhi5+VQjKBRv1j/3Y6+uycyRuJ5s23KzzXax/QOqw97PArKl2EZlx3CY3eyK5Se4pp6mV+ZeKaOBqKogXgVvsNZXZiRSn6CJh40SLhI6Tbzw+koqXIc4DQPS4soheMV7/N8cKU5J0j5BCRXe520+tBKYYupAtnRncDoF5Is9oTBFuy1PLe4kuM4t67EHQM1NFqzvCy0NdWHzqSWY/uwtoqfp8jNvfYbsCGWh/KF7ZeWazUJvOXrLkSzt1ZFzGprbidKSadzndzxy5rY495VPn2iE5nPLWSNm0oKlz98 U+3AMU9G j+ztKNvJLKCVp1h/O62eF5+aJq4yJ0J0cG2D21fGD9Cm/V/PWhWDx7Uw7qW/2qLtYdbM/sagwTvLXsjceBvLO5GEkcWCbmYpLsvQfJM9HJY9ktOJ43+YDidOLd8St6m6QmupaOf8sNrqzv0w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Boqun wrote: > On Sat, Jan 21, 2023 at 12:28:14PM +0900, Byungchul Park wrote: > > On Thu, Jan 19, 2023 at 07:07:59PM -0800, Boqun Feng wrote: > > > On Thu, Jan 19, 2023 at 06:23:49PM -0800, Boqun Feng wrote: > > > > On Fri, Jan 20, 2023 at 10:51:45AM +0900, Byungchul Park wrote: > > > > [...] > > > > > > > T0 T1 T2 > > > > > -- -- -- > > > > > unfair_read_lock(A); > > > > > write_lock(B); > > > > > write_lock(A); > > > > > write_lock(B); > > > > > fair_read_lock(A); > > > > > write_unlock(B); > > > > > read_unlock(A); > > > > > read_unlock(A); > > > > > write_unlock(B); > > > > > write_unlock(A); > > > > > > > > > > T0: read_unlock(A) cannot happen if write_lock(B) is stuck by a B owner > > > > > not doing either write_unlock(B) or read_unlock(B). In other words: > > > > > > > > > > 1. read_unlock(A) happening depends on write_unlock(B) happening. > > > > > 2. read_unlock(A) happening depends on read_unlock(B) happening. > > > > > > > > > > T1: write_unlock(B) cannot happen if fair_read_lock(A) is stuck by a A > > > > > owner not doing either write_unlock(A) or read_unlock(A). In other > > > > > words: > > > > > > > > > > 3. write_unlock(B) happening depends on write_unlock(A) happening. > > > > > 4. write_unlock(B) happening depends on read_unlock(A) happening. > > > > > > > > > > 1, 2, 3 and 4 give the following dependencies: > > > > > > > > > > 1. read_unlock(A) -> write_unlock(B) > > > > > 2. read_unlock(A) -> read_unlock(B) > > > > > 3. write_unlock(B) -> write_unlock(A) > > > > > 4. write_unlock(B) -> read_unlock(A) > > > > > > > > > > With 1 and 4, there's a circular dependency so DEPT definitely report > > > > > this as a problem. > > > > > > > > > > REMIND: DEPT focuses on waits and events. > > > > > > > > Do you have the test cases showing DEPT can detect this? > > > > > > > > > > Just tried the following on your latest GitHub branch, I commented all > > > but one deadlock case. Lockdep CAN detect it but DEPT CANNOT detect it. > > > Feel free to double check. > > > > I tried the 'queued read lock' test cases with DEPT on. I can see DEPT > > detect and report it. But yeah.. it's too verbose now. It's because DEPT > > is not aware of the test environment so it's just working hard to report > > every case. > > > > To make DEPT work with the selftest better, some works are needed. I > > will work on it later or you please work on it. > > > > The corresponding report is the following. > > > [...] > > [ 4.593037] context A's detail > > [ 4.593351] --------------------------------------------------- > > [ 4.593944] context A > > [ 4.594182] [S] lock(&rwlock_A:0) > > [ 4.594577] [W] lock(&rwlock_B:0) > > [ 4.594952] [E] unlock(&rwlock_A:0) > > [ 4.595341] > > [ 4.595501] [S] lock(&rwlock_A:0): > > [ 4.595848] [] queued_read_lock_hardirq_ER_rE+0xf4/0x170 > > [ 4.596547] stacktrace: > > [ 4.596797] _raw_read_lock+0xcf/0x110 > > [ 4.597215] queued_read_lock_hardirq_ER_rE+0xf4/0x170 > > [ 4.597766] dotest+0x30/0x7bc > > [ 4.598118] locking_selftest+0x2c6f/0x2ead > > [ 4.598602] start_kernel+0x5aa/0x6d5 > > [ 4.599017] secondary_startup_64_no_verify+0xe0/0xeb > > [ 4.599562] > [...] > > [ 4.608427] [] queued_read_lock_hardirq_RE_Er+0xf4/0x170 > > [ 4.609113] stacktrace: > > [ 4.609366] _raw_write_lock+0xc3/0xd0 > > [ 4.609788] queued_read_lock_hardirq_RE_Er+0xf4/0x170 > > [ 4.610371] dotest+0x30/0x7bc > > [ 4.610730] locking_selftest+0x2c41/0x2ead > > [ 4.611195] start_kernel+0x5aa/0x6d5 > > [ 4.611615] secondary_startup_64_no_verify+0xe0/0xeb > > [ 4.612164] > > [ 4.612325] [W] lock(&rwlock_A:0): > > [ 4.612671] [] queued_read_lock_hardirq_RE_Er+0x100/0x170 > > [ 4.613369] stacktrace: > > [ 4.613622] _raw_read_lock+0xac/0x110 > > [ 4.614047] queued_read_lock_hardirq_RE_Er+0x100/0x170 > > [ 4.614652] dotest+0x30/0x7bc > > [ 4.615007] locking_selftest+0x2c41/0x2ead > > [ 4.615468] start_kernel+0x5aa/0x6d5 > > [ 4.615879] secondary_startup_64_no_verify+0xe0/0xeb > > [ 4.616607] > [...] > > > As I told you, DEPT treats a queued lock as a normal type lock, no > > matter whether it's a read lock. That's why it prints just > > 'lock(&rwlock_A:0)' instead of 'read_lock(&rwlock_A:0)'. If needed, I'm > > gonna change the format. > > > > I checked the selftest code and found, LOCK(B) is transformed like: > > > > LOCK(B) -> WL(B) -> write_lock(&rwlock_B) > > > > That's why '&rwlock_B' is printed instead of just 'B', JFYI. > > > > Nah, you output shows that you've run at least both function > > queued_read_lock_hardirq_RE_Er() > queued_read_lock_hardirq_ER_rE() Indeed! I'm sorry for that. > but if you apply my diff > > https://lore.kernel.org/lkml/Y8oFj9A19cw3enHB@boqun-archlinux/ > > you should only run > > queued_read_lock_hardirq_RE_Er() > > one test. I checked it. DEPT doesn't assume a rwlock switches between recursive read lock and non-recursive read lock in a run time. Maybe it switches since read lock needs to switch to recursive one in interrupt context. By forcing read_lock_is_recursive() to always return false, DEPT works as we expect. Otherwise, it doesn't. Probabily I need to fix it. Thanks. Byungchul