From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D069FE66887 for ; Sun, 24 Nov 2024 13:34:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D66776B0082; Sun, 24 Nov 2024 08:34:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CEF646B0083; Sun, 24 Nov 2024 08:34:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B68AD6B0085; Sun, 24 Nov 2024 08:34:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9570A6B0082 for ; Sun, 24 Nov 2024 08:34:20 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id F2F571C81D7 for ; Sun, 24 Nov 2024 13:34:19 +0000 (UTC) X-FDA: 82821082596.26.E807218 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf02.hostedemail.com (Postfix) with ESMTP id 31A9E80017 for ; Sun, 24 Nov 2024 13:34:14 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Mb6BclrS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of yskelg@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=yskelg@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732455256; a=rsa-sha256; cv=none; b=iyTHRxXifEg7skEsTuui77+b1Ex1t212eKb10fR5tU2oHo1s0SOWylWfJHJMHf1ipUmxvk Zf/ZgNdJridKdOKjYT4FcukcGR746VeUETSIjMsKMbWjgWscY1RB6x9pVPbz5PYzVA3r3u GaiYcxXU3n9sc+uL89xoPMcWsqAyzSE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Mb6BclrS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of yskelg@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=yskelg@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732455256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qtitWj7mwZUOuapDYi1rlQN8mBEo2KiG1ezv0q6gsEE=; b=5ctEZyYfl6ZG7A7zvfGGQ2ZW68w0iZKnkvz6My5fMVO+znvtYtI9ABfLAH5fm5aRfJNnwO hQHOPM+3om5jNtUF6W91ST9Go52xC0azAWC11f9JXNWLkQkGGVtfG70OlmCoCxuRp/JLyC 6zt6paPxj99sy9m9G8xMHx4+YiBURK4= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2121858db03so4462935ad.2 for ; Sun, 24 Nov 2024 05:34:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732455257; x=1733060057; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:cc:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=qtitWj7mwZUOuapDYi1rlQN8mBEo2KiG1ezv0q6gsEE=; b=Mb6BclrSYmh6QVe7aomnpP3/BLIU38dtfuBYaMQrTybPxpVTeH6q40n9Is4ElXrw1x ctgUP7HAF+RD5bN6gTCwCMZh+Q+q2a3BRAspqfcxLpGmWly84akQMGPF5hb+tovJ5wIk o5L4KNOyth1B2vkYrPSV0Dw3MZEVmQSCzoCGdET+ZLtnBzAuemHy8eFBjzgIagQ/PvS6 tyNnMwVCm8zxNs+r1HjDWdWHRXdkhg7SNoovjZzNNn6/KJVXO6gyBZzZeiXize0GzDeO Nw7Td0oSGYdME+4VZIg8XsvrF8cNdBdzGT+lG+bMpNgPy7rfIlsFlzjCHxgLZcoSSquC Vv3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732455257; x=1733060057; h=content-transfer-encoding:in-reply-to:from:cc:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qtitWj7mwZUOuapDYi1rlQN8mBEo2KiG1ezv0q6gsEE=; b=N+HHV3y93BUALO/MIj/u58ib12Ij5p5+pipkVAMKaQwIUTvadMgz9QatowD/hjzvwJ zeOxZd1ZfUu522qlJOs8fzSwQvD0wBjs5srag4CmY31Yb+AS3FJDdIYCcjH2U0SjaAAo m8qx+hpivUYMtwyGBXadLhPE0pzwvh9bg3IAsBtKmOv+p+nq0qmBaR5SmLCMElYa205M x9cDdWk9r6ZoyU16/iW9bWcSiNkNB8tWVHFetz6zc1PgnaG8pX7pxz6ElBJ1uazNWVES j4n1JczvLi0IfKKKbsyO1etsRXZ5nWbvgxxfCgFVQc7+sFcH/GqyQ2T/CY0JxCn8ZLc5 AL6g== X-Forwarded-Encrypted: i=1; AJvYcCVDqUdeTQHpv5wyIlCUb7NpUrxymGx7Tr0cV8JqapyzgVdxTXZuFzjh2yfefKJ/4v9q7Z6TUewvWQ==@kvack.org X-Gm-Message-State: AOJu0YwEKN2yIiGJq/0MoZ1Ryo4ze/Vsz6sXcZHAnJehoaQ7mcOnXR9w MU7BidMdTXV4RXBH8sZUTBRUagSsLeM7FjuHZLQqo4AKMeyQhIH/ X-Gm-Gg: ASbGncuwb2OtiM1xS0q/a9TzZdjmBiYY/ffbWschGZmUGx970coYUb16kHcwPlSym+5 J3/ZpSlDFNSv6jf2NwYUBHt9lpV0NlKqEJyra3fBaujMqAlgFGuRcALWfi4vKbKH6upr6IBLdmd 3w6FAepuQy83pjIp2KQzJMxUR8NvQ2DijIYCRcX/P9OSty0PXRWWEQfJ9G0mAkwvj3B4icB4o6B ePPooNB7PNjoMQhrtRxLkAGrkXg8ktzw4RhVPJYe3rOdjiTxB6FHXIB8n1Fwg== X-Google-Smtp-Source: AGHT+IHLvekYbaPwVuQbj7uVMH6+WJbm/aay5oOW+hS7R6I1jLZXbkDtIA8gi64je3notGAilm7kDw== X-Received: by 2002:a05:6a20:3d8d:b0:1dc:77fc:1cd1 with SMTP id adf61e73a8af0-1e09e4004cfmr5341156637.3.1732455256584; Sun, 24 Nov 2024 05:34:16 -0800 (PST) Received: from [192.168.50.136] ([118.32.98.101]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724ed55099asm3151173b3a.49.2024.11.24.05.34.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 24 Nov 2024 05:34:16 -0800 (PST) Message-ID: <489d941f-c4e8-4d1f-92ee-02074c713dd1@gmail.com> Date: Sun, 24 Nov 2024 22:34:02 +0900 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v14 2/28] dept: Implement Dept(Dependency Tracker) To: Byungchul Park References: <20240508094726.35754-3-byungchul@sk.com> Content-Language: en-US Cc: LKML , kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com, Yeoreum Yun From: Yunseong Kim In-Reply-To: <20240508094726.35754-3-byungchul@sk.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Stat-Signature: fkd6sj5y5dpp858tk7y9xo5ehizpqwfy X-Rspamd-Queue-Id: 31A9E80017 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1732455254-507893 X-HE-Meta: U2FsdGVkX1+Q8/LSXwmPgPaW1/e2nBHFujp7kRqKM4tQdrP64FT02rRTMXxfLMcWLwe8GajiNuJSolKamLER4txz+XCa5tGp1CDNdFA9UCAlrZDGIjU3YFeEVnYxtPlg3/IKK0WNyYft7iOPk/2Jg6Exrk8HniUGz1S9uRBay9RHBkCfmEA2w+aSuRXAuUFfrRpS/tEdTsGW7s7VgtMSsmu6Tnan/RGor0iJZ4E+UBN3If7H69X7GI1b9lxYfNMRc41KX9VT2ZYiaMpZpjSg/ie0idPlEr8O6AY4u6pg9iKVGVTOVpWUlcnvfAmtgiM7DFBeDgj/NrMuowTVXr0UY8oW57C4Nr4nABnIgYOQ2yma3sAP5sYMj8W+RpEeBsjbrYjpun1EoFY5dnWBArnu0piv1HytW1eMRdW87+HWkAfgMGRW/sYOvlJcicHfmlDS8yzBymXfgi7+WVtVE9abO2Lta08809P53+EQxaoZ+bbYizDv7zlnzEuuiSrz9RY0MbYS3VkB11HzBXknuSUY1htsgOR337jkqYGsMbcrAaV50DDtqDZXtPnm4YA787DI8aKg3pIOan0abrFzL9Py715XMIcza55C+AGTXTUspWwDHxXcQ35lb7Hvr1W90rQKUsRE/pTSlzP7IR+Y4eXMkephvSjhSl/CbJaI0jPYoa2VYtNPGJjBt9WsnebWLimvNmFGv+/y2wBOQbwMfygUY9ef+ihIS44EjuL9NfkViW06xG2+J0NGUqFX4D9hsImDHqTkdVFjwEtHsWdUEsnoCvpp8YFOEETgOgLJ6Cv4SCl9lgv1IcbP+WMGSVKPCr5N2/5pfsmuaPfNiyDQOi0x4Hq1S8dj8zvHbdybbJShcE1tfQjbzvVQTOouqvmFlPpXafCd3okwFER+6rnXXX9WPUplCmMQIeLgO5Vncba2zKSKoBVvc3mHaXGL2Vegb+r2K7N5SAirPvKginFO7Xq +L1dXzzx fXruuHtEyUfbyhafz/rcsZIPrK2v4rsCP2BK1Lml/UNyx4qreFlFk2XOUw9Z1U7HmN+bnhWPMMdUEjnS2PTKnzDTXpoLqiZkGBqsH2v+GErh6P02w0upcu8bvviYUqP6UhmAAbv41Ht4C2VFMlyeA8F1lMV1QvVDankYrH5lsrRpa7u6I/vKoZd88+OCVzsCtEkL3HQpdzASchO6dznbl6auc51mUG369GwnyWaCCtUxz1Qx6GmqEOxHJ5dTdUK0cNhYyqOdWJIqvYyB2WaXemAoku2dougk5Xk4kgPVGW98m09MdQoIXdgCN7x5PbmQAxF69miMYylU7MypyAtT6T2IHJev1AbZvqKy+MBY54s+NU+li/J58a2nka4ehq2k3EM2Rz3Su77TU4KEQpwVmFLOHJXKss46N9LD2H18BjraGXNPQkBFtiblg/rCSpt9ALdqHWPfNfeAtG91is3ivdtcCS/JKF/MYWI7Nmfs0HtwtqghLY6tl91qfSkAojZDOmGa3B3OXpsnctZRcR1AhLRId5G0m95mp2t7gHU2gIHyhO44djCH8MxTJbA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Byungchul, Thank you for the great feature. Currently, DEPT has a bug in the 'dept_key_destroy()' function that must be fixed to ensure proper operation in the upstream Linux kernel. On 5/8/24 6:46 오후, Byungchul Park wrote: > CURRENT STATUS > -------------- > Lockdep tracks acquisition order of locks in order to detect deadlock, > and IRQ and IRQ enable/disable state as well to take accident > acquisitions into account. > > Lockdep should be turned off once it detects and reports a deadlock > since the data structure and algorithm are not reusable after detection > because of the complex design. > > PROBLEM > ------- > *Waits* and their *events* that never reach eventually cause deadlock. > However, Lockdep is only interested in lock acquisition order, forcing > to emulate lock acqusition even for just waits and events that have > nothing to do with real lock. > > Even worse, no one likes Lockdep's false positive detection because that > prevents further one that might be more valuable. That's why all the > kernel developers are sensitive to Lockdep's false positive. > > Besides those, by tracking acquisition order, it cannot correctly deal > with read lock and cross-event e.g. wait_for_completion()/complete() for > deadlock detection. Lockdep is no longer a good tool for that purpose. > > SOLUTION > -------- > Again, *waits* and their *events* that never reach eventually cause > deadlock. The new solution, Dept(DEPendency Tracker), focuses on waits > and events themselves. Dept tracks waits and events and report it if > any event would be never reachable. > > Dept does: > . Works with read lock in the right way. > . Works with any wait and event e.i. cross-event. > . Continue to work even after reporting multiple times. > . Provides simple and intuitive APIs. > . Does exactly what dependency checker should do. > > Q & A > ----- > Q. Is this the first try ever to address the problem? > A. No. Cross-release feature (b09be676e0ff2 locking/lockdep: Implement > the 'crossrelease' feature) addressed it 2 years ago that was a > Lockdep extension and merged but reverted shortly because: > > Cross-release started to report valuable hidden problems but started > to give report false positive reports as well. For sure, no one > likes Lockdep's false positive reports since it makes Lockdep stop, > preventing reporting further real problems. > > Q. Why not Dept was developed as an extension of Lockdep? > A. Lockdep definitely includes all the efforts great developers have > made for a long time so as to be quite stable enough. But I had to > design and implement newly because of the following: > > 1) Lockdep was designed to track lock acquisition order. The APIs and > implementation do not fit on wait-event model. > 2) Lockdep is turned off on detection including false positive. Which > is terrible and prevents developing any extension for stronger > detection. > > Q. Do you intend to totally replace Lockdep? > A. No. Lockdep also checks if lock usage is correct. Of course, the > dependency check routine should be replaced but the other functions > should be still there. > > Q. Do you mean the dependency check routine should be replaced right > away? > A. No. I admit Lockdep is stable enough thanks to great efforts kernel > developers have made. Lockdep and Dept, both should be in the kernel > until Dept gets considered stable. > > Q. Stronger detection capability would give more false positive report. > Which was a big problem when cross-release was introduced. Is it ok > with Dept? > A. It's ok. Dept allows multiple reporting thanks to simple and quite > generalized design. Of course, false positive reports should be fixed > anyway but it's no longer as a critical problem as it was. > > Signed-off-by: Byungchul Park If a module previously checked for dependencies by DEPT is loaded and then would be unloaded, a kernel panic shall occur when the kernel reuses the corresponding memory area for other purposes. This issue must be addressed as a priority to enable the use of DEPT. Testing this patch on the Ubuntu kernel confirms the problem. > +void dept_key_destroy(struct dept_key *k) > +{ > + struct dept_task *dt = dept_task(); > + unsigned long flags; > + int sub_id; > + > + if (unlikely(!dept_working())) > + return; > + > + if (dt->recursive == 1 && dt->task_exit) { > + /* > + * Need to allow to go ahead in this case where > + * ->recursive has been set to 1 by dept_off() in > + * dept_task_exit() and ->task_exit has been set to > + * true in dept_task_exit(). > + */ > + } else if (dt->recursive) { > + DEPT_STOP("Key destroying fails.\n"); > + return; > + } > + > + flags = dept_enter(); > + > + /* > + * dept_key_destroy() should not fail. > + * > + * FIXME: Should be fixed if dept_key_destroy() causes deadlock > + * with dept_lock(). > + */ > + while (unlikely(!dept_lock())) > + cpu_relax(); > + > + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { > + struct dept_class *c; > + > + c = lookup_class((unsigned long)k->base + sub_id); > + if (!c) > + continue; > + > + hash_del_class(c); > + disconnect_class(c); > + list_del(&c->all_node); > + invalidate_class(c); > + > + /* > + * Actual deletion will happen on the rcu callback > + * that has been added in disconnect_class(). > + */ > + del_class(c); > + } > + > + dept_unlock(); > + dept_exit(flags); > + > + /* > + * Wait until even lockless hash_lookup_class() for the class > + * returns NULL. > + */ > + might_sleep(); > + synchronize_rcu(); > +} > +EXPORT_SYMBOL_GPL(dept_key_destroy); Best regards, Yunseong Kim