From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6632EC30658 for ; Tue, 2 Jul 2024 07:49:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE9A06B009B; Tue, 2 Jul 2024 03:49:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D9A106B009D; Tue, 2 Jul 2024 03:49:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C61126B009E; Tue, 2 Jul 2024 03:49:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A7EE56B009B for ; Tue, 2 Jul 2024 03:49:10 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3C710A24BE for ; Tue, 2 Jul 2024 07:49:10 +0000 (UTC) X-FDA: 82294036860.21.F6ED9C7 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf22.hostedemail.com (Postfix) with ESMTP id 47530C000E for ; Tue, 2 Jul 2024 07:49:06 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of leo.lilong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=leo.lilong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719906537; a=rsa-sha256; cv=none; b=I3HlgupjuB26pPgbq0P/yAEvbertBY8oyORBkjcz41CY3kHw1cW0Jffwvc9VGxdUSN85Cy mz+Qio31sNdjD3Jf7nq++m4+z43JHN+nVrV4c8DdUfq6EoejMueB6VzKz7Of1o+H1I7ECZ +G0MUpxxVYv2AtuDmmRSWh+avuauLoQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of leo.lilong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=leo.lilong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719906537; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JEPNlXWSS+kToHnaPDHO/UFBzyqFUIxVoZhxOnasNwc=; b=S2iL5rG6HGbARgrI0egpvvCPadimhcNorFt94dVLiAH6uxyw6vI91ZgrPRVqtk9DPGD3E1 Mn2scccYh26XRlq6T9XbWxpM8oKQVlpWbr9aPpynEqd/asf2K+6/TxYczc8u/SFiSwJXn4 Y3BShaYgxV20kjQ5VdDidb03Pc2t0HU= Received: from mail.maildlp.com (unknown [172.19.163.252]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4WCw1S4g0QzQjxl; Tue, 2 Jul 2024 15:45:16 +0800 (CST) Received: from kwepemi500009.china.huawei.com (unknown [7.221.188.199]) by mail.maildlp.com (Postfix) with ESMTPS id 8ACAB18007E; Tue, 2 Jul 2024 15:49:03 +0800 (CST) Received: from localhost (10.175.127.227) by kwepemi500009.china.huawei.com (7.221.188.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Tue, 2 Jul 2024 15:49:03 +0800 Date: Tue, 2 Jul 2024 16:00:21 +0800 From: Long Li To: Dave Chinner CC: , , Subject: Re: [PATCH 07/12] xfs: use __GFP_NOLOCKDEP instead of GFP_NOFS Message-ID: <20240702080021.GA794461@ceph-admin> References: <20240115230113.4080105-1-david@fromorbit.com> <20240115230113.4080105-8-david@fromorbit.com> <20240622094411.GA830005@ceph-admin> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline In-Reply-To: X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi500009.china.huawei.com (7.221.188.199) X-Stat-Signature: tk4npip1h8o1en3mhymuqoeabouzyj4e X-Rspamd-Queue-Id: 47530C000E X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719906546-978447 X-HE-Meta: U2FsdGVkX18H3sW49MoY3/Ys32GFjtcC+DpC/hjJmniKxDRo3S1ELMpHttG8ZEIca4tKi0qGG08CoIBYWPfAZbStoueXaqFpYD1bp76Zt4WD+4YiDAvZGSF4X7hL7ku91xt1J27vSMFZaTbLPfAuj6NNKRPVLYNUjF+Zrpz1WSDkGS9c+q35o2ChJwCMiqN9keJGdP8xuTU9FIkpPr1uDqlIHBlNLMl2AFolGNvocrpablZL2a1YsOBB0zhckQVn8Q7+GCw23hKJUXXoLQ1THFtpTC2sHZTiS5uONUOVYpFShgDyLfXlSZlFpCInVLNKt2aKSF3QUJMzPDjHiNAH7/8nDfvxBg7fgrKOzu5EyYo/3p2SwfHW3t9UDNqytuG7MNfJK17Z2XamAsjRt4hDdlxMMEHfEhaPrAAm2mGnpQutMkTPvTlgDxIduJJMcPLMB1ufaR6FjXEBV9ux7WTb4yIUOtDnFdG76HsjXEK3Q1hB4XT1ECkSAXhJcQzFjUg/RSGiL/LGozQce/DDhmMbHG9fqyYV6BrDVBXTRvevu+Rh/e53P7WG124mDzb2bS6H4waEl1x4CFhYgjqI2I5APxk5RN9Cm7/nHeXhSkim6vyk4FtCLJD4dEEAKZOTktafmzxROloM8Ax0xPEX0N04THr2q6F9JLqlsTfQLxI+sNbWVfO5W226tXQbf85V9Gg9+pTA958oa3U+lLQWisNxpts3uFAxzZ5R2g1Rqj2s3L0/kzk7Ea8o5oxaB+iHDhkRDWG9UV22cI3Dgs4OLS1GxzbEkMMywfn5tqYmMVFzUa3+E1c+m0jyZ7fB2c73TeDoS4VebmCUKyTigGI5qajI595BvE4cwCS5xlA7NUp/Skgyl8nqNJ0pLFR2n/U5sgrJvQmqLD6zmOHAmUqVD8r9gKf31TNqjOzouwgIgGEWpi/AsjjpRVoF0/u8DhOASUbv+pP55NCW5nerOEH90yX Y8Ufpf5C 658OZ8vPkstF5G69LKNTPteDV0B5/gLi76yBBZVQ/QDSmJFMmmC9bi9mEpNA/CZlaEHQ7/OGeTeR2W779ciqlsd3PK570M37EQk2g/C5ZXT6SvFetEE/Fld2i6C2LsgQs7U6bxgwZDUAI4fDzDI8KK0SDI6MKdJNkPHehoPQ9JJuIr1GnVhQfBXJqwvQbO87ybt2AR1rZOzgIknb4IGy7b5oMMnstyQRK9ZRvPTpUFgSfP5xjWirfvQjDzZ2ox+qtrmfxGKFyDmE6bDce7/ozWJVl2ExNSRSioQ7BE0hBVkEmJUf5TZHWnAh4GdFoqeYe80ZP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000024, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 02, 2024 at 03:55:10PM +1000, Dave Chinner wrote: > On Sat, Jun 22, 2024 at 05:44:11PM +0800, Long Li wrote: > > On Tue, Jan 16, 2024 at 09:59:45AM +1100, Dave Chinner wrote: > > > From: Dave Chinner > > > > > > In the past we've had problems with lockdep false positives stemming > > > from inode locking occurring in memory reclaim contexts (e.g. from > > > superblock shrinkers). Lockdep doesn't know that inodes access from > > > above memory reclaim cannot be accessed from below memory reclaim > > > (and vice versa) but there has never been a good solution to solving > > > this problem with lockdep annotations. > > > > > > This situation isn't unique to inode locks - buffers are also locked > > > above and below memory reclaim, and we have to maintain lock > > > ordering for them - and against inodes - appropriately. IOWs, the > > > same code paths and locks are taken both above and below memory > > > reclaim and so we always need to make sure the lock orders are > > > consistent. We are spared the lockdep problems this might cause > > > by the fact that semaphores and bit locks aren't covered by lockdep. > > > > > > In general, this sort of lockdep false positive detection is cause > > > by code that runs GFP_KERNEL memory allocation with an actively > > > referenced inode locked. When it is run from a transaction, memory > > > allocation is automatically GFP_NOFS, so we don't have reclaim > > > recursion issues. So in the places where we do memory allocation > > > with inodes locked outside of a transaction, we have explicitly set > > > them to use GFP_NOFS allocations to prevent lockdep false positives > > > from being reported if the allocation dips into direct memory > > > reclaim. > > > > > > More recently, __GFP_NOLOCKDEP was added to the memory allocation > > > flags to tell lockdep not to track that particular allocation for > > > the purposes of reclaim recursion detection. This is a much better > > > way of preventing false positives - it allows us to use GFP_KERNEL > > > context outside of transactions, and allows direct memory reclaim to > > > proceed normally without throwing out false positive deadlock > > > warnings. > > > > Hi Dave, > > > > I recently encountered the following AA deadlock lockdep warning > > in Linux-6.9.0. This version of the kernel has currently merged > > your patch set. I believe this is a lockdep false positive warning. > > Yes, it is. > > > The xfs_dir_lookup_args() function is in a non-transactional context > > and allocates memory with the __GFP_NOLOCKDEP flag in xfs_buf_alloc_pages(). > > Even though __GFP_NOLOCKDEP can tell lockdep not to track that particular > > allocation for the purposes of reclaim recursion detection, it cannot > > completely replace __GFP_NOFS. > > We are not trying to replace GFP_NOFS with __GFP_NOLOCKDEP. What we > are trying to do is annotate the allocation sites where lockdep > false positives will occur. That way if we get a lockdep report from > a location that uses __GFP_NOLOCKDEP, we know that it is either a > false positive or there is some nested allocation that did not honor > __GFP_NOLOCKDEP. > > We've already fixed a bunch of nested allocations (e.g. kasan, > kmemleak, etc) to propagate the __GFP_NOLOCKDEP flag so they don't > generate false positives, either. So the amount of noise has already > been reduced. > > > Getting trapped in direct memory reclaim > > maybe trigger the AA deadlock warning as shown below. > > No, it can't. xfs_dir_lookup() can only lock referenced inodes. > xfs_reclaim_inodes_nr() can only lock unreferenced inodes. It is not > possible for the same inode to be both referenced and unreferenced > at the same time, therefore memory reclaim cannot self deadlock > through this path. Yes, I know. An AA deadlock couldn't happen in this situation because it's not the same inode, so it's just a lockdep false positive warning. > > I expected to see some situations like this when getting rid of > GFP_NOFS (because now memory reclaim runs in places it never used > to). Once I have an idea of the sorts of false positives that are > still being tripped over, I can formulate a plan to eradicate them, > too. Ok, memory reclaim may run in those places where GFP_NOFS is removed. Some new lockdep false positive warnings may appear. I hope this report can help you eradicate them in the future. Thanks for your reply. :) > > -Dave. > -- > Dave Chinner > david@fromorbit.com >