From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DB53C021B1 for ; Thu, 20 Feb 2025 20:37:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8E964401E4; Thu, 20 Feb 2025 15:37:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E3EC34401D8; Thu, 20 Feb 2025 15:37:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D06394401E4; Thu, 20 Feb 2025 15:37:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AFB3C4401D8 for ; Thu, 20 Feb 2025 15:37:35 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 660EE162FD3 for ; Thu, 20 Feb 2025 20:37:35 +0000 (UTC) X-FDA: 83141483670.20.DDE8289 Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) by imf17.hostedemail.com (Postfix) with ESMTP id 9242540014 for ; Thu, 20 Feb 2025 20:37:33 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=kK72U8kj; spf=pass (imf17.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740083853; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NzQmdn3Jk+AmM1L7Qreqx98YTpwYbDt8ekO/n2uIRww=; b=EXhH/Yw1xbYbezy8Ty6Mb1p1l4UBA5Vgzqm1HZuRrz8XjxwKoKpVlZHRIs1FPydN+xtUYV 15X04O2ZkgXikIkp2E0ERfAXodYM2rUm/PYTRGm6BCmhVRCLVDoYoVdlJokHFp8pGuMEjD mVKEizSLA3gZP7qjl9oqYjqvC4V7kxo= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=kK72U8kj; spf=pass (imf17.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740083853; a=rsa-sha256; cv=none; b=4+aqWcCtGagKnbLRabSlEEA/FK8cEsfJ6TtRhSmwAB54kMYQ/lG52TkaLtBprSWAv6sG4q WW26xQ13j7emsdJqaVBDh+9AYcOO+O3Inhhad+tDxEsK19hjBJ58Yo+yNikA7zYKpXQh+R FCRk+UoYAvrQgnFYogvGGA1iBnOD8W4= Date: Thu, 20 Feb 2025 15:37:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1740083851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NzQmdn3Jk+AmM1L7Qreqx98YTpwYbDt8ekO/n2uIRww=; b=kK72U8kj9C8IF5jtTLPzMID9GwNF/c0+AljB1Z7H97Be8YjGyckYRiWbSRGfNM0cnsl2NV 1q7P5mhmvVacRWAfBFZ3d8NpWQ2HRUg7rhUqZJ46AUGUZP6OSXhvhmKRFj846U2c/wOa98 96ajPFvc+7iMnZEG7Utp95tF6ovrUi0= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Vlastimil Babka Cc: Alan Huang , linux-bcachefs@vger.kernel.org, syzbot+fe63f377148a6371a9db@syzkaller.appspotmail.com, linux-mm@kvack.org, Tejun Heo , Dennis Zhou , Christoph Lameter , Michal Hocko Subject: Re: [PATCH] bcachefs: Use alloc_percpu_gfp to avoid deadlock Message-ID: References: <20250212100625.55860-1-mmpgouride@gmail.com> <25FBAAE5-8BC6-41F3-9A6D-65911BA5A5D7@gmail.com> <78d954b5-e33f-4bbc-855b-e91e96278bef@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <78d954b5-e33f-4bbc-855b-e91e96278bef@suse.cz> X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: 89jf19xmfph5g64ti59ij5ubbcyjjxiw X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9242540014 X-HE-Tag: 1740083853-52575 X-HE-Meta: U2FsdGVkX182XH7nTqNPQE5/G/F4YNyPHXeBBp4l53ZyuEvRB7uwDf2vtGJ/+YY3yQ7+iw1lsjV44e1xST0G7MMuIxwao7v2C97EC4QC47kqIlq6Cs2cgeLk5CNUQ4sEPl1pEXs0UwnCzY3rqWPVMRgtNbfQBKJqoJykGBYKnxj9VLLTD5x1Cev9ezsSMMgWh/oSj4jQJ0nG8p82qfJDBJUj1ZsBmcYfmk+ZtvofUA8VE03OS1kAmD+hVCKC/2Iwe8oL443cqqeFYGuqP4pr+OcNjlbpSaVPFN9+D7jwGKy7B+14bqMtojAAdl3vs2T3tOdwrsnd51z3jjChGRIqkvg56E9jMWHRkjn3dPgKtiyEYcPANq53sAfxDcf9fWaAzFcMhy5QF2xoNL7PvKpHODeAw7Jhsaz/ZTMC9zOfZhESZ/1vmz1PhBR280+bbN5PGBsb6cZQ6JtWfu3kESiLKJ+tOKp5phx2LS/gAPtgGvnfQGUBMo5FdDozQg8J+wA4RbzBEQGOhvOTpQw+sY8eko2M5JHsuJOF/JI6rxjUv83WCpDhkci4okhfjflCyLiDE5j/bgNizNbufCS5xV0B8hQUIuNP2LndBb9d69M4Pzws/8L3SjpEVEKYHtNEvCxgyWsqLYmVUrWfW5e7CDk0KuQSPWa1jaJM2e/uzWnAYkhseaMIkThUZR4U8jaNisTxM0j+Hx61EhQIhJl+1klchT5GCklacN2wcX8GCi095CuZ8/3+ScsmyRPffAqQYGWWq8oQRPV/fiAu5EDPoWobvItrtHoJ8Y9+nxl+osTGgJC96GWZviJq6o9dFlB9clTTz/Y3hWnrx1jYhH6M2j/FilhQxlEjg2urP2WWgh8g8wKGMYXgfHs3EBga2/4VWIf1snx/52ruyKOzrOjEsnOpZ25x0EDde04CI+RuaJfxjAejNI6jamlYtzzF7DG1tjbqUs6p7tOOvCGFWnbF2Z1 Z4pyxzy/ QRjov4dqfDEms0EIvOABUCSVCGbvJlwpMuPJ68mihvrQ/uTDS/9LAldOjuX1zFcVAC//Pibg+TX0Bsee2a9DHd64gJk4sXbuYv9F3gVtXmqbhHJ9CAwM4EasOyEKnHfYrHW/ndzfHZSyr7bgVa0Jir3i8eZ6vux4BxrQxamGDb7RTSpdLmU86QemZjfKYPEbgjxh+nSnuOfcAKC3L9WuWXmaQnXeVL3KXSkcsTdephbRbS40Fd+CiWFiOo4pCvjfYokMEyXeAHDpOpsUNah18AwNWY9s+dEB/Rif92OgkCtj7RSU4AM7YjwO/s1aR9oO39i2sTYiYFB4ydO+3wXH3JEKqfEMEL5ya2bf93XSmHccJ0i8KvqVb0Pun7NFgsVyVgtf7xhOMRT5bLdWFxpcucXktwSs8TznJo55xABmcw3ZK7AuKvUsk0Xaolw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000029, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 20, 2025 at 06:16:43PM +0100, Vlastimil Babka wrote: > On 2/20/25 11:57, Alan Huang wrote: > > Ping > > > >> On Feb 12, 2025, at 22:27, Kent Overstreet wrote: > >> > >> Adding pcpu people to the CC > >> > >> On Wed, Feb 12, 2025 at 06:06:25PM +0800, Alan Huang wrote: > >>> The cycle: > >>> > >>> CPU0: CPU1: > >>> bc->lock pcpu_alloc_mutex > >>> pcpu_alloc_mutex bc->lock > >>> > >>> Reported-by: syzbot+fe63f377148a6371a9db@syzkaller.appspotmail.com > >>> Tested-by: syzbot+fe63f377148a6371a9db@syzkaller.appspotmail.com > >>> Signed-off-by: Alan Huang > >> > >> So pcpu_alloc_mutex -> fs_reclaim? > >> > >> That's really awkward; seems like something that might invite more > >> issues. We can apply your fix if we need to, but I want to hear with the > >> percpu people have to say first. > >> > >> ====================================================== > >> WARNING: possible circular locking dependency detected > >> 6.14.0-rc2-syzkaller-00039-g09fbf3d50205 #0 Not tainted > >> ------------------------------------------------------ > >> syz.0.21/5625 is trying to acquire lock: > >> ffffffff8ea19608 (pcpu_alloc_mutex){+.+.}-{4:4}, at: pcpu_alloc_noprof+0x293/0x1760 mm/percpu.c:1782 > >> > >> but task is already holding lock: > >> ffff888051401c68 (&bc->lock){+.+.}-{4:4}, at: bch2_btree_node_mem_alloc+0x559/0x16f0 fs/bcachefs/btree_cache.c:804 > >> > >> which lock already depends on the new lock. > >> > >> > >> the existing dependency chain (in reverse order) is: > >> > >> -> #2 (&bc->lock){+.+.}-{4:4}: > >> lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5851 > >> __mutex_lock_common kernel/locking/mutex.c:585 [inline] > >> __mutex_lock+0x19c/0x1010 kernel/locking/mutex.c:730 > >> bch2_btree_cache_scan+0x184/0xec0 fs/bcachefs/btree_cache.c:482 > >> do_shrink_slab+0x72d/0x1160 mm/shrinker.c:437 > >> shrink_slab+0x1093/0x14d0 mm/shrinker.c:664 > >> shrink_one+0x43b/0x850 mm/vmscan.c:4868 > >> shrink_many mm/vmscan.c:4929 [inline] > >> lru_gen_shrink_node mm/vmscan.c:5007 [inline] > >> shrink_node+0x37c5/0x3e50 mm/vmscan.c:5978 > >> kswapd_shrink_node mm/vmscan.c:6807 [inline] > >> balance_pgdat mm/vmscan.c:6999 [inline] > >> kswapd+0x20f3/0x3b10 mm/vmscan.c:7264 > >> kthread+0x7a9/0x920 kernel/kthread.c:464 > >> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:148 > >> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 > >> > >> -> #1 (fs_reclaim){+.+.}-{0:0}: > >> lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5851 > >> __fs_reclaim_acquire mm/page_alloc.c:3853 [inline] > >> fs_reclaim_acquire+0x88/0x130 mm/page_alloc.c:3867 > >> might_alloc include/linux/sched/mm.h:318 [inline] > >> slab_pre_alloc_hook mm/slub.c:4066 [inline] > >> slab_alloc_node mm/slub.c:4144 [inline] > >> __do_kmalloc_node mm/slub.c:4293 [inline] > >> __kmalloc_noprof+0xae/0x4c0 mm/slub.c:4306 > >> kmalloc_noprof include/linux/slab.h:905 [inline] > >> kzalloc_noprof include/linux/slab.h:1037 [inline] > >> pcpu_mem_zalloc mm/percpu.c:510 [inline] > >> pcpu_alloc_chunk mm/percpu.c:1430 [inline] > >> pcpu_create_chunk+0x57/0xbc0 mm/percpu-vm.c:338 > >> pcpu_balance_populated mm/percpu.c:2063 [inline] > >> pcpu_balance_workfn+0xc4d/0xd40 mm/percpu.c:2200 > >> process_one_work kernel/workqueue.c:3236 [inline] > >> process_scheduled_works+0xa66/0x1840 kernel/workqueue.c:3317 > >> worker_thread+0x870/0xd30 kernel/workqueue.c:3398 > >> kthread+0x7a9/0x920 kernel/kthread.c:464 > >> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:148 > >> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 > > Seeing this as part of the chain (fs reclaim from a worker doing > pcpu_balance_workfn) makes me think Michal's patch could be a fix to this: > > https://lore.kernel.org/all/20250206122633.167896-1-mhocko@kernel.org/ Thanks for the link - that does look like just the thing.