From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30032C61CE8 for ; Mon, 9 Jun 2025 09:47:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD23B6B008C; Mon, 9 Jun 2025 05:47:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA9116B009B; Mon, 9 Jun 2025 05:47:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABEC06B009C; Mon, 9 Jun 2025 05:47:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 83C886B008C for ; Mon, 9 Jun 2025 05:47:36 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 23465C1813 for ; Mon, 9 Jun 2025 09:47:36 +0000 (UTC) X-FDA: 83535384912.29.44947B4 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf24.hostedemail.com (Postfix) with ESMTP id 64F62180008 for ; Mon, 9 Jun 2025 09:47:34 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hvwRkiU7; spf=pass (imf24.hostedemail.com: domain of "SRS0=kYjE=YY=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 147.75.193.91 as permitted sender) smtp.mailfrom="SRS0=kYjE=YY=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749462454; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZrMN9MhFQJG//HfxfadrmS3N8QI8lXRVQS4bWcX2PkM=; b=riJ+7aLeE4UvZK5DoCadal28hA/igUGcBKXTZ8xhq/aHpnr15cQrMWmPEzH+Qpd2V4Dv9v qsAC89WUvuEs64eC5xaQ2HReIyKvLKzUtWmUF59+0frbrBOJc7qsAaXeKcquP7kIvapB88 6d/N1oef2CuJbx8h8o4nrblTgiMediM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hvwRkiU7; spf=pass (imf24.hostedemail.com: domain of "SRS0=kYjE=YY=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 147.75.193.91 as permitted sender) smtp.mailfrom="SRS0=kYjE=YY=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749462454; a=rsa-sha256; cv=none; b=i9UnrskeQJfwApEulSzS77OIAoDWCpeJOwPoA31IMB35tqHPKeG83553DQMwtFMtsXb6FY OMgVQZakPc6hSoLGh0C8R827rRiICqtVXXra1Y3CILx9hCcZLmzLgg3vYaQ67sXFktYqZ9 sx/S4aasMwoFj95Djg23EeGWUoLqKhQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id BAD00A50AB4; Mon, 9 Jun 2025 09:47:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A536C4CEEB; Mon, 9 Jun 2025 09:47:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749462453; bh=1S9Nm6f5O7m3vI4hvRuferiAQBF4taD3Nf0gweHpbdU=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=hvwRkiU7h8ctg2xeY2aHY8qarL7QCRjdKDyKh3YyX0kfU0ngw0WWHBRiVfyxrRrvr r+/JpfqOjCh+h0WRt6zmeR6n9/wnRxoB6NK8yO5ZrGU4MjilF3x6CcXGz/RtHfzhs7 9G1L0iWfzNoWQ7XcWWa3iQmrOHJ1Qk8ClE++87ZwtFGLA3JS3ehfjwMont8BQ1CM4a d481YcTWJrtJkiE1A8XB80L/McR+q2JucMSXwecG+OHcTj24X4gF5J6HI1866WSe2k uEaopo6XsNgr2NL3j++7zBNXxfetlIOdK2d4QL48hBd+5dTr0osTsNw9YhRNgdamJo nGbsstIkBZR+g== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 02F00CE0B6C; Mon, 9 Jun 2025 02:47:33 -0700 (PDT) Date: Mon, 9 Jun 2025 02:47:32 -0700 From: "Paul E. McKenney" To: Uladzislau Rezki Cc: Kent Overstreet , syzbot , akpm@linux-foundation.org, josh@joshtriplett.org, linux-bcachefs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, syzkaller-bugs@googlegroups.com Subject: Re: [syzbot] [rcu?] [bcachefs?] BUG: unable to handle kernel NULL pointer dereference in rcu_core (3) Message-ID: Reply-To: paulmck@kernel.org References: <67a2b20a.050a0220.50516.0003.GAE@google.com> <9694d40a-072e-47c2-a950-3b258bbe04f5@paulmck-laptop> <602bb1be-f4a4-4194-803f-856e95711870@paulmck-laptop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: x3cj6wserkipganiydqnspw9gswnt3of X-Rspamd-Queue-Id: 64F62180008 X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1749462454-890541 X-HE-Meta: U2FsdGVkX1/UM2cy4sd3w5xfs6j+klzhfYgDOSA3SKfZxMmgIEVyn8qMsZbhY14Dz1CToUkbXFLA5PiU/mlOvKCEUI5Kvr/H3xtxrh1ejW6+9IADdd0UMtaYYw243ngUKWC5rqdUcv9mBLiJ5UyeP14JsJ20potzSP2ZB2xIUGamETrtBdUa5d8pxVGV9xt1avyq2QBqc4R6UEEFeM4IOOh9lF3ZnSFoXxqT6svJFGBvb2UEdZ6aETo/y+5fic7/J4EqdEIgHWK9i90Fm3G9Xkm3CLfHF6v4bb1DblZuih70bEeS8S1yRIRUklmU9UbLJUpMcKdSz/VdIcEv5eB+XOoTEOxmIE8yWALyc3slUEbGsGDa50o1XsP+8FbgslNTWTaCqMnWbUcSIjcj59+YtfcOyjLM7nSKKmfQkrbrtZbKcS2L2fGDo5gq7oBRIHzlhWUi/NH7IiJFhMtoamN4XNT0+eOFePWRciLVplY1+63osXkjucFj/ppH3+ic3Br+yNonN7KIpLDA0vMYwUuB7rhza6YrRxIfWSY4vGzu1ceUVh97P9N892nRyH2SG02Fgw02XQUH9KXyIjFdHtYkFW1RhV/6bRDEVOBpifYTcValCJJukuz+v9HjxJHAJPZrTdE6lwGhu2EAOvjLLD8p0wzLrPdGC1vAcEds/+PQdQaRMHPrYsh56QAIqf+2sIbqJlOGBvNeDcwORmczDQpTZDpb08LwF9/trohWkAO3GT9ndW86Wi4O7EOczH0najK7Zl0nJtgImEqttIim/Ms+YZmqG4vaHkn76D54VfJE+xMsoBe62ALUtSipXLvM5cFkYSK4UsPpikMMHRE/jIRjyRDLityL89zM/6n0mDoslxpZXJKhmthGvjuM0RtXeOYh2fGfdgSCQGnOLgo7WT/mNYHaMCQy5/Ghlvm+/waBp2np0zsXwaLzgR9p5TkeJRzij8iNHDrJvyErbPMsouW 7rGcgwSU wpdevOUBjhC9vmAu1m3zTsxeym9Br9YMRgNVrtv33rLIv8glJ3RiiuSGOGlTGuAfFTERMXVWNt5bHFtZnEpUGlo7KccVKTZjVz/cuWCmeMtW/VkeKNy4R7t9UWWWEoS7r5dEDSHTObDIn3rG+bT5fhE1UfrKoE+RQqrKu6KNoyY9IkRhk3DUQN+eKvl2NB+oEY91n/8IKsX2H5vDNeanbJ5v8wHY9MrB0TZakBCtxDkkd2JUEBc9Dp1pIFxAWmZWLwGYfQzGlA9Ya+P3wBrVqc+ODA/PMQJ+HPWz0imlM//QS4UoFG2lqUQlQ72B3e+6BZph3jYaA3YqbA+jGSSTfzgB6hVXmKjLJOfcdcErNoU2lqEvVKCxR+B5CLmWLjIBB1kWBkvQRHX8RmcJ2up34q0RpeHyVcLWszwhbuFgR+l/coiKR1gSW58C/Cs2Vq1/kspzlXwa0PSHhoVxRe+sDd3QoaQ3a+UzG4Cj2Hqvo9UPAIT0+0KJFNYX0lWk+Tyl2A+mVGnVN7xdQDkwOgPgV8o0FiuwQGXv28mHIgNqWz/UCvGhPYqArQ63ZJP7EDLF0MMXJ24HyUcd2M4LMKpBAAkuRkLOHjzhN10NxEynm9FmLgJ8I4y6Ig/o6DBW0wLbht2ByTj+CI8xwFLPj6P6OPxdykssSIVRDcsY05qNcTqTqTDdJvxeK2dhkhzFwYaeMpl8yhfaud27DkU79yDOGRXoKzokGA7N9G5fAKVVEapAeBN6T2Zqjxg3LwytVQ0E4wL+CRwuYsRX92oJujhpW1aQyRoLN1NZbMS/YYHs5QHGc29X9Zu9lSbbTHLYv/X+qIOcjJ+uvHPVavsZNu0nWT2lCgsPKmrzYWMX34aKgFhitrKyDT6Uk7b9y37vNr/ce3RjQkkUnSxqktiQmphKKPuQ/+g/JUQrYYJEL5PARBOcriS7h1U0GIuYb60ny/xLXeKQjeVns8EkeI5+K4RNjUiysLbTE XB0h+Q7l 0rwdUrs/mqNjom0Hn2kCygBrSXYH8+mZ+1WbGLFoM8l+WeI1ImXPdI3QxUiUNTnhGA+ZW9GZ+N1AW2u9tUMh1CGQqH2jloPqhDBMyj/GwpCamoRZcx+Vb7x9RKO8MBvTzXV8kVKblUWZQEQyVqCEzMs/uYwqTTRrJTpa/3Wf9vfLZHiGGH+5PKUoGGjXkOOcpAzvoIdTCz8zEPMrl59o5uHIg9OGuVveLV2gBLsRPTEPvoWalPAeV38Q3IMEoxE0KZwj5gLzUHj+nOauClB5cCyK934gwhU5OE7IhfOtX24gbC+smeqqgph429n2/BnDW1TngZCGuXMDzKIlrRhUZdsBvmSB3B4P X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jun 09, 2025 at 10:35:34AM +0200, Uladzislau Rezki wrote: > On Sun, Jun 08, 2025 at 05:25:05PM -0700, Paul E. McKenney wrote: > > On Sun, Jun 08, 2025 at 08:23:36PM +0200, Uladzislau Rezki wrote: > > > On Sun, Jun 08, 2025 at 11:26:28AM -0400, Kent Overstreet wrote: > > > > On Wed, Feb 05, 2025 at 06:56:19AM -0800, Paul E. McKenney wrote: > > > > > On Tue, Feb 04, 2025 at 04:34:18PM -0800, syzbot wrote: > > > > > > Hello, > > > > > > > > > > > > syzbot found the following issue on: > > > > > > > > > > > > HEAD commit: 0de63bb7d919 Merge tag 'pull-fix' of git://git.kernel.org/.. > > > > > > git tree: upstream > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=10faf5f8580000 > > > > > > kernel config: https://syzkaller.appspot.com/x/.config?x=1909f2f0d8e641ce > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=80e5d6f453f14a53383a > > > > > > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 > > > > > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16b69d18580000 > > > > > > > > > > > > Downloadable assets: > > > > > > disk image (non-bootable): https://storage.googleapis.com/syzbot-assets/7feb34a89c2a/non_bootable_disk-0de63bb7.raw.xz > > > > > > vmlinux: https://storage.googleapis.com/syzbot-assets/1142009a30a7/vmlinux-0de63bb7.xz > > > > > > kernel image: https://storage.googleapis.com/syzbot-assets/5d9e46a8998d/bzImage-0de63bb7.xz > > > > > > mounted in repro: https://storage.googleapis.com/syzbot-assets/526692501242/mount_0.gz > > > > > > > > > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > > > > > Reported-by: syzbot+80e5d6f453f14a53383a@syzkaller.appspotmail.com > > > > > > > > > > > > slab radix_tree_node start ffff88803bf382c0 pointer offset 24 size 576 > > > > > > BUG: kernel NULL pointer dereference, address: 0000000000000000 > > > > > > #PF: supervisor instruction fetch in kernel mode > > > > > > #PF: error_code(0x0010) - not-present page > > > > > > PGD 0 P4D 0 > > > > > > Oops: Oops: 0010 [#1] PREEMPT SMP KASAN NOPTI > > > > > > CPU: 0 UID: 0 PID: 5705 Comm: syz-executor Not tainted 6.14.0-rc1-syzkaller-00020-g0de63bb7d919 #0 > > > > > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014 > > > > > > RIP: 0010:0x0 > > > > > > Code: Unable to access opcode bytes at 0xffffffffffffffd6. > > > > > > RSP: 0018:ffffc90000007bd8 EFLAGS: 00010246 > > > > > > RAX: dffffc0000000000 RBX: 1ffff110077e705c RCX: 23438dd059a4b100 > > > > > > RDX: 0000000000000100 RSI: 0000000000000000 RDI: ffff88803bf382d8 > > > > > > RBP: ffffc90000007e10 R08: ffffffff819f146c R09: 1ffff11003f8519a > > > > > > R10: dffffc0000000000 R11: 0000000000000000 R12: ffffffff81a6d507 > > > > > > R13: ffff88803bf382e0 R14: 0000000000000000 R15: ffff88803bf382d8 > > > > > > FS: 0000555567992500(0000) GS:ffff88801fc00000(0000) knlGS:0000000000000000 > > > > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > > > CR2: ffffffffffffffd6 CR3: 000000004da38000 CR4: 0000000000352ef0 > > > > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > > > > > > Call Trace: > > > > > > > > > > > > rcu_do_batch kernel/rcu/tree.c:2546 [inline] > > > > > > > > > > The usual way that this happens is that someone clobbers the rcu_head > > > > > structure of something that has been passed to call_rcu(). The most > > > > > popular way of clobbering this structure is to pass the same something to > > > > > call_rcu() twice in a row, but other creative arrangements are possible. > > > > > > > > > > Building your kernel with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y can usually > > > > > spot invoking call_rcu() twice in a row. > > > > > > > > I don't think it's that - syzbot's .config already has that enabled. > > > > KASAN, too. > > > > > > > > And the only place we do call_rcu() is from rcu_pending.c, where we've > > > > got a rearming rcu callback - but we track whether it's outstanding, and > > > > we do all relevant operations with a lock held. > > > > > > > > And we only use rcu_pending.c with SRCU, not regular RCU. > > > > > > > > We do use kfree_rcu() in a few places (all boring, I expect), but that > > > > doesn't (generally?) use the rcu callback list. > > > > > > > Right, kvfree_rcu() does not intersect with regular callbacks, it has > > > its own path. > > > > > > It looks like the problem is here: > > > > > > > > > f = rhp->func; > > > debug_rcu_head_callback(rhp); > > > WRITE_ONCE(rhp->func, (rcu_callback_t)0L); > > > f(rhp); > > > > > > > > > we do not check if callback, "f", is a NULL. If it is, the kernel bug > > > is triggered right away. For example: > > > > > > call_rcu(&rh, NULL); > > > > > > @Paul, do you think it makes sense to narrow callers which apparently > > > pass NULL as a callback? To me it seems the case of this bug. But we > > > do not know the source. > > > > > > It would give at least a stack-trace of caller which passes a NULL. > > > > Adding a check for NULL func passed to __call_rcu_common(), you mean? > > > Yes. Currently there is no any check. So passing a NULL just triggers > kernel panic. > > > > > That wouldn't hurt, and would either (as you say) catch the culprit > > or show that the problem is elsewhere. > > > I can add it then and send out the patch if no objections. No objections from me! Thanx, Paul