From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D21AC00140 for ; Fri, 12 Aug 2022 09:29:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 189106B0073; Fri, 12 Aug 2022 05:29:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 139488E0003; Fri, 12 Aug 2022 05:29:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0009C8E0002; Fri, 12 Aug 2022 05:29:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E2C816B0073 for ; Fri, 12 Aug 2022 05:29:58 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B3600C0888 for ; Fri, 12 Aug 2022 09:29:58 +0000 (UTC) X-FDA: 79790418876.06.5314B26 Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by imf19.hostedemail.com (Postfix) with ESMTP id 193E61A0006 for ; Fri, 12 Aug 2022 09:29:57 +0000 (UTC) Received: by mail-lj1-f179.google.com with SMTP id v10so384855ljh.9 for ; Fri, 12 Aug 2022 02:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=92ou/u14nlAhRLAjyH+K5VTnxlst6DGZsroJdwVOovQ=; b=BYUOxr06BCh8L+j+0kdrblTLsfOAc/S1qPRVrIOUjSIteDptTVgc3VdOuPlNx8LkCE 7HL2VOP0xS7+FoFkFpGkV/2J62IDmuPMsY3+3zCsuSBJ/or6VNQAAOTVXj3RMINn3H7L VL2qIe3Z2mCX6vXter3C+GKO0XkjbhdppPRoRdPMKvVHG0EFp1ff508s8QqKCEQ8960l FGCxlwJNlnJRYyrpBQZdIr55z5UyDM3M/yL4Rfxr8HH5vIuvKOUaNXEcZ7cFA9GvA4wj egjarR98yKRN6RZPLnyEmnljddhDPuWpMH5ffg9hGkG034uwDBRtap6niExmB3h+RrUj VAbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=92ou/u14nlAhRLAjyH+K5VTnxlst6DGZsroJdwVOovQ=; b=Rbu4Vi61rXHC/2bbpSU5EB0XV7lP78M6Wghe6Z5p1DyzXgNwyr8s0y1LlMXbe4Tc7t +UXX3vJ2/SRcwfwb6sE0EnUkEn7+M3kSIN63vaX/B/NqUSwddp99X8og8bZV9w55dTJ8 ICUU0cLR1LHGxkRpoEhq9KBKviP2T8TVmfGXGoDfsoIAR1TxILLUKDu2BPfbScIHe1uP YUaVjHBVFcvFUZOfVrKZavo8VVI4sDpzZ9D2mGjAI6VXD58zuxOz3IXyDbFtR9hN6V7q JCg9y7rxZomtx1/u5nmwDgO3ScfZ4tgepwUVBhDcYxekcidkMJUMJWwpjyc7VZvIMOI4 vhcw== X-Gm-Message-State: ACgBeo2FgYr6SdLiRrRlhYa+bxJzYkmoMOBYp+PlkDjk/us9mCYwzsTm BW8bHo9OLkiITbv9T5SwjJ5oqk+qLPWuOacdN05O/g== X-Google-Smtp-Source: AA6agR6tYzV/IQQ/OxLEjd+jk1f2ZSvYtf5ruKBh0ui0nUtixe61EgEuG7yugxxBKG9qC5S4CehK8y+rA0CaaMZcuSc= X-Received: by 2002:a2e:a5ca:0:b0:25e:1c49:70f4 with SMTP id n10-20020a2ea5ca000000b0025e1c4970f4mr988263ljp.4.1660296596373; Fri, 12 Aug 2022 02:29:56 -0700 (PDT) MIME-Version: 1.0 References: <0000000000008c0ba505e5f22066@google.com> <202208110830.8F528D6737@keescook> <202208111356.97951D32@keescook> In-Reply-To: From: Dmitry Vyukov Date: Fri, 12 Aug 2022 11:29:44 +0200 Message-ID: Subject: Re: [syzbot] linux-next boot error: BUG: unable to handle kernel paging request in kernel_execve To: Ira Weiny Cc: Kees Cook , "Fabio M. De Francesco" , ebiederm@xmission.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-next@vger.kernel.org, sfr@canb.auug.org.au, syzkaller-bugs@googlegroups.com, viro@zeniv.linux.org.uk, syzbot Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660296598; a=rsa-sha256; cv=none; b=1xPgODv5MSNG3dYipMyTiGocKQfY87UYwjiVIhMElKAiY86xf53BuZHuwCrBrCS4n9ILQZ gCWIlWVKNG1GZGaEYnAu/ZTFz2C7x6S+8+4KW3ROEirl39DRr0IslMm7Qtily8uiiPJqdh IWJNY3kWZaZ3yEMFCI7CvGxmNukB7EU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660296598; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=92ou/u14nlAhRLAjyH+K5VTnxlst6DGZsroJdwVOovQ=; b=jpcPW7NX3lRA6FDguUriv6DL/pF+TcrnWtaNe33t67KDPfa4W3MO36xQZ6PClA+vSPOkZ8 kpS0kPsAw5si/BWpzeqA6mpT7DlxI6ok6+qCBNKzx2CsHlodiNVmPG6ItNNpJ3yFr2Aebp Y/Xh3K9v7tb1LR4CmeFNXKKxbpRKA6M= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BYUOxr06; spf=pass (imf19.hostedemail.com: domain of dvyukov@google.com designates 209.85.208.179 as permitted sender) smtp.mailfrom=dvyukov@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: agz4murte4odeoea6u6ruco7b36w615d X-Rspamd-Queue-Id: 193E61A0006 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BYUOxr06; spf=pass (imf19.hostedemail.com: domain of dvyukov@google.com designates 209.85.208.179 as permitted sender) smtp.mailfrom=dvyukov@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1660296597-937563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 12 Aug 2022 at 02:11, Ira Weiny wrote: > > On Thu, Aug 11, 2022 at 02:00:59PM -0700, Kees Cook wrote: > > On Thu, Aug 11, 2022 at 11:51:34AM -0700, Ira Weiny wrote: > > > On Thu, Aug 11, 2022 at 10:39:29AM -0700, Ira wrote: > > > > On Thu, Aug 11, 2022 at 08:33:16AM -0700, Kees Cook wrote: > > > > > Hi Fabio, > > > > > > > > > > It seems likely that the kmap change[1] might be causing this crash. Is > > > > > there a boot-time setup race between kmap being available and early umh > > > > > usage? > > > > > > > > I don't see how this is a setup problem with the config reported here. > > > > > > > > CONFIG_64BIT=y > > > > > > > > ...and HIGHMEM is not set. > > > > ...and PREEMPT_RT is not set. > > > > > > > > So the kmap_local_page() call in that stack should be a page_address() only. > > > > > > > > I think the issue must be some sort of race which was being prevented because > > > > of the preemption and/or pagefault disable built into kmap_atomic(). > > > > > > > > Is this reproducable? > > > > > > > > The hunk below will surely fix it but I think the pagefault_disable() is > > > > the only thing that is required. It would be nice to test it. > > > > > > Fabio and I discussed this. And he also mentioned that pagefault_disable() is > > > all that is required. > > > > Okay, sounds good. > > > > > Do we have a way to test this? > > > > It doesn't look like syzbot has a reproducer yet, so its patch testing > > system[1] will not work. But if you can send me a patch, I could land it > > in -next and we could see if the reproduction frequency drops to zero. > > (Looking at the dashboard, it's seen 2 crashes, most recently 8 hours > > ago.) > > Patch sent. > > https://lore.kernel.org/lkml/20220812000919.408614-1-ira.weiny@intel.com/ > > But I'm more confused after looking at this again. There is splat of random crashes in linux-next happened at the same time: https://groups.google.com/g/syzkaller-bugs/search?q=%22linux-next%20boot%20error%3A%22 There are 10 different crashes in completely random places. I would assume they have the same root cause, some silent memory corruption or something similar. > > [1] https://github.com/google/syzkaller/blob/master/docs/syzbot.md#testing-patches > > > > > > > > syzbot found the following issue on: > > > > > > > > > > > > HEAD commit: bc6c6584ffb2 Add linux-next specific files for 20220810 > > > > > > git tree: linux-next > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=115034c3080000 > > > > > > kernel config: https://syzkaller.appspot.com/x/.config?x=5784be4315a4403b > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=3250d9c8925ef29e975f > > > > > > compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 > > > > > > > > > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > > > > > Reported-by: syzbot+3250d9c8925ef29e975f@syzkaller.appspotmail.com > > > > > > > > > > > > BUG: unable to handle page fault for address: ffffdc0000000000 > > > > > > #PF: supervisor read access in kernel mode > > > > > > #PF: error_code(0x0000) - not-present page > > > > > > PGD 11826067 P4D 11826067 PUD 0 > > > > > > Oops: 0000 [#1] PREEMPT SMP KASAN > > > > > > CPU: 0 PID: 1100 Comm: kworker/u4:5 Not tainted 5.19.0-next-20220810-syzkaller #0 > > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022 > > > > > > RIP: 0010:strnlen+0x3b/0x70 lib/string.c:504 > > > > > > Code: 74 3c 48 bb 00 00 00 00 00 fc ff df 49 89 fc 48 89 f8 eb 09 48 83 c0 01 48 39 e8 74 1e 48 89 c2 48 89 c1 48 c1 ea 03 83 e1 07 <0f> b6 14 1a 38 ca 7f 04 84 d2 75 11 80 38 00 75 d9 4c 29 e0 48 83 > > > > > > RSP: 0000:ffffc90005c5fe10 EFLAGS: 00010246 > > > > > > RAX: ffff000000000000 RBX: dffffc0000000000 RCX: 0000000000000000 > > > > > > RDX: 1fffe00000000000 RSI: 0000000000020000 RDI: ffff000000000000 > > > > > > RBP: ffff000000020000 R08: 0000000000000005 R09: 0000000000000000 > > > > > > R10: 0000000000000006 R11: 0000000000000000 R12: ffff000000000000 > > > > > > R13: ffff88814764cc00 R14: ffff000000000000 R15: ffff88814764cc00 > > > > > > FS: 0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000 > > > > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > > > CR2: ffffdc0000000000 CR3: 000000000bc8e000 CR4: 00000000003506f0 > > > > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > > > > > > Call Trace: > > > > > > > > > > > > strnlen include/linux/fortify-string.h:119 [inline] > > > > > > copy_string_kernel+0x26/0x250 fs/exec.c:616 > > > > > > copy_strings_kernel+0xb3/0x190 fs/exec.c:655 > > > > > > kernel_execve+0x377/0x500 fs/exec.c:1998 > > > > > > call_usermodehelper_exec_async+0x2e3/0x580 kernel/umh.c:112 > > > > > > ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306 > > > > > > > > > [...] > > > > > > --- > > > > > > This report is generated by a bot. It may contain errors. > > > > > > See https://goo.gl/tpsmEJ for more information about syzbot. > > > > > > syzbot engineers can be reached at syzkaller@googlegroups.com. > > > > > > > > > > > > syzbot will keep track of this issue. See: > > > > > > https://goo.gl/tpsmEJ#status for how to communicate with syzbot. > > > > -- > > Kees Cook > > -- > You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group. > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/YvWaqhLGsBp9ynIq%40iweiny-desk3.