[syzbot] BUG: soft lockup in do_wp_page (4)

[syzbot] BUG: soft lockup in do_wp_page (4)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
userspace arch: arm64

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com

watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
Modules linked in:
irq event stamp: 10067468
hardirqs last  enabled at (10067467): [<ffff8000143de478>] exit_to_kernel_mode+0x38/0x230 arch/arm64/kernel/entry-common.c:59
hardirqs last disabled at (10067468): [<ffff8000143deb40>] enter_el1_irq_or_nmi+0x10/0x20 arch/arm64/kernel/entry-common.c:101
softirqs last  enabled at (10067362): [<ffff8000100109e0>] _stext+0x9e0/0x1084
softirqs last disabled at (10067297): [<ffff80001015976c>] do_softirq_own_stack include/asm-generic/softirq_stack.h:10 [inline]
softirqs last disabled at (10067297): [<ffff80001015976c>] invoke_softirq kernel/softirq.c:228 [inline]
softirqs last disabled at (10067297): [<ffff80001015976c>] __irq_exit_rcu+0x46c/0x510 kernel/softirq.c:422
CPU: 0 PID: 3684 Comm: syz-executor.1 Not tainted 5.12.0-rc3-syzkaller-00036-gbf152b0b41dc #0
Hardware name: linux,dummy-virt (DT)
pstate: 00000005 (nzcv daif -PAN -UAO -TCO BTYPE=--)
pc : clear_page+0x14/0x28 arch/arm64/lib/clear_page.S:23
lr : clear_highpage include/linux/highmem.h:203 [inline]
lr : kernel_init_free_pages+0x74/0x180 mm/page_alloc.c:1212
sp : ffff00000c6cf4a0
x29: ffff00000c6cf4a0 x28: fffffc0000180b00 
x27: ffff00007fc13ec0 x26: ffff00000b709458 
x25: ffff6000016e128b x24: 0000000000001458 
x23: ffff00000b708000 x22: fffffc0000180b40 
x21: ffff8000154e2938 x20: dfff800000000000 
x19: fffffc0000180b00 x18: 0000000000000000 
x17: 0000000000000000 x16: 0000000000000000 
x15: 0000000000000000 x14: 1fffe000018d9e5e 
x13: 0000000000000001 x12: ffff7f8000030167 
x11: 1fffff8000030166 x10: ffff7f8000030166 
x9 : 0000000000000000 x8 : ffff600000c05a00 
x7 : 0000000000000000 x6 : 000000000000003f 
x5 : 0000000000000040 x4 : 1ffff00002fa5748 
x3 : 1fffe000016e1001 x2 : 0000000000000004 
x1 : 0000000000000040 x0 : ffff00000602ce00 
Call trace:
 clear_page+0x14/0x28 arch/arm64/lib/clear_page.S:21
 post_alloc_hook+0x130/0x1fc mm/page_alloc.c:2305
 prep_new_page mm/page_alloc.c:2311 [inline]
 get_page_from_freelist+0x2024/0x2a40 mm/page_alloc.c:3951
 __alloc_pages_nodemask+0x2a0/0x24dc mm/page_alloc.c:5001
 alloc_pages_vma+0xb4/0x510 mm/mempolicy.c:2240
 wp_page_copy+0x198/0x16e0 mm/memory.c:2853
 do_wp_page+0x1d8/0xbbc mm/memory.c:3160
 handle_pte_fault mm/memory.c:4336 [inline]
 __handle_mm_fault+0x1020/0x21d4 mm/memory.c:4453
 handle_mm_fault+0x1cc/0x4dc mm/memory.c:4551
 __do_page_fault arch/arm64/mm/fault.c:507 [inline]
 do_page_fault+0x228/0x880 arch/arm64/mm/fault.c:591
 do_mem_abort+0x64/0x1c0 arch/arm64/mm/fault.c:805
 el0_da+0x3c/0x50 arch/arm64/kernel/entry-common.c:308
 el0_sync_handler+0x168/0x1b0 arch/arm64/kernel/entry-common.c:435
 el0_sync+0x170/0x180 arch/arm64/kernel/entry.S:699


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Dmitry Vyukov]

On Thu, Mar 18, 2021 at 8:31 PM syzbot
<syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> userspace arch: arm64
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
>
> watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]

+Will, arm

If I am reading this commit correctly, this is caused by:

commit cae118b6acc309539b33339e846cbb19187c164c
Author: Will Deacon
Date:   Wed Mar 3 13:49:27 2021 +0000
    arm64: Drop support for CMDLINE_EXTEND

syzbot passes lots of critical things in CONFIG_CMDLINE:
https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
but also wants the bootloader args to be appended.
What is the way to do it now?




> Modules linked in:
> irq event stamp: 10067468
> hardirqs last  enabled at (10067467): [<ffff8000143de478>] exit_to_kernel_mode+0x38/0x230 arch/arm64/kernel/entry-common.c:59
> hardirqs last disabled at (10067468): [<ffff8000143deb40>] enter_el1_irq_or_nmi+0x10/0x20 arch/arm64/kernel/entry-common.c:101
> softirqs last  enabled at (10067362): [<ffff8000100109e0>] _stext+0x9e0/0x1084
> softirqs last disabled at (10067297): [<ffff80001015976c>] do_softirq_own_stack include/asm-generic/softirq_stack.h:10 [inline]
> softirqs last disabled at (10067297): [<ffff80001015976c>] invoke_softirq kernel/softirq.c:228 [inline]
> softirqs last disabled at (10067297): [<ffff80001015976c>] __irq_exit_rcu+0x46c/0x510 kernel/softirq.c:422
> CPU: 0 PID: 3684 Comm: syz-executor.1 Not tainted 5.12.0-rc3-syzkaller-00036-gbf152b0b41dc #0
> Hardware name: linux,dummy-virt (DT)
> pstate: 00000005 (nzcv daif -PAN -UAO -TCO BTYPE=--)
> pc : clear_page+0x14/0x28 arch/arm64/lib/clear_page.S:23
> lr : clear_highpage include/linux/highmem.h:203 [inline]
> lr : kernel_init_free_pages+0x74/0x180 mm/page_alloc.c:1212
> sp : ffff00000c6cf4a0
> x29: ffff00000c6cf4a0 x28: fffffc0000180b00
> x27: ffff00007fc13ec0 x26: ffff00000b709458
> x25: ffff6000016e128b x24: 0000000000001458
> x23: ffff00000b708000 x22: fffffc0000180b40
> x21: ffff8000154e2938 x20: dfff800000000000
> x19: fffffc0000180b00 x18: 0000000000000000
> x17: 0000000000000000 x16: 0000000000000000
> x15: 0000000000000000 x14: 1fffe000018d9e5e
> x13: 0000000000000001 x12: ffff7f8000030167
> x11: 1fffff8000030166 x10: ffff7f8000030166
> x9 : 0000000000000000 x8 : ffff600000c05a00
> x7 : 0000000000000000 x6 : 000000000000003f
> x5 : 0000000000000040 x4 : 1ffff00002fa5748
> x3 : 1fffe000016e1001 x2 : 0000000000000004
> x1 : 0000000000000040 x0 : ffff00000602ce00
> Call trace:
>  clear_page+0x14/0x28 arch/arm64/lib/clear_page.S:21
>  post_alloc_hook+0x130/0x1fc mm/page_alloc.c:2305
>  prep_new_page mm/page_alloc.c:2311 [inline]
>  get_page_from_freelist+0x2024/0x2a40 mm/page_alloc.c:3951
>  __alloc_pages_nodemask+0x2a0/0x24dc mm/page_alloc.c:5001
>  alloc_pages_vma+0xb4/0x510 mm/mempolicy.c:2240
>  wp_page_copy+0x198/0x16e0 mm/memory.c:2853
>  do_wp_page+0x1d8/0xbbc mm/memory.c:3160
>  handle_pte_fault mm/memory.c:4336 [inline]
>  __handle_mm_fault+0x1020/0x21d4 mm/memory.c:4453
>  handle_mm_fault+0x1cc/0x4dc mm/memory.c:4551
>  __do_page_fault arch/arm64/mm/fault.c:507 [inline]
>  do_page_fault+0x228/0x880 arch/arm64/mm/fault.c:591
>  do_mem_abort+0x64/0x1c0 arch/arm64/mm/fault.c:805
>  el0_da+0x3c/0x50 arch/arm64/kernel/entry-common.c:308
>  el0_sync_handler+0x168/0x1b0 arch/arm64/kernel/entry-common.c:435
>  el0_sync+0x170/0x180 arch/arm64/kernel/entry.S:699
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/000000000000ce4c9505bdd4a48f%40google.com.

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Will Deacon]

On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> On Thu, Mar 18, 2021 at 8:31 PM syzbot
> <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> > git tree:       upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> > dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> > userspace arch: arm64
> >
> > Unfortunately, I don't have any reproducer for this issue yet.
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
> >
> > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
> 
> +Will, arm
> 
> If I am reading this commit correctly, this is caused by:
> 
> commit cae118b6acc309539b33339e846cbb19187c164c
> Author: Will Deacon
> Date:   Wed Mar 3 13:49:27 2021 +0000
>     arm64: Drop support for CMDLINE_EXTEND
> 
> syzbot passes lots of critical things in CONFIG_CMDLINE:
> https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> but also wants the bootloader args to be appended.
> What is the way to do it now?

For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
either:

  * Revert my patch for your kernels
  * Pass the arguments via QEMU's -append option
  * Take a look at one of the series which should hopefully add this
    functionality back (but with well-defined semantics) [1] [2]

Sorry for the nuisance; I did try to fix this [3] but it's a bit of a
mess.

Will

[1] https://lore.kernel.org/linux-arch/cover.1614705851.git.christophe.leroy@csgroup.eu/
[2] https://lore.kernel.org/linux-mips/20210308235319.2988609-2-danielwa@cisco.com/#t
[3] https://lore.kernel.org/r/20210225125921.13147-1-will@kernel.org

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Dmitry Vyukov]

On Thu, Mar 25, 2021 at 7:20 PM Will Deacon <will@kernel.org> wrote:
>
> On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > >
> > > Hello,
> > >
> > > syzbot found the following issue on:
> > >
> > > HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> > > git tree:       upstream
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> > > kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> > > userspace arch: arm64
> > >
> > > Unfortunately, I don't have any reproducer for this issue yet.
> > >
> > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
> > >
> > > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
> >
> > +Will, arm
> >
> > If I am reading this commit correctly, this is caused by:
> >
> > commit cae118b6acc309539b33339e846cbb19187c164c
> > Author: Will Deacon
> > Date:   Wed Mar 3 13:49:27 2021 +0000
> >     arm64: Drop support for CMDLINE_EXTEND
> >
> > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > but also wants the bootloader args to be appended.
> > What is the way to do it now?
>
> For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> either:
>
>   * Revert my patch for your kernels
>   * Pass the arguments via QEMU's -append option
>   * Take a look at one of the series which should hopefully add this
>     functionality back (but with well-defined semantics) [1] [2]

Unfortunately none of these work for syzbot (and I assume other
testing environments).

syzbot does not support custom patches by design:
http://bit.do/syzbot#no-custom-patches
As any testing system, it tests the official trees.

It's not humans who start these VMs, so it's not as easy as changing
the command line after typing...
There is no support for passing args specifically to qemu, syzkaller
support not just qemu, so these things are specifically localized in
the config. Additionally there is an issue of communicating all these
scattered details to developers in bug reports. Currently syzbot
reports the kernel config and it as well captures command line.

Could you revert the patch? Is there any point in removing the
currently supported feature before the new feature lands?



> Sorry for the nuisance; I did try to fix this [3] but it's a bit of a
> mess.
>
> Will
>
> [1] https://lore.kernel.org/linux-arch/cover.1614705851.git.christophe.leroy@csgroup.eu/
> [2] https://lore.kernel.org/linux-mips/20210308235319.2988609-2-danielwa@cisco.com/#t
> [3] https://lore.kernel.org/r/20210225125921.13147-1-will@kernel.org

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Dmitry Vyukov]

On Thu, Mar 25, 2021 at 7:34 PM Dmitry Vyukov <dvyukov@google.com> wrote:
> > On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > syzbot found the following issue on:
> > > >
> > > > HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> > > > git tree:       upstream
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> > > > userspace arch: arm64
> > > >
> > > > Unfortunately, I don't have any reproducer for this issue yet.
> > > >
> > > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > > Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
> > > >
> > > > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
> > >
> > > +Will, arm
> > >
> > > If I am reading this commit correctly, this is caused by:
> > >
> > > commit cae118b6acc309539b33339e846cbb19187c164c
> > > Author: Will Deacon
> > > Date:   Wed Mar 3 13:49:27 2021 +0000
> > >     arm64: Drop support for CMDLINE_EXTEND
> > >
> > > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > > but also wants the bootloader args to be appended.
> > > What is the way to do it now?
> >
> > For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> > either:
> >
> >   * Revert my patch for your kernels
> >   * Pass the arguments via QEMU's -append option
> >   * Take a look at one of the series which should hopefully add this
> >     functionality back (but with well-defined semantics) [1] [2]
>
> Unfortunately none of these work for syzbot (and I assume other
> testing environments).
>
> syzbot does not support custom patches by design:
> http://bit.do/syzbot#no-custom-patches
> As any testing system, it tests the official trees.
>
> It's not humans who start these VMs, so it's not as easy as changing
> the command line after typing...
> There is no support for passing args specifically to qemu, syzkaller
> support not just qemu, so these things are specifically localized in
> the config. Additionally there is an issue of communicating all these
> scattered details to developers in bug reports. Currently syzbot
> reports the kernel config and it as well captures command line.
>
> Could you revert the patch? Is there any point in removing the
> currently supported feature before the new feature lands?

FTR another instance:
https://syzkaller.appspot.com/bug?extid=b6c93d85a3f77fca7f13
https://lore.kernel.org/lkml/000000000000d7142c05be60db47@google.com/T/#u


> > Sorry for the nuisance; I did try to fix this [3] but it's a bit of a
> > mess.
> >
> > Will
> >
> > [1] https://lore.kernel.org/linux-arch/cover.1614705851.git.christophe.leroy@csgroup.eu/
> > [2] https://lore.kernel.org/linux-mips/20210308235319.2988609-2-danielwa@cisco.com/#t
> > [3] https://lore.kernel.org/r/20210225125921.13147-1-will@kernel.org

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Will Deacon]

On Thu, Mar 25, 2021 at 07:34:54PM +0100, Dmitry Vyukov wrote:
> On Thu, Mar 25, 2021 at 7:20 PM Will Deacon <will@kernel.org> wrote:
> >
> > On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > syzbot found the following issue on:
> > > >
> > > > HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> > > > git tree:       upstream
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> > > > userspace arch: arm64
> > > >
> > > > Unfortunately, I don't have any reproducer for this issue yet.
> > > >
> > > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > > Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
> > > >
> > > > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
> > >
> > > +Will, arm
> > >
> > > If I am reading this commit correctly, this is caused by:
> > >
> > > commit cae118b6acc309539b33339e846cbb19187c164c
> > > Author: Will Deacon
> > > Date:   Wed Mar 3 13:49:27 2021 +0000
> > >     arm64: Drop support for CMDLINE_EXTEND
> > >
> > > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > > but also wants the bootloader args to be appended.
> > > What is the way to do it now?
> >
> > For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> > either:
> >
> >   * Revert my patch for your kernels
> >   * Pass the arguments via QEMU's -append option
> >   * Take a look at one of the series which should hopefully add this
> >     functionality back (but with well-defined semantics) [1] [2]
> 
> Unfortunately none of these work for syzbot (and I assume other
> testing environments).
> 
> syzbot does not support custom patches by design:
> http://bit.do/syzbot#no-custom-patches
> As any testing system, it tests the official trees.
> 
> It's not humans who start these VMs, so it's not as easy as changing
> the command line after typing...
> There is no support for passing args specifically to qemu, syzkaller
> support not just qemu, so these things are specifically localized in
> the config. Additionally there is an issue of communicating all these
> scattered details to developers in bug reports. Currently syzbot
> reports the kernel config and it as well captures command line.
> 
> Could you revert the patch? Is there any point in removing the
> currently supported feature before the new feature lands?

Well, we only just merged it (in 5.10 I think?), and the semantics of the
new version will be different, so I really don't see the value in supporting
both (even worse, Android has its own implementation which is different
again). The timeline was: we merged CMDLINE_EXTEND, then we noticed it was
broken, my fixes were rejected, so we removed the feature rather than
support the broken version. In the relatively small window while it was
merged, syzbot started using it :(

So I really think the best bet is to wait until the patches are sorted out.
I think Christophe is about to spin a new version, and I reviewed his last
copy, so I don't see this being far off,

Will

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Dmitry Vyukov]

On Thu, Mar 25, 2021 at 8:10 PM Will Deacon <will@kernel.org> wrote:
>
> On Thu, Mar 25, 2021 at 07:34:54PM +0100, Dmitry Vyukov wrote:
> > On Thu, Mar 25, 2021 at 7:20 PM Will Deacon <will@kernel.org> wrote:
> > >
> > > On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > > > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > > > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > syzbot found the following issue on:
> > > > >
> > > > > HEAD commit:    bf152b0b Merge tag 'for_linus' of git://git.kernel.org/pub..
> > > > > git tree:       upstream
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=17d5264ed00000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=2c9917c41f0bc04b
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=0b036374a865ba0efa8e
> > > > > userspace arch: arm64
> > > > >
> > > > > Unfortunately, I don't have any reproducer for this issue yet.
> > > > >
> > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > > > Reported-by: syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com
> > > > >
> > > > > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor.1:3684]
> > > >
> > > > +Will, arm
> > > >
> > > > If I am reading this commit correctly, this is caused by:
> > > >
> > > > commit cae118b6acc309539b33339e846cbb19187c164c
> > > > Author: Will Deacon
> > > > Date:   Wed Mar 3 13:49:27 2021 +0000
> > > >     arm64: Drop support for CMDLINE_EXTEND
> > > >
> > > > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > > > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > > > but also wants the bootloader args to be appended.
> > > > What is the way to do it now?
> > >
> > > For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> > > either:
> > >
> > >   * Revert my patch for your kernels
> > >   * Pass the arguments via QEMU's -append option
> > >   * Take a look at one of the series which should hopefully add this
> > >     functionality back (but with well-defined semantics) [1] [2]
> >
> > Unfortunately none of these work for syzbot (and I assume other
> > testing environments).
> >
> > syzbot does not support custom patches by design:
> > http://bit.do/syzbot#no-custom-patches
> > As any testing system, it tests the official trees.
> >
> > It's not humans who start these VMs, so it's not as easy as changing
> > the command line after typing...
> > There is no support for passing args specifically to qemu, syzkaller
> > support not just qemu, so these things are specifically localized in
> > the config. Additionally there is an issue of communicating all these
> > scattered details to developers in bug reports. Currently syzbot
> > reports the kernel config and it as well captures command line.
> >
> > Could you revert the patch? Is there any point in removing the
> > currently supported feature before the new feature lands?
>
> Well, we only just merged it (in 5.10 I think?), and the semantics of the
> new version will be different, so I really don't see the value in supporting
> both (even worse, Android has its own implementation which is different
> again). The timeline was: we merged CMDLINE_EXTEND, then we noticed it was
> broken, my fixes were rejected, so we removed the feature rather than
> support the broken version. In the relatively small window while it was
> merged, syzbot started using it :(

I didn't realize it was just introduced :)
We used CMDLINE_EXTEND on x86, and I looked for a similar option for
arm64 and found it.

> So I really think the best bet is to wait until the patches are sorted out.
> I think Christophe is about to spin a new version, and I reviewed his last
> copy, so I don't see this being far off,

If it's expected to be merged soon, let's wait.

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Will Deacon]

On Thu, Mar 25, 2021 at 08:24:53PM +0100, Dmitry Vyukov wrote:
> On Thu, Mar 25, 2021 at 8:10 PM Will Deacon <will@kernel.org> wrote:
> > On Thu, Mar 25, 2021 at 07:34:54PM +0100, Dmitry Vyukov wrote:
> > > On Thu, Mar 25, 2021 at 7:20 PM Will Deacon <will@kernel.org> wrote:
> > > > On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > > > > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > > > > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > > > > commit cae118b6acc309539b33339e846cbb19187c164c
> > > > > Author: Will Deacon
> > > > > Date:   Wed Mar 3 13:49:27 2021 +0000
> > > > >     arm64: Drop support for CMDLINE_EXTEND
> > > > >
> > > > > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > > > > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > > > > but also wants the bootloader args to be appended.
> > > > > What is the way to do it now?
> > > >
> > > > For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> > > > either:
> > > >
> > > >   * Revert my patch for your kernels
> > > >   * Pass the arguments via QEMU's -append option
> > > >   * Take a look at one of the series which should hopefully add this
> > > >     functionality back (but with well-defined semantics) [1] [2]
> > >
> > > Unfortunately none of these work for syzbot (and I assume other
> > > testing environments).
> > >
> > > syzbot does not support custom patches by design:
> > > http://bit.do/syzbot#no-custom-patches
> > > As any testing system, it tests the official trees.
> > >
> > > It's not humans who start these VMs, so it's not as easy as changing
> > > the command line after typing...
> > > There is no support for passing args specifically to qemu, syzkaller
> > > support not just qemu, so these things are specifically localized in
> > > the config. Additionally there is an issue of communicating all these
> > > scattered details to developers in bug reports. Currently syzbot
> > > reports the kernel config and it as well captures command line.
> > >
> > > Could you revert the patch? Is there any point in removing the
> > > currently supported feature before the new feature lands?
> >
> > Well, we only just merged it (in 5.10 I think?), and the semantics of the
> > new version will be different, so I really don't see the value in supporting
> > both (even worse, Android has its own implementation which is different
> > again). The timeline was: we merged CMDLINE_EXTEND, then we noticed it was
> > broken, my fixes were rejected, so we removed the feature rather than
> > support the broken version. In the relatively small window while it was
> > merged, syzbot started using it :(
> 
> I didn't realize it was just introduced :)
> We used CMDLINE_EXTEND on x86, and I looked for a similar option for
> arm64 and found it.
> 
> > So I really think the best bet is to wait until the patches are sorted out.
> > I think Christophe is about to spin a new version, and I reviewed his last
> > copy, so I don't see this being far off,
> 
> If it's expected to be merged soon, let's wait.

Thanks, and knowing that we have a keen user helps to prioritise the review
:)

Will

Re: [syzbot] BUG: soft lockup in do_wp_page (4)

[Dmitry Vyukov]

On Thu, 25 Mar 2021 at 20:29, Will Deacon <will@kernel.org> wrote:
>
> On Thu, Mar 25, 2021 at 08:24:53PM +0100, Dmitry Vyukov wrote:
> > On Thu, Mar 25, 2021 at 8:10 PM Will Deacon <will@kernel.org> wrote:
> > > On Thu, Mar 25, 2021 at 07:34:54PM +0100, Dmitry Vyukov wrote:
> > > > On Thu, Mar 25, 2021 at 7:20 PM Will Deacon <will@kernel.org> wrote:
> > > > > On Thu, Mar 18, 2021 at 08:34:16PM +0100, Dmitry Vyukov wrote:
> > > > > > On Thu, Mar 18, 2021 at 8:31 PM syzbot
> > > > > > <syzbot+0b036374a865ba0efa8e@syzkaller.appspotmail.com> wrote:
> > > > > > commit cae118b6acc309539b33339e846cbb19187c164c
> > > > > > Author: Will Deacon
> > > > > > Date:   Wed Mar 3 13:49:27 2021 +0000
> > > > > >     arm64: Drop support for CMDLINE_EXTEND
> > > > > >
> > > > > > syzbot passes lots of critical things in CONFIG_CMDLINE:
> > > > > > https://github.com/google/syzkaller/blob/c3c81c94865791469d376eba84f4a2d7763d3f71/dashboard/config/linux/upstream-arm64-kasan.config#L495
> > > > > > but also wants the bootloader args to be appended.
> > > > > > What is the way to do it now?
> > > > >
> > > > > For now, there isn't a way to do it with CONFIG_CMDLINE, so I think you can
> > > > > either:
> > > > >
> > > > >   * Revert my patch for your kernels
> > > > >   * Pass the arguments via QEMU's -append option
> > > > >   * Take a look at one of the series which should hopefully add this
> > > > >     functionality back (but with well-defined semantics) [1] [2]
> > > >
> > > > Unfortunately none of these work for syzbot (and I assume other
> > > > testing environments).
> > > >
> > > > syzbot does not support custom patches by design:
> > > > http://bit.do/syzbot#no-custom-patches
> > > > As any testing system, it tests the official trees.
> > > >
> > > > It's not humans who start these VMs, so it's not as easy as changing
> > > > the command line after typing...
> > > > There is no support for passing args specifically to qemu, syzkaller
> > > > support not just qemu, so these things are specifically localized in
> > > > the config. Additionally there is an issue of communicating all these
> > > > scattered details to developers in bug reports. Currently syzbot
> > > > reports the kernel config and it as well captures command line.
> > > >
> > > > Could you revert the patch? Is there any point in removing the
> > > > currently supported feature before the new feature lands?
> > >
> > > Well, we only just merged it (in 5.10 I think?), and the semantics of the
> > > new version will be different, so I really don't see the value in supporting
> > > both (even worse, Android has its own implementation which is different
> > > again). The timeline was: we merged CMDLINE_EXTEND, then we noticed it was
> > > broken, my fixes were rejected, so we removed the feature rather than
> > > support the broken version. In the relatively small window while it was
> > > merged, syzbot started using it :(
> >
> > I didn't realize it was just introduced :)
> > We used CMDLINE_EXTEND on x86, and I looked for a similar option for
> > arm64 and found it.
> >
> > > So I really think the best bet is to wait until the patches are sorted out.
> > > I think Christophe is about to spin a new version, and I reviewed his last
> > > copy, so I don't see this being far off,
> >
> > If it's expected to be merged soon, let's wait.
>
> Thanks, and knowing that we have a keen user helps to prioritise the review
> :)

Any updates on proper CMDLINE support for arm64? This is still broken
for syzbot and it periodically reports false positives due to
incorrect cmdline.