Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2 Author: Andrey Ryabinin AuthorDate: Fri Feb 13 14:39:25 2015 -0800 Commit: Linus Torvalds CommitDate: Fri Feb 13 21:21:41 2015 -0800 x86_64: add KASan support This patch adds arch specific code for kernel address sanitizer. 16TB of virtual addressed used for shadow memory. It's located in range [ffffec0000000000 - fffffc0000000000] between vmemmap and %esp fixup stacks. At early stage we map whole shadow region with zero page. Latter, after pages mapped to direct mapping address range we unmap zero pages from corresponding shadow (see kasan_map_shadow()) and allocate and map a real shadow memory reusing vmemmap_populate() function. Also replace __pa with __pa_nodebug before shadow initialized. __pa with CONFIG_DEBUG_VIRTUAL=y make external function call (__phys_addr) __phys_addr is instrumented, so __asan_load could be called before shadow area initialized. Signed-off-by: Andrey Ryabinin Cc: Dmitry Vyukov Cc: Konstantin Serebryany Cc: Dmitry Chernenkov Signed-off-by: Andrey Konovalov Cc: Yuri Gribov Cc: Konstantin Khlebnikov Cc: Sasha Levin Cc: Christoph Lameter Cc: Joonsoo Kim Cc: Dave Hansen Cc: Andi Kleen Cc: Ingo Molnar Cc: Thomas Gleixner Cc: "H. Peter Anvin" Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Jim Davis Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds +------------------------------------------------+------------+------------+-----------------+ | | 786a895991 | ef7f0d6a6c | v4.2-rc4_080210 | +------------------------------------------------+------------+------------+-----------------+ | boot_successes | 910 | 77 | 40 | | boot_failures | 0 | 233 | 18 | | RIP:rb_insert_color | 0 | 17 | | | Kernel_panic-not_syncing:softlockup:hung_tasks | 0 | 233 | 17 | | backtrace:insert | 0 | 72 | 1 | | backtrace:rbtree_test_init | 0 | 232 | 17 | | backtrace:kernel_init_freeable | 0 | 233 | 17 | | RIP:rb_erase | 0 | 17 | | | backtrace:apic_timer_interrupt | 0 | 57 | 2 | | RIP:__asan_load8 | 0 | 44 | 2 | | backtrace:rb_erase | 0 | 45 | | | RIP:__asan_loadN | 0 | 72 | 8 | | backtrace:erase_augmented | 0 | 33 | 5 | | RIP:insert_augmented | 0 | 7 | 1 | | RIP:__asan_store8 | 0 | 24 | 2 | | RIP:__asan_store4 | 0 | 5 | | | backtrace:insert_augmented | 0 | 26 | 9 | | RIP:augment_recompute | 0 | 4 | | | RIP:augment_callbacks_propagate | 0 | 1 | | | RIP:erase_augmented | 0 | 2 | 1 | | RIP:__rb_insert_augmented | 0 | 4 | | | RIP:augment_callbacks_rotate | 0 | 6 | | | RIP:insert | 0 | 8 | | | RIP:__asan_storeN | 0 | 4 | | | RIP:__asan_load4 | 0 | 14 | 3 | | RIP:rbtree_test_init | 0 | 1 | | | RIP:__rb_erase_color | 0 | 2 | | | RIP:__rb_change_child | 0 | 1 | | | BUG:kernel_boot_hang | 0 | 0 | 1 | +------------------------------------------------+------------+------------+-----------------+ [ 53.667591] xz_dec_test: Create a device node with 'mknod xz_dec_test c 250 0' and write .xz files to it. [ 53.671288] rbtree testing [ 80.140009] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swapper:1] [ 80.140009] Modules linked in: [ 80.140009] CPU: 0 PID: 1 Comm: swapper Not tainted 3.19.0-05243-gef7f0d6 #4 [ 80.140009] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 80.140009] task: ffff88000e4d0000 ti: ffff88000e4d8000 task.ti: ffff88000e4d8000 [ 80.140009] RIP: 0010:[] [] __asan_store8+0x4c/0x140 [ 80.140009] RSP: 0018:ffff88000e4dbd88 EFLAGS: 00000206 [ 80.140009] RAX: 0000000086dfa090 RBX: ffffffff8413730f RCX: dffffc0000000000 [ 80.140009] RDX: 0000000086dfa08f RSI: 0000000000000008 RDI: ffffffff84a3d468 [ 80.140009] RBP: ffff88000e4dbdb8 R08: fffffbfff0826e61 R09: ffffffff8413730f [ 80.140009] R10: 0000000026d79129 R11: 0000000026d6454e R12: 0000000026d6bb7e [ 80.140009] R13: 1ffffffff0826e61 R14: 0000000000000010 R15: ffff88000e4dbdb8 [ 80.140009] FS: 0000000000000000(0000) GS:ffffffff838b0000(0000) knlGS:0000000000000000 [ 80.140009] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 80.140009] CR2: 0000000000000000 CR3: 000000000388a000 CR4: 00000000000006b0 [ 80.140009] Stack: [ 80.140009] ffff88000e4dbdf8 ffffffff84a3d440 ffffffff84a3cfb8 0000000000000002 [ 80.140009] 00000000b4f0c03a ffffffff84a3d460 ffff88000e4dbdf8 ffffffff82d2a679 [ 80.140009] 0000000026d79123 00000000000005c8 0000000000004094 00000034c1e9ef3c [ 80.140009] Call Trace: [ 80.140009] [] insert+0x9d/0xf1 [ 80.140009] [] rbtree_test_init+0x98/0x32c [ 80.140009] [] do_one_initcall+0x409/0x570 [ 80.140009] [] ? dynamic_debug_init+0x52a/0x52a [ 80.140009] [] kernel_init_freeable+0x25a/0x3e4 [ 80.140009] [] ? finish_task_switch+0x274/0x4c0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] [] kernel_init+0x1f/0x2b0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] [] ret_from_fork+0x7a/0xb0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] Code: 01 48 39 c7 76 49 48 8b 15 42 ce 3f 03 48 b9 00 00 00 00 00 fc ff df 48 83 05 00 d2 3f 03 01 48 8d 42 01 48 83 05 04 d2 3f 03 01 <48> 89 05 1d ce 3f 03 48 89 f8 48 c1 e8 03 48 01 c8 66 83 38 00 [ 80.140009] Kernel panic - not syncing: softlockup: hung tasks [ 80.140009] CPU: 0 PID: 1 Comm: swapper Tainted: G L 3.19.0-05243-gef7f0d6 #4 [ 80.140009] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 80.140009] 0000000000000003 0000000000000003 0000000000000001 ffff88000e4dbc01 [ 80.140009] 0000000000000000 ffffffff838b3da8 ffffffff82d28acb ffffffff838b3e38 [ 80.140009] ffffffff82d1c785 ffffffff838b3e38 ffffffff00000008 ffffffff838b3e48 [ 80.140009] Call Trace: [ 80.140009] [] dump_stack+0x2e/0x3e [ 80.140009] [] panic+0x1bb/0x4d6 [ 80.140009] [] watchdog_timer_fn+0x46e/0x470 [ 80.140009] [] hrtimer_run_queues+0x5aa/0xb30 [ 80.140009] [] ? watchdog+0x40/0x40 [ 80.140009] [] update_process_times+0x3b/0xe0 [ 80.140009] [] tick_nohz_handler+0x15e/0x350 [ 80.140009] [] local_apic_timer_interrupt+0x65/0xb0 [ 80.140009] [] smp_apic_timer_interrupt+0x85/0xb0 [ 80.140009] [] apic_timer_interrupt+0x6b/0x70 [ 80.140009] [] ? __asan_store8+0x4c/0x140 [ 80.140009] [] insert+0x9d/0xf1 [ 80.140009] [] rbtree_test_init+0x98/0x32c [ 80.140009] [] do_one_initcall+0x409/0x570 [ 80.140009] [] ? dynamic_debug_init+0x52a/0x52a [ 80.140009] [] kernel_init_freeable+0x25a/0x3e4 [ 80.140009] [] ? finish_task_switch+0x274/0x4c0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] [] kernel_init+0x1f/0x2b0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] [] ret_from_fork+0x7a/0xb0 [ 80.140009] [] ? rest_init+0xe0/0xe0 [ 80.140009] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff) Elapsed time: 110 git bisect start v4.0 v2.6.39 -- git bisect good 5abcd76f5d896de014bd8d1486107c483659d40d # 13:13 310+ 310 Merge branch 'x86-boot-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip git bisect good 6a4d07f85ba9da5b6eab6e60a493d459c4296176 # 13:35 310+ 156 Merge branch 'for-3.14-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup git bisect good 9f47112975fdc32e545e079f42a17bbd0be236fc # 14:09 310+ 0 Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus git bisect good c0f486fde3f353232c1cc2fd4d62783ac782a467 # 14:30 310+ 0 Merge tag 'pm+acpi-3.19-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect good a42cf70eb81558082e9a26fe8541d160b6c2a694 # 14:51 301+ 0 Merge tag 'modules-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux git bisect bad ecddad64d4ca427c71598cc23183f48bc9cc4568 # 15:07 47- 17 Merge tag 'fbdev-fixes-4.0' of git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux git bisect bad d34696c2208b2dc1b27ec8f0a017a91e4e6eb85d # 15:15 6- 1 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux git bisect bad 66dc830d14a222c9214a8557e9feb1e4a67a3857 # 15:25 15- 8 Merge branch 'iov_iter' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect bad 8c334ce8f0fec7122fc3059c52a697b669a01b41 # 15:36 34- 38 Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip git bisect bad 4ba63072b998cc31515cc6305c25f3b808b50c01 # 15:48 36- 36 Merge tag 'char-misc-3.20-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc git bisect bad fee5429e028c414d80d036198db30454cfd91b7a # 16:00 52- 44 Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 git bisect good 18320f2a6871aaf2522f793fee4a67eccf5e131a # 16:21 310+ 0 Merge tag 'pm+acpi-3.20-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect bad 83e047c104aa95a8a683d6bd421df1551c17dbd2 # 16:57 39- 40 Merge branch 'akpm' (patches from Andrew) git bisect good 327953e9af6c59ad111b28359e59e3ec0cbd71b6 # 17:37 310+ 0 checkpatch: add check for keyword 'boolean' in Kconfig definitions git bisect bad 3f15801cdc2379ca4bf507f48bffd788f9e508ae # 17:47 20- 21 lib: add kasan test module git bisect good 0f3c5aab5e00527eb3167aa9d1725cca9320e01e # 18:14 300+ 1 checkpatch: add of_device_id to structs that should be const git bisect bad b8c73fc2493d42517be95cf2c89659fc6c6f4d02 # 18:24 8- 10 mm: page_alloc: add kasan hooks on alloc and free paths git bisect good cb4188ac8e5779f66b9f55888ac2c75b391cde44 # 18:47 310+ 0 compiler: introduce __alias(symbol) shortcut git bisect good 786a8959912eb94fc2381c2ae487a96ce55dabca # 19:10 306+ 0 kasan: disable memory hotplug git bisect bad ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2 # 19:20 14- 12 x86_64: add KASan support # first bad commit: [ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2] x86_64: add KASan support git bisect good 786a8959912eb94fc2381c2ae487a96ce55dabca # 19:56 910+ 0 kasan: disable memory hotplug # extra tests with DEBUG_INFO git bisect bad ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2 # 20:08 31- 22 x86_64: add KASan support # extra tests on HEAD of linux-devel/devel-hourly-2015080210 git bisect bad 8fc06a4ce2b4a6828d0a8d70daaf9d999c72fb8a # 20:08 0- 18 0day head guard for 'devel-hourly-2015080210' # extra tests on tree/branch linus/master git bisect bad 01183609ab61d11f1c310d42552a97be3051cc0f # 20:47 54- 31 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs # extra tests on tree/branch linus/master git bisect bad 01183609ab61d11f1c310d42552a97be3051cc0f # 20:47 0- 31 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs # extra tests on tree/branch linux-next/master This script may reproduce the error. ---------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=quantal-core-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd kvm=( qemu-system-x86_64 -enable-kvm -cpu kvm64 -kernel $kernel -initrd $initrd -m 300 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -serial stdio -display none -monitor null ) append=( hung_task_panic=1 earlyprintk=ttyS0,115200 systemd.log_level=err debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw drbd.minor_count=8 ) "${kvm[@]}" --append "${append[*]}" ---------------------------------------------------------------------------- --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/lkp Intel Corporation