Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 80a9201a5965f4715d5c09790862e0df84ce0614 Author: Alexander Potapenko AuthorDate: Thu Jul 28 15:49:07 2016 -0700 Commit: Linus Torvalds CommitDate: Thu Jul 28 16:07:41 2016 -0700 mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB For KASAN builds: - switch SLUB allocator to using stackdepot instead of storing the allocation/deallocation stacks in the objects; - change the freelist hook so that parts of the freelist can be put into the quarantine. [aryabinin@virtuozzo.com: fixes] Link: http://lkml.kernel.org/r/1468601423-28676-1-git-send-email-aryabinin@virtuozzo.com Link: http://lkml.kernel.org/r/1468347165-41906-3-git-send-email-glider@google.com Signed-off-by: Alexander Potapenko Cc: Andrey Konovalov Cc: Christoph Lameter Cc: Dmitry Vyukov Cc: Steven Rostedt (Red Hat) Cc: Joonsoo Kim Cc: Kostya Serebryany Cc: Andrey Ryabinin Cc: Kuthonuzo Luruo Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds +------------------------------------------------+------------+------------+------------+ | | c146a2b98e | 80a9201a59 | 4fc0672d18 | +------------------------------------------------+------------+------------+------------+ | boot_successes | 106 | 0 | 17 | | boot_failures | 874 | 250 | 30 | | RIP:T | 201 | 53 | 3 | | Kernel_panic-not_syncing:softlockup:hung_tasks | 874 | 250 | 30 | | backtrace:eata2x_detect | 490 | 84 | 15 | | backtrace:init_this_scsi_driver | 490 | 84 | 15 | | backtrace:do_basic_setup | 506 | 250 | 23 | | backtrace:kernel_init_freeable | 506 | 250 | 23 | | backtrace:ret_from_fork | 874 | 250 | 30 | | RIP:_raw_spin_unlock_irqrestore | 244 | 11 | 9 | | backtrace:pci_enable_device_flags | 16 | 166 | 8 | | backtrace:__pci_register_driver | 16 | 166 | 8 | | backtrace:virtio_pci_driver_init | 16 | 166 | 8 | | RIP:note_page | 248 | 0 | 5 | | backtrace:mark_rodata_ro | 368 | 0 | 7 | | RIP:walk_pmd_level | 120 | 0 | 2 | | RIP:kmem_cache_free | 7 | 2 | 2 | | RIP:check_bytes_and_report | 1 | | | | backtrace:acpi_ut_update_object_reference | 2 | 1 | | | RIP:kasan_kmalloc | 2 | 1 | | | RIP:acpi_ut_update_object_reference | 3 | 2 | | | RIP:port_detect | 19 | | | | RIP:delay_tsc | 3 | | | | RIP:lockdep_trace_alloc | 1 | | | | RIP:free_debug_processing | 2 | | | | RIP:__slab_free | 6 | 0 | 1 | | RIP:kasan_slab_free | 1 | 3 | | | RIP:___might_sleep | 1 | | | | RIP:__memset | 3 | 2 | | | RIP:acpi_ps_push_scope | 1 | | | | RIP:debug_lockdep_rcu_enabled | 1 | | | | RIP:lock_is_held | 2 | 2 | | | RIP:memset_erms | 1 | | | | RIP:should_failslab | 2 | | | | RIP:acpi_ut_update_ref_count | 2 | | | | RIP:acpi_ds_result_push | 1 | | | | RIP:acpi_ps_get_arg | 1 | | | | RIP:memchr_inv | 1 | | | | RIP:print_context_stack | 0 | 36 | 3 | | RIP:qlist_free_all | 0 | 65 | | | RIP:__kernel_text_address | 0 | 37 | 3 | | RIP:memcmp | 0 | 18 | 1 | | RIP:depot_save_stack | 0 | 5 | | | backtrace:apic_timer_interrupt | 0 | 17 | 2 | | RIP:get_page_from_freelist | 0 | 1 | | | RIP:quarantine_put | 0 | 1 | | | RIP:save_stack_address | 0 | 4 | | | RIP:kasan_unpoison_shadow | 0 | 1 | | | RIP:dump_trace | 0 | 2 | | | RIP:acpi_ut_create_generic_state | 0 | 1 | | | RIP:acpi_ds_exec_begin_op | 0 | 1 | | | RIP:__do_softirq | 0 | 1 | | | backtrace:new_slab | 0 | 1 | | | RIP:acpi_ns_search_one_scope | 0 | 1 | | | RIP:acpi_ut_delete_generic_state | 0 | 0 | 1 | +------------------------------------------------+------------+------------+------------+ [ 64.298576] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swapper/0:1] [ 64.300827] irq event stamp: 5606950 [ 64.301377] hardirqs last enabled at (5606949): [] T.2097+0x9a/0xbe [ 64.302586] hardirqs last disabled at (5606950): [] apic_timer_interrupt+0x89/0xa0 [ 64.303991] softirqs last enabled at (5605564): [] __do_softirq+0x23e/0x2bb [ 64.305308] softirqs last disabled at (5605557): [] irq_exit+0x73/0x108 [ 64.306598] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0-05999-g80a9201 #1 [ 64.307678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 64.326233] task: ffff88000ea19ec0 task.stack: ffff88000ea20000 [ 64.327137] RIP: 0010:[] [] __kernel_text_address+0xb/0xa1 [ 64.328504] RSP: 0000:ffff88000ea27348 EFLAGS: 00000207 [ 64.329320] RAX: 0000000000000001 RBX: ffff88000ea275c0 RCX: 0000000000000001 [ 64.330426] RDX: ffff88000ea27ff8 RSI: 024080c099733d8f RDI: 024080c099733d8f [ 64.331496] RBP: ffff88000ea27348 R08: ffff88000ea27678 R09: 0000000000000000 [ 64.332567] R10: 0000000000021298 R11: ffffffff990f235c R12: ffff88000ea276c8 [ 64.333635] R13: ffffffff99805e20 R14: ffff88000ea19ec0 R15: 0000000000000000 [ 64.334706] FS: 0000000000000000(0000) GS:ffff88000ee00000(0000) knlGS:0000000000000000 [ 64.335916] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 64.336782] CR2: 0000000000000000 CR3: 000000000aa0a000 CR4: 00000000000406b0 [ 64.337846] Stack: [ 64.338206] ffff88000ea273a8 ffffffff9881f3dd 024080c099733d8f ffffffffffff8000 [ 64.339410] ffff88000ea27678 ffff88000ea276c8 000000020e81a4d8 ffff88000ea273f8 [ 64.340602] ffffffff99805e20 ffff88000ea19ec0 ffff88000ea27438 ffff88000ee07fc0 [ 64.348993] Call Trace: [ 64.349380] [] print_context_stack+0x68/0x13e [ 64.350295] [] dump_trace+0x3ab/0x3d6 [ 64.351102] [] save_stack_trace+0x31/0x5c [ 64.351964] [] kasan_kmalloc+0x126/0x1f6 [ 64.365727] [] ? save_stack_trace+0x31/0x5c [ 64.366675] [] ? kasan_kmalloc+0x126/0x1f6 [ 64.367560] [] ? acpi_ut_create_generic_state+0x43/0x5c git bisect start 29b4817d4018df78086157ea3a55c1d9424a7cfc v4.7 -- git bisect bad 574c7e233344b58c6b14b305c93de361d3e7d35d # 23:23 2- 4 Merge branch 'for-4.7-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup git bisect good 0e06f5c0deeef0332a5da2ecb8f1fcf3e024d958 # 23:53 205+ 114 Merge branch 'akpm' (patches from Andrew) git bisect good 76d5b28bbad1c5502a24f94c2beafc468690b2ba # 08:23 213+ 198 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs git bisect bad c624c86615fb8aa61fa76ed8c935446d06c80e77 # 08:37 27- 32 Merge tag 'trace-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace git bisect good 6039b80eb50a893476fea7d56e86ed2d19290054 # 09:05 206+ 170 Merge tag 'dmaengine-4.8-rc1' of git://git.infradead.org/users/vkoul/slave-dma git bisect bad f0c98ebc57c2d5e535bc4f9167f35650d2ba3c90 # 09:24 42- 46 Merge tag 'libnvdimm-for-4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm git bisect bad 1c88e19b0f6a8471ee50d5062721ba30b8fd4ba9 # 09:47 43- 53 Merge branch 'akpm' (patches from Andrew) git bisect good bca6759258dbef378bcf5b872177bcd2259ceb68 # 09:58 245+ 243 mm, vmstat: remove zone and node double accounting by approximating retries git bisect good efdc94907977d2db84b4b00cb9bd98ca011f6819 # 10:15 240+ 240 mm: fix memcg stack accounting for sub-page stacks git bisect good fb399b4854d2159a4d23fbfbd7daaed914fd54fa # 11:48 250+ 249 mm/memblock.c: fix index adjustment error in __next_mem_range_rev() git bisect bad 31a6c1909f51dbe9bf08eb40dc64e3db90cf6f79 # 11:53 47- 52 mm, page_alloc: set alloc_flags only once in slowpath git bisect good c146a2b98eb5898eb0fab15a332257a4102ecae9 # 12:03 245+ 178 mm, kasan: account for object redzone in SLUB's nearest_obj() git bisect bad 87cc271d5e4320d705cfdf59f68d4d037b3511b2 # 12:07 3- 6 lib/stackdepot.c: use __GFP_NOWARN for stack allocations git bisect bad 80a9201a5965f4715d5c09790862e0df84ce0614 # 12:14 14- 16 mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB # first bad commit: [80a9201a5965f4715d5c09790862e0df84ce0614] mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB git bisect good c146a2b98eb5898eb0fab15a332257a4102ecae9 # 12:27 726+ 874 mm, kasan: account for object redzone in SLUB's nearest_obj() # extra tests with CONFIG_DEBUG_INFO_REDUCED git bisect bad 80a9201a5965f4715d5c09790862e0df84ce0614 # 12:33 14- 17 mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB # extra tests on HEAD of linux-devel/devel-spot-201608102121 git bisect bad 4fc0672d1847abd92df3ce73f61a1f0a1cc83e58 # 12:33 0- 30 0day head guard for 'devel-spot-201608102121' # extra tests on tree/branch linus/master git bisect bad 85e97be32c6242c98dbbc7a241b4a78c1b93327b # 12:40 33- 37 Merge branch 'akpm' (patches from Andrew) # extra tests on tree/branch linus/master git bisect bad 85e97be32c6242c98dbbc7a241b4a78c1b93327b # 12:40 0- 37 Merge branch 'akpm' (patches from Andrew) # extra tests on tree/branch linux-next/master git bisect bad c0a5420a2efbfebd3cb90b000aeb953068b4da20 # 12:50 15- 20 Add linux-next specific files for 20160811 This script may reproduce the error. ---------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd kvm=( qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap -kernel $kernel -initrd $initrd -m 300 -smp 1 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc -no-reboot -watchdog i6300esb -watchdog-action debug -rtc base=localtime -serial stdio -display none -monitor null ) append=( root=/dev/ram0 hung_task_panic=1 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 systemd.log_level=err ignore_loglevel earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal rw drbd.minor_count=8 ) "${kvm[@]}" --append "${append[*]}" ---------------------------------------------------------------------------- --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/lkp Intel Corporation