On Thu, Aug 11, 2016 at 01:35:03PM -0700, Andrew Morton wrote: >On Thu, 11 Aug 2016 12:52:27 +0800 kernel test robot wrote: > >> Greetings, >> >> 0day kernel testing robot got the below dmesg and the first bad commit is >> >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master >> >> commit 80a9201a5965f4715d5c09790862e0df84ce0614 >> Author: Alexander Potapenko >> AuthorDate: Thu Jul 28 15:49:07 2016 -0700 >> Commit: Linus Torvalds >> CommitDate: Thu Jul 28 16:07:41 2016 -0700 >> >> mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB >> >> For KASAN builds: >> - switch SLUB allocator to using stackdepot instead of storing the >> allocation/deallocation stacks in the objects; >> - change the freelist hook so that parts of the freelist can be put >> into the quarantine. >> >> ... >> >> [ 64.298576] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swapper/0:1] >> [ 64.300827] irq event stamp: 5606950 >> [ 64.301377] hardirqs last enabled at (5606949): [] T.2097+0x9a/0xbe >> [ 64.302586] hardirqs last disabled at (5606950): [] apic_timer_interrupt+0x89/0xa0 >> [ 64.303991] softirqs last enabled at (5605564): [] __do_softirq+0x23e/0x2bb >> [ 64.305308] softirqs last disabled at (5605557): [] irq_exit+0x73/0x108 >> [ 64.306598] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0-05999-g80a9201 #1 >> [ 64.307678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 >> [ 64.326233] task: ffff88000ea19ec0 task.stack: ffff88000ea20000 >> [ 64.327137] RIP: 0010:[] [] __kernel_text_address+0xb/0xa1 >> [ 64.328504] RSP: 0000:ffff88000ea27348 EFLAGS: 00000207 >> [ 64.329320] RAX: 0000000000000001 RBX: ffff88000ea275c0 RCX: 0000000000000001 >> [ 64.330426] RDX: ffff88000ea27ff8 RSI: 024080c099733d8f RDI: 024080c099733d8f >> [ 64.331496] RBP: ffff88000ea27348 R08: ffff88000ea27678 R09: 0000000000000000 >> [ 64.332567] R10: 0000000000021298 R11: ffffffff990f235c R12: ffff88000ea276c8 >> [ 64.333635] R13: ffffffff99805e20 R14: ffff88000ea19ec0 R15: 0000000000000000 >> [ 64.334706] FS: 0000000000000000(0000) GS:ffff88000ee00000(0000) knlGS:0000000000000000 >> [ 64.335916] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [ 64.336782] CR2: 0000000000000000 CR3: 000000000aa0a000 CR4: 00000000000406b0 >> [ 64.337846] Stack: >> [ 64.338206] ffff88000ea273a8 ffffffff9881f3dd 024080c099733d8f ffffffffffff8000 >> [ 64.339410] ffff88000ea27678 ffff88000ea276c8 000000020e81a4d8 ffff88000ea273f8 >> [ 64.340602] ffffffff99805e20 ffff88000ea19ec0 ffff88000ea27438 ffff88000ee07fc0 >> [ 64.348993] Call Trace: >> [ 64.349380] [] print_context_stack+0x68/0x13e >> [ 64.350295] [] dump_trace+0x3ab/0x3d6 >> [ 64.351102] [] save_stack_trace+0x31/0x5c >> [ 64.351964] [] kasan_kmalloc+0x126/0x1f6 >> [ 64.365727] [] ? save_stack_trace+0x31/0x5c >> [ 64.366675] [] ? kasan_kmalloc+0x126/0x1f6 >> [ 64.367560] [] ? acpi_ut_create_generic_state+0x43/0x5c >> > >At a guess I'd say that >arch/x86/kernel/dumpstack.c:print_context_stack() failed to terminate, >or took a super long time. Is that a thing that is known to be possible? Andrew, note that this kernel is compiled with gcc-4.4. This commit caused the below problems, too, with gcc-4.4. However they no longer show up in mainline HEAD, so not reported before. Thanks, Fengguang +------------------------------------------------------------------+------------+------------+------------+ | | c146a2b98e | 80a9201a59 | 8403fd82f1 | +------------------------------------------------------------------+------------+------------+------------+ | boot_successes | 138 | 5 | 12 | | boot_failures | 2 | 33 | 29 | | BUG:kernel_test_oversize | 2 | 0 | 16 | | Mem-Info | 0 | 4 | 2 | | Out_of_memory:Kill_process | 0 | 4 | | | backtrace:SYSC_newfstatat | 0 | 3 | | | backtrace:SyS_newfstatat | 0 | 4 | | | BUG_anon_vma_chain(Not_tainted):Poison_overwritten | 0 | 9 | | | INFO:#-#.First_byte#instead_of | 0 | 24 | 6 | | INFO:Allocated_in_anon_vma_fork_age=#cpu=#pid= | 0 | 5 | | | INFO:Freed_in_qlist_free_all_age=#cpu=#pid= | 0 | 22 | 5 | | INFO:Slab#objects=#used=#fp=#flags= | 0 | 1 | | | INFO:Object#@offset=#fp= | 0 | 22 | 6 | | backtrace:SyS_open | 0 | 2 | 3 | | INFO:Allocated_in_anon_vma_clone_age=#cpu=#pid= | 0 | 5 | | | INFO:Slab#objects=#used=#fp=0x(null)flags= | 0 | 21 | 6 | | BUG_anon_vma_chain(Tainted:G_B):Poison_overwritten | 0 | 4 | | | INFO:Allocated_in_anon_vma_prepare_age=#cpu=#pid= | 0 | 4 | | | BUG_vm_area_struct(Tainted:G_B):Poison_overwritten | 0 | 8 | | | INFO:Allocated_in_copy_process_age=#cpu=#pid= | 0 | 11 | | | backtrace:SyS_read | 0 | 2 | | | backtrace:SyS_clone | 0 | 16 | 1 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 9 | 2 | | Oops | 0 | 4 | | | RIP:vt_console_print | 0 | 5 | | | BUG_vm_area_struct(Not_tainted):Poison_overwritten | 0 | 10 | 1 | | backtrace:do_execve | 0 | 7 | 1 | | backtrace:SyS_execve | 0 | 9 | 1 | | INFO:Object#@offset=#fp=0x(null) | 0 | 7 | 1 | | INFO:Allocated_in__split_vma_age=#cpu=#pid= | 0 | 2 | | | BUG_buffer_head(Not_tainted):Poison_overwritten | 0 | 1 | | | INFO:Allocated_in_alloc_buffer_head_age=#cpu=#pid= | 0 | 2 | | | BUG_names_cache(Tainted:G_B):Poison_overwritten | 0 | 2 | | | INFO:Allocated_in_getname_flags_age=#cpu=#pid= | 0 | 3 | 1 | | BUG_kmalloc-#(Tainted:G_B):Poison_overwritten | 0 | 3 | 1 | | INFO:Allocated_in__alloc_skb_age=#cpu=#pid= | 0 | 3 | | | backtrace:SyS_mount | 0 | 2 | | | backtrace:SYSC_newstat | 0 | 1 | | | backtrace:SyS_newstat | 0 | 2 | | | backtrace:__sys_sendmsg | 0 | 2 | 1 | | backtrace:SyS_sendmsg | 0 | 2 | 1 | | backtrace:mprotect_fixup | 0 | 2 | 1 | | backtrace:SyS_mprotect | 0 | 2 | 1 | | backtrace:_do_fork | 0 | 8 | 1 | | general_protection_fault:#[##]PREEMPT_KASAN | 0 | 5 | 2 | | RIP:lock_anon_vma_root | 0 | 4 | 1 | | backtrace:SyS_exit_group | 0 | 1 | | | BUG_skbuff_head_cache(Not_tainted):Poison_overwritten | 0 | 1 | | | backtrace:SyS_connect | 0 | 1 | | | INFO:Allocated_in_mmap_region_age=#cpu=#pid= | 0 | 2 | 1 | | backtrace:vm_mmap_pgoff | 0 | 2 | | | backtrace:SyS_mmap_pgoff | 0 | 2 | | | backtrace:SyS_mmap | 0 | 2 | | | INFO:Allocated_in_load_elf_phdrs_age=#cpu=#pid= | 0 | 1 | | | BUG_fs_cache(Tainted:G_B):Poison_overwritten | 0 | 1 | | | INFO:Allocated_in_copy_fs_struct_age=#cpu=#pid= | 0 | 1 | | | INFO:Allocated_in_seq_buf_alloc_age=#cpu=#pid= | 0 | 1 | | | RIP:__slab_free | 0 | 1 | 1 | | backtrace:user_path_at_empty | 0 | 1 | | | backtrace:SyS_readlinkat | 0 | 1 | | | backtrace:SyS_readlink | 0 | 1 | | | BUG_kmalloc-#(Not_tainted):Poison_overwritten | 0 | 2 | 3 | | INFO:Allocated_in__install_special_mapping_age=#cpu=#pid= | 0 | 1 | | | BUG:Bad_page_map_in_process | 0 | 2 | | | BUG:unable_to_handle_kernel | 0 | 2 | | | BUG:kernel_boot_crashed | 0 | 1 | | | BUG_names_cache(Not_tainted):Poison_overwritten | 0 | 1 | 1 | | BUG_buffer_head(Tainted:G_B):Poison_overwritten | 0 | 1 | | | backtrace:do_sys_open | 0 | 1 | 3 | | backtrace:alloc_debug_processing | 0 | 1 | | | BUG:KASAN:use-after-free_in__probe_kernel_read_at_addr | 0 | 1 | | | backtrace:cpuset_init_smp | 0 | 1 | | | backtrace:kernel_init_freeable | 0 | 1 | | | INFO:Allocated_in_load_elf_binary_age=#cpu=#pid= | 0 | 1 | | | IP-Config:Auto-configuration_of_network_failed | 0 | 0 | 4 | | invoked_oom-killer:gfp_mask=0x | 0 | 0 | 2 | | Kernel_panic-not_syncing:Out_of_memory_and_no_killable_processes | 0 | 0 | 2 | | BUG_dentry(Tainted:G_B):Poison_overwritten | 0 | 0 | 1 | | INFO:Allocated_in__d_alloc_age=#cpu=#pid= | 0 | 0 | 1 | | INFO:Allocated_in_kernfs_fop_open_age=#cpu=#pid= | 0 | 0 | 3 | | backtrace:user_path_create | 0 | 0 | 1 | | backtrace:SyS_symlinkat | 0 | 0 | 1 | | backtrace:SyS_symlink | 0 | 0 | 1 | | BUG_task_struct(Not_tainted):Poison_overwritten | 0 | 0 | 1 | | INFO:Allocated_in_kzalloc_age=#cpu=#pid= | 0 | 0 | 1 | | backtrace:vfs_write | 0 | 0 | 1 | | backtrace:SyS_write | 0 | 0 | 1 | +------------------------------------------------------------------+------------+------------+------------+ [ 105.829566] blk_update_request: I/O error, dev fd0, sector 0 [ 105.829567] floppy: error -5 while reading block 0 [ 105.944521] ============================================================================= [ 105.944524] BUG vm_area_struct (Not tainted): Poison overwritten [ 105.944524] ----------------------------------------------------------------------------- [ 105.944524] [ 105.944525] Disabling lock debugging due to kernel taint [ 105.944527] INFO: 0xffff880009318ca5-0xffff880009318ca7. First byte 0x1 instead of 0x6b [ 105.944532] INFO: Allocated in copy_process+0xc65/0x1797 age=114 cpu=0 pid=518 [ 105.944550] INFO: Freed in qlist_free_all+0x7a/0x100 age=29 cpu=0 pid=596 [ 105.944582] INFO: Slab 0xffffea000024c600 objects=15 used=15 fp=0x (null) flags=0x4000000000004080 [ 105.944583] INFO: Object 0xffff880009318c98 @offset=3224 fp=0xffff8800093192e0 [ 105.944583] [ 105.944585] Redzone ffff880009318c90: bb bb bb bb bb bb bb bb ........ [ 105.944586] Object ffff880009318c98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 01 80 dd kkkkkkkkkkkkk... [ 105.944587] Object ffff880009318ca8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk [ 105.944588] Object ffff880009318cb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk git bisect start 8403fd82f1c6320dfbe7cfa5f35362ddb3966e30 523d939ef98fd712632d93a5a2b588e477a7565e -- git bisect bad faaab93209a0298c605bbfa0489867ae031da494 # 09:48 4- 11 Merge 'mips-sjhill/linux-4.7-stable' into devel-spot-201608040312 git bisect bad c5f0c3e1c933cb4982ef6d58813b495c154fe07a # 10:06 0- 1 Merge 'linux-review/Johannes-Thumshirn/mpt3sas-Don-t-spam-logs-if-logging-level-is-0/20160803-212128' into devel-spot-201608040312 git bisect bad a26d7f6d37364e54715c6cff79b9f1b854713440 # 10:17 1- 9 Merge 'gfs2/master' into devel-spot-201608040312 git bisect bad 4db40b95edbc23164ed76301bef5f67a77f87edc # 11:01 1- 2 Merge 'linux-review/Bimmy-Pujari/i40e-i40evf-updates/20160804-025159' into devel-spot-201608040312 git bisect good e3ba205b84679c6d01e3c921b3b6665f31227a95 # 11:07 32+ 4 Merge 'linux-review/Steve-Longerbeam/adv7180-subdev-fixes-v4/20160804-030449' into devel-spot-201608040312 git bisect bad bc489156b44eb2b68d937d28b750149c507e0bac # 11:16 0- 5 Merge 'kees/for-next/lkdtm' into devel-spot-201608040312 git bisect bad 1056c9bd2702ea1bb79abf9bd1e78c578589d247 # 11:22 1- 13 Merge tag 'clk-for-linus-4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux git bisect bad c624c86615fb8aa61fa76ed8c935446d06c80e77 # 11:30 7- 13 Merge tag 'trace-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace git bisect bad 1c88e19b0f6a8471ee50d5062721ba30b8fd4ba9 # 11:37 0- 1 Merge branch 'akpm' (patches from Andrew) git bisect bad c3486f5376696034d0fcbef8ba70c70cfcb26f51 # 11:40 1- 12 mm, compaction: simplify contended compaction handling git bisect good 1e6b10857f91685c60c341703ece4ae9bb775cf3 # 11:45 33+ 0 mm, workingset: make working set detection node-aware git bisect good b4fd07a0864a06d7a8b20a624d851736330d6fd8 # 11:53 34+ 1 mm/zsmalloc: use class->objs_per_zspage to get num of max objects git bisect good 7c7fd82556c61113b6327c9696b347a82b215072 # 11:58 34+ 0 mm: hwpoison: remove incorrect comments git bisect good c146a2b98eb5898eb0fab15a332257a4102ecae9 # 12:09 34+ 0 mm, kasan: account for object redzone in SLUB's nearest_obj() git bisect bad 23771235bb569c4999ff077d2c38eaee5763193a # 12:13 2- 19 mm, page_alloc: don't retry initial attempt in slowpath git bisect bad 87cc271d5e4320d705cfdf59f68d4d037b3511b2 # 12:18 5- 10 lib/stackdepot.c: use __GFP_NOWARN for stack allocations git bisect bad 80a9201a5965f4715d5c09790862e0df84ce0614 # 12:24 0- 3 mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB # first bad commit: [80a9201a5965f4715d5c09790862e0df84ce0614] mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB git bisect good c146a2b98eb5898eb0fab15a332257a4102ecae9 # 12:29 101+ 2 mm, kasan: account for object redzone in SLUB's nearest_obj() # extra tests with CONFIG_DEBUG_INFO_REDUCED git bisect bad 80a9201a5965f4715d5c09790862e0df84ce0614 # 12:37 4- 11 mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB # extra tests on HEAD of linux-devel/devel-spot-201608040312 git bisect bad 8403fd82f1c6320dfbe7cfa5f35362ddb3966e30 # 12:37 0- 29 0day head guard for 'devel-spot-201608040312' # extra tests on tree/branch linus/master git bisect good 96b585267f552d4b6a28ea8bd75e5ed03deb6e71 # 12:46 99+ 2 Revert "ACPI / hotplug / PCI: Runtime resume bridge before rescan" # extra tests on tree/branch linus/master git bisect good 96b585267f552d4b6a28ea8bd75e5ed03deb6e71 # 12:50 101+ 5 Revert "ACPI / hotplug / PCI: Runtime resume bridge before rescan" # extra tests on tree/branch linux-next/master git bisect good 7a4be45ba2ccc2dc5a15d0b0c5bfba05ad672ff8 # 13:03 102+ 77 Add linux-next specific files for 20160803 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/lkp Intel Corporation