Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master commit ddc5bfec501f4be3f9e89084c2db270c0c45d1d6 Author: Johannes Weiner AuthorDate: Fri Jun 20 10:27:58 2014 +1000 Commit: Stephen Rothwell CommitDate: Fri Jun 20 10:27:58 2014 +1000 mm: memcontrol: rewrite uncharge API The memcg uncharging code that is involved towards the end of a page's lifetime - truncation, reclaim, swapout, migration - is impressively complicated and fragile. Because anonymous and file pages were always charged before they had their page->mapping established, uncharges had to happen when the page type could still be known from the context; as in unmap for anonymous, page cache removal for file and shmem pages, and swap cache truncation for swap pages. However, these operations happen well before the page is actually freed, and so a lot of synchronization is necessary: - Charging, uncharging, page migration, and charge migration all need to take a per-page bit spinlock as they could race with uncharging. - Swap cache truncation happens during both swap-in and swap-out, and possibly repeatedly before the page is actually freed. This means that the memcg swapout code is called from many contexts that make no sense and it has to figure out the direction from page state to make sure memory and memory+swap are always correctly charged. - On page migration, the old page might be unmapped but then reused, so memcg code has to prevent untimely uncharging in that case. Because this code - which should be a simple charge transfer - is so special-cased, it is not reusable for replace_page_cache(). But now that charged pages always have a page->mapping, introduce mem_cgroup_uncharge(), which is called after the final put_page(), when we know for sure that nobody is looking at the page anymore. For page migration, introduce mem_cgroup_migrate(), which is called after the migration is successful and the new page is fully rmapped. Because the old page is no longer uncharged after migration, prevent double charges by decoupling the page's memcg association (PCG_USED and pc->mem_cgroup) from the page holding an actual charge. The new bits PCG_MEM and PCG_MEMSW represent the respective charges and are transferred to the new page during migration. mem_cgroup_migrate() is suitable for replace_page_cache() as well, which gets rid of mem_cgroup_replace_page_cache(). Swap accounting is massively simplified: because the page is no longer uncharged as early as swap cache deletion, a new mem_cgroup_swapout() can transfer the page's memory+swap charge (PCG_MEMSW) to the swap entry before the final put_page() in page reclaim. Finally, page_cgroup changes are now protected by whatever protection the page itself offers: anonymous pages are charged under the page table lock, whereas page cache insertions, swapin, and migration hold the page lock. Uncharging happens under full exclusion with no outstanding references. Charging and uncharging also ensure that the page is off-LRU, which serializes against charge migration. Remove the very costly page_cgroup lock and set pc->flags non-atomically. Signed-off-by: Johannes Weiner Cc: Michal Hocko Cc: Hugh Dickins Cc: Tejun Heo Cc: Vladimir Davydov Signed-off-by: Andrew Morton +-----------------------------------------------------------------------+------------+------------+---------------+ | | 5b647620c6 | ddc5bfec50 | next-20140620 | +-----------------------------------------------------------------------+------------+------------+---------------+ | boot_successes | 60 | 0 | 0 | | boot_failures | 0 | 20 | 13 | | WARNING:CPU:PID:at_kernel/res_counter.c:res_counter_uncharge_locked() | 0 | 20 | 13 | | backtrace:vm_munmap | 0 | 20 | 13 | | backtrace:SyS_munmap | 0 | 20 | 13 | | backtrace:do_sys_open | 0 | 20 | 13 | | backtrace:SyS_open | 0 | 20 | 13 | | backtrace:do_execve | 0 | 20 | 13 | | backtrace:SyS_execve | 0 | 20 | 13 | | backtrace:do_group_exit | 0 | 20 | 13 | | backtrace:SyS_exit_group | 0 | 20 | 13 | | backtrace:SYSC_renameat2 | 0 | 11 | 8 | | backtrace:SyS_rename | 0 | 11 | 8 | | backtrace:do_munmap | 0 | 11 | 8 | | backtrace:SyS_brk | 0 | 11 | 8 | | Out_of_memory:Kill_process | 0 | 1 | | | backtrace:do_unlinkat | 0 | 9 | 5 | | backtrace:SyS_unlink | 0 | 9 | 5 | | backtrace:SYSC_umount | 0 | 9 | | | backtrace:SyS_umount | 0 | 9 | | | backtrace:cleanup_mnt_work | 0 | 0 | 5 | +-----------------------------------------------------------------------+------------+------------+---------------+ [ 2.747397] debug: unmapping init [mem 0xffff880001a3a000-0xffff880001bfffff] [ 2.748630] debug: unmapping init [mem 0xffff8800021ad000-0xffff8800021fffff] [ 2.752857] ------------[ cut here ]------------ [ 2.753355] WARNING: CPU: 0 PID: 1 at kernel/res_counter.c:28 res_counter_uncharge_locked+0x48/0x74() [ 2.753355] CPU: 0 PID: 1 Comm: init Not tainted 3.16.0-rc1-00238-gddc5bfe #1 [ 2.753355] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 2.753355] 0000000000000000 ffff880012073c50 ffffffff81a23b9d ffff880012073c88 [ 2.753355] ffffffff810bc765 ffffffff8111fac8 0000000000001000 ffff88001200fa50 [ 2.753355] 0000000000000001 ffff88001200fa01 ffff880012073c98 ffffffff810bc84b [ 2.753355] Call Trace: [ 2.753355] [] dump_stack+0x19/0x1b [ 2.753355] [] warn_slowpath_common+0x73/0x8c [ 2.753355] [] ? res_counter_uncharge_locked+0x48/0x74 [ 2.753355] [] warn_slowpath_null+0x1a/0x1c [ 2.753355] [] res_counter_uncharge_locked+0x48/0x74 [ 2.753355] [] res_counter_uncharge_until+0x4e/0xa9 [ 2.753355] [] res_counter_uncharge+0x13/0x15 [ 2.753355] [] mem_cgroup_uncharge_end+0x73/0x8d [ 2.753355] [] release_pages+0x1f2/0x20d [ 2.753355] [] tlb_flush_mmu_free+0x28/0x43 [ 2.753355] [] tlb_flush_mmu+0x20/0x23 [ 2.753355] [] tlb_finish_mmu+0x14/0x39 [ 2.753355] [] unmap_region+0xcd/0xdf [ 2.753355] [] ? vma_gap_callbacks_propagate+0x18/0x33 [ 2.753355] [] do_munmap+0x252/0x2e0 [ 2.753355] [] vm_munmap+0x44/0x5c [ 2.753355] [] SyS_munmap+0x23/0x29 [ 2.753355] [] system_call_fastpath+0x16/0x1b [ 2.753355] ---[ end trace cfeb07101f6fbdfb ]--- [ 2.780913] ------------[ cut here ]------------ git bisect start 633594bb2d3890711a887897f2003f41735f0dfa 71d273fa769ea21f2422a18482e002a07ab9f8f3 -- git bisect bad df2c04c68831d13d505c127b5aa172361a17c7e3 # 14:51 0- 4 Revert "mm, CMA: change cma_declare_contiguous() to obey coding convention" git bisect bad dc8a26d69d2039a81985549b00fc7e7e2bd34dd4 # 14:58 0- 2 Merge branch 'akpm/master' git bisect bad fe297b4d6987d04e8b3878b3ee47efd26b95114d # 15:16 0- 8 Merge branch 'akpm-current/current' git bisect good 6b11d02e25c79a8961983a966b7fafcdc36c7a91 # 15:24 20+ 0 slab: do not keep free objects/slabs on dead memcg caches git bisect bad 11709212b3a5479fcc63dda3160f4f4b0251f914 # 16:02 0- 4 mm/util.c: add kstrimdup() git bisect good d070bd175fccaab0616d8aec75acbde480531fee # 16:11 20+ 0 mm: memcontrol: catch root bypass in move precharge git bisect bad e77f4c327c7aa19d2c9ea28ebeb3a7166db418ad # 16:27 0- 12 m68k: call find_vma with the mmap_sem held in sys_cacheflush() git bisect bad ddc5bfec501f4be3f9e89084c2db270c0c45d1d6 # 16:48 0- 1 mm: memcontrol: rewrite uncharge API git bisect good 737f5b9367a254a3b3149b3abae65470f5ed941e # 17:10 20+ 0 mm: memcontrol: do not acquire page_cgroup lock for kmem pages git bisect good 5b647620c6cae14cc27782c3491c2da0f1cf245c # 17:40 20+ 0 mm-memcontrol-rewrite-charge-api-fix # first bad commit: [ddc5bfec501f4be3f9e89084c2db270c0c45d1d6] mm: memcontrol: rewrite uncharge API git bisect good 5b647620c6cae14cc27782c3491c2da0f1cf245c # 17:43 60+ 0 mm-memcontrol-rewrite-charge-api-fix git bisect bad 633594bb2d3890711a887897f2003f41735f0dfa # 17:43 0- 13 Add linux-next specific files for 20140620 git bisect good 3c8fb50445833b93f69b6b703a29aae3523cad0c # 18:06 60+ 0 Merge tag 'pm+acpi-3.16-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect bad 633594bb2d3890711a887897f2003f41735f0dfa # 18:06 0- 13 Add linux-next specific files for 20140620 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=quantal-core-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/blob/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Fengguang