From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id F37266B0033 for ; Tue, 7 Feb 2017 03:43:06 -0500 (EST) Received: by mail-wr0-f199.google.com with SMTP id o16so2358189wra.2 for ; Tue, 07 Feb 2017 00:43:06 -0800 (PST) Received: from mx2.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id p94si4171003wrc.161.2017.02.07.00.43.05 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 07 Feb 2017 00:43:05 -0800 (PST) Date: Tue, 7 Feb 2017 09:43:03 +0100 From: Michal Hocko Subject: Re: mm: deadlock between get_online_cpus/pcpu_alloc Message-ID: <20170207084302.GA5810@dhcp22.suse.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Dmitry Vyukov Cc: Vlastimil Babka , Tejun Heo , Christoph Lameter , "linux-mm@kvack.org" , LKML , Thomas Gleixner , Ingo Molnar , Peter Zijlstra , syzkaller , Mel Gorman , Andrew Morton On Mon 06-02-17 20:13:35, Dmitry Vyukov wrote: [...] > Fuzzer now runs on 510948533b059f4f5033464f9f4a0c32d4ab0c08 of > mmotm/auto-latest > (git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git): > > commit 510948533b059f4f5033464f9f4a0c32d4ab0c08 > Date: Thu Feb 2 10:08:47 2017 +0100 > mmotm: userfaultfd-non-cooperative-add-event-for-memory-unmaps-fix > > The commit you referenced is already there: > > commit 806b158031ca0b4714e775898396529a758ebc2c > Date: Thu Feb 2 08:53:16 2017 +0100 > mm, page_alloc: use static global work_struct for draining per-cpu pages > > But I still got: > > [ INFO: possible circular locking dependency detected ] > 4.9.0 #6 Not tainted > ------------------------------------------------------- > syz-executor1/8199 is trying to acquire lock: > (cpu_hotplug.dep_map){++++++}, at: [] get_online_cpus+0x37/0x90 kernel/cpu.c:246 > but task is already holding lock: > (pcpu_alloc_mutex){+.+.+.}, at: [] pcpu_alloc+0xbda/0x1280 mm/percpu.c:896 > which lock already depends on the new lock. > the original was too hard to read so here is the reformated output. > the existing dependency chain (in reverse order) is: > > [ 403.953319] [] validate_chain kernel/locking/lockdep.c:2265 [inline] > [ 403.953319] [] __lock_acquire+0x2149/0x3430 kernel/locking/lockdep.c:3338 > [ 403.961232] [] lock_acquire+0x2a1/0x630 kernel/locking/lockdep.c:3753 > [ 403.968788] [] __mutex_lock_common kernel/locking/mutex.c:521 [inline] > [ 403.968788] [] mutex_lock_nested+0x24e/0xff0 kernel/locking/mutex.c:621 > [ 403.976782] [] pcpu_alloc+0xbda/0x1280 mm/percpu.c:896 > [ 403.984266] [] __alloc_percpu+0x24/0x30 mm/percpu.c:1075 > [ 403.991873] [] smpcfd_prepare_cpu+0x73/0xd0 kernel/smp.c:44 > [ 403.999799] [] cpuhp_invoke_callback+0x254/0x1480 kernel/cpu.c:136 > [ 404.008253] [] cpuhp_up_callbacks+0x81/0x2a0 kernel/cpu.c:493 > [ 404.016365] [] _cpu_up+0x1e3/0x2a0 kernel/cpu.c:1057 > [ 404.023507] [] do_cpu_up+0x73/0xa0 kernel/cpu.c:1087 > [ 404.030647] [] cpu_up+0x18/0x20 kernel/cpu.c:1095 > [ 404.037523] [] smp_init+0xe9/0xee kernel/smp.c:564 > [ 404.044559] [] kernel_init_freeable+0x439/0x690 init/main.c:1010 > [ 404.052811] [] kernel_init+0x13/0x180 init/main.c:941 > [ 404.060198] [] ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:433 cpu_hotplug_begin cpu_hotplug.lock pcpu_alloc pcpu_alloc_mutex > [ 404.072827] [] validate_chain kernel/locking/lockdep.c:2265 [inline] > [ 404.072827] [] __lock_acquire+0x2149/0x3430 kernel/locking/lockdep.c:3338 > [ 404.080733] [] lock_acquire+0x2a1/0x630 kernel/locking/lockdep.c:3753 > [ 404.088311] [] __mutex_lock_common kernel/locking/mutex.c:521 [inline] > [ 404.088311] [] mutex_lock_nested+0x24e/0xff0 kernel/locking/mutex.c:621 > [ 404.096318] [] cpu_hotplug_begin+0x206/0x2e0 kernel/cpu.c:304 > [ 404.104321] [] _cpu_up+0xca/0x2a0 kernel/cpu.c:1011 > [ 404.111357] [] do_cpu_up+0x73/0xa0 kernel/cpu.c:1087 > [ 404.118480] [] cpu_up+0x18/0x20 kernel/cpu.c:1095 > [ 404.125360] [] smp_init+0xe9/0xee kernel/smp.c:564 > [ 404.132393] [] kernel_init_freeable+0x439/0x690 init/main.c:1010 > [ 404.140668] [] kernel_init+0x13/0x180 init/main.c:941 > [ 404.148079] [] ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:433 cpu_hotplug_begin cpu_hotplug.lock > [ 404.160977] [] check_prev_add kernel/locking/lockdep.c:1828 [inline] > [ 404.160977] [] check_prevs_add+0xa8d/0x1c00 kernel/locking/lockdep.c:1938 > [ 404.168898] [] validate_chain kernel/locking/lockdep.c:2265 [inline] > [ 404.168898] [] __lock_acquire+0x2149/0x3430 kernel/locking/lockdep.c:3338 > [ 404.176844] [] lock_acquire+0x2a1/0x630 kernel/locking/lockdep.c:3753 > [ 404.184416] [] get_online_cpus+0x62/0x90 kernel/cpu.c:248 > [ 404.192103] [] drain_all_pages+0xf8/0x710 mm/page_alloc.c:2385 > [ 404.199880] [] __alloc_pages_direct_reclaim mm/page_alloc.c:3440 [inline] > [ 404.199880] [] __alloc_pages_slowpath+0x8fd/0x2370 mm/page_alloc.c:3778 > [ 404.208406] [] __alloc_pages_nodemask+0x8f5/0xc60 mm/page_alloc.c:3980 > [ 404.216851] [] __alloc_pages include/linux/gfp.h:426 [inline] > [ 404.216851] [] __alloc_pages_node include/linux/gfp.h:439 [inline] > [ 404.216851] [] alloc_pages_node include/linux/gfp.h:453 [inline] > [ 404.216851] [] pcpu_alloc_pages mm/percpu-vm.c:93 [inline] > [ 404.216851] [] pcpu_populate_chunk+0x1e1/0x900 mm/percpu-vm.c:282 > [ 404.225015] [] pcpu_alloc+0xe01/0x1280 mm/percpu.c:998 > [ 404.232482] [] __alloc_percpu_gfp+0x27/0x30 mm/percpu.c:1062 > [ 404.240389] [] bpf_array_alloc_percpu kernel/bpf/arraymap.c:34 [inline] > [ 404.240389] [] array_map_alloc+0x532/0x710 kernel/bpf/arraymap.c:99 > [ 404.248224] [] find_and_alloc_map kernel/bpf/syscall.c:34 [inline] > [ 404.248224] [] map_create kernel/bpf/syscall.c:188 [inline] > [ 404.248224] [] SYSC_bpf kernel/bpf/syscall.c:870 [inline] > [ 404.248224] [] SyS_bpf+0xd64/0x2500 kernel/bpf/syscall.c:827 > [ 404.255434] [] entry_SYSCALL_64_fastpath+0x1f/0xc2 pcpu_alloc pcpu_alloc_mutex drain_all_pages pcpu_drain_mutex get_online_cpus cpu_hotplug.lock so the deadlock is real! -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org