From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 316EAC47404 for ; Wed, 9 Oct 2019 14:34:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F0DF2206BB for ; Wed, 9 Oct 2019 14:34:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F0DF2206BB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C1618E0005; Wed, 9 Oct 2019 10:34:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 772538E0003; Wed, 9 Oct 2019 10:34:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 660DA8E0005; Wed, 9 Oct 2019 10:34:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0109.hostedemail.com [216.40.44.109]) by kanga.kvack.org (Postfix) with ESMTP id 491688E0003 for ; Wed, 9 Oct 2019 10:34:42 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 0221252A3 for ; Wed, 9 Oct 2019 14:34:42 +0000 (UTC) X-FDA: 76024492404.07.scale91_6330611fb831d X-HE-Tag: scale91_6330611fb831d X-Filterd-Recvd-Size: 3976 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Wed, 9 Oct 2019 14:34:41 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 55699AD12; Wed, 9 Oct 2019 14:34:40 +0000 (UTC) Date: Wed, 9 Oct 2019 16:34:39 +0200 From: Michal Hocko To: Qian Cai Cc: Petr Mladek , Christian Borntraeger , Heiko Carstens , sergey.senozhatsky.work@gmail.com, rostedt@goodmis.org, peterz@infradead.org, linux-mm@kvack.org, john.ogness@linutronix.de, akpm@linux-foundation.org, Vasily Gorbik , Peter Oberparleiter , david@redhat.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] mm/page_isolation: fix a deadlock with printk() Message-ID: <20191009143439.GF6681@dhcp22.suse.cz> References: <20191008183525.GQ6681@dhcp22.suse.cz> <1570561573.5576.307.camel@lca.pw> <20191008191728.GS6681@dhcp22.suse.cz> <1570563324.5576.309.camel@lca.pw> <20191009114903.aa6j6sa56z2cssom@pathway.suse.cz> <1570626402.5937.1.camel@lca.pw> <20191009132746.GA6681@dhcp22.suse.cz> <1570628593.5937.3.camel@lca.pw> <20191009135155.GC6681@dhcp22.suse.cz> <1570630784.5937.5.camel@lca.pw> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1570630784.5937.5.camel@lca.pw> User-Agent: Mutt/1.10.1 (2018-07-13) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 09-10-19 10:19:44, Qian Cai wrote: > On Wed, 2019-10-09 at 15:51 +0200, Michal Hocko wrote: [...] > > Can you paste the full lock chain graph to be sure we are on the same > > page? >=20 > WARNING: possible circular locking dependency detected > 5.3.0-next-20190917 #8 Not tainted > ------------------------------------------------------ > test.sh/8653 is trying to acquire lock: > ffffffff865a4460 (console_owner){-.-.}, at: > console_unlock+0x207/0x750 >=20 > but task is already holding lock: > ffff88883fff3c58 (&(&zone->lock)->rlock){-.-.}, at: > __offline_isolated_pages+0x179/0x3e0 >=20 > which lock already depends on the new lock. >=20 >=20 > the existing dependency chain (in reverse order) is: >=20 > -> #3 (&(&zone->lock)->rlock){-.-.}: > =A0=A0=A0=A0=A0=A0=A0__lock_acquire+0x5b3/0xb40 > =A0=A0=A0=A0=A0=A0=A0lock_acquire+0x126/0x280 > =A0=A0=A0=A0=A0=A0=A0_raw_spin_lock+0x2f/0x40 > =A0=A0=A0=A0=A0=A0=A0rmqueue_bulk.constprop.21+0xb6/0x1160 > =A0=A0=A0=A0=A0=A0=A0get_page_from_freelist+0x898/0x22c0 > =A0=A0=A0=A0=A0=A0=A0__alloc_pages_nodemask+0x2f3/0x1cd0 > =A0=A0=A0=A0=A0=A0=A0alloc_pages_current+0x9c/0x110 > =A0=A0=A0=A0=A0=A0=A0allocate_slab+0x4c6/0x19c0 > =A0=A0=A0=A0=A0=A0=A0new_slab+0x46/0x70 > =A0=A0=A0=A0=A0=A0=A0___slab_alloc+0x58b/0x960 > =A0=A0=A0=A0=A0=A0=A0__slab_alloc+0x43/0x70 > =A0=A0=A0=A0=A0=A0=A0__kmalloc+0x3ad/0x4b0 > =A0=A0=A0=A0=A0=A0=A0__tty_buffer_request_room+0x100/0x250 > =A0=A0=A0=A0=A0=A0=A0tty_insert_flip_string_fixed_flag+0x67/0x110 > =A0=A0=A0=A0=A0=A0=A0pty_write+0xa2/0xf0 > =A0=A0=A0=A0=A0=A0=A0n_tty_write+0x36b/0x7b0 > =A0=A0=A0=A0=A0=A0=A0tty_write+0x284/0x4c0 > =A0=A0=A0=A0=A0=A0=A0__vfs_write+0x50/0xa0 > =A0=A0=A0=A0=A0=A0=A0vfs_write+0x105/0x290 > =A0=A0=A0=A0=A0=A0=A0redirected_tty_write+0x6a/0xc0 > =A0=A0=A0=A0=A0=A0=A0do_iter_write+0x248/0x2a0 > =A0=A0=A0=A0=A0=A0=A0vfs_writev+0x106/0x1e0 > =A0=A0=A0=A0=A0=A0=A0do_writev+0xd4/0x180 > =A0=A0=A0=A0=A0=A0=A0__x64_sys_writev+0x45/0x50 > =A0=A0=A0=A0=A0=A0=A0do_syscall_64+0xcc/0x76c > =A0=A0=A0=A0=A0=A0=A0entry_SYSCALL_64_after_hwframe+0x49/0xbe This one looks indeed legit. pty_write is allocating memory from inside the port->lock. But this seems to be quite broken, right? The forward progress depends on GFP_ATOMIC allocation which might fail easily under memory pressure. So the preferred way to fix this should be to change the allocation scheme to use the preallocated buffer and size it from a context when it doesn't hold internal locks. It might be a more complex fix than using printk_deferred or other games but addressing that would make the pty code more robust as well. --=20 Michal Hocko SUSE Labs