From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09484ECE58D for ; Mon, 7 Oct 2019 14:11:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDC442173B for ; Mon, 7 Oct 2019 14:11:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDC442173B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6A7298E0005; Mon, 7 Oct 2019 10:11:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 630CD8E0003; Mon, 7 Oct 2019 10:11:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51FA88E0005; Mon, 7 Oct 2019 10:11:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 32A0A8E0003 for ; Mon, 7 Oct 2019 10:11:03 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id CE338180AD803 for ; Mon, 7 Oct 2019 14:11:02 +0000 (UTC) X-FDA: 76017175164.07.bird86_4343df495312b X-HE-Tag: bird86_4343df495312b X-Filterd-Recvd-Size: 5249 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Oct 2019 14:11:02 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 999A6ADE0; Mon, 7 Oct 2019 14:11:00 +0000 (UTC) Date: Mon, 7 Oct 2019 16:10:59 +0200 From: Petr Mladek To: Qian Cai Cc: Michal Hocko , sergey.senozhatsky.work@gmail.com, rostedt@goodmis.org, peterz@infradead.org, linux-mm@kvack.org, john.ogness@linutronix.de, akpm@linux-foundation.org, david@redhat.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] mm/page_isolation: fix a deadlock with printk() Message-ID: <20191007141059.friotqx2ymwvlo3j@pathway.suse.cz> References:<20191007080742.GD2381@dhcp22.suse.cz> <20191007113710.GH2381@dhcp22.suse.cz> <1570450304.5576.283.camel@lca.pw> <20191007124356.GJ2381@dhcp22.suse.cz> <1570453622.5576.288.camel@lca.pw> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To:<1570453622.5576.288.camel@lca.pw> User-Agent: NeoMutt/20170912 (1.9.0) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 2019-10-07 09:07:02, Qian Cai wrote: > On Mon, 2019-10-07 at 14:43 +0200, Michal Hocko wrote: > > On Mon 07-10-19 08:11:44, Qian Cai wrote: > > > On Mon, 2019-10-07 at 13:37 +0200, Michal Hocko wrote: > > > > On Mon 07-10-19 07:04:00, Qian Cai wrote: > > > > >=20 > > > > >=20 > > > > > > On Oct 7, 2019, at 4:07 AM, Michal Hocko = wrote: > > > > > >=20 > > > > > > I do not think that removing the printk is the right long ter= m solution. > > > > > > While I do agree that removing the debugging printk __offline= _isolated_pages > > > > > > does make sense because it is essentially of a very limited u= se, this > > > > > > doesn't really solve the underlying problem. There are likel= y other > > > > > > printks from zone->lock. It would be much more saner to actua= lly > > > > > > disallow consoles to allocate any memory while printk is call= ed from an > > > > > > atomic context. > > > > >=20 > > > > > No, there is only a handful of places called printk() from > > > > > zone->lock. It is normal that the callers will quietly process > > > > > =E2=80=9Cstruct zone=E2=80=9D modification in a short section w= ith zone->lock > > > > > held. > > > >=20 > > > > It is extremely error prone to have any zone->lock vs. printk > > > > dependency. I do not want to play an endless whack a mole. > > > >=20 > > > > > No, it is not about =E2=80=9Callocate any memory while printk i= s called from an > > > > > atomic context=E2=80=9D. It is opposite lock chain from differ= ent processors which has the same effect. For example, > > > > >=20 > > > > > CPU0: CPU1: CPU2: > > > > > console_owner > > > > > sclp_lock > > > > > sclp_lock zone_lock > > > > > zone_lock > > > > > console_owner > > > >=20 > > > > Why would sclp_lock ever take a zone->lock (apart from an allocat= ion). > > > > So really if sclp_lock is a lock that might be taken from many co= ntexts > > > > and generate very subtle lock dependencies then it should better = be > > > > really careful what it is calling into. > > > >=20 > > > > In other words you are trying to fix a wrong end of the problem. = Fix the > > > > console to not allocate or depend on MM by other means. > > >=20 > > > It looks there are way too many places that could generate those in= direct lock > > > chains that are hard to eliminate them all. Here is anther example,= where it > > > has, > >=20 > > Yeah and I strongly suspect they are consoles which are broken and ne= ed > > to be fixed rathert than the problem papered over. > >=20 > > I do realize how tempting it is to remove all printks from the > > zone->lock but do realize that as soon as the allocator starts using = any > > other locks then we are back to square one and the problem is there > > again. We would have to drop _all_ printks from any locked section in > > the allocator and I do not think this is viable. > >=20 > > Really, the only way forward is to make these consoles be more carefu= l > > of external dependencies. >=20 > Even with the new printk() Petr proposed. There is no guarantee it will= fix it > properly. It looks like just reduce the chance of this kind of deadlock= s as it > may or may not call=C2=A0wake_up_klogd() in=C2=A0vprintk_emit() depends= on timing. The chain below is wrong: > zone->lock > printk_deferred() > vprintk_emit() > wake_up_klogd() wake_up_klogd() calls irq_work_queue(). It queues the work for an interrupt handler and triggers the interrupt. > wake_up_klogd_work_func() > console_unlock() The work is done in the interrupt context. The interrupt could never be handled under zone->lock. So, printk_deferred() would help. But I do not think that it is really needed. I am going to answer the original mail with all the full lockdep report. Best Regards, Petr