From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94FC7CA9EAF for ; Thu, 24 Oct 2019 23:34:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2F11D20650 for ; Thu, 24 Oct 2019 23:34:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Bv5nspXZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F11D20650 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 80C7E6B0003; Thu, 24 Oct 2019 19:34:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BC8B6B0006; Thu, 24 Oct 2019 19:34:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 684A56B0007; Thu, 24 Oct 2019 19:34:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0034.hostedemail.com [216.40.44.34]) by kanga.kvack.org (Postfix) with ESMTP id 41DD06B0003 for ; Thu, 24 Oct 2019 19:34:54 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id A621D52B4 for ; Thu, 24 Oct 2019 23:34:53 +0000 (UTC) X-FDA: 76080285666.30.store29_d8f94eda1752 X-HE-Tag: store29_d8f94eda1752 X-Filterd-Recvd-Size: 6016 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 24 Oct 2019 23:34:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:Cc:References:To: Subject:Sender:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=OJw5ZmjGXAsA3sm9+sitk/amoBIrD9+6eN7vsgzpRII=; b=Bv5nspXZoUneRGQhj6pwSf/NB wHanD+G8xtrEdxoLpdS4ujDhnbewXxhtblHghT3lqLR5GBhCaNCO2n/WSkGAYoJXdtLvsEYeC+SUx qGILMiiDQQ38Ad+fBPNVY1DWQ0QeuAHlW7Xu5VlSIymdhkmJUXOkRBZ2cYbv46coIPdSEIrcCKS2B Udpo1lMWrUAIrQDrRU7A+9w+X3qrbnsI1DoNLahrbL9birpNGja+rO1GodiMI2AFr8EOJg6yyBNiN zM2YPrMTacxKZTGFPA+gQt8dpT5dGMU03idEKTcdmAP+VxMYAd57CtKazSFQTyeIK86uSWdCMYAhr gSNnywJ7A==; Received: from [2601:1c0:6280:3f0::9ef4] by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1iNmcw-0007S0-JD; Thu, 24 Oct 2019 23:34:46 +0000 Subject: Re: mlockall(MCL_CURRENT) blocking infinitely To: Robert Stupp , linux-kernel@vger.kernel.org, Linux MM References: <4576b336-66e6-e2bb-cd6a-51300ed74ab8@snazy.de> Cc: Andrew Morton , "Potyra, Stefan" From: Randy Dunlap Message-ID: Date: Thu, 24 Oct 2019 16:34:46 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: <4576b336-66e6-e2bb-cd6a-51300ed74ab8@snazy.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: [adding linux-mm + people] I see only one change in the last 4 years: commit dedca63504a204dc8410d98883fdc16dffa8cb80 Author: Potyra, Stefan Date: Thu Jun 13 15:55:55 2019 -0700 mm/mlock.c: mlockall error for flag MCL_ONFAULT On 10/24/19 12:36 AM, Robert Stupp wrote: > Hi guys, >=20 > I've got an issue with `mlockall(MCL_CURRENT)` after upgrading Ubuntu 1= 9.04 to 19.10 - i.e. kernel version change from 5.0.x to 5.3.x. >=20 > The following simple program hangs forever with one CPU running at 100%= (kernel): >=20 > #include > #include > int main(char** argv) { > =C2=A0 printf("Before mlockall(MCL_CURRENT)\n"); > =C2=A0 // works in 5.0 > =C2=A0 // hangs forever w/ 5.1 and newer > =C2=A0 mlockall(MCL_CURRENT); > =C2=A0 printf("After mlockall(MCL_CURRENT)\n"); > } >=20 > All kernel versions since 5.1 (tried 5.1.0, 5.1.21, 5.2.21, 5.3.0-19, 5= .3.7, 5.4-rc4) show the same symptom (hanging in mlockall(MCL_CURRENT) wi= th 100% kernel-CPU). 5.0 kernel versions (5.0.21) are fine. >=20 > First, I thought, that it's something generic, so I tried the above pro= gram in a fresh install of Ubuntu eoan (5.3.x) in a VM in virtualbox, but= it works fine there. So I suspect, that it has to do with something that= 's specific to my machine. >=20 > My first suspicion was that some library "hijacks" mlockall(), but call= ing the test program with `LD_DEBUG=3Dall` shows that glibc gets called d= irectly: > =C2=A0=C2=A0=C2=A0=C2=A0 12248:=C2=A0=C2=A0=C2=A0 symbol=3Dmlockall;=C2= =A0 lookup in file=3D./test [0] > =C2=A0=C2=A0=C2=A0=C2=A0 12248:=C2=A0=C2=A0=C2=A0 symbol=3Dmlockall;=C2= =A0 lookup in file=3D/lib/x86_64-linux-gnu/libc.so.6 [0] > =C2=A0=C2=A0=C2=A0=C2=A0 12248:=C2=A0=C2=A0=C2=A0 binding file ./test [= 0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `mlockall' [GLIB= C_2.2.5] > An `strace` doesn't show anything meaningful (beside that mlockall's be= en called but never returns). dmesg and syslog don't show anything obviou= s (to me) as well. >=20 > Some information about the machine: > - Intel(R) Core(TM) i7-6900K, Intel X99 chipset > - NVMe 1.1b > - 64GB RAM (4x 16GB) >=20 > I've also reverted all changes for sysctl and ld.conf and checked for o= ther suspicious software without any luck. >=20 > I also tried a bunch of variations of the above program, but only `mloc= kall(MCL_CURRENT)` or `mlockall(MCL_FUTURE | MCL_CURRENT)` hang. >=20 > A `git diff v5.0..v5.1 mm/` doesn't show anything obvious (to me). >=20 > It seems, there's no debug/trace information that would help to find ou= t what exactly it's doing. >=20 > I'm kinda lost at the moment. >=20 >=20 > PS: Variations of the above test program: >=20 > #include > #include > char foo[65536]; > int main(char** argv) { > =C2=A0 printf("Before mlock()\n"); > =C2=A0 int e =3D mlock(foo, 8192); // works in 5.0, 5.1, 5.2, 5.3, 5.4 > =C2=A0 printf("After mlock()=3D%d\n", e); > } >=20 >=20 > #include > #include > #include > int main(char** argv) { > =C2=A0 printf("Before mlockall(MCL_FUTURE)\n"); > =C2=A0 int e =3D mlockall(MCL_FUTURE); // works in 5.0, 5.1, 5.2, 5.3, = 5.4 > =C2=A0 printf("After mlockall(MCL_FUTURE) =3D %d\n", e); > =C2=A0 void* mem =3D malloc(1024 * 1024 * 1024); > =C2=A0 printf("After malloc()\n"); > =C2=A0 mem =3D malloc(1024 * 1024 * 1024); > =C2=A0 printf("After malloc()\n"); > =C2=A0 mem =3D malloc(1024 * 1024 * 1024); > =C2=A0 printf("After malloc()\n"); > =C2=A0 // works in 5.0, 5.1, 5.2, 5.3, 5.4 > } >=20 >=20 > #include > #include > int main(char** argv) { > =C2=A0 printf("Before munlockall()\n"); > =C2=A0 int e =3D munlockall(); // works in 5.0, 5.1, 5.2, 5.3, 5.4 > =C2=A0 printf("After munlockall() =3D %d\n", e); > } >=20 >=20 > #include > #include > int main(char** argv) { > =C2=A0 printf("Before mlockall(MCL_CURRENT|MCL_FUTURE)\n"); > =C2=A0 // works in 5.0 > =C2=A0 // hangs forever w/ 5.1 and newer > =C2=A0 int e =3D mlockall(MCL_CURRENT|MCL_FUTURE); > =C2=A0 printf("After mlockall(MCL_CURRENT|MCL_FUTURE) =3D %d\n", e); > } >=20 > PPS: Kernel version images installed from https://kernel.ubuntu.com/~ke= rnel-ppa/mainline/?C=3DN;O=3DD >=20 --=20 ~Randy