From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2875C33C9E for ; Tue, 7 Jan 2020 21:26:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B3F51222D9 for ; Tue, 7 Jan 2020 21:26:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=colorremedies-com.20150623.gappssmtp.com header.i=@colorremedies-com.20150623.gappssmtp.com header.b="W0iweNgh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3F51222D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=colorremedies.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 45EB18E0005; Tue, 7 Jan 2020 16:26:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E7FD8E0001; Tue, 7 Jan 2020 16:26:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AF5A8E0005; Tue, 7 Jan 2020 16:26:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0020.hostedemail.com [216.40.44.20]) by kanga.kvack.org (Postfix) with ESMTP id 10C768E0001 for ; Tue, 7 Jan 2020 16:26:05 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id C08DB49961F for ; Tue, 7 Jan 2020 21:26:04 +0000 (UTC) X-FDA: 76352121048.23.grade53_3500efce1aa25 X-HE-Tag: grade53_3500efce1aa25 X-Filterd-Recvd-Size: 4967 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 7 Jan 2020 21:26:04 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id m24so340531wmc.3 for ; Tue, 07 Jan 2020 13:26:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=colorremedies-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MTYEANQTpjNz6385TWgavIxFsLfxQlBLIjkp2BdU32c=; b=W0iweNghlbqfoHKJnY1Fn5c+wec0FvxHPVRy0XcErFzmoYQuK6jz9QIPlCyb24h9HO XQXbGMOSam/fGrJygzllJpkI+osbUHf6aGp9bCphBHo/appu3K3plUhG+E2xAjR6v4bn uU11Y/+2MeenBH4SCSvMYE23BVQbzkm27hiJ22wqTrBLg8VhuiAw3OXuqcap+cBMsDFT rTaaaB7v10uC3ShmDPOL+waflik1hwACztzj50Rq8qzFBWppFaLYCorQ8zzKQUoEqERf nzK0blC75FxjozT0Mgugh9d88leZpi2fVeVOVLsRCUrPLXWWBQbfCO0nqiVn4Pn5u47g 7ArQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MTYEANQTpjNz6385TWgavIxFsLfxQlBLIjkp2BdU32c=; b=J5LlOgVgTsRcnDdV9ckSI2SExajSG4reEUc+Y6vTWFzwyWfw4RdUhLyZYxAKp4vRFQ GL+tx44Hhw9vrTZtYv0FE+21GJoP55Y5A383pYZTWjgXdJ0pq6rqaPN5Zo5gE6elHklv 2J1vs2IhvQgmNmulWA53ISUPjO3YJVsMKjBPKcqv+9YsBFkSFmhH+TmIeYVvFFfyD1fo OR0E1P/tzNDEE9xqve2tcNHP/lkupKivgajCtQl96k5tAsKfOusFgnJhhyLLtbq6wWBq 6ciZZCan0YMfQzjc9o+d8hiiKEK+KBS/Q6Gp7lHgsWV/hvQXBdPhCbLzY9rEtjvkei/w y79g== X-Gm-Message-State: APjAAAU6Hl/hazmSBoJ59j3N6NEOKk8N5r66HLpjw42qCC/3fA7jG2sT Bw9sfnmCQxDuGixCJ/GqC5nXEHPKsyxVArEhl/eV9Q== X-Google-Smtp-Source: APXvYqz//A6usWpLU1LQIqw5FOq/uDLrbOEEJyTCvRR23iHDVrd1z8Dukn7Ftg5Z7Py7hZ8P89LWz0YdG4xyJyc2Vwc= X-Received: by 2002:a05:600c:294:: with SMTP id 20mr314771wmk.97.1578432362851; Tue, 07 Jan 2020 13:26:02 -0800 (PST) MIME-Version: 1.0 References: <20200107205824.GM32178@dhcp22.suse.cz> In-Reply-To: <20200107205824.GM32178@dhcp22.suse.cz> From: Chris Murphy Date: Tue, 7 Jan 2020 14:25:46 -0700 Message-ID: Subject: Re: user space unresponsive, followup: lsf/mm congestion To: Michal Hocko Cc: Chris Murphy , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 7, 2020 at 1:58 PM Michal Hocko wrote: > > On Tue 07-01-20 13:29:20, Chris Murphy wrote: > > Hi, > > > > This is in response to: > > https://lore.kernel.org/linux-fsdevel/20200104090955.GF23195@dread.disaster.area/T/#m8b25fd42501d780d8053fc7aa9f4e3a28a19c49f > > > > I decided to open a bug report for tracking and attachments but I'm > > also subscribed now to this list so - either here or there. > > > > "loss of responsiveness during heavy swap" > > https://bugzilla.kernel.org/show_bug.cgi?id=206117 > > Please collect more snapshots of /proc/vmstat (e.g. in 1s intervals) OK. > Btw. from a quick look at the sysrq output there seems to be quite a lot > of tasks (more than 1k) running on the system. Only handful of them > belong to the compilation. kswapd is busy and 13 processes in direct > reclaim all swapping out to the disk. There might be many dozens of tabs in Firefox with nothing loaded in them, trying to keep the testing more real world (a compile while browsing) rather than being too deferential to the compile. That does clutter the sysrq+t but it doesn't change the outcome of the central culprit which is the ninja compile, which by default does n+2 jobs where n is the number of virtual CPUs. > From the above, my first guess would be that you are over subscribing > memory you have available. I would focus on who is consuming all that > memory. ninja - I have made the argument that it is in some sense sabotaging the system, and I think they're trying to do something a little smarter with their defaults; however, it's an unprivileged task acting as a kind of fork bomb that takes down the system. It's a really eyebrow raising and remarkable experience. And it's common within the somewhat vertical use case of developers compiling things on their own systems. Many IDE's use a ton of resources, as much as they can get. It's not clear to me by what mechanism either the user or these processes are supposed to effectively negotiate for limited resources, other than resource restriction. But anyway, they aren't contrived or malicious examples. A much more synthetic example is 'tail /dev/zero' which is much more quickly arrrested by the kernel oom-killer, at least on recent kernels. -- Chris Murphy