From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00400C33CA1 for ; Tue, 7 Jan 2020 20:58:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BF423214D8 for ; Tue, 7 Jan 2020 20:58:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF423214D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D2928E0007; Tue, 7 Jan 2020 15:58:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 583458E0006; Tue, 7 Jan 2020 15:58:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 471F88E0007; Tue, 7 Jan 2020 15:58:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id 2F41B8E0006 for ; Tue, 7 Jan 2020 15:58:29 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id E486652A7 for ; Tue, 7 Jan 2020 20:58:28 +0000 (UTC) X-FDA: 76352051496.27.dog95_671d6145c3f4d X-HE-Tag: dog95_671d6145c3f4d X-Filterd-Recvd-Size: 3695 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Tue, 7 Jan 2020 20:58:28 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id z3so1101737wru.3 for ; Tue, 07 Jan 2020 12:58:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Tln4Uo4AT3yXBjWpYb2vUIO4YNsB1eJLcZD+kLsghqk=; b=HdXlbQy4USkBvQ625PMQG2M1QCYpfwCDvcl2nF1ZOGphVo7jZq/NY9oL7iNx0BqPrc h4dubMrCC1yGKcM+o5s//2We0E+Ln4cl3cTTMLfV3vB+VRs5pKSOOeYQQ+sta1yVxL4R Kc9Ot8gTjTNG9GACXTUusAR5U8+HsS3+OU+XqQbO3Arm1MBq1UOuKR3m6c/sEvmruBQg DQCx0XyrymmCV9OktyHR79n1YWDbvjngcI9JYahwa6K6ifVNHt/mQSzBEraPdUFXR2mv wQq+MLMe3bN0Z+luKeOB9rj9JkpdT6BtMxhQBDJHkLgD6dSLR4hZ9lJ8ywjZOGsrOIU+ 9O9Q== X-Gm-Message-State: APjAAAUHASlcqw1n+ImjJu3FTH3ZXSCawzErQgA7w4i/LPKp0AZbDiK2 F1BJuUmmf0pRwF8R6EkF8xY= X-Google-Smtp-Source: APXvYqybGu4lP7++sUDZ4TgEgx+V60OUxVIUSmZzbc7BK18AEP3miYgqqG7Y5IkQ3w4SaV6ECaozbw== X-Received: by 2002:a05:6000:118d:: with SMTP id g13mr1015552wrx.141.1578430707359; Tue, 07 Jan 2020 12:58:27 -0800 (PST) Received: from localhost (ip-37-188-146-105.eurotel.cz. [37.188.146.105]) by smtp.gmail.com with ESMTPSA id k8sm1347953wrl.3.2020.01.07.12.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2020 12:58:26 -0800 (PST) Date: Tue, 7 Jan 2020 21:58:24 +0100 From: Michal Hocko To: Chris Murphy Cc: linux-mm@kvack.org Subject: Re: user space unresponsive, followup: lsf/mm congestion Message-ID: <20200107205824.GM32178@dhcp22.suse.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.2 (2019-09-21) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 07-01-20 13:29:20, Chris Murphy wrote: > Hi, > > This is in response to: > https://lore.kernel.org/linux-fsdevel/20200104090955.GF23195@dread.disaster.area/T/#m8b25fd42501d780d8053fc7aa9f4e3a28a19c49f > > I decided to open a bug report for tracking and attachments but I'm > also subscribed now to this list so - either here or there. > > "loss of responsiveness during heavy swap" > https://bugzilla.kernel.org/show_bug.cgi?id=206117 Please collect more snapshots of /proc/vmstat (e.g. in 1s intervals) since you start your workload. This will give us an insight on how the memory characteristics change over time. From the single snapshot we can see nr_zone_inactive_anon 247598 nr_zone_active_anon 1551129 nr_zone_inactive_file 25283 nr_zone_active_file 21563 pswpin 6002574 pswpout 11199783 which tells us that there is not all that much page cache resident (less than 200MB) while the majority of the memory is anonymous (7GB) most of it tracked as active. Btw. from a quick look at the sysrq output there seems to be quite a lot of tasks (more than 1k) running on the system. Only handful of them belong to the compilation. kswapd is busy and 13 processes in direct reclaim all swapping out to the disk. >From the above, my first guess would be that you are over subscribing memory you have available. I would focus on who is consuming all that memory. -- Michal Hocko SUSE Labs