From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 840A8C2D0DB for ; Thu, 23 Jan 2020 12:31:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4D89324655 for ; Thu, 23 Jan 2020 12:31:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D89324655 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F08D66B0005; Thu, 23 Jan 2020 07:31:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E91E26B0006; Thu, 23 Jan 2020 07:31:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA9776B0007; Thu, 23 Jan 2020 07:31:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id C18DA6B0005 for ; Thu, 23 Jan 2020 07:31:31 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 69079181AC1F5 for ; Thu, 23 Jan 2020 12:31:31 +0000 (UTC) X-FDA: 76408834782.22.year59_64907709efc12 X-HE-Tag: year59_64907709efc12 X-Filterd-Recvd-Size: 3906 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Thu, 23 Jan 2020 12:31:30 +0000 (UTC) Received: by mail-wm1-f50.google.com with SMTP id p17so2354812wma.1 for ; Thu, 23 Jan 2020 04:31:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Vl2tCa7+ozhGzVdeMfMTn2Hbrtnd6ugFua7G3LeWw8Y=; b=BJFh7rkV6mtTdJRcyF+2B/RQ4pPwEETuR1zCfk/1XWom4mDVwtoqtIB6MRRj9o5Wp7 8ml/H3XpLXzC50LT1kzKSh4vb18V06iWFEIm5IjS57HuGDSJnhTYNKObwA6bdv0SP85O sHydvyxwQydJCA2Lyf/H630/VtOdE7GqlCcfpwUCe0o4m00A1E9qO4nS8JUydGAHk1wc PNFqtFIQCgb4jqDE+7c34x2DZKsgZ6oDS3+ogQTmq2bkztdASNfGgCFjzOHZKSKxjL9K KzUJWjcZm1mGF5vMWNI0e+nWq92GEpcSy42r/zbKfAICW+S/kG2OSbmeGLsmYw5L7ZAP aMLw== X-Gm-Message-State: APjAAAXwIImHY8KAqRxNDFcu/Jszhu1aO/5gNHtd8fx59XIsyJhiFO74 aOAMgDKAegnmNIouPVyQO7U= X-Google-Smtp-Source: APXvYqw2TbPPAMmzUbl/2fBZfYrM/wyJ59wezVPZikEjclkaKY7av0ix0Fvvzo/8lRPgvirkY/h+fA== X-Received: by 2002:a1c:3906:: with SMTP id g6mr4205795wma.49.1579782689827; Thu, 23 Jan 2020 04:31:29 -0800 (PST) Received: from localhost (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id p7sm2452527wmp.31.2020.01.23.04.31.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2020 04:31:28 -0800 (PST) Date: Thu, 23 Jan 2020 13:31:27 +0100 From: Michal Hocko To: Chris Edwards Cc: "linux-mm@kvack.org" Subject: Re: Paging out when free memory is low but not exhausted (and available memory remains high) Message-ID: <20200123123127.GK29276@dhcp22.suse.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 22-01-20 00:40:12, Chris Edwards wrote: > Hi all, > > I've observed a strange behaviour in recent kernels whereby the system will page out, continuously and seemingly needlessly, when free (cf. available) memory is low but not exhausted. Strangely, swap space utilisation does not increase over time, despite the continuous page-outs with little or no page-in activity. > > I've seen this on 5.3 and 5.4 series kernels on Arch Linux on a Dell Optiplex 9020 (Intel Q87 Express, i5-4670 CPU, 12 GiB RAM, no NUMA). I've also replicated the behaviour on a colleague's machine (same configuration). > > > To reproduce: > > > 1. Get the system in a low-memory state, but not so low that it has to start paging. A convenient way is to use stress/stress-ng: > > stress --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.95;}' < /proc/meminfo)k --vm-keep -m 1 > > Confirm that the system isn't paging, and that memory pressure reported by /proc/pressure/memory remains low: > > cat /proc/pressure/memory > some avg10=0.00 avg60=0.00 avg300=1.54 total=414331152 > full avg10=0.00 avg60=0.00 avg300=1.42 total=377927386 > > > 2. Perform an additional task that will fill up the filesystem cache and/or I/O buffers, such as dd. > > dd if=/dev/sda of=/dev/null bs=1048576 > > > 3. Observe system behaviour (with `vmstat 1`, `xosview`, etc.) while the free memory decreases. Here's what I witness using vmstat: Could you collect /proc/vmstat every second or so while you observe this behavior? This should give us more information that vmstat(8) output. Thanks! -- Michal Hocko SUSE Labs