From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D1AC3524E for ; Wed, 19 Feb 2020 20:42:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B8A1E207FD for ; Wed, 19 Feb 2020 20:42:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8A1E207FD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kerneltoast.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 60A546B0005; Wed, 19 Feb 2020 15:42:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B96B6B0006; Wed, 19 Feb 2020 15:42:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A91C6B0007; Wed, 19 Feb 2020 15:42:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 331E76B0005 for ; Wed, 19 Feb 2020 15:42:25 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E2257181AEF1A for ; Wed, 19 Feb 2020 20:42:24 +0000 (UTC) X-FDA: 76508049408.01.offer42_8970a93a14302 X-HE-Tag: offer42_8970a93a14302 X-Filterd-Recvd-Size: 5438 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Wed, 19 Feb 2020 20:42:24 +0000 (UTC) Received: by mail-pj1-f51.google.com with SMTP id n96so567217pjc.3 for ; Wed, 19 Feb 2020 12:42:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XlRxO87zkf/AZ6fh20OMIQSRLp7vbnlHaFnZNks6GNI=; b=B7SXpt+GzXwkAzF7kKw3YqgM2FiCRp7ZjG8K4RIQXjgup2pHij46AzCtmdpOBTpSH1 UfcIQXV2HMCeuMQd4HS6rENqg8MQn234RwAvr05Jm4hpeWLilAE/OTmpXPmGRg8BPVnx r3cD2m5ScrnU7ho6BNszD/cSsRhHgpaotKFPFdBlvJg6noNch9r4VwZhm9dTxCT/6gB5 NPSfHZYhx39AbTv2+55ms0zrhri+vUv/4hA5FpRFQE39Qr/QqX2A/S67W7MP/Uy4KdWf I75GRApqcg68mlF9nKKSFQ1SS6TImaNWWP5HmBy1B1CXvUwiFZcP95TQwh2brPN9Wu2i AI7g== X-Gm-Message-State: APjAAAXw/2YD0VtgLYNK2O7NXTdvWbi7/TPEfn8B8Mwx5AuXhF5XQVIw WovrDvtsHJUStUVILPBu6sQ= X-Google-Smtp-Source: APXvYqwkDymfw8D/L9PhjjHvG7D/OoNLTVBSTKhGrP8xP+IqgehdJyfbntSfF/pc3LWqhSUHIAKqPQ== X-Received: by 2002:a17:90a:fd85:: with SMTP id cx5mr10894444pjb.80.1582144943130; Wed, 19 Feb 2020 12:42:23 -0800 (PST) Received: from sultan-book.localdomain ([104.200.129.62]) by smtp.gmail.com with ESMTPSA id q8sm713147pje.2.2020.02.19.12.42.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2020 12:42:22 -0800 (PST) Date: Wed, 19 Feb 2020 12:42:20 -0800 From: Sultan Alsawaf To: Michal Hocko Cc: Dave Hansen , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mel Gorman , Johannes Weiner Subject: Re: [PATCH] mm: Stop kswapd early when nothing's waiting for it to free pages Message-ID: <20200219204220.GA3488@sultan-book.localdomain> References: <20200219182522.1960-1-sultan@kerneltoast.com> <20200219194006.GA3075@sultan-book.localdomain> <20200219200527.GF11847@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200219200527.GF11847@dhcp22.suse.cz> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Feb 19, 2020 at 09:05:27PM +0100, Michal Hocko wrote: > Could you be more specific please? kspwad should stop as soon as the > high watermark is reached. If that is not the case then there is a bug > which should be fixed. No, there is no bug causing kswapd to continue beyond the high watermark. > Sure it is quite possible that kswapd is busy for extended amount of > time if the memory pressure is continuous. > > > On a constrained system I tested (mem=2G), this patch had the positive effect of > > improving overall responsiveness at high memory pressure. > > Again, do you have more details about the workload and what was the > cause of responsiveness issues? Because I would expect that the > situation would be quite opposite because it is usually the direct > reclaim that is a source of stalls visible from userspace. Or is this > about a single CPU situation where kswapd saturates the single CPU and > all other tasks are just not getting enough CPU cycles? The workload was having lots of applications open at once. At a certain point when memory ran low, my system became sluggish and kswapd CPU usage skyrocketed. I added printks into kswapd with this patch, and my premature exit in kswapd kicked in quite often. > > On systems with more memory I tested (>=4G), kswapd becomes more expensive to > > run at its higher scan depths, so stopping kswapd prematurely when there aren't > > any memory allocations waiting for it prevents it from reaching the *really* > > expensive scan depths and burning through even more resources. > > > > Combine a large amount of memory with a slow CPU and the current problematic > > behavior of kswapd at high memory pressure shows. My personal test scenario for > > this was an arm64 CPU with a variable amount of memory (up to 4G RAM + 2G swap). > > But still, somebody has to put the system into balanced state so who is > going to do all the work? All the work will be done by kswapd of course, but only if it's needed. The real problem is that a single memory allocation failure, and free memory being some amount below the high watermark, are not good heuristics to predict *future* memory allocation needs. They are good for determining how to steer kswapd to help satisfy a failed allocation in the present, but anything more is pure speculation (which turns out to be wrong speculation, since this behavior causes problems). If there are outstanding failed allocations that won't go away, then it's perfectly reasonable to keep kswapd running until it frees pages up to the high watermark. But beyond that is unnecessary, since there's no way to know if or when kswapd will need to fire up again. This makes sense considering how kswapd is currently invoked: it's fired up due to a failed allocation of some sort, not because the amount of free memory dropped below the high watermark. Sultan