From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DA79C433F5 for ; Thu, 2 Dec 2021 16:52:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5B8F6B0072; Thu, 2 Dec 2021 11:52:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AE3E46B0073; Thu, 2 Dec 2021 11:52:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9845E6B0074; Thu, 2 Dec 2021 11:52:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 82E746B0072 for ; Thu, 2 Dec 2021 11:52:35 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3DC2681387 for ; Thu, 2 Dec 2021 16:52:25 +0000 (UTC) X-FDA: 78873447450.25.AE6D56D Received: from outbound-smtp16.blacknight.com (outbound-smtp16.blacknight.com [46.22.139.233]) by imf05.hostedemail.com (Postfix) with ESMTP id ADC63100003 for ; Thu, 2 Dec 2021 16:52:24 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail03.blacknight.ie [81.17.254.16]) by outbound-smtp16.blacknight.com (Postfix) with ESMTPS id 19C841C3BB8 for ; Thu, 2 Dec 2021 16:52:23 +0000 (GMT) Received: (qmail 30231 invoked from network); 2 Dec 2021 16:52:22 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.17.29]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 2 Dec 2021 16:52:22 -0000 Date: Thu, 2 Dec 2021 16:52:20 +0000 From: Mel Gorman To: Shakeel Butt Cc: Andrew Morton , Michal Hocko , Vlastimil Babka , Alexey Avramov , Rik van Riel , Mike Galbraith , Darrick Wong , regressions@lists.linux.dev, Linux-fsdevel , Linux-MM , LKML Subject: Re: [PATCH v4 1/1] mm: vmscan: Reduce throttling due to a failure to make progress Message-ID: <20211202165220.GZ3366@techsingularity.net> References: <20211202150614.22440-1-mgorman@techsingularity.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: ADC63100003 X-Stat-Signature: a9wwbq9wz4azskcm76ojremsyigg3cra Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.233 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none X-HE-Tag: 1638463944-36884 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Dec 02, 2021 at 08:30:51AM -0800, Shakeel Butt wrote: > Hi Mel, > > On Thu, Dec 2, 2021 at 7:07 AM Mel Gorman wrote: > > > > Mike Galbraith, Alexey Avramov and Darrick Wong all reported similar > > problems due to reclaim throttling for excessive lengths of time. > > In Alexey's case, a memory hog that should go OOM quickly stalls for > > several minutes before stalling. In Mike and Darrick's cases, a small > > memcg environment stalled excessively even though the system had enough > > memory overall. > > > > Commit 69392a403f49 ("mm/vmscan: throttle reclaim when no progress is being > > made") introduced the problem although commit a19594ca4a8b ("mm/vmscan: > > increase the timeout if page reclaim is not making progress") made it > > worse. Systems at or near an OOM state that cannot be recovered must > > reach OOM quickly and memcg should kill tasks if a memcg is near OOM. > > > > Is there a reason we can't simply revert 69392a403f49 instead of adding > more code/heuristics? Looking more into 69392a403f49, I don't think the > code and commit message are in sync. > > For the memcg reclaim, instead of just removing congestion_wait or > replacing it with schedule_timeout in mem_cgroup_force_empty(), why > change the behavior of all memcg reclaim. Also this patch effectively > reverts that behavior of 69392a403f49. > It doesn't fully revert it but I did consider reverting it. The reason why I preserved it because the intent originally was to throttle somewhat when progress is not being made to avoid a premature OOM and I wanted to preserve that charactersistic. Right now, this is the least harmful way of doing it. As more memcg, I removed the NOTHROTTLE because the primary reason why a memcg might fail to make progress is excessive writeback and that should still throttle. Completely failing to make progress in a memcg is most likely due to a memcg-OOM. > For direct reclaimers under global pressure, why is page allocator a bad > place for stalling on no progress reclaim? IMHO the callers of the > reclaim should decide what to do if reclaim is not making progress. Because it's a layering violation and the caller has little direct control over the reclaim retry logic. The page allocator has no visibility on why reclaim failed only that it did fail. -- Mel Gorman SUSE Labs