From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43079C55199 for ; Mon, 27 Apr 2020 15:01:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 016D8206BF for ; Mon, 27 Apr 2020 15:01:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 016D8206BF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 755DA8E0005; Mon, 27 Apr 2020 11:01:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 705288E0001; Mon, 27 Apr 2020 11:01:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F4478E0005; Mon, 27 Apr 2020 11:01:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id 480628E0001 for ; Mon, 27 Apr 2020 11:01:41 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 6D818180AD811 for ; Mon, 27 Apr 2020 15:01:39 +0000 (UTC) X-FDA: 76753949118.11.stamp00_65ba0c44a7134 X-HE-Tag: stamp00_65ba0c44a7134 X-Filterd-Recvd-Size: 6919 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Mon, 27 Apr 2020 15:01:38 +0000 (UTC) Received: by mail-wr1-f42.google.com with SMTP id f13so20889511wrm.13 for ; Mon, 27 Apr 2020 08:01:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=jErfeyJqy2ekOgktMYV3Gdiye62PCw4i+yFqIFRALNg=; b=kMjSbJpKfSS1dhG3EAl3JpE2tO2RayCEBn0KZbxkdQ2d4fteK58IamJehA8UwL/zSS eid884ihWSaT/sW8dZYMyjBwRFHkJ+lOd+/JB1yDnFugm462FhHph0pHb9nQTGlWT5fT TGnBz09RMMaTo6o00ScJwwwdHVPPuMCS5IAaDIKkGnkc3XxQ1jt7AmQEFX81Eautb7Y1 sZdEq2ypQNjkLxLeg+K2EyfJUTBw4COiFcpT8pLbIsOhJPhgWvTX8DnE+H1g707zQ71Y mhFGK9CCPXSEr9lhsCS3Q3yTDrysMH+lXgdIIfJ6X7ISkuWznxEYDRY2PUwdP021yW43 EGJQ== X-Gm-Message-State: AGi0PubH2buY18+40JJhF4+dqkeoxcBy83Fujymp3i3J2lH+ZutzboB7 D6VfE7JJ7/XqspuNDd4vYDk= X-Google-Smtp-Source: APiQypLT6kjKf4hBKZqFJgg9CyylQTcsdBIBOYagqZJnOSd5QqpsUj8DBCzfTIPLV0YwLABkIdgj/w== X-Received: by 2002:a5d:404a:: with SMTP id w10mr27187756wrp.397.1587999696474; Mon, 27 Apr 2020 08:01:36 -0700 (PDT) Received: from localhost (ip-37-188-130-62.eurotel.cz. [37.188.130.62]) by smtp.gmail.com with ESMTPSA id j10sm16883320wmi.18.2020.04.27.08.01.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Apr 2020 08:01:35 -0700 (PDT) Date: Mon, 27 Apr 2020 17:01:33 +0200 From: Michal Hocko To: David Rientjes Cc: Andrew Morton , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [patch] mm, oom: stop reclaiming if GFP_ATOMIC will start failing soon Message-ID: <20200427150133.GG28637@dhcp22.suse.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 24-04-20 13:48:06, David Rientjes wrote: > If GFP_ATOMIC allocations will start failing soon because the amount of > free memory is substantially under per-zone min watermarks, it is better > to oom kill a process rather than continue to reclaim. > > This intends to significantly reduce the number of page allocation > failures that are encountered when the demands of user and atomic > allocations overwhelm the ability of reclaim to keep up. We can see this > with a high ingress of networking traffic where memory allocated in irq > context can overwhelm the ability to reclaim fast enough such that user > memory consistently loops. In that case, we have reclaimable memory, and > reclaiming is successful, but we've fully depleted memory reserves that > are allowed for non-blockable allocations. > > Commit 400e22499dd9 ("mm: don't warn about allocations which stall for > too long") removed evidence of user allocations stalling because of this, > but the situation can apply anytime we get "page allocation failures" > where reclaim is happening but per-zone min watermarks are starved: > > Node 0 Normal free:87356kB min:221984kB low:416984kB high:611984kB active_anon:123009936kB inactive_anon:67647652kB active_file:429612kB inactive_file:209980kB unevictable:112348kB writepending:260kB present:198180864kB managed:195027624kB mlocked:81756kB kernel_stack:24040kB pagetables:11460kB bounce:0kB free_pcp:940kB local_pcp:96kB free_cma:0kB > lowmem_reserve[]: 0 0 0 0 > Node 1 Normal free:105616kB min:225568kB low:423716kB high:621864kB active_anon:122124196kB inactive_anon:74112696kB active_file:39172kB inactive_file:103696kB unevictable:204480kB writepending:180kB present:201326592kB managed:198174372kB mlocked:204480kB kernel_stack:11328kB pagetables:3680kB bounce:0kB free_pcp:1140kB local_pcp:0kB free_cma:0kB > lowmem_reserve[]: 0 0 0 0 > > Without this patch, there is no guarantee that user memory allocations > will ever be successful when non-blockable allocations overwhelm the > ability to get above per-zone min watermarks. We have never had any guarantee and we will not have any after this patch either. The fundamental problem is that direct reclaim doesn't provide any guarantee that the reclaimed memory is going to be used for the reclaimer. You can see the same if the memory demand is higher than the reclaim. GFP_ATOMIC is only different in that aspect that they are not throttled by reclaiming and consume what is availble right away which makes the problem worse. > This doesn't solve page allocation failures entirely since it's a > preemptive measure based on watermarks that requires concurrent blockable > allocations to trigger the oom kill. To complete solve page allocation > failures, it would be possible to do the same watermark check for non- > blockable allocations and then queue a worker to asynchronously oom kill > if it finds watermarks to be sufficiently low as well. I do not think this is the right approach. This patch is also quite dangerous as pointed out by Tetsuo. You are effectively allowing a remote DoS via OOM killer. If the reclaim is making progress then the issue seem to be more on the configuration side I believe. Have you tried to tune watermarks resp. watermark_scale_factor? Another potential problem might be that the kswapd is not making sufficient progress because it is blocked on something. > Signed-off-by: David Rientjes > --- > mm/page_alloc.c | 12 +++++++++++- > 1 file changed, 11 insertions(+), 1 deletion(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -4372,11 +4372,21 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order, > ac->nodemask) { > unsigned long available; > unsigned long reclaimable; > + unsigned long free; > unsigned long min_wmark = min_wmark_pages(zone); > bool wmark; > > + free = zone_page_state_snapshot(zone, NR_FREE_PAGES); > + /* > + * If this zone is approaching the point where even order-0 > + * GFP_ATOMIC allocations will fail, stop considering reclaim. > + */ > + if (!__zone_watermark_ok(zone, 0, min_wmark, ac_classzone_idx(ac), > + alloc_flags | ALLOC_HIGH, free)) > + continue; > + > available = reclaimable = zone_reclaimable_pages(zone); > - available += zone_page_state_snapshot(zone, NR_FREE_PAGES); > + available += free; > > /* > * Would the allocation succeed if we reclaimed all -- Michal Hocko SUSE Labs