From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 607D7C83007 for ; Tue, 28 Apr 2020 07:43:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 09CDA206B9 for ; Tue, 28 Apr 2020 07:43:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09CDA206B9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 133458E0005; Tue, 28 Apr 2020 03:43:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BD5C8E0001; Tue, 28 Apr 2020 03:43:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC5B08E0005; Tue, 28 Apr 2020 03:43:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id D09028E0001 for ; Tue, 28 Apr 2020 03:43:05 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9064F4821 for ; Tue, 28 Apr 2020 07:43:05 +0000 (UTC) X-FDA: 76756472730.03.lift61_7245bca448d00 X-HE-Tag: lift61_7245bca448d00 X-Filterd-Recvd-Size: 4311 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Tue, 28 Apr 2020 07:43:05 +0000 (UTC) Received: by mail-wm1-f41.google.com with SMTP id u127so1680476wmg.1 for ; Tue, 28 Apr 2020 00:43:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=4+wiF+sn2LhlnwSYXd62Wli00bERAEfEXjhtyHabBWQ=; b=Otal2YHa4hb0DX5bAC4vdMgGaWP9KaOH2zgo2H5VyOCigkh0doUpM0NwZoUKJSoi3U hwyk5StLjJ50+7OWsluKzHRBZH3D1eI+uZhD0XOIbO6GwnJ2Vbx0ModjS6lJYy6zoKju Q8TubI5Y0edBB2dMsZCequzGsB2PSNyvg68SApRJWZ8Y9mA8NpPdcaVSIv9FF+FI2ISZ /jS4hWdr1kz74yD8axCgHCCDNG9CysOQEYFs27ayO4zKGJAKlgPH5kP6rl8L0d8nHuhe 19D7+nDqXuibwak5cnQ6n44+6FPL+Mby0Yc2bHbRb3dTmDwhVHse5O/2hdDF6pX/R4J8 Vo1w== X-Gm-Message-State: AGi0PuYGtDY+riG0XZIVOEG592SBHN2U2vdNU8kcbH0BZ/YAEmfOc9bi V00Q2jez6WJIuY+0naOMekw= X-Google-Smtp-Source: APiQypLsML2yykyX0NvZF5rI/vYISgHp62mPExEOmWKxWnhYyT5PuMEFfiAwaVwJTrYa2VvzGUnYeQ== X-Received: by 2002:a1c:4956:: with SMTP id w83mr2926649wma.43.1588059784050; Tue, 28 Apr 2020 00:43:04 -0700 (PDT) Received: from localhost (ip-37-188-130-62.eurotel.cz. [37.188.130.62]) by smtp.gmail.com with ESMTPSA id c190sm2202066wme.10.2020.04.28.00.43.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2020 00:43:02 -0700 (PDT) Date: Tue, 28 Apr 2020 09:43:01 +0200 From: Michal Hocko To: Andrew Morton Cc: David Rientjes , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [patch] mm, oom: stop reclaiming if GFP_ATOMIC will start failing soon Message-ID: <20200428074301.GK28637@dhcp22.suse.cz> References: <20200425172706.26b5011293e8dc77b1dccaf3@linux-foundation.org> <20200427133051.b71f961c1bc53a8e72c4f003@linux-foundation.org> <20200427163558.5b08487d63da3cc7a89bf50b@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200427163558.5b08487d63da3cc7a89bf50b@linux-foundation.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 27-04-20 16:35:58, Andrew Morton wrote: [...] > No consumer of GFP_ATOMIC memory should consume an unbounded amount of > it. > Subsystems such as networking will consume a certain amount and > will then start recycling it. The total amount in-flight will vary > over the longer term as workloads change. A dynamically tuning > threshold system will need to adapt rapidly enough to sudden load > shifts, which might require unreasonable amounts of headroom. I do agree. __GFP_HIGH/__GFP_ATOMIC are bound by the size of the reserves under memory pressure. Then allocatios start failing very quickly and users have to cope with that, usually by deferring to a sleepable context. Tuning reserves dynamically for heavy reserves consumers would be possible but I am worried that this is far from trivial. We definitely need to understand what is going on here. Why doesn't kswapd + N*direct reclaimers do not provide enough memory to satisfy both N threads + reserves consumers? How many times those direct reclaimers have to retry? We used to have the allocation stall warning as David mentioned in the patch description and I have seen it triggering without heavy reserves consumers (aka reported free pages corresponded to the min watermark). The underlying problem was usually kswapd being stuck on some FS locks, direct reclaimers stuck in shrinkers or way too overloaded system with dozens if not hundreds of processes stuck in the page allocator each racing with the reclaim and betting on luck. The last problem was the most annoying because it is really hard to tune for. -- Michal Hocko SUSE Labs