From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 489DAC433E0 for ; Wed, 10 Jun 2020 02:39:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DC89C2072E for ; Wed, 10 Jun 2020 02:39:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="R5nHtFKt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DC89C2072E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 04B3E6B0002; Tue, 9 Jun 2020 22:39:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3D446B0005; Tue, 9 Jun 2020 22:39:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E54C96B0006; Tue, 9 Jun 2020 22:39:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id CE7DD6B0002 for ; Tue, 9 Jun 2020 22:39:42 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8759C181ABEB6 for ; Wed, 10 Jun 2020 02:39:42 +0000 (UTC) X-FDA: 76911746604.16.cause55_5d0795326dc7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 58DFB100E690C for ; Wed, 10 Jun 2020 02:39:42 +0000 (UTC) X-HE-Tag: cause55_5d0795326dc7 X-Filterd-Recvd-Size: 3941 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Wed, 10 Jun 2020 02:39:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:Message-ID: Subject:Cc:To:From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=Vv8bvXiX1qPH7NGlI8CBFw5+3LL/jDCA14X6pY9LJzA=; b=R5nHtFKtqZeoG+gcc0fGS3gaOL y89AMGAupuYd7etNIIFBQELJ1UHM/DCMVX8ujCe5NzM2yAmJWnrlWPeFemJw1U1JCxZZ01ESpZYFr 054q+WBfj9en/w+vBbPZ/pbZJ5l+Ug51QcaBbgVEdSPQDJbK51eaYBXPHS4FO1LVMI4ojLwqi7JR9 4hPhxLNgtyxbMQBGoDlB1U4RCsY9lDmVKA2LGYVtw1IRYM6/n7KyL+yqEbYRlIo9G4CdjucM2OREM mfHjfRIfycMQfL5ZeuObmP7iZVhOaRNs37cttj0V8nZ7vwN+SYjCENOvsi6Mx4dCF1XXcDDXmqeAf ZALu5hsQ==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1jiqeS-0002lr-2e; Wed, 10 Jun 2020 02:39:40 +0000 Date: Tue, 9 Jun 2020 19:39:39 -0700 From: Matthew Wilcox To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andres Freund Subject: [RFC] page cache drop-behind Message-ID: <20200610023939.GI19604@bombadil.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 58DFB100E690C X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Andres reported a problem recently where reading a file several times the size of memory causes intermittent stalls. My suspicion is that page allocation eventually runs into the low watermark and starts to do reclaim. Some shrinkers take a long time to run and have a low chance of actually freeing a page (eg the dentry cache needs to free 21 dentries which all happen to be on the same pair of pages to free those two pages). This patch attempts to free pages from the file that we're currently reading from if there are no pages readily available. If that doesn't work, we'll run all the shrinkers just as we did before. This should solve Andres' problem, although it's a bit narrow in scope. It might be better to look through the inactive page list, regardless of which file they were allocated for. That could solve the "weekly backup" problem with lots of little files. I'm not really set up to do performance testing at the moment, so this is just me thinking hard about the problem. diff --git a/mm/readahead.c b/mm/readahead.c index 3c9a8dd7c56c..3531e1808e24 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -111,9 +111,24 @@ int read_cache_pages(struct address_space *mapping, struct list_head *pages, } return ret; } - EXPORT_SYMBOL(read_cache_pages); +/* + * Attempt to detect a streaming workload which exceeds memory and + * handle it by dropping the page cache behind the active part of the + * file. + */ +static void discard_behind(struct file *file, struct address_space *mapping) +{ + unsigned long keep = file->f_ra.ra_pages * 2; + + if (mapping->nrpages < 1000) + return; + if (file->f_ra.start < keep) + return; + invalidate_mapping_pages(mapping, 0, file->f_ra.start - keep); +} + static void read_pages(struct readahead_control *rac, struct list_head *pages, bool skip_page) { @@ -179,6 +194,7 @@ void page_cache_readahead_unbounded(struct address_space *mapping, { LIST_HEAD(page_pool); gfp_t gfp_mask = readahead_gfp_mask(mapping); + gfp_t light_gfp = gfp_mask & ~__GFP_DIRECT_RECLAIM; struct readahead_control rac = { .mapping = mapping, .file = file, @@ -219,7 +235,11 @@ void page_cache_readahead_unbounded(struct address_space *mapping, continue; } - page = __page_cache_alloc(gfp_mask); + page = __page_cache_alloc(light_gfp); + if (!page) { + discard_behind(file, mapping); + page = __page_cache_alloc(gfp_mask); + } if (!page) break; if (mapping->a_ops->readpages) {