From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7444EC43334 for ; Fri, 17 Jun 2022 15:43:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 087526B0095; Fri, 17 Jun 2022 11:43:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F2BB36B0096; Fri, 17 Jun 2022 11:43:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF37C6B0098; Fri, 17 Jun 2022 11:43:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CEA196B0095 for ; Fri, 17 Jun 2022 11:43:01 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A146321827 for ; Fri, 17 Jun 2022 15:43:01 +0000 (UTC) X-FDA: 79588146162.18.77ABBFA Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf30.hostedemail.com (Postfix) with ESMTP id 0DCAE80098 for ; Fri, 17 Jun 2022 15:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=q9jnfXr7dSxAhKDKUUhPH87t77ACgmIW7yxpumzvBpI=; b=afK2QEE7g70R0UsagfJ2QHqnHP uQQJzgZiRMK0r8ibW+DCBTXz6m26FbSzhbyuEOcWVV6K8/RrcCH1301mkWKvt1kKvGwNoC6VfWZPn Z1yHFgbkFL0J/iTzQfowe03g27mhqeRtL1RLPxe9o1SADcAsLWP8m5+QGgFX3V0pu+iis+OFro3nx 6t/A5GmbOr3HuMlpb7YVeP3nrbWgSmF2n4VQVmKCMgZ/WpvSVQ/L+GtuMZk2qZ6Xy+7LZTs3lkM3S 9NJxLM5BBhGaqe7tGrrdRNiy/gZiEj87FKaSzuPyknMWMtxna0V6O5Po4HpoNtGdx0XOwSKvnR3RL fn5tknJQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1o2E7W-002wEj-D3; Fri, 17 Jun 2022 15:42:50 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 2/5] mm/vmscan: Convert isolate_lru_pages() to use a folio Date: Fri, 17 Jun 2022 16:42:45 +0100 Message-Id: <20220617154248.700416-3-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220617154248.700416-1-willy@infradead.org> References: <20220617154248.700416-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655480581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q9jnfXr7dSxAhKDKUUhPH87t77ACgmIW7yxpumzvBpI=; b=hHkbN/O0Hco8JG3LJhqzIlk5pl0hs66QO1PV6jgNQiV/FPklVQiSNj2YZRSR9Bzj+ep13X 4+YejJGzR3Ij/BB23niHIy1zWazu8MqQSUnxRWjJNlyC2w4/VFuv3QR4ua2PnoC5wIVC59 RPR4xMMhMlxiwQt+LUxgSVLDmEtKv28= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655480581; a=rsa-sha256; cv=none; b=Bl/E6idjbNSN4WyjEiTMlbP3uQXUQ7uW35nMTNVFH44qbYSwy6B7A2Qu3E7PCTQ01i/w2u BDZe5kmut7UDmLcfffNVpu9/nZA+jz9yazbF6jmy4Xx/Dy/EzQM4dPwO70ME3swPwY0HLQ fq9EhmiAg6Ole+RcumdGbZQs+IZ6oQM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=afK2QEE7; dmarc=none; spf=none (imf30.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=afK2QEE7; dmarc=none; spf=none (imf30.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 0DCAE80098 X-Stat-Signature: qdhxmpjd993ydyea4wkdmx9x1kp5jext X-HE-Tag: 1655480580-900835 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Remove a few hidden calls to compound_head, saving 279 bytes of text. Signed-off-by: Matthew Wilcox (Oracle) --- mm/vmscan.c | 66 ++++++++++++++++++++++++++--------------------------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 6c7184f333bf..81d61139bbb1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -160,17 +160,17 @@ struct scan_control { }; #ifdef ARCH_HAS_PREFETCHW -#define prefetchw_prev_lru_page(_page, _base, _field) \ +#define prefetchw_prev_lru_folio(_folio, _base, _field) \ do { \ - if ((_page)->lru.prev != _base) { \ - struct page *prev; \ + if ((_folio)->lru.prev != _base) { \ + struct folio *prev; \ \ - prev = lru_to_page(&(_page->lru)); \ + prev = lru_to_folio(&(_folio->lru)); \ prefetchw(&prev->_field); \ } \ } while (0) #else -#define prefetchw_prev_lru_page(_page, _base, _field) do { } while (0) +#define prefetchw_prev_lru_folio(_folio, _base, _field) do { } while (0) #endif /* @@ -2085,72 +2085,72 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, unsigned long nr_skipped[MAX_NR_ZONES] = { 0, }; unsigned long skipped = 0; unsigned long scan, total_scan, nr_pages; - LIST_HEAD(pages_skipped); + LIST_HEAD(folios_skipped); total_scan = 0; scan = 0; while (scan < nr_to_scan && !list_empty(src)) { struct list_head *move_to = src; - struct page *page; + struct folio *folio; - page = lru_to_page(src); - prefetchw_prev_lru_page(page, src, flags); + folio = lru_to_folio(src); + prefetchw_prev_lru_folio(folio, src, flags); - nr_pages = compound_nr(page); + nr_pages = folio_nr_pages(folio); total_scan += nr_pages; - if (page_zonenum(page) > sc->reclaim_idx) { - nr_skipped[page_zonenum(page)] += nr_pages; - move_to = &pages_skipped; + if (folio_zonenum(folio) > sc->reclaim_idx) { + nr_skipped[folio_zonenum(folio)] += nr_pages; + move_to = &folios_skipped; goto move; } /* - * Do not count skipped pages because that makes the function - * return with no isolated pages if the LRU mostly contains - * ineligible pages. This causes the VM to not reclaim any - * pages, triggering a premature OOM. - * Account all tail pages of THP. + * Do not count skipped folios because that makes the function + * return with no isolated folios if the LRU mostly contains + * ineligible folios. This causes the VM to not reclaim any + * folios, triggering a premature OOM. + * Account all pages in a folio. */ scan += nr_pages; - if (!PageLRU(page)) + if (!folio_test_lru(folio)) goto move; - if (!sc->may_unmap && page_mapped(page)) + if (!sc->may_unmap && folio_mapped(folio)) goto move; /* - * Be careful not to clear PageLRU until after we're - * sure the page is not being freed elsewhere -- the - * page release code relies on it. + * Be careful not to clear the lru flag until after we're + * sure the folio is not being freed elsewhere -- the + * folio release code relies on it. */ - if (unlikely(!get_page_unless_zero(page))) + if (unlikely(!folio_try_get(folio))) goto move; - if (!TestClearPageLRU(page)) { - /* Another thread is already isolating this page */ - put_page(page); + if (!folio_test_clear_lru(folio)) { + /* Another thread is already isolating this folio */ + folio_put(folio); goto move; } nr_taken += nr_pages; - nr_zone_taken[page_zonenum(page)] += nr_pages; + nr_zone_taken[folio_zonenum(folio)] += nr_pages; move_to = dst; move: - list_move(&page->lru, move_to); + list_move(&folio->lru, move_to); } /* - * Splice any skipped pages to the start of the LRU list. Note that + * Splice any skipped folios to the start of the LRU list. Note that * this disrupts the LRU order when reclaiming for lower zones but * we cannot splice to the tail. If we did then the SWAP_CLUSTER_MAX - * scanning would soon rescan the same pages to skip and waste lots + * scanning would soon rescan the same folios to skip and waste lots * of cpu cycles. */ - if (!list_empty(&pages_skipped)) { + if (!list_empty(&folios_skipped)) { int zid; - list_splice(&pages_skipped, src); + list_splice(&folios_skipped, src); for (zid = 0; zid < MAX_NR_ZONES; zid++) { if (!nr_skipped[zid]) continue; -- 2.35.1