From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D784BC433EF for ; Fri, 17 Jun 2022 17:50:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C30596B0085; Fri, 17 Jun 2022 13:50:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3291C6B0087; Fri, 17 Jun 2022 13:50:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E24156B0087; Fri, 17 Jun 2022 13:50:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6B9E16B009A for ; Fri, 17 Jun 2022 13:50:27 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 406D135CB7 for ; Fri, 17 Jun 2022 17:50:27 +0000 (UTC) X-FDA: 79588467294.13.B0C7781 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id AEEA840089 for ; Fri, 17 Jun 2022 17:50:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=hYsjeaVMxzLAu27qyf2VCCnSr/zDrVqguzP+Eyb8MNM=; b=ceCvV3HhI/CTwaliJOnoPnAoJW EfmkRclw5LrV2LjVpvK+2cT9Uyu0jj/R5DvYKYz+Outi6bJ4D49rJrAFO7mzbn/GLfVBH/ipxaAOy u1EsZcQ161dnYM+54O7JYDryYt9aKtLgiyBozAVmWBxQeJmIllhj8vrN9FvUyk3V84FCdIMlcQsXG Zs6ZKdJV4XT6mRPuC6mEskorWaOamaXGK6Q2YC/GYk4nB5m4MhdW0LmxhsfQ7/RMfm+fNqGNIzwz2 Izs+h4F9ptFQ5Qe7SMjB9XBYh249jNCu+ll2OvuUka/LINaZcMKDlQWHj7fQb8yQi/712CDnLok9C KFIm/AOw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1o2G6y-0030ap-RP; Fri, 17 Jun 2022 17:50:24 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 13/22] mm/swap: Convert release_pages to use a folio internally Date: Fri, 17 Jun 2022 18:50:11 +0100 Message-Id: <20220617175020.717127-14-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220617175020.717127-1-willy@infradead.org> References: <20220617175020.717127-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655488226; a=rsa-sha256; cv=none; b=WgCi6wKmKuxIqBdZCyhk8aK0s2gfskZZRlC0hfsdqH9SCQR0lgRChqmPqAwJT1vmYtVJqr 6A8XD1q2whrVJUicwduTEvSalwxjm1qQTXfTCUfrYK81Kq+K8BEpGxJhmMYjRyQ6Vf4WBG Awa4uQr0gUVXFa+3c456bl2DH8woKjA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ceCvV3Hh; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655488226; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hYsjeaVMxzLAu27qyf2VCCnSr/zDrVqguzP+Eyb8MNM=; b=tq7bGQqRY82S4FAyE5bsHGk4F6go8TjCnYCHiPgUikXZoW62LKPfO7PntFYIrG8vGTuP+J z1Ny27mST71Dngz+4NKyvfOheT2lHOMByOXWK7ISV/cH62dXPRNv0KoYOrY3p9vMQbQKFb Fbd52ftBhZITjhBgR3wv4pD1sn3aGxQ= X-Stat-Signature: 4hwk47iyqpahcqmq998bwnzc8py6udmc X-Rspamd-Queue-Id: AEEA840089 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ceCvV3Hh; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam07 X-Rspam-User: X-HE-Tag: 1655488226-678440 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This function was already calling compound_head(), but now it can cache the result of calling compound_head() and avoid calling it again. Saves 299 bytes of text by avoiding various calls to compound_page() and avoiding checks of PageTail. Signed-off-by: Matthew Wilcox (Oracle) --- mm/swap.c | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 5ea6ae795f0d..15ed7584ccb9 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -941,8 +941,7 @@ void release_pages(struct page **pages, int nr) unsigned int lock_batch; for (i = 0; i < nr; i++) { - struct page *page = pages[i]; - struct folio *folio = page_folio(page); + struct folio *folio = page_folio(pages[i]); /* * Make sure the IRQ-safe lock-holding time does not get @@ -954,35 +953,34 @@ void release_pages(struct page **pages, int nr) lruvec = NULL; } - page = &folio->page; - if (is_huge_zero_page(page)) + if (is_huge_zero_page(&folio->page)) continue; - if (is_zone_device_page(page)) { + if (folio_is_zone_device(folio)) { if (lruvec) { unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - if (put_devmap_managed_page(page)) + if (put_devmap_managed_page(&folio->page)) continue; - if (put_page_testzero(page)) - free_zone_device_page(page); + if (folio_put_testzero(folio)) + free_zone_device_page(&folio->page); continue; } - if (!put_page_testzero(page)) + if (!folio_put_testzero(folio)) continue; - if (PageCompound(page)) { + if (folio_test_large(folio)) { if (lruvec) { unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - __put_compound_page(page); + __put_compound_page(&folio->page); continue; } - if (PageLRU(page)) { + if (folio_test_lru(folio)) { struct lruvec *prev_lruvec = lruvec; lruvec = folio_lruvec_relock_irqsave(folio, lruvec, @@ -990,8 +988,8 @@ void release_pages(struct page **pages, int nr) if (prev_lruvec != lruvec) lock_batch = 0; - del_page_from_lru_list(page, lruvec); - __clear_page_lru_flags(page); + lruvec_del_folio(lruvec, folio); + __folio_clear_lru_flags(folio); } /* @@ -1000,13 +998,13 @@ void release_pages(struct page **pages, int nr) * found set here. This does not indicate a problem, unless * "unevictable_pgs_cleared" appears worryingly large. */ - if (unlikely(PageMlocked(page))) { - __ClearPageMlocked(page); - dec_zone_page_state(page, NR_MLOCK); + if (unlikely(folio_test_mlocked(folio))) { + __folio_clear_mlocked(folio); + zone_stat_sub_folio(folio, NR_MLOCK); count_vm_event(UNEVICTABLE_PGCLEARED); } - list_add(&page->lru, &pages_to_free); + list_add(&folio->lru, &pages_to_free); } if (lruvec) unlock_page_lruvec_irqrestore(lruvec, flags); -- 2.35.1