From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64917C6FA82 for ; Fri, 2 Sep 2022 19:48:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BE6E8010A; Fri, 2 Sep 2022 15:47:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E14E380118; Fri, 2 Sep 2022 15:47:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEF6C80119; Fri, 2 Sep 2022 15:47:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A715480118 for ; Fri, 2 Sep 2022 15:47:08 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 897C2A09BE for ; Fri, 2 Sep 2022 19:47:08 +0000 (UTC) X-FDA: 79868178936.08.D309174 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id 4A2DE18004D for ; Fri, 2 Sep 2022 19:47:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RwARIESPVJRvPjdBwRi4bFYQLWzQ2fXgslMpG4V504o=; b=AKs8AOvBwDG8jU9p6FGeL2KKt5 S7UWiJ6nzMzsQBbSKk6npx3TRb08F1HNjKO2mRwxAhOyYFmRtVmae642IpW7m+VtLl31KBd4iZQQy o0GbahNSjnvCdE2uKKAiFpQPy6f4HzryFwr94iYyufAFJNHIsHdJlvm0TodHP2A7KDyMkGowwzqpP dBngILO0N4ONkFWT/wdRvlMQt82GLNpQhQFbfkELlWyvjKJoA3546qq6AVGjkDLfHEkDMhrRLZ3lw NtZ7O2CK41+7pkxJEC165DVsBjHtaeEPbG80rAIMsYtVLAK9O/0ElKMYn/63ivJ54kmgFFeMZMLBK G4CemPDA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oUCd3-007If1-Sp; Fri, 02 Sep 2022 19:47:01 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH v2 36/57] swapfile: Convert unuse_pte_range() to use a folio Date: Fri, 2 Sep 2022 20:46:32 +0100 Message-Id: <20220902194653.1739778-37-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220902194653.1739778-1-willy@infradead.org> References: <20220902194653.1739778-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662148028; a=rsa-sha256; cv=none; b=mxOwWYOtroXWdU3boScQRx+efl8BRAcFIU4gP1STQQlpxz4R1Xt5OBCU3GSxGu3PQA4Xhe zR2BJDxtN5OKtAbCLp1FQSBxDbcK7yrfA8x9sJ0Zz99wmArSxWGJOCJxd1tIxBNUjn086e GQsa0Imqr2eIIVAq4y8/L9qusNDElHA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=AKs8AOvB; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662148028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RwARIESPVJRvPjdBwRi4bFYQLWzQ2fXgslMpG4V504o=; b=zBneueNUHG1Qa6V2v9r5tDVdYuREJ4DtetooA4K3EBLgQCTjAQ+TzzlYPqeGzjMDK3rN8/ kZzEv7jBbIQJdisol9g4q5ww0nTx5DzvNSioeih3h+bptkujytBU4HHBk2QpcGOhK68pU/ y6xXkMGVdplUrYtRlCRY/UQkbb6o/pw= X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 4A2DE18004D Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=AKs8AOvB; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: X-Stat-Signature: i1p5hrt4zs6as51qacbpuw7hdkd3czfb X-HE-Tag: 1662148028-862213 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Delay fetching the precise page from the folio until we're in unuse_pte(). Saves many calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/swapfile.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index ce538c3f9161..9ee42a12cffc 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1754,8 +1754,9 @@ static inline int pte_same_as_swp(pte_t pte, pte_t swp_pte) * force COW, vm_page_prot omits write permission from any private vma. */ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, - unsigned long addr, swp_entry_t entry, struct page *page) + unsigned long addr, swp_entry_t entry, struct folio *folio) { + struct page *page = folio_file_page(folio, swp_offset(entry)); struct page *swapcache; spinlock_t *ptl; pte_t *pte, new_pte; @@ -1827,17 +1828,18 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, unsigned int type) { - struct page *page; swp_entry_t entry; pte_t *pte; struct swap_info_struct *si; - unsigned long offset; int ret = 0; volatile unsigned char *swap_map; si = swap_info[type]; pte = pte_offset_map(pmd, addr); do { + struct folio *folio; + unsigned long offset; + if (!is_swap_pte(*pte)) continue; @@ -1848,8 +1850,9 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); swap_map = &si->swap_map[offset]; - page = lookup_swap_cache(entry, vma, addr); - if (!page) { + folio = swap_cache_get_folio(entry, vma, addr); + if (!folio) { + struct page *page; struct vm_fault vmf = { .vma = vma, .address = addr, @@ -1859,25 +1862,27 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, &vmf); + if (page) + folio = page_folio(page); } - if (!page) { + if (!folio) { if (*swap_map == 0 || *swap_map == SWAP_MAP_BAD) goto try_next; return -ENOMEM; } - lock_page(page); - wait_on_page_writeback(page); - ret = unuse_pte(vma, pmd, addr, entry, page); + folio_lock(folio); + folio_wait_writeback(folio); + ret = unuse_pte(vma, pmd, addr, entry, folio); if (ret < 0) { - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); goto out; } - try_to_free_swap(page); - unlock_page(page); - put_page(page); + folio_free_swap(folio); + folio_unlock(folio); + folio_put(folio); try_next: pte = pte_offset_map(pmd, addr); } while (pte++, addr += PAGE_SIZE, addr != end); -- 2.35.1