From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D65BBC6FA82 for ; Fri, 2 Sep 2022 19:47:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7826280102; Fri, 2 Sep 2022 15:47:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1A7980105; Fri, 2 Sep 2022 15:47:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C0EB800F4; Fri, 2 Sep 2022 15:47:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E3ECB800FC for ; Fri, 2 Sep 2022 15:47:02 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D21F9A121F for ; Fri, 2 Sep 2022 19:47:02 +0000 (UTC) X-FDA: 79868178684.25.A612EFC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 7B67018007B for ; Fri, 2 Sep 2022 19:47:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vb5v6SCiW0r2u9p6AY6WdMhWa7Kyh2K4USUZrofwcsc=; b=mOwvg7Pa/OqFYqAQxBjyL2UYo9 p57go/VwNybiz9BbPNEgYhih8D1o8hvAorMJOGsYb0A/LZcGvTnjPHkgt5ivUNTrmfZR1otaGT7PW 6V4oxOhmz+aPCM+UziLDitmHsJPxanHAMJ0saN1EawtIGHAEY3t0SIFrOdmUD8Jy6HDluf8ttw/RX bT99qUg9zICIzdCDGKyFHk6oOZFTujQXrNBSMwFAsOLcY+IP1Ls7J7oQmQc4yUwN2Cm0JUVt8DEE7 euRQiyS05rZ0/mMAsX++MFJPpU61VdfvdcWD4AMmvDYmh65zPYZJJrpVccMpx4CgJhD9SoVDrRezD pn87F23Q==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oUCd4-007Ifc-IL; Fri, 02 Sep 2022 19:47:02 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH v2 41/57] mm: Convert do_wp_page() to use a folio Date: Fri, 2 Sep 2022 20:46:37 +0100 Message-Id: <20220902194653.1739778-42-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220902194653.1739778-1-willy@infradead.org> References: <20220902194653.1739778-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=mOwvg7Pa; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662148022; a=rsa-sha256; cv=none; b=cIJ9desLnyJ3TCrxzDhFyU6fQnys9nWoIdQbpAXCLQ6eyCELCV3QxzGCfkSRkOrBrhiEn9 WgSY4ASTPTTWUet6RM2NqmjrcECcg/3HLaT/9XVUPdZ5fZjeoWSCiEhL1Nw7jlv7ICJhaI 8TgE0jI6sjhelrcfcjrZlkcwFxxCpl4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662148022; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vb5v6SCiW0r2u9p6AY6WdMhWa7Kyh2K4USUZrofwcsc=; b=rW0J1+jeiJrGXT5jcG9upm4MzrAtBMDWQmj5PKjztCPLIJbf/FWtkzB5F+rz4dnY1Y5D3T UEd5D2Cj6E9vG9X1TuG2mFNXYhDFh+41XVJdbNdlgXX6CeAaz/xtCv6TevQrxYegP5L7Oa 04693jBl6emi+RzwbDQVdLtWvspvABE= X-Rspamd-Queue-Id: 7B67018007B X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=mOwvg7Pa; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam09 X-Stat-Signature: w4fup9hoxptosbitnabisx5qmdasck1g X-HE-Tag: 1662148022-701479 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Saves many calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/memory.c | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 10834effb32d..4ec9827a2d37 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3362,6 +3362,7 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) { const bool unshare = vmf->flags & FAULT_FLAG_UNSHARE; struct vm_area_struct *vma = vmf->vma; + struct folio *folio; VM_BUG_ON(unshare && (vmf->flags & FAULT_FLAG_WRITE)); VM_BUG_ON(!unshare && !(vmf->flags & FAULT_FLAG_WRITE)); @@ -3408,48 +3409,47 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) * Take out anonymous pages first, anonymous shared vmas are * not dirty accountable. */ - if (PageAnon(vmf->page)) { - struct page *page = vmf->page; - + folio = page_folio(vmf->page); + if (folio_test_anon(folio)) { /* * If the page is exclusive to this process we must reuse the * page without further checks. */ - if (PageAnonExclusive(page)) + if (PageAnonExclusive(vmf->page)) goto reuse; /* - * We have to verify under page lock: these early checks are - * just an optimization to avoid locking the page and freeing + * We have to verify under folio lock: these early checks are + * just an optimization to avoid locking the folio and freeing * the swapcache if there is little hope that we can reuse. * - * PageKsm() doesn't necessarily raise the page refcount. + * KSM doesn't necessarily raise the folio refcount. */ - if (PageKsm(page) || page_count(page) > 3) + if (folio_test_ksm(folio) || folio_ref_count(folio) > 3) goto copy; - if (!PageLRU(page)) + if (!folio_test_lru(folio)) /* * Note: We cannot easily detect+handle references from - * remote LRU pagevecs or references to PageLRU() pages. + * remote LRU pagevecs or references to LRU folios. */ lru_add_drain(); - if (page_count(page) > 1 + PageSwapCache(page)) + if (folio_ref_count(folio) > 1 + folio_test_swapcache(folio)) goto copy; - if (!trylock_page(page)) + if (!folio_trylock(folio)) goto copy; - if (PageSwapCache(page)) - try_to_free_swap(page); - if (PageKsm(page) || page_count(page) != 1) { - unlock_page(page); + if (folio_test_swapcache(folio)) + folio_free_swap(folio); + if (folio_test_ksm(folio) || folio_ref_count(folio) != 1) { + folio_unlock(folio); goto copy; } /* - * Ok, we've got the only page reference from our mapping - * and the page is locked, it's dark out, and we're wearing + * Ok, we've got the only folio reference from our mapping + * and the folio is locked, it's dark out, and we're wearing * sunglasses. Hit it. */ - page_move_anon_rmap(page, vma); - unlock_page(page); + page_move_anon_rmap(vmf->page, vma); + folio_unlock(folio); reuse: if (unlikely(unshare)) { pte_unmap_unlock(vmf->pte, vmf->ptl); -- 2.35.1