From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B9A4C47DD9 for ; Mon, 22 Jan 2024 16:58:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8ABC16B0078; Mon, 22 Jan 2024 11:58:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 863DA6B007B; Mon, 22 Jan 2024 11:58:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74AC06B007D; Mon, 22 Jan 2024 11:58:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 66A826B0078 for ; Mon, 22 Jan 2024 11:58:38 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3823AA1E29 for ; Mon, 22 Jan 2024 16:58:38 +0000 (UTC) X-FDA: 81707555916.03.87846DC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf25.hostedemail.com (Postfix) with ESMTP id 6130AA0016 for ; Mon, 22 Jan 2024 16:58:36 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705942716; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nyc5u/Phksy01OQQZ/glB3+fXiGJr+tc+QYc/oajTYY=; b=EfxdUNuePPEkZIrXjnq4ngacCc48qdRVVjF5eK7udzipsMKGjlsmvKBQjZK8qC2LVLcuAj BTW1pqzsJBPSmtasyzNcOYYWr66MT/x9WMy75FGBJPgJJvdhBQ2+bCnQJGFWY1mQKokjgE Qtwja2RgZVQuyehGndWN8cNfKHOU6eg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705942716; a=rsa-sha256; cv=none; b=nj2UdWNhqWpFEYValLL7jm1YTkS9RENA3bjgikFxwalNE4Cdel2TiVPzYuw3wvkMrr5kPJ VoWWrkPcAu2mfcZOIt/qi285WxNUYTT6sBqKVTKVYr/Cfpr+XgvWt9yYIMLRq4Cyv+d/1S nI4jIT9/t/gLTfogROOuXAIyvzxTOzk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8E07A1FB; Mon, 22 Jan 2024 08:59:21 -0800 (PST) Received: from [10.1.33.151] (XHFQ2J9959.cambridge.arm.com [10.1.33.151]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 362A93F5A1; Mon, 22 Jan 2024 08:58:34 -0800 (PST) Message-ID: <2445cedb-61fb-422c-8bfb-caf0a2beed62@arm.com> Date: Mon, 22 Jan 2024 16:58:33 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 28/40] mm/memory: page_remove_rmap() -> folio_remove_rmap_pte() Content-Language: en-GB To: David Hildenbrand , linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andrew Morton , "Matthew Wilcox (Oracle)" , Hugh Dickins , Yin Fengwei , Mike Kravetz , Muchun Song , Peter Xu References: <20231220224504.646757-1-david@redhat.com> <20231220224504.646757-29-david@redhat.com> From: Ryan Roberts In-Reply-To: <20231220224504.646757-29-david@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 6130AA0016 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5rw393qdfy17ontuc6dzxes59ymprmq9 X-HE-Tag: 1705942716-240801 X-HE-Meta: U2FsdGVkX190s2ynsWbtz1PLA8gDPD8iCT8TQYQwrtgQuSzxtcVEYdKnaFihzBqXz6ivmSuS55bXIvRJs9XctQcEKxEZhTxAGas2Nj6p3r1zFMUMsy5WGHnNcFGChYElcrdEn0LglM7YLGbqyC1zzfFFx0s1vfRxJYzeEr6vgIMzpw8c73WjrPrldU9SQlyw/WTdPQWZJOj4gcCQSC+oC2+EPY2nhyAxRVbgtA9kuWTEp73BpjCfMxGyAqEvGp8fqhSJsi8ChTnTygWFxUvVODOuRtCUnNnLYylM762eeAVO+PR2ylGS04ue77iWRjZ2I0+zif1x7C/pda/k27kgaLyKfagZA2KycDP5FYS9tvPdQWOKFCsYsTdwayTyFVv8x3Sb2vLfzPbJtvlipQaXzXR88kMDI9FDVsF/HuiiBsyBTt/6nFGLBnIbmTJrdQs4N91x7aATmGCNOm9+dC3uSCmPgZfHjs1/5B8tS4Y98/WlNm5beBBZ7AzE6mhfZA8LKDchjKVuVffYaTVr+kFB3fksjv88hNzHkm8a6wbf1+QtojVyLyECpDc4Xl/1sPBOVPWBWb8mWTL2e89cZhra/NmGc7EXfDS+LoanA8N/ckHOE9+tOWZe0UAIEsQ1WvjOnBlPQiVxC3LhBeqIZgTuh6aFuP3NH8IC+zxonDPlg9zvH+UkpF3+3gIuZe0Gp2mdMx1R6hPiCmsIlCcDWh5AOSc1pEqbxtgM2w0o/wG+L10NEFNKyRnHKWvNYdL4rCHS2dqgt1O3C0U84hZibEM6XOWnrEu2HJReyIcyC3axxQjHMObPnVQVYRI5WM0RBlxKWGQTHHL2lPDyvKxRUpbzra9XxVh5e+QvjAMnUabjyIOx2aOGaFJ+tdDIR7aO3tnZHZA14uL4MnYQjx0HCZ74kCTGhvmgl0A6mx5LTj0kvEhY9zCOv/64EdcCAWeXnyf7A+7OugvWNv3hD5gBsCh guAn8F41 J4GhiKgE/bdCCQB/9Tu9rQYgR3rEs4wTDg/761AQTXvewx/sqiCHOUe9smpXRxQj55j7+AiMQ41YzL5F+oA9XkKqoojqie2DZ4mzK2Zcr2ssD11y22PqDT0tyMabvPq0J23w7Yh7wIN8Zrme23yakYZ2DZPIfZXfHd9MgczXRx2wqjnUce0xaQkhPt5HrvqYS3u2+rFJnGBHo2pp8XT7Lr7GjbQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 20/12/2023 22:44, David Hildenbrand wrote: > Let's convert zap_pte_range() and closely-related > tlb_flush_rmap_batch(). While at it, perform some more folio conversion > in zap_pte_range(). > > Signed-off-by: David Hildenbrand > --- > mm/memory.c | 23 +++++++++++++---------- > mm/mmu_gather.c | 2 +- > 2 files changed, 14 insertions(+), 11 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 6552ea27b0bfa..eda2181275d9b 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -1434,6 +1434,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > arch_enter_lazy_mmu_mode(); > do { > pte_t ptent = ptep_get(pte); > + struct folio *folio; > struct page *page; > > if (pte_none(ptent)) > @@ -1459,21 +1460,22 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > continue; > } > > + folio = page_folio(page); > delay_rmap = 0; > - if (!PageAnon(page)) { > + if (!folio_test_anon(folio)) { > if (pte_dirty(ptent)) { > - set_page_dirty(page); > + folio_set_dirty(folio); Is this foliation change definitely correct? I note that set_page_dirty() is defined as this: bool set_page_dirty(struct page *page) { return folio_mark_dirty(page_folio(page)); } And folio_mark_dirty() is doing more than just setting teh PG_dirty bit. In my equivalent change, as part of the contpte series, I've swapped set_page_dirty() for folio_mark_dirty(). > if (tlb_delay_rmap(tlb)) { > delay_rmap = 1; > force_flush = 1; > } > } > if (pte_young(ptent) && likely(vma_has_recency(vma))) > - mark_page_accessed(page); > + folio_mark_accessed(folio); > } > rss[mm_counter(page)]--; > if (!delay_rmap) { > - page_remove_rmap(page, vma, false); > + folio_remove_rmap_pte(folio, page, vma); > if (unlikely(page_mapcount(page) < 0)) > print_bad_pte(vma, addr, ptent, page); > } > @@ -1489,6 +1491,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > if (is_device_private_entry(entry) || > is_device_exclusive_entry(entry)) { > page = pfn_swap_entry_to_page(entry); > + folio = page_folio(page); > if (unlikely(!should_zap_page(details, page))) > continue; > /* > @@ -1500,8 +1503,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > WARN_ON_ONCE(!vma_is_anonymous(vma)); > rss[mm_counter(page)]--; > if (is_device_private_entry(entry)) > - page_remove_rmap(page, vma, false); > - put_page(page); > + folio_remove_rmap_pte(folio, page, vma); > + folio_put(folio); > } else if (!non_swap_entry(entry)) { > /* Genuine swap entry, hence a private anon page */ > if (!should_zap_cows(details)) > @@ -3220,10 +3223,10 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) > * threads. > * > * The critical issue is to order this > - * page_remove_rmap with the ptp_clear_flush above. > - * Those stores are ordered by (if nothing else,) > + * folio_remove_rmap_pte() with the ptp_clear_flush > + * above. Those stores are ordered by (if nothing else,) > * the barrier present in the atomic_add_negative > - * in page_remove_rmap. > + * in folio_remove_rmap_pte(); > * > * Then the TLB flush in ptep_clear_flush ensures that > * no process can access the old page before the > @@ -3232,7 +3235,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) > * mapcount is visible. So transitively, TLBs to > * old page will be flushed before it can be reused. > */ > - page_remove_rmap(vmf->page, vma, false); > + folio_remove_rmap_pte(old_folio, vmf->page, vma); > } > > /* Free the old page.. */ > diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c > index 4f559f4ddd217..604ddf08affed 100644 > --- a/mm/mmu_gather.c > +++ b/mm/mmu_gather.c > @@ -55,7 +55,7 @@ static void tlb_flush_rmap_batch(struct mmu_gather_batch *batch, struct vm_area_ > > if (encoded_page_flags(enc)) { > struct page *page = encoded_page_ptr(enc); > - page_remove_rmap(page, vma, false); > + folio_remove_rmap_pte(page_folio(page), page, vma); > } > } > }