From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7453C4829B for ; Mon, 12 Feb 2024 08:37:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DDFC8D0002; Mon, 12 Feb 2024 03:37:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2693A6B007B; Mon, 12 Feb 2024 03:37:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10A3B8D0002; Mon, 12 Feb 2024 03:37:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EE2AF6B0078 for ; Mon, 12 Feb 2024 03:37:38 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BCD5EA1E6A for ; Mon, 12 Feb 2024 08:37:38 +0000 (UTC) X-FDA: 81782498196.25.81F93DC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf10.hostedemail.com (Postfix) with ESMTP id E79EDC0009 for ; Mon, 12 Feb 2024 08:37:36 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707727057; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WZbo6DkxozIBrYfyEfieDKnWm3+ElUe0ZnyDKdiEH4s=; b=RSQkW2pSLU1XUAfzP0Ov/7rKoLPfg9BnyBM8ryUTJLKDLp6383OZ/+wkguY8pORNdIpw82 okfD7g8PMHlbBNjK0vjAYDQN6buDn4hI1tZJE/ViLQqsV0FMkkIpqutduJcl8cNNy4xsBK 11TXcUBUWMVee1hxdJ3LBclrfTTr0pE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707727057; a=rsa-sha256; cv=none; b=1qrrbiN3ehDPJfP8vXvREAkxYs896wIzhB4fE2727QQwcPHw+U8j8wA0zX1aOZObft04QU kFQ33AeUHSM+pe6vKZr9gHVKWtkSr88xiiPvFt8eRQQqmTASpx+T0Xipned2BmAF2Mge4r wZTRct97Ypd8qYbLZ5vwbmLh9hX6Me8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C332DA7; Mon, 12 Feb 2024 00:38:17 -0800 (PST) Received: from [10.57.78.115] (unknown [10.57.78.115]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB68C3F762; Mon, 12 Feb 2024 00:37:32 -0800 (PST) Message-ID: Date: Mon, 12 Feb 2024 08:37:31 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 01/10] mm/memory: factor out zapping of present pte into zap_present_pte() Content-Language: en-GB To: David Hildenbrand , linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andrew Morton , Matthew Wilcox , Catalin Marinas , Yin Fengwei , Michal Hocko , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , Michael Ellerman , Christophe Leroy , "Naveen N. Rao" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Arnd Bergmann , linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org References: <20240209221509.585251-1-david@redhat.com> <20240209221509.585251-2-david@redhat.com> From: Ryan Roberts In-Reply-To: <20240209221509.585251-2-david@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: E79EDC0009 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5nfmk36uyxkoiinqjsfuj5ukhabmm8km X-HE-Tag: 1707727056-207073 X-HE-Meta: U2FsdGVkX19ta6e42KURRMC1FvgK/IiLgFqfsnVOq5gbyV/WBAWBaEMOWUka/rn+3u7p5PJAa/7qfYWJLVBa8gnnyLVDz+hyrfV1nmbcMkQOoDIT+HZOnlRUAEZD1vahFSfBwLE2LYevfYi8iIZ2CycJ3aiFACmn0GAbnlX9EfpM8b5/TXAn7sVhQdhs8Zp/HHC06QIPCsNm+lbVLLhybtlv2AdpoZUSnnE3tCQqqSJCASuBhqySWk0tBsqRRp9nwBpzexJc8uzquglVwnHTKGZqpETtaxjHB8X15CQ4Ye91RgM3eAFb84vD0sc57Sbcr6wuDS6sjFy+5G3wSMmzLkAc03dpWJgkJJNmPdnrWs5VxQ3/tWclrEwii1LPIkke0tLnCjVngCBBAY6pI8NQihKejctP89NWg3rpnSCUHjU70i5swExOq0Ct7wF+zMbQHb7SCR69cnSv5CNs+iEa7IhYHkTF2s/J6UdjvffKP9T0hXNLenK0DlBY18P/xoSYofuzonUy6+8LHKO1flccKOZX9EGiI6MyLhCXVeUY8X0qC06kO3piC5dP/0tCP4qnPckUoNWqarzxpYqVljjfujlFwaGMxK+aoiItYFgx+wUmmHWiuTskyx++u+Bhr2DmBOAOvffd2ael52DbYdjTbOhpcjlUtbz0QfyU7gSjKl36Wacfs8d1MxtMi9co2iWJrwVlGnH818+A1YY09zPAR62CBW6Xw6PSZflsFWplScsLQlsZhcN6S+mlXdfun5HhSQvINoHY74SFzmDk/PBl+dtqlRth6iwsvfqsl1svhbn4ALi2lpMHpLz6vyFOZbj7DreLZHAr5byiGGzdZyPDlVPUd1v9m1CVk7bTqUK6IoT9CZIFy59d0HtdUstOnSdkiQxmZ+X2npqjFRW5InFWqQtoSIdZMTV+G0XmEPJ5rf3ezljft6ZLd1LDvVaXz58uWxFhmwpqKgRU4sSwviM kzM9kfP3 yruxDyMVUBT7G6J2ddPLKzCYfI60KxEgppEnp4Ll3HMquAcgG3Yuxs3EKqBkH1C7FaD4DL++kIS7WjqknLbCTT5XB4ogDzKABO+g5fisBmsrgggosateSStl32WkkFA3k0kiv72YuFlWqeII7d3y99IeAsoegafg9hK7HgpCix0bwdNEP+KestyaQWeMySrciAhHULDpYR9TiypG4hTWwsOkDxCXWr1LW1kvBEPD8LTgF9YANcFgb0zIgSGy5e29/qQx74L9iqQixOl2VtW3+9mwImA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 09/02/2024 22:15, David Hildenbrand wrote: > Let's prepare for further changes by factoring out processing of present > PTEs. > > Signed-off-by: David Hildenbrand Reviewed-by: Ryan Roberts > --- > mm/memory.c | 94 ++++++++++++++++++++++++++++++----------------------- > 1 file changed, 53 insertions(+), 41 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 7c3ca41a7610..5b0dc33133a6 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -1532,13 +1532,61 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, > pte_install_uffd_wp_if_needed(vma, addr, pte, pteval); > } > > +static inline void zap_present_pte(struct mmu_gather *tlb, > + struct vm_area_struct *vma, pte_t *pte, pte_t ptent, > + unsigned long addr, struct zap_details *details, > + int *rss, bool *force_flush, bool *force_break) > +{ > + struct mm_struct *mm = tlb->mm; > + struct folio *folio = NULL; > + bool delay_rmap = false; > + struct page *page; > + > + page = vm_normal_page(vma, addr, ptent); > + if (page) > + folio = page_folio(page); > + > + if (unlikely(!should_zap_folio(details, folio))) > + return; > + ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); > + arch_check_zapped_pte(vma, ptent); > + tlb_remove_tlb_entry(tlb, pte, addr); > + zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent); > + if (unlikely(!page)) { > + ksm_might_unmap_zero_page(mm, ptent); > + return; > + } > + > + if (!folio_test_anon(folio)) { > + if (pte_dirty(ptent)) { > + folio_mark_dirty(folio); > + if (tlb_delay_rmap(tlb)) { > + delay_rmap = true; > + *force_flush = true; > + } > + } > + if (pte_young(ptent) && likely(vma_has_recency(vma))) > + folio_mark_accessed(folio); > + } > + rss[mm_counter(folio)]--; > + if (!delay_rmap) { > + folio_remove_rmap_pte(folio, page, vma); > + if (unlikely(page_mapcount(page) < 0)) > + print_bad_pte(vma, addr, ptent, page); > + } > + if (unlikely(__tlb_remove_page(tlb, page, delay_rmap))) { > + *force_flush = true; > + *force_break = true; > + } > +} > + > static unsigned long zap_pte_range(struct mmu_gather *tlb, > struct vm_area_struct *vma, pmd_t *pmd, > unsigned long addr, unsigned long end, > struct zap_details *details) > { > + bool force_flush = false, force_break = false; > struct mm_struct *mm = tlb->mm; > - int force_flush = 0; > int rss[NR_MM_COUNTERS]; > spinlock_t *ptl; > pte_t *start_pte; > @@ -1555,7 +1603,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > arch_enter_lazy_mmu_mode(); > do { > pte_t ptent = ptep_get(pte); > - struct folio *folio = NULL; > + struct folio *folio; > struct page *page; > > if (pte_none(ptent)) > @@ -1565,45 +1613,9 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > break; > > if (pte_present(ptent)) { > - unsigned int delay_rmap; > - > - page = vm_normal_page(vma, addr, ptent); > - if (page) > - folio = page_folio(page); > - > - if (unlikely(!should_zap_folio(details, folio))) > - continue; > - ptent = ptep_get_and_clear_full(mm, addr, pte, > - tlb->fullmm); > - arch_check_zapped_pte(vma, ptent); > - tlb_remove_tlb_entry(tlb, pte, addr); > - zap_install_uffd_wp_if_needed(vma, addr, pte, details, > - ptent); > - if (unlikely(!page)) { > - ksm_might_unmap_zero_page(mm, ptent); > - continue; > - } > - > - delay_rmap = 0; > - if (!folio_test_anon(folio)) { > - if (pte_dirty(ptent)) { > - folio_mark_dirty(folio); > - if (tlb_delay_rmap(tlb)) { > - delay_rmap = 1; > - force_flush = 1; > - } > - } > - if (pte_young(ptent) && likely(vma_has_recency(vma))) > - folio_mark_accessed(folio); > - } > - rss[mm_counter(folio)]--; > - if (!delay_rmap) { > - folio_remove_rmap_pte(folio, page, vma); > - if (unlikely(page_mapcount(page) < 0)) > - print_bad_pte(vma, addr, ptent, page); > - } > - if (unlikely(__tlb_remove_page(tlb, page, delay_rmap))) { > - force_flush = 1; > + zap_present_pte(tlb, vma, pte, ptent, addr, details, > + rss, &force_flush, &force_break); > + if (unlikely(force_break)) { > addr += PAGE_SIZE; > break; > }