From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D3E6C8303F for ; Wed, 2 Jul 2025 10:49:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14CAD6B0095; Wed, 2 Jul 2025 06:49:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D62E6B00C1; Wed, 2 Jul 2025 06:49:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6B378E0007; Wed, 2 Jul 2025 06:49:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CE0E38E0001 for ; Wed, 2 Jul 2025 06:49:36 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 742F21A015F for ; Wed, 2 Jul 2025 10:49:36 +0000 (UTC) X-FDA: 83619003552.06.E518A28 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 657254000C for ; Wed, 2 Jul 2025 10:49:34 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TuJD3nSg; spf=pass (imf12.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751453374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/J+W0F15YhPX18Zq/VlVVl9dXxQqgtke2+o/p/5jgRs=; b=mb9hBDs1Znuzrc7XWuZRlfmYv3Faeomjub2nArgv850V5puPxa12NVPsB34IHCmm7+bX0n rNdhMxUFgR1pTHBFZb9WaKrnHiRfRNAVWolgsxuzM8a0AHifVRT13pwj0JYYx8ccS65/La syEeQfS5ZWRpjnmDGyqWCcossa7+B+A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751453374; a=rsa-sha256; cv=none; b=qPZJqjdCa8nkJ2HldQLpgImuhSdmvqtaFEsEKWptw22vBWeG3yxeZH3hChc0WQTYjzTx5a 6IrrUZX9MnYdqjGFRVum44sMTJ0vamtnHyb31HxGeGf93IXEeesH3hYZoiuKlOrJMONCnK r2DxFpop7JW8GrOgw7l88ziVk2DADck= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TuJD3nSg; spf=pass (imf12.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751453373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/J+W0F15YhPX18Zq/VlVVl9dXxQqgtke2+o/p/5jgRs=; b=TuJD3nSgLX/gQ9SxHAmQX7XXmOND+hOUHIUnrq3XAQuJR92ITBdd1e68Hkf/6SvCGtV9OM +hfHbD50GtrJSiX2ccetYQB8O5LfZn1SE9HiAno2NRuzDaxvz1D3LwlBfJgYh0bKiYZRsB fOsbQ/+j1tRl/Pttg8am0HMZqH2Vntc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-279-b9UhE4hsOqalBH88I1Gu7Q-1; Wed, 02 Jul 2025 06:49:32 -0400 X-MC-Unique: b9UhE4hsOqalBH88I1Gu7Q-1 X-Mimecast-MFC-AGG-ID: b9UhE4hsOqalBH88I1Gu7Q_1751453372 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-45320bfc18dso33580375e9.1 for ; Wed, 02 Jul 2025 03:49:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751453371; x=1752058171; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/J+W0F15YhPX18Zq/VlVVl9dXxQqgtke2+o/p/5jgRs=; b=GcOUR4LS7wph2qxpbQvi/5BSYkTlmqt0pqnBqvw1m09kZ0BhqWiGdw3MhIJJHjlHj0 tcP8+3GXJ0EGtrhzR6BWxa01lohiFufVYW6nh3Gk5mkvUMvRwn0/SJEQmwaQ5m9rArrU loIesupq9PJRjedTHSqjaEiDG7CteD1fSGHjZWd9fUgfKAHf3hFNeoquHRkQ3UV0ALEJ DrzsKKsw8YZRxQn2dsgJpk4A7qj9lh3Y6LK25A51hYzeyH+74xAl2//eDkHGHukMjuzk zS6/pLcAWhGlri9dx98sho2TBk97Sa0F7k4MHxBrbPdsxeHhQiEYH4WQHvq1ij+YmbeN nOTQ== X-Gm-Message-State: AOJu0YzqCV/d70AUg9DjRklsiYxNf0ohAA9KUV31CEE+IeTS5GGyjRYX hOSzzN1iTp91NV4I5Sfb5iqaWHvfl4tyjOGfPdWEd8pJkw/z6+IwUvJShkhQeFCZsjQsopZeY1x ZamU6VKjiuOx+NGcrTF9vS7s0+1uCgINVs3vuf5pM5ikP+8aunP8A X-Gm-Gg: ASbGncthEuI6OMo4PjxaetSPtSdbmeLXdqkG2zvnkdZI8FQCP/kZzgkoiH0n8PJtJgu XL4ylqcSW2LuXLZsQCzex1+cK81Jq3iWj7kVqgKuSnYOxRc09X0P1V6fhpuDy2HR6kN+Pk+w4Pl H8V33g0AT+qhIG101KtKgKNyK1/bUARiDo5Y/+UKMGz7bBCkr8y6KDGreh8pt23nAhoFY8yMRuF PrFYoIKcRMzPHTHWZDoEN5SnntMpQgDKusMC8VtVGTjlIf0x8fC890Q54vzsZoecbi2e+QIyS28 wF5KkaRCyW5reWG1 X-Received: by 2002:a05:600c:c176:b0:442:f4a3:a2c0 with SMTP id 5b1f17b1804b1-454a41609efmr20817005e9.13.1751453371308; Wed, 02 Jul 2025 03:49:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEPxIF24NU0cgA7LgQoEvYVTqVBLrm0QWneoTQ4N5UtqQayGKXh6ikNCsFoGZpvs78Obi1muw== X-Received: by 2002:a05:600c:c176:b0:442:f4a3:a2c0 with SMTP id 5b1f17b1804b1-454a41609efmr20816655e9.13.1751453370775; Wed, 02 Jul 2025 03:49:30 -0700 (PDT) Received: from localhost ([2a09:80c0:192:0:5dac:bf3d:c41:c3e7]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a892e52ebcsm15430008f8f.46.2025.07.02.03.49.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Jul 2025 03:49:30 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Pedro Falcato , Rik van Riel , Harry Yoo , Lance Yang , Oscar Salvador , Lance Yang Subject: [PATCH v2 1/4] mm: convert FPB_IGNORE_* into FPB_RESPECT_* Date: Wed, 2 Jul 2025 12:49:23 +0200 Message-ID: <20250702104926.212243-2-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250702104926.212243-1-david@redhat.com> References: <20250702104926.212243-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: aJ-AIRucFIexX983uB-BOKpV_jSCX01slV_Nions5HM_1751453372 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 657254000C X-Stat-Signature: 7bcoogabrot3asr3jg95iztnja8k3kz4 X-Rspam-User: X-HE-Tag: 1751453374-647222 X-HE-Meta: U2FsdGVkX19hgCu7jlh0RRIMZJH/YNqj8kYCzkawicPSIgNbfLbpQTq9xBNrp2x54LeafGIw00cSW1UwdNpUMh6cdrXBjPHf6YKUvqlnoToBzaAhOb3pDS4jLSaVoB0ei01xxr/E56JjAADSGA1V/mrCQGHwCCjlbrFogrevmnlNSbw5TctPfGl64QT04aIBcRf/5W+L5O5V5Yn6P/wu00ZWSK3qmA+jRx3kT7qtCsJu3FBF1OZtSWMEv1wYQpdSZVD/rViWwxIFfImCNuRKCVs881xIRHCfuU1zOH7RiebF/g7Y3wz+RP+lKalogg5rxjEK7PSK1qbGQGcOzVP4AIh8+2A/PJ0RnGpHLpgANQ11G6yQ3phvOkLYEKhE9bEmTuqgYbSf/vcI5bxWSmfMxHpUOoZOQJSUpd4fSQhmfVW9YD1NCnV58l76/akGSiZVlDtTgvZlz8fSu03wHw3Yn86l2KYJ9FvnDbCqKidzybJZifGsGfy5/z+TW9fS7tpxMhwEJlnJmOTq5Tg5NjXFwjm6R1rvSSm+vsuIIfK8GNHG4ACt6cPq+2JPSx5dBEv6MuyJbvbCjwP6UZLcXEGvIjdUF/hLA+maegGG0DrobSxkSuONb2haKFnE2yf4YdNFUycqjLDOM9pht4I+yOb3oMnkpsBIfR71OTNYfruhHMKnZ/lcQik4/Cdhxt8CQXcQU4irn0F4oTks+ikQdbETiRMLo0tHyc4kOBOL9ZZvGejCxfIFM/04bkDDXst3qnkrRW75ySH0h0NXEdoEFRDTirdgpalFRAzZF5WfWyITd6wxQR8eGjP7rBCQHc+9YyNzs3FXFUby/V/DZzOt+jK0+tVO0ycU/Ju33G1XhnMylMLIp3hE70UyPlAIqf3E4FkIB7lpciz34UfRhoSR+bOfHaWhe6U3EvTg9DQbzqRZNOznYrxLSyjfLyKdT8RyF2NVYAgxYtbjDiDecPm8WfI +nUDNO6H 30D7tI4rEYnUDmaryH/9eIReRu8CTg5WnHqflX42XPDvVm/x+VgwTDZT5UCzzbQwoLAeOOXDlAg70rklUzWHwe/tpb0HPfeD16dblpyOUoFezRmiUuWc6iSfalLd+a9rhXYwzLIkVpL3c4vKltQUJ8nvNYnrSK7HeyqFAirJD7yt+UfMZ54SmidS/w/4BujYngQw6GP2+ODwVC4JIeDk9ujMizSNqInG0Xrtek0cGBBSSL1Hi48/6UrfQZCDfazuGzoUyVPWNKouE/JbgtImvSR4HJF3lqHNICKGu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Respecting these PTE bits is the exception, so let's invert the meaning. With this change, most callers don't have to pass any flags. This is a preparation for splitting folio_pte_batch() into a non-inlined variant that doesn't consume any flags. Long-term, we want folio_pte_batch() to probably ignore most common PTE bits (e.g., write/dirty/young/soft-dirty) that are not relevant for most page table walkers: uffd-wp and protnone might be bits to consider in the future. Only walkers that care about them can opt-in to respect them. No functional change intended. Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand --- mm/internal.h | 16 ++++++++-------- mm/madvise.c | 3 +-- mm/memory.c | 11 +++++------ mm/mempolicy.c | 4 +--- mm/mlock.c | 3 +-- mm/mremap.c | 3 +-- mm/rmap.c | 3 +-- 7 files changed, 18 insertions(+), 25 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index e84217e27778d..170d55b6851ff 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -202,17 +202,17 @@ static inline void vma_close(struct vm_area_struct *vma) /* Flags for folio_pte_batch(). */ typedef int __bitwise fpb_t; -/* Compare PTEs after pte_mkclean(), ignoring the dirty bit. */ -#define FPB_IGNORE_DIRTY ((__force fpb_t)BIT(0)) +/* Compare PTEs respecting the dirty bit. */ +#define FPB_RESPECT_DIRTY ((__force fpb_t)BIT(0)) -/* Compare PTEs after pte_clear_soft_dirty(), ignoring the soft-dirty bit. */ -#define FPB_IGNORE_SOFT_DIRTY ((__force fpb_t)BIT(1)) +/* Compare PTEs respecting the soft-dirty bit. */ +#define FPB_RESPECT_SOFT_DIRTY ((__force fpb_t)BIT(1)) static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags) { - if (flags & FPB_IGNORE_DIRTY) + if (!(flags & FPB_RESPECT_DIRTY)) pte = pte_mkclean(pte); - if (likely(flags & FPB_IGNORE_SOFT_DIRTY)) + if (likely(!(flags & FPB_RESPECT_SOFT_DIRTY))) pte = pte_clear_soft_dirty(pte); return pte_wrprotect(pte_mkold(pte)); } @@ -236,8 +236,8 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags) * pages of the same large folio. * * All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN, - * the accessed bit, writable bit, dirty bit (with FPB_IGNORE_DIRTY) and - * soft-dirty bit (with FPB_IGNORE_SOFT_DIRTY). + * the accessed bit, writable bit, dirty bit (unless FPB_RESPECT_DIRTY is set) + * and soft-dirty bit (unless FPB_RESPECT_SOFT_DIRTY is set). * * start_ptep must map any page of the folio. max_nr must be at least one and * must be limited by the caller so scanning cannot exceed a single page table. diff --git a/mm/madvise.c b/mm/madvise.c index e61e32b2cd91f..661bb743d2216 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -347,10 +347,9 @@ static inline int madvise_folio_pte_batch(unsigned long addr, unsigned long end, pte_t pte, bool *any_young, bool *any_dirty) { - const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; int max_nr = (end - addr) / PAGE_SIZE; - return folio_pte_batch(folio, addr, ptep, pte, max_nr, fpb_flags, NULL, + return folio_pte_batch(folio, addr, ptep, pte, max_nr, 0, NULL, any_young, any_dirty); } diff --git a/mm/memory.c b/mm/memory.c index 0f9b32a20e5b7..0a47583ca9937 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -990,10 +990,10 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma * by keeping the batching logic separate. */ if (unlikely(!*prealloc && folio_test_large(folio) && max_nr != 1)) { - if (src_vma->vm_flags & VM_SHARED) - flags |= FPB_IGNORE_DIRTY; - if (!vma_soft_dirty_enabled(src_vma)) - flags |= FPB_IGNORE_SOFT_DIRTY; + if (!(src_vma->vm_flags & VM_SHARED)) + flags |= FPB_RESPECT_DIRTY; + if (vma_soft_dirty_enabled(src_vma)) + flags |= FPB_RESPECT_SOFT_DIRTY; nr = folio_pte_batch(folio, addr, src_pte, pte, max_nr, flags, &any_writable, NULL, NULL); @@ -1535,7 +1535,6 @@ static inline int zap_present_ptes(struct mmu_gather *tlb, struct zap_details *details, int *rss, bool *force_flush, bool *force_break, bool *any_skipped) { - const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; struct mm_struct *mm = tlb->mm; struct folio *folio; struct page *page; @@ -1565,7 +1564,7 @@ static inline int zap_present_ptes(struct mmu_gather *tlb, * by keeping the batching logic separate. */ if (unlikely(folio_test_large(folio) && max_nr != 1)) { - nr = folio_pte_batch(folio, addr, pte, ptent, max_nr, fpb_flags, + nr = folio_pte_batch(folio, addr, pte, ptent, max_nr, 0, NULL, NULL, NULL); zap_present_folio_ptes(tlb, vma, folio, page, pte, ptent, nr, diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 1ff7b2174eb77..2a25eedc3b1c0 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -675,7 +675,6 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { - const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; struct vm_area_struct *vma = walk->vma; struct folio *folio; struct queue_pages *qp = walk->private; @@ -713,8 +712,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, continue; if (folio_test_large(folio) && max_nr != 1) nr = folio_pte_batch(folio, addr, pte, ptent, - max_nr, fpb_flags, - NULL, NULL, NULL); + max_nr, 0, NULL, NULL, NULL); /* * vm_normal_folio() filters out zero pages, but there might * still be reserved folios to skip, perhaps in a VDSO. diff --git a/mm/mlock.c b/mm/mlock.c index 3cb72b579ffd3..2238cdc5eb1c1 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -307,14 +307,13 @@ void munlock_folio(struct folio *folio) static inline unsigned int folio_mlock_step(struct folio *folio, pte_t *pte, unsigned long addr, unsigned long end) { - const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; unsigned int count = (end - addr) >> PAGE_SHIFT; pte_t ptent = ptep_get(pte); if (!folio_test_large(folio)) return 1; - return folio_pte_batch(folio, addr, pte, ptent, count, fpb_flags, NULL, + return folio_pte_batch(folio, addr, pte, ptent, count, 0, NULL, NULL, NULL); } diff --git a/mm/mremap.c b/mm/mremap.c index 36585041c760d..d4d3ffc931502 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -173,7 +173,6 @@ static pte_t move_soft_dirty_pte(pte_t pte) static int mremap_folio_pte_batch(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int max_nr) { - const fpb_t flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; struct folio *folio; if (max_nr == 1) @@ -183,7 +182,7 @@ static int mremap_folio_pte_batch(struct vm_area_struct *vma, unsigned long addr if (!folio || !folio_test_large(folio)) return 1; - return folio_pte_batch(folio, addr, ptep, pte, max_nr, flags, NULL, + return folio_pte_batch(folio, addr, ptep, pte, max_nr, 0, NULL, NULL, NULL); } diff --git a/mm/rmap.c b/mm/rmap.c index 34311f654d0c2..98ed3e49e8bbe 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1849,7 +1849,6 @@ static inline unsigned int folio_unmap_pte_batch(struct folio *folio, struct page_vma_mapped_walk *pvmw, enum ttu_flags flags, pte_t pte) { - const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; unsigned long end_addr, addr = pvmw->address; struct vm_area_struct *vma = pvmw->vma; unsigned int max_nr; @@ -1869,7 +1868,7 @@ static inline unsigned int folio_unmap_pte_batch(struct folio *folio, if (pte_unused(pte)) return 1; - return folio_pte_batch(folio, addr, pvmw->pte, pte, max_nr, fpb_flags, + return folio_pte_batch(folio, addr, pvmw->pte, pte, max_nr, 0, NULL, NULL, NULL); } -- 2.49.0