[PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lance Yang]

From: Lance Yang <lance.yang@linux.dev>

When splitting an mTHP and replacing a zero-filled subpage with the shared
zeropage, try_to_map_unused_to_zeropage() currently drops several
important PTE bits.

For userspace tools like CRIU, which rely on the soft-dirty mechanism for
incremental snapshots, losing the soft-dirty bit means modified pages are
missed, leading to inconsistent memory state after restore.

As pointed out by David, the more critical uffd-wp bit is also dropped.
This breaks the userfaultfd write-protection mechanism, causing writes to
be silently missed by monitoring applications, which can lead to data
corruption.

Preserve both the soft-dirty and uffd-wp bits from the old PTE when
creating the new zeropage mapping to ensure they are correctly tracked.

Link: https://lkml.kernel.org/r/20250930081040.80926-1-lance.yang@linux.dev
Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")
Signed-off-by: Lance Yang <lance.yang@linux.dev>
Suggested-by: David Hildenbrand <david@redhat.com>
Suggested-by: Dev Jain <dev.jain@arm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Jann Horn <jannh@google.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Mariano Pache <npache@redhat.com>
Cc: Mathew Brost <matthew.brost@intel.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Usama Arif <usamaarif642@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Yu Zhao <yuzhao@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
(cherry picked from commit 9658d698a8a83540bf6a6c80d13c9a61590ee985)
---
 mm/migrate.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 8619aa884eaa..603330ad8e0b 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -198,8 +198,7 @@ bool isolate_folio_to_list(struct folio *folio, struct list_head *list)
 }
 
 static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
-					  struct folio *folio,
-					  unsigned long idx)
+		struct folio *folio, pte_t old_pte, unsigned long idx)
 {
 	struct page *page = folio_page(folio, idx);
 	bool contains_data;
@@ -210,7 +209,7 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
 		return false;
 	VM_BUG_ON_PAGE(!PageAnon(page), page);
 	VM_BUG_ON_PAGE(!PageLocked(page), page);
-	VM_BUG_ON_PAGE(pte_present(*pvmw->pte), page);
+	VM_BUG_ON_PAGE(pte_present(old_pte), page);
 
 	if (folio_test_mlocked(folio) || (pvmw->vma->vm_flags & VM_LOCKED) ||
 	    mm_forbids_zeropage(pvmw->vma->vm_mm))
@@ -230,6 +229,12 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
 
 	newpte = pte_mkspecial(pfn_pte(my_zero_pfn(pvmw->address),
 					pvmw->vma->vm_page_prot));
+
+	if (pte_swp_soft_dirty(old_pte))
+		newpte = pte_mksoft_dirty(newpte);
+	if (pte_swp_uffd_wp(old_pte))
+		newpte = pte_mkuffd_wp(newpte);
+
 	set_pte_at(pvmw->vma->vm_mm, pvmw->address, pvmw->pte, newpte);
 
 	dec_mm_counter(pvmw->vma->vm_mm, mm_counter(folio));
@@ -272,13 +277,13 @@ static bool remove_migration_pte(struct folio *folio,
 			continue;
 		}
 #endif
+		old_pte = ptep_get(pvmw.pte);
 		if (rmap_walk_arg->map_unused_to_zeropage &&
-		    try_to_map_unused_to_zeropage(&pvmw, folio, idx))
+		    try_to_map_unused_to_zeropage(&pvmw, folio, old_pte, idx))
 			continue;
 
 		folio_get(folio);
 		pte = mk_pte(new, READ_ONCE(vma->vm_page_prot));
-		old_pte = ptep_get(pvmw.pte);
 
 		entry = pte_to_swp_entry(old_pte);
 		if (!is_migration_entry_young(entry))
-- 
2.49.0

Re: [PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lorenzo Stoakes]

On Fri, Oct 17, 2025 at 04:51:06PM +0800, Lance Yang wrote:
> From: Lance Yang <lance.yang@linux.dev>
>
> When splitting an mTHP and replacing a zero-filled subpage with the shared
> zeropage, try_to_map_unused_to_zeropage() currently drops several
> important PTE bits.
>
> For userspace tools like CRIU, which rely on the soft-dirty mechanism for
> incremental snapshots, losing the soft-dirty bit means modified pages are
> missed, leading to inconsistent memory state after restore.
>
> As pointed out by David, the more critical uffd-wp bit is also dropped.
> This breaks the userfaultfd write-protection mechanism, causing writes to
> be silently missed by monitoring applications, which can lead to data
> corruption.
>
> Preserve both the soft-dirty and uffd-wp bits from the old PTE when
> creating the new zeropage mapping to ensure they are correctly tracked.
>
> Link: https://lkml.kernel.org/r/20250930081040.80926-1-lance.yang@linux.dev
> Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")
> Signed-off-by: Lance Yang <lance.yang@linux.dev>
> Suggested-by: David Hildenbrand <david@redhat.com>
> Suggested-by: Dev Jain <dev.jain@arm.com>
> Acked-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Dev Jain <dev.jain@arm.com>
> Acked-by: Zi Yan <ziy@nvidia.com>
> Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

You're missing my R-b...

Re: [PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lance Yang]

On 2025/10/17 17:52, Lorenzo Stoakes wrote:
> On Fri, Oct 17, 2025 at 04:51:06PM +0800, Lance Yang wrote:
>> From: Lance Yang <lance.yang@linux.dev>
>>
>> When splitting an mTHP and replacing a zero-filled subpage with the shared
>> zeropage, try_to_map_unused_to_zeropage() currently drops several
>> important PTE bits.
>>
>> For userspace tools like CRIU, which rely on the soft-dirty mechanism for
>> incremental snapshots, losing the soft-dirty bit means modified pages are
>> missed, leading to inconsistent memory state after restore.
>>
>> As pointed out by David, the more critical uffd-wp bit is also dropped.
>> This breaks the userfaultfd write-protection mechanism, causing writes to
>> be silently missed by monitoring applications, which can lead to data
>> corruption.
>>
>> Preserve both the soft-dirty and uffd-wp bits from the old PTE when
>> creating the new zeropage mapping to ensure they are correctly tracked.
>>
>> Link: https://lkml.kernel.org/r/20250930081040.80926-1-lance.yang@linux.dev
>> Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")
>> Signed-off-by: Lance Yang <lance.yang@linux.dev>
>> Suggested-by: David Hildenbrand <david@redhat.com>
>> Suggested-by: Dev Jain <dev.jain@arm.com>
>> Acked-by: David Hildenbrand <david@redhat.com>
>> Reviewed-by: Dev Jain <dev.jain@arm.com>
>> Acked-by: Zi Yan <ziy@nvidia.com>
>> Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
>> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
> 
> You're missing my R-b...

Sorry, I missed it! I just cherry-picked the commit from
upstream and didn't notice ...

Hopefully Greg can add your Reviewed-by when applying.

Re: [PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lorenzo Stoakes]

On Fri, Oct 17, 2025 at 06:25:42PM +0800, Lance Yang wrote:
> > You're missing my R-b...
>
> Sorry, I missed it! I just cherry-picked the commit from
> upstream and didn't notice ...
>
> Hopefully Greg can add your Reviewed-by when applying.

OK disregard, I must have reviewed it after it got merged due to my
vacation.

Re: [PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lance Yang]

On 2025/10/17 18:25, Lance Yang wrote:
> 
> 
> On 2025/10/17 17:52, Lorenzo Stoakes wrote:
>> On Fri, Oct 17, 2025 at 04:51:06PM +0800, Lance Yang wrote:
>>> From: Lance Yang <lance.yang@linux.dev>
>>>
>>> When splitting an mTHP and replacing a zero-filled subpage with the 
>>> shared
>>> zeropage, try_to_map_unused_to_zeropage() currently drops several
>>> important PTE bits.
>>>
>>> For userspace tools like CRIU, which rely on the soft-dirty mechanism 
>>> for
>>> incremental snapshots, losing the soft-dirty bit means modified pages 
>>> are
>>> missed, leading to inconsistent memory state after restore.
>>>
>>> As pointed out by David, the more critical uffd-wp bit is also dropped.
>>> This breaks the userfaultfd write-protection mechanism, causing 
>>> writes to
>>> be silently missed by monitoring applications, which can lead to data
>>> corruption.
>>>
>>> Preserve both the soft-dirty and uffd-wp bits from the old PTE when
>>> creating the new zeropage mapping to ensure they are correctly tracked.
>>>
>>> Link: https://lkml.kernel.org/r/20250930081040.80926-1- 
>>> lance.yang@linux.dev
>>> Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage 
>>> when splitting isolated thp")
>>> Signed-off-by: Lance Yang <lance.yang@linux.dev>
>>> Suggested-by: David Hildenbrand <david@redhat.com>
>>> Suggested-by: Dev Jain <dev.jain@arm.com>
>>> Acked-by: David Hildenbrand <david@redhat.com>
>>> Reviewed-by: Dev Jain <dev.jain@arm.com>
>>> Acked-by: Zi Yan <ziy@nvidia.com>
>>> Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
>>> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
>>
>> You're missing my R-b...
> 
> Sorry, I missed it! I just cherry-picked the commit from
> upstream and didn't notice ...
> 
> Hopefully Greg can add your Reviewed-by when applying.

Looking at the timeline again, the fix was actually merged
upstream before your review arrived, so the commit I
cherry-picked never had your tag to begin with :(

Still hoping Greg can add it!

Re: [PATCH 6.12.y 1/1] mm/rmap: fix soft-dirty and uffd-wp bit loss when remapping zero-filled mTHP subpage to shared zeropage

[Lorenzo Stoakes]

On Fri, Oct 17, 2025 at 07:14:32PM +0800, Lance Yang wrote:
>
>
> On 2025/10/17 18:25, Lance Yang wrote:
> >
> >
> > On 2025/10/17 17:52, Lorenzo Stoakes wrote:
> > > On Fri, Oct 17, 2025 at 04:51:06PM +0800, Lance Yang wrote:
> > > > From: Lance Yang <lance.yang@linux.dev>
> > > >
> > > > When splitting an mTHP and replacing a zero-filled subpage with
> > > > the shared
> > > > zeropage, try_to_map_unused_to_zeropage() currently drops several
> > > > important PTE bits.
> > > >
> > > > For userspace tools like CRIU, which rely on the soft-dirty
> > > > mechanism for
> > > > incremental snapshots, losing the soft-dirty bit means modified
> > > > pages are
> > > > missed, leading to inconsistent memory state after restore.
> > > >
> > > > As pointed out by David, the more critical uffd-wp bit is also dropped.
> > > > This breaks the userfaultfd write-protection mechanism, causing
> > > > writes to
> > > > be silently missed by monitoring applications, which can lead to data
> > > > corruption.
> > > >
> > > > Preserve both the soft-dirty and uffd-wp bits from the old PTE when
> > > > creating the new zeropage mapping to ensure they are correctly tracked.
> > > >
> > > > Link: https://lkml.kernel.org/r/20250930081040.80926-1-
> > > > lance.yang@linux.dev
> > > > Fixes: b1f202060afe ("mm: remap unused subpages to shared
> > > > zeropage when splitting isolated thp")
> > > > Signed-off-by: Lance Yang <lance.yang@linux.dev>
> > > > Suggested-by: David Hildenbrand <david@redhat.com>
> > > > Suggested-by: Dev Jain <dev.jain@arm.com>
> > > > Acked-by: David Hildenbrand <david@redhat.com>
> > > > Reviewed-by: Dev Jain <dev.jain@arm.com>
> > > > Acked-by: Zi Yan <ziy@nvidia.com>
> > > > Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> > > > Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
> > >
> > > You're missing my R-b...
> >
> > Sorry, I missed it! I just cherry-picked the commit from
> > upstream and didn't notice ...
> >
> > Hopefully Greg can add your Reviewed-by when applying.
>
> Looking at the timeline again, the fix was actually merged
> upstream before your review arrived, so the commit I
> cherry-picked never had your tag to begin with :(
>
> Still hoping Greg can add it!
>

No, I don't want that in that case it'd be odd unless I had separately reviewed
the backport.

I was just unaware this had been taken during the merge window.