[PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[Chris Down]

After commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of the
huge zero folio special"), moved huge zero PMDs must remain special so
vm_normal_page_pmd() continues to treat them as special mappings.

move_pages_huge_pmd() currently reconstructs the destination PMD in the
huge zero page branch, which drops PMD state such as pmd_special() on
architectures with CONFIG_ARCH_HAS_PTE_SPECIAL. As a result,
vm_normal_page_pmd() can treat the moved huge zero PMD as a normal page
and corrupt its refcount.

Instead of reconstructing the PMD from the folio, derive the destination
entry from src_pmdval after pmdp_huge_clear_flush(), then handle the PMD
metadata the same way move_huge_pmd() does for moved entries by marking
it soft-dirty and clearing uffd-wp.

Fixes: d82d09e48219 ("mm/huge_memory: mark PMD mappings of the huge zero folio special")
Cc: stable@vger.kernel.org
Signed-off-by: Chris Down <chris@chrisdown.name>
---
 mm/huge_memory.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index fed57951a7cd..8166b5e871ad 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2794,7 +2794,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
 		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
 	} else {
 		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
-		_dst_pmd = folio_mk_pmd(page_folio(src_page), dst_vma->vm_page_prot);
+		_dst_pmd = move_soft_dirty_pmd(src_pmdval);
+		_dst_pmd = clear_uffd_wp_pmd(_dst_pmd);
 	}
 	set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd);
 
-- 
2.51.2

Re: [PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[David Hildenbrand (Arm)]

On 2/26/26 15:16, Chris Down wrote:
> After commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of the
> huge zero folio special"), moved huge zero PMDs must remain special so
> vm_normal_page_pmd() continues to treat them as special mappings.
> 
> move_pages_huge_pmd() currently reconstructs the destination PMD in the
> huge zero page branch, which drops PMD state such as pmd_special() on
> architectures with CONFIG_ARCH_HAS_PTE_SPECIAL. As a result,
> vm_normal_page_pmd() can treat the moved huge zero PMD as a normal page
> and corrupt its refcount.
> 
> Instead of reconstructing the PMD from the folio, derive the destination
> entry from src_pmdval after pmdp_huge_clear_flush(), then handle the PMD
> metadata the same way move_huge_pmd() does for moved entries by marking
> it soft-dirty and clearing uffd-wp.
> 
> Fixes: d82d09e48219 ("mm/huge_memory: mark PMD mappings of the huge zero folio special")
> Cc: stable@vger.kernel.org
> Signed-off-by: Chris Down <chris@chrisdown.name>
> ---
>  mm/huge_memory.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index fed57951a7cd..8166b5e871ad 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2794,7 +2794,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
>  		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
>  	} else {
>  		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
> -		_dst_pmd = folio_mk_pmd(page_folio(src_page), dst_vma->vm_page_prot);
> +		_dst_pmd = move_soft_dirty_pmd(src_pmdval);
> +		_dst_pmd = clear_uffd_wp_pmd(_dst_pmd);

Please squash that patch directly in #1.

It doesn't make sense to leave something partially fixed in #1. It's
been completely broken from the start. folio_mk_pmd() should never have
been used.

Apart from that, the end results LGTM, thanks

-- 
Cheers,

David

Re: [PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[Matthew Wilcox]

On Thu, Feb 26, 2026 at 04:17:31PM +0100, David Hildenbrand (Arm) wrote:
> It doesn't make sense to leave something partially fixed in #1. It's
> been completely broken from the start. folio_mk_pmd() should never have
> been used.

... mk_huge_pmd() should never have been used ;-)

Re: [PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[David Hildenbrand (Arm)]

On 3/2/26 18:42, Matthew Wilcox wrote:
> On Thu, Feb 26, 2026 at 04:17:31PM +0100, David Hildenbrand (Arm) wrote:
>> It doesn't make sense to leave something partially fixed in #1. It's
>> been completely broken from the start. folio_mk_pmd() should never have
>> been used.
> 
> ... mk_huge_pmd() should never have been used ;-)

Heh, yes!

You made me grep for it and find some leftovers:

arch/powerpc/mm/book3s64/pgtable.c: * pmd_mkhuge() and mk_huge_pmd()
when we update all the

Anybody fancy sending a cleanup? :)

-- 
Cheers,

David

Re: [PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[Lorenzo Stoakes]

On Thu, Feb 26, 2026 at 10:16:47PM +0800, Chris Down wrote:
> After commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of the
> huge zero folio special"), moved huge zero PMDs must remain special so
> vm_normal_page_pmd() continues to treat them as special mappings.
>
> move_pages_huge_pmd() currently reconstructs the destination PMD in the
> huge zero page branch, which drops PMD state such as pmd_special() on
> architectures with CONFIG_ARCH_HAS_PTE_SPECIAL. As a result,
> vm_normal_page_pmd() can treat the moved huge zero PMD as a normal page
> and corrupt its refcount.
>
> Instead of reconstructing the PMD from the folio, derive the destination
> entry from src_pmdval after pmdp_huge_clear_flush(), then handle the PMD
> metadata the same way move_huge_pmd() does for moved entries by marking
> it soft-dirty and clearing uffd-wp.
>
> Fixes: d82d09e48219 ("mm/huge_memory: mark PMD mappings of the huge zero folio special")
> Cc: stable@vger.kernel.org
> Signed-off-by: Chris Down <chris@chrisdown.name>
> ---
>  mm/huge_memory.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index fed57951a7cd..8166b5e871ad 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2794,7 +2794,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
>  		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
>  	} else {
>  		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
> -		_dst_pmd = folio_mk_pmd(page_folio(src_page), dst_vma->vm_page_prot);
> +		_dst_pmd = move_soft_dirty_pmd(src_pmdval);
> +		_dst_pmd = clear_uffd_wp_pmd(_dst_pmd);

I'm confused as to what's going on here, it seems like the 2/3 is simply
updating the 1/3 with a different fixes?

I agree with David that just moving it is probably completely fine, so I think
this should be the only actual patch you need, and you can just Fixes:
e3981db444a0 with it? Then make this a v3 series with 2 patches this + the test
right (but maybe best not backport the test :)?

>  	}
>  	set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd);
>
> --
> 2.51.2
>
>
>

Re: [PATCH v2 2/3] mm/huge_memory: Prevent huge zeropage refcount corruption in PMD move

[Lorenzo Stoakes]

TL;DR - To make life easier, I squashed the two patches and asked Andrew to take
it with my R-b,T-b tags attached, hope that's ok with you Chris.

It takes your work, combines commit msgs and the code into one patch with
correct attribution to you.

See https://lore.kernel.org/linux-mm/a1e787dd-b911-474d-8570-f37685357d86@lucifer.local/

Thanks, Lorenzo

On Mon, Mar 02, 2026 at 05:34:47PM +0000, Lorenzo Stoakes wrote:
> On Thu, Feb 26, 2026 at 10:16:47PM +0800, Chris Down wrote:
> > After commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of the
> > huge zero folio special"), moved huge zero PMDs must remain special so
> > vm_normal_page_pmd() continues to treat them as special mappings.
> >
> > move_pages_huge_pmd() currently reconstructs the destination PMD in the
> > huge zero page branch, which drops PMD state such as pmd_special() on
> > architectures with CONFIG_ARCH_HAS_PTE_SPECIAL. As a result,
> > vm_normal_page_pmd() can treat the moved huge zero PMD as a normal page
> > and corrupt its refcount.
> >
> > Instead of reconstructing the PMD from the folio, derive the destination
> > entry from src_pmdval after pmdp_huge_clear_flush(), then handle the PMD
> > metadata the same way move_huge_pmd() does for moved entries by marking
> > it soft-dirty and clearing uffd-wp.
> >
> > Fixes: d82d09e48219 ("mm/huge_memory: mark PMD mappings of the huge zero folio special")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Chris Down <chris@chrisdown.name>
> > ---
> >  mm/huge_memory.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > index fed57951a7cd..8166b5e871ad 100644
> > --- a/mm/huge_memory.c
> > +++ b/mm/huge_memory.c
> > @@ -2794,7 +2794,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
> >  		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
> >  	} else {
> >  		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
> > -		_dst_pmd = folio_mk_pmd(page_folio(src_page), dst_vma->vm_page_prot);
> > +		_dst_pmd = move_soft_dirty_pmd(src_pmdval);
> > +		_dst_pmd = clear_uffd_wp_pmd(_dst_pmd);
>
> I'm confused as to what's going on here, it seems like the 2/3 is simply
> updating the 1/3 with a different fixes?
>
> I agree with David that just moving it is probably completely fine, so I think
> this should be the only actual patch you need, and you can just Fixes:
> e3981db444a0 with it? Then make this a v3 series with 2 patches this + the test
> right (but maybe best not backport the test :)?
>
> >  	}
> >  	set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd);
> >
> > --
> > 2.51.2
> >
> >
> >