[PATCH mm-hotfixes] mm/huge_memory: fix memory corruption on huge zero page move

[PATCH mm-hotfixes] mm/huge_memory: fix memory corruption on huge zero page move

[Lorenzo Stoakes]

In commit eb1521dad8f3 ("userfaultfd: handle zeropage moves by
UFFDIO_MOVE"), handling was added to enable the moving of huge zero pages
in move_pages_huge_pmd().

This achieves this by setting src_folio to NULL, and adding subsequent
checks for src_folio being NULL to determine whether to perform the usual
move operations or to simply establish the huge zero page in the
destination.

As part of this change, when installing the destination huge zero page it
invoked mk_huge_pmd() on src_page, correctly.

However, commit e3981db444a0 ("mm: add folio_mk_pmd()") updated the code in
the huge zero page branch from mk_huge_pmd(src_page, ...) to
folio_mk_pmd(src_folio, ...), where src_folio is guaranteed to be NULL at
this point.

This resulted in an invocation of folio_mk_pmd(NULL, ...) in effect, which
causes an invocation of page_to_pfn(0) and results in the installation of a
corrupted PMD entry and undefined behaviour.

This patch fixes the issue by obtaining the zero folio via
page_folio(src_page) and feeding this into folio_mk_pmd(). This retains the
use of folio_mk_pmd() whilst avoiding the memory corruption.

Additionally, this code path was not updated to reflect the changes
introduced by commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of
the huge zero folio special"), meaning a zero huge folio was installed but
not marked special in this case.

This patch additionally fixes that issue by invoking pmd_mkspecial().

With thanks to Chris Down who exposed this bug by adding an explicit test
for UFFDIO_MOVE in commit f07254dce67d ("selftests/mm: add UFFDIO_MOVE huge
zeropage PMD regression test").

Fixes: e3981db444a0 ("mm: add folio_mk_pmd()")
Cc: <stable@vger.kernel.org>
Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
---
 mm/huge_memory.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index c17965f5682f..de2a775590f1 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2796,8 +2796,12 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
 		/* Follow mremap() behavior and treat the entry dirty after the move */
 		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
 	} else {
+		struct folio *zero_folio = page_folio(src_page);
+
+		VM_WARN_ON_ONCE_FOLIO(!is_huge_zero_folio(zero_folio), zero_folio);
 		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
-		_dst_pmd = folio_mk_pmd(src_folio, dst_vma->vm_page_prot);
+		_dst_pmd = folio_mk_pmd(zero_folio, dst_vma->vm_page_prot);
+		_dst_pmd = pmd_mkspecial(_dst_pmd);
 	}
 	set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd);

--
2.53.0

Re: [PATCH mm-hotfixes] mm/huge_memory: fix memory corruption on huge zero page move

[David Hildenbrand (Arm)]

On 3/2/26 18:06, Lorenzo Stoakes wrote:
> In commit eb1521dad8f3 ("userfaultfd: handle zeropage moves by
> UFFDIO_MOVE"), handling was added to enable the moving of huge zero pages
> in move_pages_huge_pmd().
> 
> This achieves this by setting src_folio to NULL, and adding subsequent
> checks for src_folio being NULL to determine whether to perform the usual
> move operations or to simply establish the huge zero page in the
> destination.
> 
> As part of this change, when installing the destination huge zero page it
> invoked mk_huge_pmd() on src_page, correctly.
> 
> However, commit e3981db444a0 ("mm: add folio_mk_pmd()") updated the code in
> the huge zero page branch from mk_huge_pmd(src_page, ...) to
> folio_mk_pmd(src_folio, ...), where src_folio is guaranteed to be NULL at
> this point.
> 
> This resulted in an invocation of folio_mk_pmd(NULL, ...) in effect, which
> causes an invocation of page_to_pfn(0) and results in the installation of a
> corrupted PMD entry and undefined behaviour.
> 
> This patch fixes the issue by obtaining the zero folio via
> page_folio(src_page) and feeding this into folio_mk_pmd(). This retains the
> use of folio_mk_pmd() whilst avoiding the memory corruption.
> 
> Additionally, this code path was not updated to reflect the changes
> introduced by commit d82d09e48219 ("mm/huge_memory: mark PMD mappings of
> the huge zero folio special"), meaning a zero huge folio was installed but
> not marked special in this case.
> 
> This patch additionally fixes that issue by invoking pmd_mkspecial().
> 
> With thanks to Chris Down who exposed this bug by adding an explicit test
> for UFFDIO_MOVE in commit f07254dce67d ("selftests/mm: add UFFDIO_MOVE huge
> zeropage PMD regression test").
> 
> Fixes: e3981db444a0 ("mm: add folio_mk_pmd()")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> ---
>  mm/huge_memory.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index c17965f5682f..de2a775590f1 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2796,8 +2796,12 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
>  		/* Follow mremap() behavior and treat the entry dirty after the move */
>  		_dst_pmd = pmd_mkwrite(pmd_mkdirty(_dst_pmd), dst_vma);
>  	} else {
> +		struct folio *zero_folio = page_folio(src_page);
> +
> +		VM_WARN_ON_ONCE_FOLIO(!is_huge_zero_folio(zero_folio), zero_folio);
>  		src_pmdval = pmdp_huge_clear_flush(src_vma, src_addr, src_pmd);
> -		_dst_pmd = folio_mk_pmd(src_folio, dst_vma->vm_page_prot);
> +		_dst_pmd = folio_mk_pmd(zero_folio, dst_vma->vm_page_prot);
> +		_dst_pmd = pmd_mkspecial(_dst_pmd);
>  	}
>  	set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd);

There are already patches in flight:

https://lore.kernel.org/r/aaBVaHs8rIkNcwM0@chrisdown.name

:)

-- 
Cheers,

David

Re: [PATCH mm-hotfixes] mm/huge_memory: fix memory corruption on huge zero page move

[Lorenzo Stoakes]

On Mon, Mar 02, 2026 at 06:10:06PM +0100, David Hildenbrand (Arm) wrote:
> There are already patches in flight:
>
> https://lore.kernel.org/r/aaBVaHs8rIkNcwM0@chrisdown.name

Yup, if people would actually cc the right people I'd not have wasted my
afternoon.

Replying there.

>
> --
> Cheers,
>
> David

Thanks, Lorenzo

Re: [PATCH mm-hotfixes] mm/huge_memory: fix memory corruption on huge zero page move

[Andrew Morton]

On Mon, 2 Mar 2026 17:19:15 +0000 Lorenzo Stoakes <lorenzo.stoakes@oracle.com> wrote:

> On Mon, Mar 02, 2026 at 06:10:06PM +0100, David Hildenbrand (Arm) wrote:
> > There are already patches in flight:
> >
> > https://lore.kernel.org/r/aaBVaHs8rIkNcwM0@chrisdown.name
> 
> Yup, if people would actually cc the right people I'd not have wasted my
> afternoon.
> 
> Replying there.
> 

I'm not sure where this is headed but I'll queue this patch for now, to 
get it some testing, to lessen duplicated bug reports and to generally track things.