[PATCH] mm/userfaultfd: fix likely/unlikely annotation in move_pages()

[PATCH] mm/userfaultfd: fix likely/unlikely annotation in move_pages()

[Aswin Kumar]

All other userfaultfd paths use unlikely() for the mmap_changing check,
since normally mmap is not changing. The move_pages() path incorrectly
uses likely(), which is the opposite of the intended branch prediction
hint.

This is a performance-only fix - the logic is correct but the branch
prediction annotation is wrong, potentially causing a minor performance
penalty on the fast path.

Fixes: e0a58ef0faa7 ("userfaultfd: UFFDIO_MOVE uABI")
Signed-off-by: Aswin Kumar <aswinkumar3301@gmail.com>
---
 mm/userfaultfd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index e6dfd5f28..d27080348 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -1793,7 +1793,7 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,
 	/* Re-check after taking map_changing_lock */
 	err = -EAGAIN;
 	down_read(&ctx->map_changing_lock);
-	if (likely(atomic_read(&ctx->mmap_changing)))
+	if (unlikely(atomic_read(&ctx->mmap_changing)))
 		goto out_unlock;
 	/*
 	 * Make sure the vma is not shared, that the src and dst remap
-- 
2.43.0

Re: [PATCH] mm/userfaultfd: fix likely/unlikely annotation in move_pages()

[Matthew Wilcox]

On Wed, Feb 04, 2026 at 02:23:50PM +0000, Aswin Kumar wrote:
> All other userfaultfd paths use unlikely() for the mmap_changing check,

No they don't?

$ git grep mmap_changing mm
mm/userfaultfd.c:               if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (likely(atomic_read(&ctx->mmap_changing)))

> This is a performance-only fix - the logic is correct but the branch
> prediction annotation is wrong, potentially causing a minor performance
> penalty on the fast path.

Can you measure it?  Bet you can't.

Honestly, I'd just remove the likely() annotation, not change it to
unlikely().

Re: [PATCH] mm/userfaultfd: fix likely/unlikely annotation in move_pages()

[David Hildenbrand (Arm)]

On 2/4/26 15:58, Matthew Wilcox wrote:
> On Wed, Feb 04, 2026 at 02:23:50PM +0000, Aswin Kumar wrote:
>> All other userfaultfd paths use unlikely() for the mmap_changing check,
> 
> No they don't?
> 
> $ git grep mmap_changing mm
> mm/userfaultfd.c:               if (atomic_read(&ctx->mmap_changing))
> mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
> mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
> mm/userfaultfd.c:       if (likely(atomic_read(&ctx->mmap_changing)))
> 
>> This is a performance-only fix - the logic is correct but the branch
>> prediction annotation is wrong, potentially causing a minor performance
>> penalty on the fast path.
> 
> Can you measure it?  Bet you can't.
> 
> Honestly, I'd just remove the likely() annotation, not change it to
> unlikely().

Agreed.

-- 
Cheers,

David