Re: [PATCH v3] userfaultfd: provide unmasked address on page-fault

[Nadav Amit <namit@vmware.com>]

> On Mar 3, 2022, at 11:05 AM, Nadav Amit <namit@vmware.com> wrote:
> 
> 
> 
>> On Mar 3, 2022, at 12:03 AM, Peter Xu <peterx@redhat.com> wrote:
>> 
>> On Sat, Feb 26, 2022 at 02:26:55AM +0000, Nadav Amit wrote:
>>> From: Nadav Amit <namit@vmware.com>
>>> 
>>> Userfaultfd is supposed to provide the full address (i.e., unmasked) of
>>> the faulting access back to userspace. However, that is not the case for
>>> quite some time.
>>> 
>>> Even running "userfaultfd_demo" from the userfaultfd man page provides
>>> the wrong output (and contradicts the man page). Notice that
>>> "UFFD_EVENT_PAGEFAULT event" shows the masked address (7fc5e30b3000)
>>> and not the first read address (0x7fc5e30b300f).
>>> 
>>> 	Address returned by mmap() = 0x7fc5e30b3000
>>> 
>>> 	fault_handler_thread():
>>> 	    poll() returns: nready = 1; POLLIN = 1; POLLERR = 0
>>> 	    UFFD_EVENT_PAGEFAULT event: flags = 0; address = 7fc5e30b3000
>>> 		(uffdio_copy.copy returned 4096)
>>> 	Read address 0x7fc5e30b300f in main(): A
>>> 	Read address 0x7fc5e30b340f in main(): A
>>> 	Read address 0x7fc5e30b380f in main(): A
>>> 	Read address 0x7fc5e30b3c0f in main(): A
>>> 
>>> The exact address is useful for various reasons and specifically for
>>> prefetching decisions. If it is known that the memory is populated by
>>> certain objects whose size is not page-aligned, then based on the
>>> faulting address, the uffd-monitor can decide whether to prefetch and
>>> prefault the adjacent page.
>>> 
>>> This bug has been for quite some time in the kernel: since commit
>>> 1a29d85eb0f1 ("mm: use vmf->address instead of of vmf->virtual_address")
>>> vmf->virtual_address"), which dates back to 2016. A concern has been
>>> raised that existing userspace application might rely on the old/wrong
>>> behavior in which the address is masked. Therefore, it was suggested to
>>> provide the masked address unless the user explicitly asks for the exact
>>> address.
>>> 
>>> Add a new userfaultfd feature UFFD_FEATURE_EXACT_ADDRESS to direct
>>> userfaultfd to provide the exact address. Add a new "real_address" field
>>> to vmf to hold the unmasked address. Provide the address to userspace
>>> accordingly.
>>> 
>>> Initialize real_address in various code-paths to be consistent with
>>> address, even when it is not used, to be on the safe side.
>>> 
>>> Acked-by: Peter Xu <peterx@redhat.com>
>>> Reviewed-by: David Hildenbrand <david@redhat.com>
>>> Cc: Andrea Arcangeli <aarcange@redhat.com>
>>> Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
>>> Cc: Jan Kara <jack@suse.cz>
>>> Signed-off-by: Nadav Amit <namit@vmware.com>
>> 
>> Hi, Andrew,
>> 
>> Just a heads-up that this version has not yet been updated in -mm I think,
>> while the queued one is the old version.
>> 
>> IOW, uffd is currently broken on latest linux-next on hugetlb.
> 
> Thanks Peter for reminding Andrew.
> 
> Andrew, please acknowledge it would be queue for the next version and
> I will submit a patch to the man pages.

Peter (et. al),

I’ll send it in a more orderly fashion later, but let me know if I got
something completely wrong for the man page change:

[ Thanks as usual; sorry - limited experience changing man pages ]

-- >8 --

From: Nadav Amit <namit@vmware.com>
Date: Thu, 3 Mar 2022 19:44:37 +0000
Subject: [PATCH] ioctl_userfaultfd: add UFFD_FEATURE_EXACT_ADDRESS

Describe the new UFFD_FEATURE_EXACT_ADDRESS API feature.

Signed-off-by: Nadav Amit <namit@vmware.com>
---
 man2/ioctl_userfaultfd.2 | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 504f61d4b..2d065504e 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -214,6 +214,12 @@ memory accesses to the regions registered with userfaultfd.
 If this feature bit is set,
 .I uffd_msg.pagefault.feat.ptid
 will be set to the faulted thread ID for each page-fault message.
+.TP
+.BR UFFD_FEATURE_EXACT_ADDRESS " (since Linux 5.18)"
+If this feature bit is set,
+.I uffd_msg.pagefault.address
+will be set to the exact page-fault address that was reported by the hardware,
+and will not mask the offset within the page.
 .PP
 The returned
 .I ioctls
-- 
2.25.1