Re: [PATCH v3] userfaultfd: provide unmasked address on page-fault

[David Hildenbrand <david@redhat.com>]

On 03.03.22 20:51, Nadav Amit wrote:
> 
> 
>> On Mar 3, 2022, at 11:05 AM, Nadav Amit <namit@vmware.com> wrote:
>>
>>
>>
>>> On Mar 3, 2022, at 12:03 AM, Peter Xu <peterx@redhat.com> wrote:
>>>
>>> On Sat, Feb 26, 2022 at 02:26:55AM +0000, Nadav Amit wrote:
>>>> From: Nadav Amit <namit@vmware.com>
>>>>
>>>> Userfaultfd is supposed to provide the full address (i.e., unmasked) of
>>>> the faulting access back to userspace. However, that is not the case for
>>>> quite some time.
>>>>
>>>> Even running "userfaultfd_demo" from the userfaultfd man page provides
>>>> the wrong output (and contradicts the man page). Notice that
>>>> "UFFD_EVENT_PAGEFAULT event" shows the masked address (7fc5e30b3000)
>>>> and not the first read address (0x7fc5e30b300f).
>>>>
>>>> 	Address returned by mmap() = 0x7fc5e30b3000
>>>>
>>>> 	fault_handler_thread():
>>>> 	    poll() returns: nready = 1; POLLIN = 1; POLLERR = 0
>>>> 	    UFFD_EVENT_PAGEFAULT event: flags = 0; address = 7fc5e30b3000
>>>> 		(uffdio_copy.copy returned 4096)
>>>> 	Read address 0x7fc5e30b300f in main(): A
>>>> 	Read address 0x7fc5e30b340f in main(): A
>>>> 	Read address 0x7fc5e30b380f in main(): A
>>>> 	Read address 0x7fc5e30b3c0f in main(): A
>>>>
>>>> The exact address is useful for various reasons and specifically for
>>>> prefetching decisions. If it is known that the memory is populated by
>>>> certain objects whose size is not page-aligned, then based on the
>>>> faulting address, the uffd-monitor can decide whether to prefetch and
>>>> prefault the adjacent page.
>>>>
>>>> This bug has been for quite some time in the kernel: since commit
>>>> 1a29d85eb0f1 ("mm: use vmf->address instead of of vmf->virtual_address")
>>>> vmf->virtual_address"), which dates back to 2016. A concern has been
>>>> raised that existing userspace application might rely on the old/wrong
>>>> behavior in which the address is masked. Therefore, it was suggested to
>>>> provide the masked address unless the user explicitly asks for the exact
>>>> address.
>>>>
>>>> Add a new userfaultfd feature UFFD_FEATURE_EXACT_ADDRESS to direct
>>>> userfaultfd to provide the exact address. Add a new "real_address" field
>>>> to vmf to hold the unmasked address. Provide the address to userspace
>>>> accordingly.
>>>>
>>>> Initialize real_address in various code-paths to be consistent with
>>>> address, even when it is not used, to be on the safe side.
>>>>
>>>> Acked-by: Peter Xu <peterx@redhat.com>
>>>> Reviewed-by: David Hildenbrand <david@redhat.com>
>>>> Cc: Andrea Arcangeli <aarcange@redhat.com>
>>>> Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
>>>> Cc: Jan Kara <jack@suse.cz>
>>>> Signed-off-by: Nadav Amit <namit@vmware.com>
>>>
>>> Hi, Andrew,
>>>
>>> Just a heads-up that this version has not yet been updated in -mm I think,
>>> while the queued one is the old version.
>>>
>>> IOW, uffd is currently broken on latest linux-next on hugetlb.
>>
>> Thanks Peter for reminding Andrew.
>>
>> Andrew, please acknowledge it would be queue for the next version and
>> I will submit a patch to the man pages.
> 
> Peter (et. al),
> 
> I’ll send it in a more orderly fashion later, but let me know if I got
> something completely wrong for the man page change:
> 
> [ Thanks as usual; sorry - limited experience changing man pages ]
> 
> -- >8 --
> 
> From: Nadav Amit <namit@vmware.com>
> Date: Thu, 3 Mar 2022 19:44:37 +0000
> Subject: [PATCH] ioctl_userfaultfd: add UFFD_FEATURE_EXACT_ADDRESS
> 
> Describe the new UFFD_FEATURE_EXACT_ADDRESS API feature.
> 
> Signed-off-by: Nadav Amit <namit@vmware.com>
> ---
>  man2/ioctl_userfaultfd.2 | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index 504f61d4b..2d065504e 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -214,6 +214,12 @@ memory accesses to the regions registered with userfaultfd.
>  If this feature bit is set,
>  .I uffd_msg.pagefault.feat.ptid
>  will be set to the faulted thread ID for each page-fault message.
> +.TP
> +.BR UFFD_FEATURE_EXACT_ADDRESS " (since Linux 5.18)"
> +If this feature bit is set,
> +.I uffd_msg.pagefault.address
> +will be set to the exact page-fault address that was reported by the hardware,
> +and will not mask the offset within the page.
>  .PP
>  The returned
>  .I ioctls

Do we want to add a comment about early uffd code that did this as well?

"Note that old Linux versions might indicate the exact address as well,
even though the feature bit is not set."

-- 
Thanks,

David / dhildenb