Re: PROBLEM: userfaultfd REGISTER minor mode on MAP_PRIVATE range fails

["David P. Reed" <dpreed@deepplum.com>]

On Tuesday, September 16, 2025 15:10, "James Houghton" <jthoughton@google.com> said:

> On Tue, Sep 16, 2025 at 11:35 AM Axel Rasmussen
> <axelrasmussen@google.com> wrote:
>>
>>
>>
>> On Tue, Sep 16, 2025 at 10:27 AM David P. Reed <dpreed@deepplum.com>
>> wrote:
>>>
>>> Than -
>>>
>>> Just to clarify -
>>> Looking at the man page for UFFDIO_API, there are two "feature bits" that
>>> indicate cases where "minor" handling is now supported, and can be enabled.
>>> UFFD_FEATURE_MINOR_HUGETLBFS and UFFD_FEATURE_MINOR_SHMEM
>>> In my reading of the documents, these seem to imply that before they were added
>>> as new features, that MAP_PRIVATE|MAP_ANONYMOUS mappings were supported, and
>>> that the "new" additions to the MINOR mode were just for HUGETLBFS and
>>> MAP_SHARED cases.
>>
>>
>> Actually minor fault support didn't exist at all before those two features were
>> added. :)
>>
>> You are right that userfaultfd's use of "minor fault" is (unfortunately) slightly
>> different from the meaning in other contexts. I think the more normal meaning is,
>> faults which do not incur I/O (i.e., swap faults and file faults [i.e., faults on
>> non-swap-backed pages] are major, other faults are minor).
>>
>> For userfaultfd, a minor fault is a fault where the page already exists in the
>> page cache, but the page table entry wasn't setup. I don't think that scenario
>> can ever happen for anonymous, private mappings, so it doesn't really make sense
>> to be able to register such mappings in this mode. If you create a mapping with
>> mmap(MAP_ANON|MAP_PRIVATE) and then access it (read or write), that fault
>> requires allocation of a new page, so userfaultfd does not consider that a "minor
>> fault". My recollection though is if you make a file on tmpfs or hugetlbfs,
>> fallocate() it or whatever, and you MAP_PRIVATE that file, *that* registration
>> will work.
> 
> Ah! You're right... MAP_PRIVATE *is* supported (for tmpfs and
> hugetlbfs only), and UFFDIO_CONTINUE will, upon finding the page in
> the page cache, install a RO PTE for it.
> 
> But what happens when the write comes after installing the RO PTE? My
> reading of the code today makes me think that we'd get a minor
> userfault and then be unable to continue...! (The only reasonable
> behavior is that CoW is done without triggering a userfault... I
> assumed/thought this was the behavior today. I wish I had time to test
> this -- I hope I'm misreading it.)
> 
> :( Here I was thinking I understood how userfaultfd minor faults worked.
> 

So did I. It's kind of confusing. I suppose `git blame` might suggest when the code that doesn't like MAP_PRIVATE|MAP_ANONYMOUS pages in the UFFDIO_REGISTER minor call was introduced... I know that MAP_SHARED|MAP_ANONYMOUS pages allows minor faults to be registered on them - and the only difference really is the passing of the mapping to forked clones (and the COW behavior resulting from sharing until written, but minor mapping has nothing to do with write faults particularly - reading can cause minor faults too).