Re: PROBLEM: userfaultfd REGISTER minor mode on MAP_PRIVATE range fails

[Axel Rasmussen <axelrasmussen@google.com>]

Thanks for linking the ExtMem paper David, that makes it a lot more
clear to me what your expectations are.

I think basically, userfaultfd has evolved incrementally, and it only
has a handful of features needed to address pretty specific use cases,
it doesn't have the full flexibility / generality you would need to do
"full memory management in userspace". Not to say I think it shouldn't
be able to do that from a philosophical point of view, I just mean to
say it would take quite a lot of work to get there.

Performance is also a big concern. Userfaultfd performance is not
great, in fact scalability issues are one of the reasons we have been
pursuing guest_memfd based approaches to VM demand paging, instead of
userfaultfd.

I don't disagree that in principle it makes sense for anon private
swap faults to generate userfaultfd minor fault events, it's just
until now nobody had ever wanted to do that, so it hasn't been
implemented yet. :) For what it's worth, I don't think this would get
you where you want to go by itself though, because the only action you
could take in response to such an event today is UFFDIO_CONTINUE,
which would simply swap in + map the page, you would have no
opportunity to e.g. populate the page contents from elsewhere, you'd
be delegating all of that to the existing in-kernel swap
implementation. So it doesn't really get you all the way to "full
userspace memory management".


On Mon, Sep 29, 2025 at 1:30 PM Peter Xu <peterx@redhat.com> wrote:
>
> On Mon, Sep 29, 2025 at 03:44:52PM -0400, David P. Reed wrote:
> > I thought it was a general purpose interface. My mistake. But I think it
> > can be more general, at least encompassing my goal of having a userspace
> > "interface" that monitors processes' page faults.
>
> To James: thanks for the great writeup.  Somehow, I just feel like userfaultfd
> (as a linux submodule) got some sheer luck to have you around. :)
>
> To David: just to say, I still think it's a general purpose interface, at
> least that's the hope..
>
> I agree with you at least on one point you mentioned, that shmem also can
> swap, and that was accounted as minor faults when swapin happens at a
> specific virtual address.  It doesn't sound fair if anon isn't doing the
> same. Indeed.
>
> It was just not in the radar when minor fault was introduced by Axel, even
> it was for a solo purpose for live migration at that time.. but the hope is
> the interface designed should service a generic purpose.
>
> Now the problem is, userfaultfd wasn't initially used for monitoring system
> activities.  As its name implies, it provides the userspace a way to
> resolve a fault, but only if a fault happens first..
>
> Meanwhile, system activities should definitely at least involve swapouts,
> which unfortunately doesn't involve page faults, but only happen the other
> way round when the system wants to secretly move things out.. that is what
> userfaultfd is out of control.
>
> It just sounds like it won't suffice your need even if we could add minor
> fault support for anon private memories on swap cache. However, if
> userfaultfd is used to do everything (including swap in/outs), then it's by
> nature all trappable + accountable, on both swap in/outs to/from any media.
> Then swapout will be driven by the userspace too, then everything will be
> in solid control, including monitoring of the activities.
>
> Thanks,
>
> --
> Peter Xu
>