Re: Userfaultfd doesn't seem to break out of poll on fd close

[Hillf Danton <hdanton@sina.com>]

On Tue, 14 Apr 2020 19:34:10 -0300 Jason Gunthorpe wrote:
> 
> On Tue, Apr 14, 2020 at 05:45:16PM -0400, Peter Xu wrote:
> > On Sun, Apr 12, 2020 at 01:10:40PM -0700, Brian Geffon wrote:
> > > Hi,
> > > It seems that userfaultfd isn't woken from a poll when the file
> > > descriptor is closed. It seems that it should be from the code in
> > > userfault_ctx_release, but it appears that's not actually called
> > > immediately. I have a simple standalone example that shows this
> > > behavior. It's straight forward: one thread creates a userfaultfd and
> > > then closes it after a second thread has entered a poll syscall, some
> > > abbreviated strace output is below showing this and the code can be
> > > seen here: https://gist.github.com/bgaff/9a8fbbe8af79c0e18502430d416df77e
> > > 
> > > Given that it's probably very common to have a dedicated thread remain
> > > blocked indefinitely in a poll(2) waiting for faults there must be a
> > > way to break it out early when it's closed. Am I missing something?
> > 
> > Hi, Brian,
> > 
> > I might be wrong below, just to share my understanding...
> > 
> > IMHO a well-behaved userspace should not close() on a file descriptor
> > if it's still in use within another thread.  In this case, the poll()
> > thread is still using the userfaultfd handle
> 
> I also don't think concurrant close() on a file descriptor that is
> under poll() is well defined, or should be relied upon.
> 
> > IIUC userfaultfd_release() is only called when the file descriptor
> > destructs itself.  But shouldn't the poll() take a refcount of that
> > file descriptor too before waiting?  Not sure userfaultfd_release() is
> > the place to kick then, because if so, close() will only decrease the
> > fd refcount from 2->1, and I'm not sure userfaultfd_release() will be
> > triggered.
> 
> This is most probably true.
> 
> eventfd, epoll and pthread_join is the robust answer to these
> problems.
> 

See the difference EPOLLHUP makes.

--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -937,7 +937,7 @@ wakeup:
 	/* Flush pending events that may still wait on event_wqh */
 	wake_up_all(&ctx->event_wqh);
 
-	wake_up_poll(&ctx->fd_wqh, EPOLLHUP);
+	wake_up_all(&ctx->fd_wqh);
 	userfaultfd_ctx_put(ctx);
 	return 0;
 }