Re: [PATCH v3 17/28] userfaultfd: wp: support swap and page migration

[Peter Xu <peterx@redhat.com>]

On Fri, Apr 19, 2019 at 11:08:02AM -0400, Jerome Glisse wrote:
> On Fri, Apr 19, 2019 at 03:42:20PM +0800, Peter Xu wrote:
> > On Thu, Apr 18, 2019 at 04:59:07PM -0400, Jerome Glisse wrote:
> > > On Wed, Mar 20, 2019 at 10:06:31AM +0800, Peter Xu wrote:
> > > > For either swap and page migration, we all use the bit 2 of the entry to
> > > > identify whether this entry is uffd write-protected.  It plays a similar
> > > > role as the existing soft dirty bit in swap entries but only for keeping
> > > > the uffd-wp tracking for a specific PTE/PMD.
> > > > 
> > > > Something special here is that when we want to recover the uffd-wp bit
> > > > from a swap/migration entry to the PTE bit we'll also need to take care
> > > > of the _PAGE_RW bit and make sure it's cleared, otherwise even with the
> > > > _PAGE_UFFD_WP bit we can't trap it at all.
> > > > 
> > > > Note that this patch removed two lines from "userfaultfd: wp: hook
> > > > userfault handler to write protection fault" where we try to remove the
> > > > VM_FAULT_WRITE from vmf->flags when uffd-wp is set for the VMA.  This
> > > > patch will still keep the write flag there.
> > > > 
> > > > Reviewed-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> > > > Signed-off-by: Peter Xu <peterx@redhat.com>
> > > 
> > > Some missing thing see below.
> > > 
> > > [...]
> > > 
> > > > diff --git a/mm/memory.c b/mm/memory.c
> > > > index 6405d56debee..c3d57fa890f2 100644
> > > > --- a/mm/memory.c
> > > > +++ b/mm/memory.c
> > > > @@ -736,6 +736,8 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
> > > >  				pte = swp_entry_to_pte(entry);
> > > >  				if (pte_swp_soft_dirty(*src_pte))
> > > >  					pte = pte_swp_mksoft_dirty(pte);
> > > > +				if (pte_swp_uffd_wp(*src_pte))
> > > > +					pte = pte_swp_mkuffd_wp(pte);
> > > >  				set_pte_at(src_mm, addr, src_pte, pte);
> > > >  			}
> > > >  		} else if (is_device_private_entry(entry)) {
> > > 
> > > You need to handle the is_device_private_entry() as the migration case
> > > too.
> > 
> > Hi, Jerome,
> > 
> > Yes I can simply add the handling, but I'd confess I haven't thought
> > clearly yet on how userfault-wp will be used with HMM (and that's
> > mostly because my unfamiliarity so far with HMM).  Could you give me
> > some hint on a most general and possible scenario?
> 
> device private is just a temporary state with HMM you can have thing
> like GPU or FPGA migrate some anonymous page to their local memory
> because it is use by the GPU or the FPGA. The GPU or FPGA behave like
> a CPU from mm POV so if it wants to write it will fault and go through
> the regular CPU page fault.
> 
> That said it can still migrate a page that is UFD write protected just
> because the device only care about reading. So if you have a UFD pte
> to a regular page that get migrated to some device memory you want to
> keep the UFD WP flags after the migration (in both direction when going
> to device memory and from coming back from it).
> 
> As far as UFD is concern this is just another page, it just does not
> have a valid pte entry because CPU can not access such memory. But from
> mm point of view it just another page.

I see the point.  Thanks for explaining that!

-- 
Peter Xu