Re: [PATCH v2 1/2] mm/migrate_device.c: Copy pte dirty bit to page

[Alistair Popple <apopple@nvidia.com>]

Peter Xu <peterx@redhat.com> writes:

> On Wed, Aug 24, 2022 at 04:25:44PM -0400, Peter Xu wrote:
>> On Wed, Aug 24, 2022 at 11:56:25AM +1000, Alistair Popple wrote:
>> > >> Still I don't know whether there'll be any side effect of having stall tlbs
>> > >> in !present ptes because I'm not familiar enough with the private dev swap
>> > >> migration code.  But I think having them will be safe, even if redundant.
>> >
>> > What side-effect were you thinking of? I don't see any issue with not
>> > TLB flushing stale device-private TLBs prior to the migration because
>> > they're not accessible anyway and shouldn't be in any TLB.
>>
>> Sorry to be misleading, I never meant we must add them.  As I said it's
>> just that I don't know the code well so I don't know whether it's safe to
>> not have it.
>>
>> IIUC it's about whether having stall system-ram stall tlb in other
>> processor would matter or not here.  E.g. some none pte that this code
>> collected (boosted both "cpages" and "npages" for a none pte) could have
>> stall tlb in other cores that makes the page writable there.
>
> For this one, let me give a more detailed example.

Thanks, I would have been completely lost about what you were talking
about without this :-)

> It's about whether below could happen:
>
>        thread 1                thread 2                 thread 3
>        --------                --------                 --------
>                           write to page P (data=P1)
>                             (cached TLB writable)
>   zap_pte_range()
>     pgtable lock
>     clear pte for page P
>     pgtable unlock
>     ...
>                                                      migrate_vma_collect
>                                                        pte none, npages++, cpages++
>                                                        allocate device page
>                                                        copy data (with P1)
>                                                        map pte as device swap
>                           write to page P again
>                           (data updated from P1->P2)
>   flush tlb
>
> Then at last from processor side P should have data P2 but actually from
> device memory it's P1. Data corrupt.

In the above scenario migrate_vma_collect_pmd() will observe pte_none.
This will mark the src_pfn[] array as needing a new zero page which will
be installed by migrate_vma_pages()->migrate_vma_insert_page().

So there is no data to be copied hence there can't be any data
corruption. Remember these are private anonymous pages, so any
zap_pte_range() indicates the data is no longer needed (eg.
MADV_DONTNEED).

>>
>> When I said I'm not familiar with the code, it's majorly about one thing I
>> never figured out myself, in that migrate_vma_collect_pmd() has this
>> optimization to trylock on the page, collect if it succeeded:
>>
>>   /*
>>    * Optimize for the common case where page is only mapped once
>>    * in one process. If we can lock the page, then we can safely
>>    * set up a special migration page table entry now.
>>    */
>>    if (trylock_page(page)) {
>>           ...
>>    } else {
>>           put_page(page);
>>           mpfn = 0;
>>    }
>>
>> But it's kind of against a pure "optimization" in that if trylock failed,
>> we'll clear the mpfn so the src[i] will be zero at last.  Then will we
>> directly give up on this page, or will we try to lock_page() again
>> somewhere?

That comment is out dated. We used to try locking the page again but
that was removed by ab09243aa95a ("mm/migrate.c: remove
MIGRATE_PFN_LOCKED"). See
https://lkml.kernel.org/r/20211025041608.289017-1-apopple@nvidia.com

Will post a clean-up for it.

>> The future unmap op is also based on this "cpages", not "npages":
>>
>> 	if (args->cpages)
>> 		migrate_vma_unmap(args);
>>
>> So I never figured out how this code really works.  It'll be great if you
>> could shed some light to it.
>>
>> Thanks,
>>
>> --
>> Peter Xu