Question about ptep_get_and_clear and TLB flush

[Joel Fernandes <joelaf@google.com>]

Hello friends,
I was trying to understand the safety of this piece of code in
move_ptes in mremap.c
Here we have some code that does this in a loop:

for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE,
 new_pte++, new_addr += PAGE_SIZE) {
  if (pte_none(*old_pte))
       continue;
    pte = ptep_get_and_clear(mm, old_addr, old_pte);
    if (pte_present(pte) && pte_dirty(pte))
         force_flush = true;
    pte = move_pte(pte, new_vma->vm_page_prot, old_addr, new_addr);
    pte = move_soft_dirty_pte(pte);
    set_pte_at(mm, new_addr, new_pte, pte);
}

If I understand correctly, the ptep_get_and_clear is needed to
atomically get and clear the page table entry so that we do not miss
any other bits in PTE that may get set but have not been read, before
we clear it. Such as the dirty bit.

My question is, After the ptep_get_and_clear runs, what happens if
another CPU has a valid TLB entry for this old_addr and does a
memory-write *before* the TLBs are flushed. Would that not cause us to
lose the dirty bit? Once set_pte_at runs, it would be using the PTE
fetched earlier which did not have the dirty bit set. This seems wrong
to me. What do you think?

Thanks,
Joel