Re: Splitting the mmap_sem

[Vlastimil Babka <vbabka@suse.cz>]

On 12/10/19 5:07 PM, Jerome Glisse wrote:
> On Tue, Dec 10, 2019 at 04:26:40PM +0100, Vlastimil Babka wrote:
>> On 12/5/19 6:21 PM, Jerome Glisse wrote:
>>>>
>>>> So calling mmap() looks like this:
>>>>
>>>>       1 allocate a new VMA
>>>>       2 update pointer(s) in maple tree
>>>>       3 sleep until old VMAs have a zero refcount
>>>>       4 synchronize_rcu()
>>>>       5 free old VMAs
>>>>       6 flush caches for affected range
>>>>       7 return to userspace
>>>>
>>>> While one thread is calling mmap(MAP_FIXED), two other threads which are
>>>> accessing the same address may see different data from each other and
>>>> have different page translations in their respective CPU caches until
>>>> the thread calling mmap() returns.  I believe this is OK, but would
>>>> greatly appreciate hearing from people who know better.
>>>
>>> I do not believe this is OK, i believe this is wrong (not even considering
>>> possible hardware issues that can arise from such aliasing).
>>
>> But is it true that the races can happen in the above such that multiple CPU's
>> have different translations? I think it's impossible to tell from above - there
>> are no details about when and which pte modifications happen, where ptl lock is
>> taken... perhaps after filling those details, we could be able to see that
>> there's no race.
>>
> 
> My assumption reading Matthew was that as step 6 is making progress
> (flushing caches and i assume TLB too) then you can have a CPU which
> is already flushed and that do take a fault against the new VMA and
> thus get a new TLB entry that do not match a CPU which is not yet
> flushed.

We already have to protect against CPU's that access page tables (and
thus fill their TLBs) by hardware, taking no mmap_sem at all. For the
mmap(MAP_FIXED) overwriting existing mapping case this means that the
old stuff will first be munmapped - zap_pte_range() will take pte lock,
clear pte and flush tlb's before installing any new mappings.
A parallel fault trying to install pte for the new VMA should thus
serialize on the pte lock and only install the new pte after everyone
was flushed, and this should be fine?

What might IMHO perhaps need care is a parallel fault that started with
the old VMA, because some PTE was unpopulated. We need to make sure it
doesn't end up being the last to install the PTE for the VMA that's
going away.

> Today this can not happens because page fault will serialize on the
> mmap_sem (ie until the write mode is release when returning to user-
> space).