Re: [PATCH v2] mmu_gather: move tlb flush for VM_PFNMAP/VM_MIXEDMAP vmas into free_pgtables()

[Roman Gushchin <roman.gushchin@linux.dev>]

On Wed, Jan 22, 2025 at 11:42:13PM -0800, Hugh Dickins wrote:
> On Wed, 22 Jan 2025, Roman Gushchin wrote:
> 
> > Commit b67fbebd4cf9 ("mmu_gather: Force tlb-flush VM_PFNMAP vmas")
> > added a forced tlbflush to tlb_vma_end(), which is required to avoid a
> > race between munmap() and unmap_mapping_range(). However it added some
> > overhead to other paths where tlb_vma_end() is used, but vmas are not
> > removed, e.g. madvise(MADV_DONTNEED).
> > 
> > Fix this by moving the tlb flush out of tlb_end_vma() into
> > free_pgtables(), somewhat similar to the stable version of the
> > original commit: e.g. stable commit 895428ee124a ("mm: Force TLB flush
> > for PFNMAP mappings before unlink_file_vma()").
> > 
> > Note, that if tlb->fullmm is set, no flush is required, as the whole
> > mm is about to be destroyed.
> > 
> > v2:
> >   - moved vma_pfn flag handling into tlb.h (by Peter Z.)
> >   - added comments (by Peter Z.)
> >   - fixed the vma_pfn flag setting (by Hugh D.)
> > 
> > Suggested-by: Jann Horn <jannh@google.com>
> > Signed-off-by: Roman Gushchin <roman.gushchin@linux.dev>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Will Deacon <will@kernel.org>
> > Cc: "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Nick Piggin <npiggin@gmail.com>
> > Cc: Hugh Dickins <hughd@google.com>
> > Cc: linux-arch@vger.kernel.org
> > Cc: linux-mm@kvack.org
> > ---
> >  include/asm-generic/tlb.h | 41 ++++++++++++++++++++++++++-------------
> >  mm/memory.c               |  7 +++++++
> >  2 files changed, 35 insertions(+), 13 deletions(-)
> > 
> > diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
> > index 709830274b75..fbe31f49a5af 100644
> > --- a/include/asm-generic/tlb.h
> > +++ b/include/asm-generic/tlb.h
> > @@ -449,7 +449,14 @@ tlb_update_vma_flags(struct mmu_gather *tlb, struct vm_area_struct *vma)
> >  	 */
> >  	tlb->vma_huge = is_vm_hugetlb_page(vma);
> >  	tlb->vma_exec = !!(vma->vm_flags & VM_EXEC);
> > -	tlb->vma_pfn  = !!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP));
> > +
> > +	/*
> > +	 * vma_pfn is checked and cleared by tlb_flush_mmu_pfnmap()
> > +	 * for a set of vma's, so it should be set if at least one vma
> > +	 * has VM_PFNMAP or VM_MIXEDMAP flags set.
> > +	 */
> > +	if (vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))
> > +		tlb->vma_pfn = 1;
> 
> Okay, but struct mmu_gather is usually on the caller's stack,
> containing junk initially, and there's nothing in this patch
> yet to initialize tlb->vma_pfn to 0.
> 
> __tlb_reset_range() needs to do that, doesn't it?  With some adjustment
> to its comment about not resetting mmu_gather::vma_* fields.  Or would
> it be better to get around that by renaming vma_pfn to, er, something
> else - I'd have to understand the essence of Jann's race better to
> come up with the right name - untracked_mappings? would that be right?
> I still haven't grasped the essence of that race.

Yeah, this is a really good point.

> 
> (Panic attack: where is, for example, tlb->need_flush_all initialized
> to 0?  Ah, over in mm/mmu_gather.c, __tlb_gather_mmu().  Phew.)
> 
> And if __tlb_reset_range() resets tlb->vma_pfn to 0, then that has the
> side-effect that any TLB flush cancels the vma_pfn state: which is a
> desirable side-effect, isn't it? avoiding the possibility of doing an
> unnecessary extra TLB flush in free_pgtables(), as I criticized before.

Good point.

Just sent out v3 with your suggestions.

Thank you for your help here!

Roman