[v3 PATCH] mm/memory.c: skip spurious TLB flush for retried page fault

[v3 PATCH] mm/memory.c: skip spurious TLB flush for retried page fault

[Yang Shi]

Recently we found regression when running will_it_scale/page_fault3 test
on ARM64.  Over 70% down for the multi processes cases and over 20% down
for the multi threads cases.  It turns out the regression is caused by
commit 89b15332af7c0312a41e50846819ca6613b58b4c ("mm: drop mmap_sem before
calling balance_dirty_pages() in write fault").

The test mmaps a memory size file then write to the mapping, this would
make all memory dirty and trigger dirty pages throttle, that upstream
commit would release mmap_sem then retry the page fault.  The retried page
fault would see correct PTEs installed then just fall through to spurious TLB
flush.  The regression is caused by the excessive spurious TLB flush.  It is
fine on x86 since x86's spurious TLB flush is no-op.

We could just skip the spurious TLB flush to mitigate the regression.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reported-by: Xu Yu <xuyu@linux.alibaba.com>
Debugged-by: Xu Yu <xuyu@linux.alibaba.com>
Tested-by: Xu Yu <xuyu@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Yang Shi <shy828301@gmail.com>
---
v3: Incorporated Linus's suggestion
v2: Incorporated Will Deacon's suggestion

 mm/memory.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/memory.c b/mm/memory.c
index 3a7779d9891d..602f4283122f 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4247,6 +4247,9 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
 				vmf->flags & FAULT_FLAG_WRITE)) {
 		update_mmu_cache(vmf->vma, vmf->address, vmf->pte);
 	} else {
+		/* Skip spurious TLB flush for retried page fault */
+		if (vmf->flags & FAULT_FLAG_TRIED)
+			goto unlock;
 		/*
 		 * This is needed only for protection faults but the arch code
 		 * is not yet telling us if this is a protection fault or not.
-- 
2.26.2

Re: [v3 PATCH] mm/memory.c: skip spurious TLB flush for retried page fault

[Linus Torvalds]

On Mon, Aug 17, 2020 at 2:04 PM Yang Shi <shy828301@gmail.com> wrote:
>
> We could just skip the spurious TLB flush to mitigate the regression.

Ok, this patch I will apply.

I still hope that arm64 fixes (maybe already fixed) their spurious TLB
function, and I think we should rename it to make sure everybody
understands it's local, but in the meantime this patch hides the
regression and isn't wrong.

Thanks,

                Linus

Re: [v3 PATCH] mm/memory.c: skip spurious TLB flush for retried page fault

[Will Deacon]

On Tue, Aug 18, 2020 at 12:01:43PM -0700, Linus Torvalds wrote:
> On Mon, Aug 17, 2020 at 2:04 PM Yang Shi <shy828301@gmail.com> wrote:
> >
> > We could just skip the spurious TLB flush to mitigate the regression.
> 
> Ok, this patch I will apply.

Cheers.

> I still hope that arm64 fixes (maybe already fixed) their spurious TLB
> function, and I think we should rename it to make sure everybody
> understands it's local, but in the meantime this patch hides the
> regression and isn't wrong.

I'll look at it for 5.10.

Will