[PATCH 3/9] x86/mm: Address feedback

[Peter Zijlstra <peterz@infradead.org>]

[-- Attachment #1: peterz-tlb-invalidate-other-fix.patch --]
[-- Type: text/plain, Size: 2007 bytes --]

fold into: ("x86/mm: Allow flushing for future ASID switches")

Andy asked for the KPTI check to be pulled out of the API such that
the function always does as advertised.

Rename to: invalidate_other_asid() because the pcid name is actually
wrong if we consider our ASID to represend two PCID values (as it does
with KPTI). In specific, it will not flush the user PCID of the
current ASID.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/include/asm/tlbflush.h |   24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -270,16 +270,11 @@ static inline unsigned long cr4_read_sha
 	return this_cpu_read(cpu_tlbstate.cr4);
 }
 
-static inline void invalidate_pcid_other(void)
+/*
+ * Mark all other ASIDs as invalid, preserves the current.
+ */
+static inline void invalidate_other_asid(void)
 {
-	/*
-	 * With global pages, all of the shared kenel page tables
-	 * are set as _PAGE_GLOBAL.  We have no shared nonglobals
-	 * and nothing to do here.
-	 */
-	if (!static_cpu_has_bug(X86_BUG_CPU_SECURE_MODE_KPTI))
-		return;
-
 	this_cpu_write(cpu_tlbstate.invalidate_other, true);
 }
 
@@ -411,11 +406,16 @@ static inline void __flush_tlb_one(unsig
 {
 	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
 	__flush_tlb_single(addr);
+
+	if (!static_cpu_has_bug(X86_BUG_CPU_SECURE_MODE_KPTI))
+		return;
+
 	/*
-	 * Invalidate other address spaces inaccessible to single-page
-	 * invalidation:
+	 * __flush_tlb_single() will have cleared the TLB entry for this ASID,
+	 * but since kernel space is replicated across all, we must also
+	 * invalidate all others.
 	 */
-	invalidate_pcid_other();
+	invalidate_other_asid();
 }
 
 #define TLB_FLUSH_ALL	-1UL


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>