* [PATCH 11/24] x86/mm: Give each mm TLB flush generation a unique ID [not found] <1524021512-24022-1-git-send-email-youquan.song@intel.com> @ 2018-04-18 3:18 ` Youquan Song 0 siblings, 0 replies; 2+ messages in thread From: Youquan Song @ 2018-04-18 3:18 UTC (permalink / raw) To: stable, gregkh Cc: tim.c.chen, ashok.raj, dave.hansen, yi.y.sun, youquan.song, youquan.song, Andy Lutomirski, Andrew Morton, Arjan van de Ven, Borislav Petkov, Linus Torvalds, Mel Gorman, Peter Zijlstra, Rik van Riel, linux-mm From: Andy Lutomirski <luto@kernel.org> (cherry picked from commit f39681ed0f48498b80455095376f11535feea332) This adds two new variables to mmu_context_t: ctx_id and tlb_gen. ctx_id uniquely identifies the mm_struct and will never be reused. For a given mm_struct (and hence ctx_id), tlb_gen is a monotonic count of the number of times that a TLB flush has been requested. The pair (ctx_id, tlb_gen) can be used as an identifier for TLB flush actions and will be used in subsequent patches to reliably determine whether all needed TLB flushes have occurred on a given CPU. This patch is split out for ease of review. By itself, it has no real effect other than creating and updating the new variables. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/413a91c24dab3ed0caa5f4e4d017d87b0857f920.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Youquan Song <youquan.song@linux.intel.com> [v4.4 backport] --- arch/x86/include/asm/mmu.h | 15 +++++++++++++-- arch/x86/include/asm/mmu_context.h | 4 ++++ arch/x86/kernel/ldt.c | 1 + arch/x86/mm/tlb.c | 2 ++ 4 files changed, 20 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 7680b76..3359dfe 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -3,12 +3,18 @@ #include <linux/spinlock.h> #include <linux/mutex.h> +#include <linux/atomic.h> /* - * The x86 doesn't have a mmu context, but - * we put the segment information here. + * x86 has arch-specific MMU state beyond what lives in mm_struct. */ typedef struct { + /* + * ctx_id uniquely identifies this mm_struct. A ctx_id will never + * be reused, and zero is not a valid ctx_id. + */ + u64 ctx_id; + #ifdef CONFIG_MODIFY_LDT_SYSCALL struct ldt_struct *ldt; #endif @@ -24,6 +30,11 @@ typedef struct { atomic_t perf_rdpmc_allowed; /* nonzero if rdpmc is allowed */ } mm_context_t; +#define INIT_MM_CONTEXT(mm) \ + .context = { \ + .ctx_id = 1, \ + } + void leave_mm(int cpu); #endif /* _ASM_X86_MMU_H */ diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 9bfc5fd..24bc41a 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -11,6 +11,9 @@ #include <asm/tlbflush.h> #include <asm/paravirt.h> #include <asm/mpx.h> + +extern atomic64_t last_mm_ctx_id; + #ifndef CONFIG_PARAVIRT static inline void paravirt_activate_mm(struct mm_struct *prev, struct mm_struct *next) @@ -58,6 +61,7 @@ void destroy_context(struct mm_struct *mm); static inline int init_new_context(struct task_struct *tsk, struct mm_struct *mm) { + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); return 0; } static inline void destroy_context(struct mm_struct *mm) {} diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c index bc42936..323590e 100644 --- a/arch/x86/kernel/ldt.c +++ b/arch/x86/kernel/ldt.c @@ -125,6 +125,7 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm) struct mm_struct *old_mm; int retval = 0; + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); mutex_init(&mm->context.lock); old_mm = current->mm; if (!old_mm) { diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 7cad01af..efec198 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -29,6 +29,8 @@ * Implement flush IPI by CALL_FUNCTION_VECTOR, Alex Shi */ +atomic64_t last_mm_ctx_id = ATOMIC64_INIT(1); + struct flush_tlb_info { struct mm_struct *flush_mm; unsigned long flush_start; -- 1.8.3.1 ^ permalink raw reply [flat|nested] 2+ messages in thread
[parent not found: <1523939240-16508-1-git-send-email-youquan.song@intel.com>]
* [PATCH 11/24] x86/mm: Give each mm TLB flush generation a unique ID [not found] <1523939240-16508-1-git-send-email-youquan.song@intel.com> @ 2018-04-17 4:27 ` Youquan Song 0 siblings, 0 replies; 2+ messages in thread From: Youquan Song @ 2018-04-17 4:27 UTC (permalink / raw) To: stable, gregkh Cc: tim.c.chen, ashok.raj, dave.hansen, yi.y.sun, youquan.song, youquan.song, Andy Lutomirski, Andrew Morton, Arjan van de Ven, Borislav Petkov, Linus Torvalds, Mel Gorman, Peter Zijlstra, Rik van Riel, linux-mm From: Andy Lutomirski <luto@kernel.org> (cherry picked from commit f39681ed0f48498b80455095376f11535feea332) This adds two new variables to mmu_context_t: ctx_id and tlb_gen. ctx_id uniquely identifies the mm_struct and will never be reused. For a given mm_struct (and hence ctx_id), tlb_gen is a monotonic count of the number of times that a TLB flush has been requested. The pair (ctx_id, tlb_gen) can be used as an identifier for TLB flush actions and will be used in subsequent patches to reliably determine whether all needed TLB flushes have occurred on a given CPU. This patch is split out for ease of review. By itself, it has no real effect other than creating and updating the new variables. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/413a91c24dab3ed0caa5f4e4d017d87b0857f920.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Youquan Song <youquan.song@linux.intel.com> [v4.4 backport] --- arch/x86/include/asm/mmu.h | 15 +++++++++++++-- arch/x86/include/asm/mmu_context.h | 4 ++++ arch/x86/kernel/ldt.c | 1 + arch/x86/mm/tlb.c | 2 ++ 4 files changed, 20 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 7680b76..3359dfe 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -3,12 +3,18 @@ #include <linux/spinlock.h> #include <linux/mutex.h> +#include <linux/atomic.h> /* - * The x86 doesn't have a mmu context, but - * we put the segment information here. + * x86 has arch-specific MMU state beyond what lives in mm_struct. */ typedef struct { + /* + * ctx_id uniquely identifies this mm_struct. A ctx_id will never + * be reused, and zero is not a valid ctx_id. + */ + u64 ctx_id; + #ifdef CONFIG_MODIFY_LDT_SYSCALL struct ldt_struct *ldt; #endif @@ -24,6 +30,11 @@ typedef struct { atomic_t perf_rdpmc_allowed; /* nonzero if rdpmc is allowed */ } mm_context_t; +#define INIT_MM_CONTEXT(mm) \ + .context = { \ + .ctx_id = 1, \ + } + void leave_mm(int cpu); #endif /* _ASM_X86_MMU_H */ diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 9bfc5fd..24bc41a 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -11,6 +11,9 @@ #include <asm/tlbflush.h> #include <asm/paravirt.h> #include <asm/mpx.h> + +extern atomic64_t last_mm_ctx_id; + #ifndef CONFIG_PARAVIRT static inline void paravirt_activate_mm(struct mm_struct *prev, struct mm_struct *next) @@ -58,6 +61,7 @@ void destroy_context(struct mm_struct *mm); static inline int init_new_context(struct task_struct *tsk, struct mm_struct *mm) { + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); return 0; } static inline void destroy_context(struct mm_struct *mm) {} diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c index bc42936..323590e 100644 --- a/arch/x86/kernel/ldt.c +++ b/arch/x86/kernel/ldt.c @@ -125,6 +125,7 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm) struct mm_struct *old_mm; int retval = 0; + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); mutex_init(&mm->context.lock); old_mm = current->mm; if (!old_mm) { diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 7cad01af..efec198 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -29,6 +29,8 @@ * Implement flush IPI by CALL_FUNCTION_VECTOR, Alex Shi */ +atomic64_t last_mm_ctx_id = ATOMIC64_INIT(1); + struct flush_tlb_info { struct mm_struct *flush_mm; unsigned long flush_start; -- 1.8.3.1 ^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2018-04-18 3:19 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1524021512-24022-1-git-send-email-youquan.song@intel.com>
2018-04-18 3:18 ` [PATCH 11/24] x86/mm: Give each mm TLB flush generation a unique ID Youquan Song
[not found] <1523939240-16508-1-git-send-email-youquan.song@intel.com>
2018-04-17 4:27 ` Youquan Song
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox