From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl0-f70.google.com (mail-pl0-f70.google.com [209.85.160.70]) by kanga.kvack.org (Postfix) with ESMTP id 219E16B002E for ; Mon, 2 Apr 2018 13:29:56 -0400 (EDT) Received: by mail-pl0-f70.google.com with SMTP id w9-v6so3892115plp.0 for ; Mon, 02 Apr 2018 10:29:56 -0700 (PDT) Received: from mga07.intel.com (mga07.intel.com. [134.134.136.100]) by mx.google.com with ESMTPS id v8-v6si745263plg.68.2018.04.02.10.29.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 10:29:55 -0700 (PDT) Subject: [PATCH 09/11] x86/pti: enable global pages for shared areas From: Dave Hansen Date: Mon, 02 Apr 2018 10:27:13 -0700 References: <20180402172700.65CAE838@viggo.jf.intel.com> In-Reply-To: <20180402172700.65CAE838@viggo.jf.intel.com> Message-Id: <20180402172713.B7D6F0C0@viggo.jf.intel.com> Sender: owner-linux-mm@kvack.org List-ID: To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Dave Hansen , aarcange@redhat.com, luto@kernel.org, torvalds@linux-foundation.org, keescook@google.com, hughd@google.com, jgross@suse.com, x86@kernel.org, namit@vmware.com From: Dave Hansen The entry/exit text and cpu_entry_area are mapped into userspace and the kernel. But, they are not _PAGE_GLOBAL. This creates unnecessary TLB misses. Add the _PAGE_GLOBAL flag for these areas. Signed-off-by: Dave Hansen Cc: Andrea Arcangeli Cc: Andy Lutomirski Cc: Linus Torvalds Cc: Kees Cook Cc: Hugh Dickins Cc: Juergen Gross Cc: x86@kernel.org Cc: Nadav Amit --- b/arch/x86/mm/cpu_entry_area.c | 10 +++++++++- b/arch/x86/mm/pti.c | 10 +++++++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff -puN arch/x86/mm/cpu_entry_area.c~kpti-why-no-global arch/x86/mm/cpu_entry_area.c --- a/arch/x86/mm/cpu_entry_area.c~kpti-why-no-global 2018-04-02 10:26:47.185661207 -0700 +++ b/arch/x86/mm/cpu_entry_area.c 2018-04-02 10:26:47.190661207 -0700 @@ -27,8 +27,16 @@ EXPORT_SYMBOL(get_cpu_entry_area); void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags) { unsigned long va = (unsigned long) cea_vaddr; + pte_t pte = pfn_pte(pa >> PAGE_SHIFT, flags); - set_pte_vaddr(va, pfn_pte(pa >> PAGE_SHIFT, flags)); + /* + * The cpu_entry_area is shared between the user and kernel + * page tables. All of its ptes can safely be global. + */ + if (boot_cpu_has(X86_FEATURE_PGE)) + pte = pte_set_flags(pte, _PAGE_GLOBAL); + + set_pte_vaddr(va, pte); } static void __init diff -puN arch/x86/mm/pti.c~kpti-why-no-global arch/x86/mm/pti.c --- a/arch/x86/mm/pti.c~kpti-why-no-global 2018-04-02 10:26:47.187661207 -0700 +++ b/arch/x86/mm/pti.c 2018-04-02 10:26:47.191661207 -0700 @@ -300,6 +300,14 @@ pti_clone_pmds(unsigned long start, unsi return; /* + * Setting 'target_pmd' below creates a mapping in both + * the user and kernel page tables. It is effectively + * global, so set it as global in both copies. + */ + if (boot_cpu_has(X86_FEATURE_PGE)) + *pmd = pmd_set_flags(*pmd, _PAGE_GLOBAL); + + /* * Copy the PMD. That is, the kernelmode and usermode * tables will share the last-level page tables of this * address range @@ -348,7 +356,7 @@ static void __init pti_clone_entry_text( { pti_clone_pmds((unsigned long) __entry_text_start, (unsigned long) __irqentry_text_end, - _PAGE_RW | _PAGE_GLOBAL); + _PAGE_RW); } /* _