From: Catalin Marinas <catalin.marinas@arm.com>
To: linux-arm-kernel@lists.infradead.org
Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org,
Will Deacon <will@kernel.org>,
Dave P Martin <Dave.Martin@arm.com>,
Vincenzo Frascino <vincenzo.frascino@arm.com>,
Szabolcs Nagy <szabolcs.nagy@arm.com>,
Kevin Brodsky <kevin.brodsky@arm.com>,
Andrey Konovalov <andreyknvl@google.com>,
Peter Collingbourne <pcc@google.com>
Subject: [PATCH v4 05/26] arm64: mte: Handle synchronous and asynchronous tag check faults
Date: Fri, 15 May 2020 18:15:51 +0100 [thread overview]
Message-ID: <20200515171612.1020-6-catalin.marinas@arm.com> (raw)
In-Reply-To: <20200515171612.1020-1-catalin.marinas@arm.com>
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
The Memory Tagging Extension has two modes of notifying a tag check
fault at EL0, configurable through the SCTLR_EL1.TCF0 field:
1. Synchronous raising of a Data Abort exception with DFSC 17.
2. Asynchronous setting of a cumulative bit in TFSRE0_EL1.
Add the exception handler for the synchronous exception and handling of
the asynchronous TFSRE0_EL1.TF0 bit setting via a new TIF flag in
do_notify_resume().
On a tag check failure in user-space, whether synchronous or
asynchronous, a SIGSEGV will be raised on the faulting thread.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
---
Notes:
v4:
- Use send_signal_fault() instead of fault_signal_inject() for
asynchronous tag check faults as execution can continue even if this
signal is masked.
- Add DSB ISH prior to writing TFSRE0_EL1 in the clear_mte_async_tcf
macro.
- Move clear_mte_async_tcf just after returning to user since
do_notify_resume() may still cause async tag faults via do_signal().
v3:
- Asynchronous tag check faults during the uaccess routines in the
kernel are ignored.
- Fix check_mte_async_tcf calling site as it expects the first argument
to be the thread flags.
- Move the mte_thread_switch() definition and call to a later patch as
this became empty with the removal of async uaccess checking.
- Add dsb() and clearing of TFSRE0_EL1 in flush_mte_state(), in case
execve() triggered a asynchronous tag check fault.
- Clear TIF_MTE_ASYNC_FAULT in arch_dup_task_struct() so that the child
does not inherit any pending tag fault in the parent.
v2:
- Clear PSTATE.TCO on exception entry (automatically set by the hardware).
- On syscall entry, for asynchronous tag check faults from user space,
generate the signal early via syscall restarting.
- Before context switch, save any potential async tag check fault
generated by the kernel to the TIF flag (this follows an architecture
update where the uaccess routines use the TCF0 mode).
- Moved the flush_mte_state() and mte_thread_switch() function to a new
mte.c file.
arch/arm64/include/asm/mte.h | 23 +++++++++++++++++
arch/arm64/include/asm/thread_info.h | 4 ++-
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/entry.S | 37 ++++++++++++++++++++++++++++
arch/arm64/kernel/mte.c | 21 ++++++++++++++++
arch/arm64/kernel/process.c | 5 ++++
arch/arm64/kernel/signal.c | 8 ++++++
arch/arm64/kernel/syscall.c | 10 ++++++++
arch/arm64/mm/fault.c | 9 ++++++-
9 files changed, 116 insertions(+), 2 deletions(-)
create mode 100644 arch/arm64/include/asm/mte.h
create mode 100644 arch/arm64/kernel/mte.c
diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h
new file mode 100644
index 000000000000..a0bf310da74b
--- /dev/null
+++ b/arch/arm64/include/asm/mte.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef __ASM_MTE_H
+#define __ASM_MTE_H
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARM64_MTE
+
+void flush_mte_state(void);
+
+#else
+
+static inline void flush_mte_state(void)
+{
+}
+
+#endif
+
+#endif /* __ASSEMBLY__ */
+#endif /* __ASM_MTE_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 512174a8e789..0c6e5523b932 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -63,6 +63,7 @@ void arch_release_task_struct(struct task_struct *tsk);
#define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */
#define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */
#define TIF_FSCHECK 5 /* Check FS is USER_DS on return */
+#define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */
#define TIF_SYSCALL_TRACE 8 /* syscall trace active */
#define TIF_SYSCALL_AUDIT 9 /* syscall auditing */
#define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */
@@ -91,10 +92,11 @@ void arch_release_task_struct(struct task_struct *tsk);
#define _TIF_FSCHECK (1 << TIF_FSCHECK)
#define _TIF_32BIT (1 << TIF_32BIT)
#define _TIF_SVE (1 << TIF_SVE)
+#define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT)
#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \
_TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \
- _TIF_UPROBE | _TIF_FSCHECK)
+ _TIF_UPROBE | _TIF_FSCHECK | _TIF_MTE_ASYNC_FAULT)
#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \
_TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4e5b8ee31442..dbede7a4c5fb 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -63,6 +63,7 @@ obj-$(CONFIG_CRASH_CORE) += crash_core.o
obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
+obj-$(CONFIG_ARM64_MTE) += mte.o
obj-y += vdso/ probes/
obj-$(CONFIG_COMPAT_VDSO) += vdso32/
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index ddcde093c433..cbb3cacdf79f 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -145,6 +145,32 @@ alternative_cb_end
#endif
.endm
+ /* Check for MTE asynchronous tag check faults */
+ .macro check_mte_async_tcf, flgs, tmp
+#ifdef CONFIG_ARM64_MTE
+alternative_if_not ARM64_MTE
+ b 1f
+alternative_else_nop_endif
+ mrs_s \tmp, SYS_TFSRE0_EL1
+ tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f
+ /* Asynchronous TCF occurred for TTBR0 access, set the TI flag */
+ orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT
+ str \flgs, [tsk, #TSK_TI_FLAGS]
+ msr_s SYS_TFSRE0_EL1, xzr
+1:
+#endif
+ .endm
+
+ /* Clear the MTE asynchronous tag check faults */
+ .macro clear_mte_async_tcf
+#ifdef CONFIG_ARM64_MTE
+alternative_if ARM64_MTE
+ dsb ish
+ msr_s SYS_TFSRE0_EL1, xzr
+alternative_else_nop_endif
+#endif
+ .endm
+
.macro kernel_entry, el, regsize = 64
.if \regsize == 32
mov w0, w0 // zero upper 32 bits of x0
@@ -176,6 +202,8 @@ alternative_cb_end
ldr x19, [tsk, #TSK_TI_FLAGS]
disable_step_tsk x19, x20
+ /* Check for asynchronous tag check faults in user space */
+ check_mte_async_tcf x19, x22
apply_ssbd 1, x22, x23
ptrauth_keys_install_kernel tsk, 1, x20, x22, x23
@@ -244,6 +272,13 @@ alternative_if ARM64_HAS_IRQ_PRIO_MASKING
str x20, [sp, #S_PMR_SAVE]
alternative_else_nop_endif
+ /* Re-enable tag checking (TCO set on exception entry) */
+#ifdef CONFIG_ARM64_MTE
+alternative_if ARM64_MTE
+ SET_PSTATE_TCO(0)
+alternative_else_nop_endif
+#endif
+
/*
* Registers that may be useful after this macro is invoked:
*
@@ -748,6 +783,8 @@ ret_to_user:
and x2, x1, #_TIF_WORK_MASK
cbnz x2, work_pending
finish_ret_to_user:
+ /* Ignore asynchronous tag check faults in the uaccess routines */
+ clear_mte_async_tcf
enable_step_tsk x1, x2
#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
bl stackleak_erase
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
new file mode 100644
index 000000000000..032016823957
--- /dev/null
+++ b/arch/arm64/kernel/mte.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+
+#include <linux/thread_info.h>
+
+#include <asm/cpufeature.h>
+#include <asm/mte.h>
+#include <asm/sysreg.h>
+
+void flush_mte_state(void)
+{
+ if (!system_supports_mte())
+ return;
+
+ /* clear any pending asynchronous tag fault */
+ dsb(ish);
+ write_sysreg_s(0, SYS_TFSRE0_EL1);
+ clear_thread_flag(TIF_MTE_ASYNC_FAULT);
+}
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 56be4cbf771f..740047c9cd13 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -50,6 +50,7 @@
#include <asm/exec.h>
#include <asm/fpsimd.h>
#include <asm/mmu_context.h>
+#include <asm/mte.h>
#include <asm/processor.h>
#include <asm/pointer_auth.h>
#include <asm/stacktrace.h>
@@ -323,6 +324,7 @@ void flush_thread(void)
tls_thread_flush();
flush_ptrace_hw_breakpoint(current);
flush_tagged_addr_state();
+ flush_mte_state();
}
void release_thread(struct task_struct *dead_task)
@@ -355,6 +357,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
dst->thread.sve_state = NULL;
clear_tsk_thread_flag(dst, TIF_SVE);
+ /* clear any pending asynchronous tag fault raised by the parent */
+ clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT);
+
return 0;
}
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 339882db5a91..149334d5df02 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -732,6 +732,9 @@ static void setup_return(struct pt_regs *regs, struct k_sigaction *ka,
regs->regs[29] = (unsigned long)&user->next_frame->fp;
regs->pc = (unsigned long)ka->sa.sa_handler;
+ /* TCO (Tag Check Override) always cleared for signal handlers */
+ regs->pstate &= ~PSR_TCO_BIT;
+
if (ka->sa.sa_flags & SA_RESTORER)
sigtramp = ka->sa.sa_restorer;
else
@@ -923,6 +926,11 @@ asmlinkage void do_notify_resume(struct pt_regs *regs,
if (thread_flags & _TIF_UPROBE)
uprobe_notify_resume(regs);
+ if (thread_flags & _TIF_MTE_ASYNC_FAULT) {
+ clear_thread_flag(TIF_MTE_ASYNC_FAULT);
+ send_sig_fault(SIGSEGV, SEGV_MTEAERR, 0, current);
+ }
+
if (thread_flags & _TIF_SIGPENDING)
do_signal(regs);
diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c
index a12c0c88d345..db25f5d6a07c 100644
--- a/arch/arm64/kernel/syscall.c
+++ b/arch/arm64/kernel/syscall.c
@@ -102,6 +102,16 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
local_daif_restore(DAIF_PROCCTX);
user_exit();
+ if (system_supports_mte() && (flags & _TIF_MTE_ASYNC_FAULT)) {
+ /*
+ * Process the asynchronous tag check fault before the actual
+ * syscall. do_notify_resume() will send a signal to userspace
+ * before the syscall is restarted.
+ */
+ regs->regs[0] = -ERESTARTNOINTR;
+ return;
+ }
+
if (has_syscall_work(flags)) {
/* set default errno for user-issued syscall(-1) */
if (scno == NO_SYSCALL)
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index c9cedc0432d2..38b59cace3e3 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -650,6 +650,13 @@ static int do_sea(unsigned long addr, unsigned int esr, struct pt_regs *regs)
return 0;
}
+static int do_tag_check_fault(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ do_bad_area(addr, esr, regs);
+ return 0;
+}
+
static const struct fault_info fault_info[] = {
{ do_bad, SIGKILL, SI_KERNEL, "ttbr address size fault" },
{ do_bad, SIGKILL, SI_KERNEL, "level 1 address size fault" },
@@ -668,7 +675,7 @@ static const struct fault_info fault_info[] = {
{ do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" },
{ do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" },
{ do_sea, SIGBUS, BUS_OBJERR, "synchronous external abort" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 17" },
+ { do_tag_check_fault, SIGSEGV, SEGV_MTESERR, "synchronous tag check fault" },
{ do_bad, SIGKILL, SI_KERNEL, "unknown 18" },
{ do_bad, SIGKILL, SI_KERNEL, "unknown 19" },
{ do_sea, SIGKILL, SI_KERNEL, "level 0 (translation table walk)" },
next prev parent reply other threads:[~2020-05-15 17:16 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-15 17:15 [PATCH v4 00/26] arm64: Memory Tagging Extension user-space support Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 01/26] arm64: mte: system register definitions Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 02/26] arm64: mte: CPU feature detection and initial sysreg configuration Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 03/26] arm64: mte: Use Normal Tagged attributes for the linear map Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 04/26] arm64: mte: Add specific SIGSEGV codes Catalin Marinas
2020-05-15 17:15 ` Catalin Marinas [this message]
2020-05-15 17:15 ` [PATCH v4 06/26] mm: Add PG_ARCH_2 page flag Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 07/26] arm64: mte: Clear the tags when a page is mapped in user-space with PROT_MTE Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 08/26] arm64: mte: Tags-aware copy_page() implementation Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 09/26] arm64: mte: Tags-aware aware memcmp_pages() implementation Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 10/26] mm: Introduce arch_calc_vm_flag_bits() Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 11/26] arm64: mte: Add PROT_MTE support to mmap() and mprotect() Catalin Marinas
2020-05-27 18:57 ` Peter Collingbourne
2020-05-28 9:14 ` Catalin Marinas
2020-05-28 11:05 ` Szabolcs Nagy
2020-05-28 16:34 ` Catalin Marinas
2020-05-28 18:35 ` Evgenii Stepanov
2020-05-29 11:19 ` Catalin Marinas
2020-06-01 8:55 ` Dave Martin
2020-06-01 14:45 ` Catalin Marinas
2020-06-01 15:04 ` Dave Martin
2020-05-15 17:15 ` [PATCH v4 12/26] mm: Introduce arch_validate_flags() Catalin Marinas
2020-05-15 17:15 ` [PATCH v4 13/26] arm64: mte: Validate the PROT_MTE request via arch_validate_flags() Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 14/26] mm: Allow arm64 mmap(PROT_MTE) on RAM-based files Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 15/26] arm64: mte: Allow user control of the tag check mode via prctl() Catalin Marinas
2020-05-27 7:46 ` Will Deacon
2020-05-27 8:32 ` Dave Martin
2020-05-27 8:48 ` Will Deacon
2020-05-27 11:16 ` Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 16/26] arm64: mte: Allow user control of the generated random tags " Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 17/26] arm64: mte: Restore the GCR_EL1 register after a suspend Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 18/26] arm64: mte: Add PTRACE_{PEEK,POKE}MTETAGS support Catalin Marinas
2020-05-29 21:25 ` Luis Machado
2020-06-01 12:07 ` Catalin Marinas
2020-06-01 15:17 ` Luis Machado
2020-06-01 16:33 ` Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 19/26] fs: Handle intra-page faults in copy_mount_options() Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 20/26] mm: Add arch hooks for saving/restoring tags Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 21/26] arm64: mte: Enable swap of tagged pages Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 22/26] arm64: mte: Save tags when hibernating Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 23/26] arm64: mte: Check the DT memory nodes for MTE support Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 24/26] arm64: mte: Introduce early param to disable " Catalin Marinas
2020-05-18 11:26 ` Vladimir Murzin
2020-05-18 11:31 ` Will Deacon
2020-05-18 17:20 ` Catalin Marinas
2020-05-22 5:57 ` Patrick Daly
2020-05-22 10:37 ` Catalin Marinas
2020-05-27 2:11 ` Patrick Daly
2020-05-27 9:55 ` Will Deacon
2020-05-27 10:37 ` Szabolcs Nagy
2020-05-27 11:12 ` Catalin Marinas
2020-05-19 16:14 ` Catalin Marinas
2021-01-21 19:37 ` Andrey Konovalov
2021-01-22 2:03 ` Andrey Konovalov
2021-01-22 14:41 ` Catalin Marinas
2021-01-22 17:28 ` Andrey Konovalov
2020-05-15 17:16 ` [PATCH v4 25/26] arm64: mte: Kconfig entry Catalin Marinas
2020-05-15 17:16 ` [PATCH v4 26/26] arm64: mte: Add Memory Tagging Extension documentation Catalin Marinas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200515171612.1020-6-catalin.marinas@arm.com \
--to=catalin.marinas@arm.com \
--cc=Dave.Martin@arm.com \
--cc=andreyknvl@google.com \
--cc=kevin.brodsky@arm.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-mm@kvack.org \
--cc=pcc@google.com \
--cc=szabolcs.nagy@arm.com \
--cc=vincenzo.frascino@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox