From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BE25C88CBA for ; Tue, 13 Jun 2023 00:12:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 18C458E0015; Mon, 12 Jun 2023 20:12:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C8A18E000B; Mon, 12 Jun 2023 20:12:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE7498E0015; Mon, 12 Jun 2023 20:12:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BAF868E000B for ; Mon, 12 Jun 2023 20:12:28 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9325014035B for ; Tue, 13 Jun 2023 00:12:28 +0000 (UTC) X-FDA: 80895797976.10.8807531 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf08.hostedemail.com (Postfix) with ESMTP id 62B3C16001F for ; Tue, 13 Jun 2023 00:12:26 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QIwfNVHS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686615146; a=rsa-sha256; cv=none; b=wOxJbmPs6Z4bhb8JqhrdUGll2RY4WpxQCl7ZtFaUH4hcBKvtvTv8qdCMgm0pq8ZhUWZpxi qjSbQ1zztbKRpW1nDQmCVNTaXTOk8a5AvmBpZ88WV7R36JU3Ry3A/2jJKQYxVUebuCSu55 UFq8knrIrXJCF6Dkln/rVsWJbPPM0cA= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QIwfNVHS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686615146; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eSpfbzj81RJHeXXJwuREGCXBeYOUsrYFIe5C3KzovZA=; b=pkXOhsT8nxlZW3byPQmagOqKWJQwvH5IuyAmbN9SnOhJqwasRGpuPMzjzIJMraI/GfjYgd sfTkFF9Ub9J9ccMfXK8xUqJIMugPWBLKRkh6ZOVMjNrX2XMymQo+zGM0BdDnnyx2FCfhG4 ntt1sh84nO07LYYTMs/NgHMqfcT8TKA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686615146; x=1718151146; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EOoWyIFrsGRkjg2/O3hP5P9SjQgryxLlQpl2aAV/598=; b=QIwfNVHSnmdVrL/U9bd6je3EgR5A31QT9ua6exiMSxeJFOJnv+XTXOb4 EYNcpsoFqeepxMtWjAsClgGWdPWjH9JQjxTuMsqmfO5pBHtFHM/MOSSfL PwKeq+TFOH1PVTojZUlh96pKXpKkLO7NBaTVh7iAEItsJtl4soI6nf1yC e/6Leo3s+LGZS3rv6RxpsOnQgKvoViDNlfGNi6ezlgCYFL6ttgtSOzcrU Xb+XzHAK31LHLPN6fjdtBnSswVhTlpUuLnZNgIN9PtkuUeraLKvigY/dC UVPAmHaNGTI7zEtXqhkr/YA2jIgo4nhYve03F8aQ2dVbUF8xw5Fs6bMOh w==; X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="361557128" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="361557128" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="835671048" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="835671048" Received: from almeisch-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.amr.corp.intel.com) ([10.209.42.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:24 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, broonie@kernel.org Cc: rick.p.edgecombe@intel.com, Pengfei Xu Subject: [PATCH v9 20/42] x86/mm: Introduce MAP_ABOVE4G Date: Mon, 12 Jun 2023 17:10:46 -0700 Message-Id: <20230613001108.3040476-21-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> References: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 62B3C16001F X-Stat-Signature: rtde1eb878gg39kuufibu3nyapqn5jb1 X-HE-Tag: 1686615146-607166 X-HE-Meta: U2FsdGVkX1++MOhOb9gDgAcKXvBTHLpI4TSPAb2wG4c3SF9cLJDR/sFgR0iRnamaxyNeN6EbMW9BXI4zooQQgpvepjmYPMo1dWeIXvzWYmL+2ak5fGPWuf0rWP012yclvdHQezILdf9HExQ3gzjG+V8+yal1k7+T9EBerEbTSvg6d0ulVya1qMFVkTSt6X5vPdMMK+7Dw5QvyimdfZM+36rn08ZKut+jLFRRKzxIAvDPJCDsVE+H/NcbVIGEioy/Rs/Ws+jtsrpVfcsO/mFgEgmWd/x4VknaGqkNBtp0lvX951g7cToUyFoDK71hmTydphyq+645AYixsf0BaYdZLulJQz7qpoKOrSK0vJRq4fcecvndYBpuQUgUh/gd4lM/8twdWHt0gnfG30IS+kgd2xJzZT7TGRMOofsHa0Crjj9SWa9luTyWYFPrn6huOh0QT+haEpvfozWJWV1hgf9tokoPk8Y6xMZid1c3HBYj9RPDlCTCaOAuvjDOUz1Y5DWmYm43PasYKV6JoevRGpAnLGa8DpSOvyD6vBtrcnF5R4t80d7KiO4QTDGP0c5ehLuR6mbA1Amer3hmR3+Z1cGuD5ZFXixNhgoQfq1leVocBCDdxJq/tOVzPjyfPetb6x93XqW4MnDfPoUa+dZ7Ay8/UYD+n8M5EUJK1a2NTTXzato3G6ynq3Wk7Xhq2eiT0q/PE7ikigGFW5qKy+zHadEGh3CCSCkUjhW0Mav8xani58059FWFH1ORSbye3XjOO0jL/tBYS0QxI5K9VvZCL5UygWKb5UIjuynazcKVnsJfSsBZ2zAgHRGtG/l1CHhV/3uTkctRY9Nez4JczT4wuSxAbO95kO5jyHJqTbe7Xufy5eXCZ0snm4mihWNd15Y0MgkoZQKwgcL4eTmMP107nUL0Z7qIpY6pQyBPxgHQmhS+TFwx46VyNLLMO+t8/B+2hm09BjlJAdw/d883Oq4FxNE Flo6DVDo gab5wvYY4GsbAhNbu2FVNrKApoREkqq5G0OhvXF20eDX51VEkjlfHQJnYD/SmOmkPBOtB0avT0Ds5MOkF/nngu+X3B7yFOz5xH9prKd23QMZ6NXr8UasaKx+dS39TSqq5Ny8V7B+fIbmtwL9QhKBKb6RbTrjQESeiEBixoPG1qAIvyJpsNI0K1/J2SbvOQYJ6UIGeiNW7zig7e2Z3hTb3rK5voZx3oBBdxNO0jE+Dn/UEmkRG58lyw+SB26tYiYyqJ/+XpdWtX+Jln98Mkf9nJKCnYqKSx1OUojQxwz+AnMdbgnqH9VQRiESyv80m094MHUAteIr8VO9kFBsQMkNn37hvUsKSCDBGi8+AdV0iRuKBQ3o7xgUjvu+nHEC9xHJ7HgsQyW2ACuadc0znSb9MNsEEaWyTztHcUNqczQpf4awc250= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The x86 Control-flow Enforcement Technology (CET) feature includes a new type of memory called shadow stack. This shadow stack memory has some unusual properties, which require some core mm changes to function properly. One of the properties is that the shadow stack pointer (SSP), which is a CPU register that points to the shadow stack like the stack pointer points to the stack, can't be pointing outside of the 32 bit address space when the CPU is executing in 32 bit mode. It is desirable to prevent executing in 32 bit mode when shadow stack is enabled because the kernel can't easily support 32 bit signals. On x86 it is possible to transition to 32 bit mode without any special interaction with the kernel, by doing a "far call" to a 32 bit segment. So the shadow stack implementation can use this address space behavior as a feature, by enforcing that shadow stack memory is always mapped outside of the 32 bit address space. This way userspace will trigger a general protection fault which will in turn trigger a segfault if it tries to transition to 32 bit mode with shadow stack enabled. This provides a clean error generating border for the user if they try attempt to do 32 bit mode shadow stack, rather than leave the kernel in a half working state for userspace to be surprised by. So to allow future shadow stack enabling patches to map shadow stacks out of the 32 bit address space, introduce MAP_ABOVE4G. The behavior is pretty much like MAP_32BIT, except that it has the opposite address range. The are a few differences though. If both MAP_32BIT and MAP_ABOVE4G are provided, the kernel will use the MAP_ABOVE4G behavior. Like MAP_32BIT, MAP_ABOVE4G is ignored in a 32 bit syscall. Since the default search behavior is top down, the normal kaslr base can be used for MAP_ABOVE4G. This is unlike MAP_32BIT which has to add its own randomization in the bottom up case. For MAP_32BIT, only the bottom up search path is used. For MAP_ABOVE4G both are potentially valid, so both are used. In the bottomup search path, the default behavior is already consistent with MAP_ABOVE4G since mmap base should be above 4GB. Without MAP_ABOVE4G, the shadow stack will already normally be above 4GB. So without introducing MAP_ABOVE4G, trying to transition to 32 bit mode with shadow stack enabled would usually segfault anyway. This is already pretty decent guard rails. But the addition of MAP_ABOVE4G is some small complexity spent to make it make it more complete. Signed-off-by: Rick Edgecombe Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- arch/x86/include/uapi/asm/mman.h | 1 + arch/x86/kernel/sys_x86_64.c | 6 +++++- include/linux/mman.h | 4 ++++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 775dbd3aff73..5a0256e73f1e 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -3,6 +3,7 @@ #define _ASM_X86_MMAN_H #define MAP_32BIT 0x40 /* only give out 32bit addresses */ +#define MAP_ABOVE4G 0x80 /* only map above 4GB */ #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS #define arch_calc_vm_prot_bits(prot, key) ( \ diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c index 8cc653ffdccd..c783aeb37dce 100644 --- a/arch/x86/kernel/sys_x86_64.c +++ b/arch/x86/kernel/sys_x86_64.c @@ -193,7 +193,11 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, info.flags = VM_UNMAPPED_AREA_TOPDOWN; info.length = len; - info.low_limit = PAGE_SIZE; + if (!in_32bit_syscall() && (flags & MAP_ABOVE4G)) + info.low_limit = SZ_4G; + else + info.low_limit = PAGE_SIZE; + info.high_limit = get_mmap_base(0); /* diff --git a/include/linux/mman.h b/include/linux/mman.h index cee1e4b566d8..40d94411d492 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -15,6 +15,9 @@ #ifndef MAP_32BIT #define MAP_32BIT 0 #endif +#ifndef MAP_ABOVE4G +#define MAP_ABOVE4G 0 +#endif #ifndef MAP_HUGE_2MB #define MAP_HUGE_2MB 0 #endif @@ -50,6 +53,7 @@ | MAP_STACK \ | MAP_HUGETLB \ | MAP_32BIT \ + | MAP_ABOVE4G \ | MAP_HUGE_2MB \ | MAP_HUGE_1GB) -- 2.34.1