From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AFC1C54E5D for ; Tue, 12 Mar 2024 22:29:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0448B8E0024; Tue, 12 Mar 2024 18:29:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0E668E0025; Tue, 12 Mar 2024 18:29:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC0BF8E0024; Tue, 12 Mar 2024 18:29:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B2EA98E0011 for ; Tue, 12 Mar 2024 18:29:10 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6E57EA0740 for ; Tue, 12 Mar 2024 22:29:10 +0000 (UTC) X-FDA: 81889828860.09.87C4D4D Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) by imf24.hostedemail.com (Postfix) with ESMTP id 6109518000C for ; Tue, 12 Mar 2024 22:29:08 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hCZuqNvY; spf=pass (imf24.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710282548; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=heTClq2fhft0JrBvJbtw5Y/342RH8KsSHyD4E4sYLCU=; b=m6ae8UGTJZQHEPbkIeqt4sP378Ny5yyYjh8ntstG/zbwLKhvYKvzXW+ZamjGDNN3exlh4o bAIm9t2vjh6DbbWzbS7YJdCeiWNkD4k7aoQdeZBl0v14h6bpbQMcDki8mkXkba/HvdLUzc RRHpSstkECirrGG4zqkM3Pune3Q20HI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710282548; a=rsa-sha256; cv=none; b=oSaIY1IWAFbJ25zzYN5Cm/ow17sYi2pFtMsBzVfJDE7hPMS8V5cOfZzU3eitWF/s6d3BtK pcHJVwjRi7BtH/aQqSjUUJtukmMtTxZf4qzLGnhfNji0bwLHhIdcNPnR7s+6/pWQXOXJJ0 qx4Ytj23wFDl4le8+1bEHce5gZo5Dv8= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hCZuqNvY; spf=pass (imf24.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710282549; x=1741818549; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/il3sOjTRp5Oqe4q9Enzhux+EAXqchUHAcOIqSHR2bA=; b=hCZuqNvYrqgHDYc5shfPIgNq2P6v8mdHMJOvZVGQVHThSZO4k4otXBhS 0zaJcT/RvK9jmfXszQtgHSlVEjT/YFtXzWdhpOB+3kaSmRsmyvxjlx6ti /z11ZKi1IZ2XFmC+JaXi3KSyyJOnVW5hL+aC1TlqAUL7pvYUsBuCkJD23 vvyULm4Un+g6RWl3jaxn7S0ttU6Mhgc5UzhmXOWkKaK8eUAlgB46k34pZ DeqH5beqsuErDiM4032IbAWCwMaIktwCCijdeO0sINuhTjoJtx5eLIVkD M4deWUGQ85pQ3cAT0lc0SjwVoSO7i0A3a/KKUaSvRvROEZ/a0shFjGmVq g==; X-IronPort-AV: E=McAfee;i="6600,9927,11011"; a="5191943" X-IronPort-AV: E=Sophos;i="6.07,119,1708416000"; d="scan'208";a="5191943" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2024 15:29:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,119,1708416000"; d="scan'208";a="16356836" Received: from gargayus-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.intel.com) ([10.255.231.196]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2024 15:29:02 -0700 From: Rick Edgecombe To: Liam.Howlett@oracle.com, akpm@linux-foundation.org, bp@alien8.de, broonie@kernel.org, dave.hansen@linux.intel.com, debug@rivosinc.com, hpa@zytor.com, keescook@chromium.org, kirill.shutemov@linux.intel.com, luto@kernel.org, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, x86@kernel.org, christophe.leroy@csgroup.eu Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rick.p.edgecombe@intel.com Subject: [PATCH v3 03/12] mm: Use get_unmapped_area_vmflags() Date: Tue, 12 Mar 2024 15:28:34 -0700 Message-Id: <20240312222843.2505560-4-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240312222843.2505560-1-rick.p.edgecombe@intel.com> References: <20240312222843.2505560-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 6109518000C X-Rspam-User: X-Stat-Signature: twip3w818nxi4fz7cnc8ipiq97opcou1 X-Rspamd-Server: rspam03 X-HE-Tag: 1710282548-45846 X-HE-Meta: U2FsdGVkX1+IdAdc48g3x3xMiEjy1wZO9aRHBGHSOSP8VyqCJuJ8hzH/OG85CTMRe6vdjwla6hHB8mz3BlUJ46Ke14onkrLypM8f+P2XNmX12nJjGlSr9whrPEvsu7bK6GrTl4+fTzRxHYLmPiMK5LJWE2K+lE1xB4fMKAGwoirc7SB23gUdVC/gKK/G8HhmCNkS+HXpFnwN64SXxpvx8H3WAJhNbIPGH6bih5WsXg+v0SLdI19TIY6xs2775tPHkGyPo+Jq1HR6ZI1BBm9QcuTJNpcs1eqcqpochYfczrU/rPI+gLlkWZkudOmcX9rCzMOf3ydpU4flO4c0wksQF/aWEeyqZhtA7IMrro0qrfMr6UFUsgUKsuTVUcCsvx5VT4Ja70fRRsETWbqo9sunIJ33z7MEn7c9jDz13cMYyzTiVkcQHF6KzxiO/dtWQ10H9FY+B2YtEBfwPH46XWLnEX0dtSQcr8UMl/N52DO3O66mt1uV7rlqmXHTZrjyX6I0EYFyGY8hLHa2OGLrxozErtLKx66dOM9MjbFCRSrBezk/ofZz4pGoKDo1+RNNFGC7MSA805sHjw2l2WKVrm8/jBxhls4T8by3XKkXkNnfLvDuXznbdPZSBRd7pU4VaeLnKOqnXKmuPUCvu4hZJV4KchenJP7gAb9VrTCQ0f5Fj7WjYldOzhDk5rUosO95BY1Ue/EQ6KfOYBdVdALDbhz9BKlyu3H98UlgwBlnQXYYh8iHjhZKQQUYQcCzItFoxcioc25IOxOqB0UE+GObWEtO1s8ukqpd76hRW3wb+7frjY2KR4QCpjJ2gTP71Wg74pNM+46KpfQX6xUW8u/UGccO8cXFHn1SRvTE7kwsESVorowKALwZ6Z7b/EJmMUDvJ55MZV8oI1t6Ke4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When memory is being placed, mmap() will take care to respect the guard gaps of certain types of memory (VM_SHADOWSTACK, VM_GROWSUP and VM_GROWSDOWN). In order to ensure guard gaps between mappings, mmap() needs to consider two things: 1. That the new mapping isn’t placed in an any existing mappings guard gaps. 2. That the new mapping isn’t placed such that any existing mappings are not in *its* guard gaps. The long standing behavior of mmap() is to ensure 1, but not take any care around 2. So for example, if there is a PAGE_SIZE free area, and a mmap() with a PAGE_SIZE size, and a type that has a guard gap is being placed, mmap() may place the shadow stack in the PAGE_SIZE free area. Then the mapping that is supposed to have a guard gap will not have a gap to the adjacent VMA. Use mm_get_unmapped_area_vmflags() in the do_mmap() so future changes can cause shadow stack mappings to be placed with a guard gap. Also use the THP variant that takes vm_flags, such that THP shadow stack can get the same treatment. Adjust the vm_flags calculation to happen earlier so that the vm_flags can be passed into __get_unmapped_area(). Signed-off-by: Rick Edgecombe --- v2: - Make get_unmapped_area() a static inline (Kirill) --- include/linux/mm.h | 11 ++++++++++- mm/mmap.c | 34 ++++++++++++++++------------------ 2 files changed, 26 insertions(+), 19 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index f5a97dec5169..d91cde79aaee 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3363,7 +3363,16 @@ extern int install_special_mapping(struct mm_struct *mm, unsigned long randomize_stack_top(unsigned long stack_top); unsigned long randomize_page(unsigned long start, unsigned long range); -extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned long, unsigned long, unsigned long); +unsigned long +__get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, + unsigned long pgoff, unsigned long flags, vm_flags_t vm_flags); + +static inline unsigned long +get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, + unsigned long pgoff, unsigned long flags) +{ + return __get_unmapped_area(file, addr, len, pgoff, flags, 0); +} extern unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, diff --git a/mm/mmap.c b/mm/mmap.c index e23ce8ca24c9..a3128ed26676 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1257,18 +1257,6 @@ unsigned long do_mmap(struct file *file, unsigned long addr, if (mm->map_count > sysctl_max_map_count) return -ENOMEM; - /* Obtain the address to map to. we verify (or select) it and ensure - * that it represents a valid section of the address space. - */ - addr = get_unmapped_area(file, addr, len, pgoff, flags); - if (IS_ERR_VALUE(addr)) - return addr; - - if (flags & MAP_FIXED_NOREPLACE) { - if (find_vma_intersection(mm, addr, addr + len)) - return -EEXIST; - } - if (prot == PROT_EXEC) { pkey = execute_only_pkey(mm); if (pkey < 0) @@ -1282,6 +1270,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr, vm_flags |= calc_vm_prot_bits(prot, pkey) | calc_vm_flag_bits(flags) | mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC; + /* Obtain the address to map to. we verify (or select) it and ensure + * that it represents a valid section of the address space. + */ + addr = __get_unmapped_area(file, addr, len, pgoff, flags, vm_flags); + if (IS_ERR_VALUE(addr)) + return addr; + + if (flags & MAP_FIXED_NOREPLACE) { + if (find_vma_intersection(mm, addr, addr + len)) + return -EEXIST; + } + if (flags & MAP_LOCKED) if (!can_do_mlock()) return -EPERM; @@ -1839,8 +1839,8 @@ unsigned long mm_get_unmapped_area_vmflags(struct mm_struct *mm, struct file *fi } unsigned long -get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, - unsigned long pgoff, unsigned long flags) +__get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, + unsigned long pgoff, unsigned long flags, vm_flags_t vm_flags) { unsigned long (*get_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long) @@ -1875,8 +1875,8 @@ get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (get_area) addr = get_area(file, addr, len, pgoff, flags); else - addr = mm_get_unmapped_area(current->mm, file, addr, len, - pgoff, flags); + addr = mm_get_unmapped_area_vmflags(current->mm, file, addr, len, + pgoff, flags, vm_flags); if (IS_ERR_VALUE(addr)) return addr; @@ -1889,8 +1889,6 @@ get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, return error ? error : addr; } -EXPORT_SYMBOL(get_unmapped_area); - unsigned long mm_get_unmapped_area(struct mm_struct *mm, struct file *file, unsigned long addr, unsigned long len, -- 2.34.1