From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43B61C4345F for ; Tue, 16 Apr 2024 17:12:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B25E26B0089; Tue, 16 Apr 2024 13:12:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AD5086B008C; Tue, 16 Apr 2024 13:12:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99C906B0093; Tue, 16 Apr 2024 13:12:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7A8C16B0089 for ; Tue, 16 Apr 2024 13:12:55 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 27451A1719 for ; Tue, 16 Apr 2024 17:12:55 +0000 (UTC) X-FDA: 82016039910.07.F2CF2E9 Received: from mx0a-00823401.pphosted.com (mx0a-00823401.pphosted.com [148.163.148.104]) by imf16.hostedemail.com (Postfix) with ESMTP id 50E1D180011 for ; Tue, 16 Apr 2024 17:12:52 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=fail ("headers rsa verify failed") header.d=motorola.com header.s=DKIM202306 header.b=C+iV7Q9m; spf=pass (imf16.hostedemail.com: domain of mbland@motorola.com designates 148.163.148.104 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713287572; h=from:from:sender:reply-to:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1bbiwhjSp0cC1+TX0aaOYyIv3i00GGwNyol8tBO0SV4=; b=sppVdkOf8eOJCSRkE/3Aosl22bCVcLyMhqaOj3EExx7N71qtTaCc0EZ4+62hD5B/y4Rj2U QMkJxWMGsauC7hj+2DeDRTRJK84QZSjugDLNkXCVtlU6GVnWutynFaHGp/jxGpmKlnXatP 6lE88WKGtisLeDfpExVFiqMtPtaBBXY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=fail ("headers rsa verify failed") header.d=motorola.com header.s=DKIM202306 header.b=C+iV7Q9m; spf=pass (imf16.hostedemail.com: domain of mbland@motorola.com designates 148.163.148.104 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713287572; a=rsa-sha256; cv=none; b=XqF0QpUVKRiwimC/miJimRY/vhMj9C4nRhQaryLKRTjkUCSFT1o0TLHlaFmrPldT+iR9mb u9DNyaD0Fadqb6ujWyF9z/iMd22QvAbmGvjVTabcTf2JfZXWlN1c1xRHLcImKdIOKT/ORM U9y0vLdYZ3IYHgCcUKN+Z74geQw/RhU= Received: from pps.filterd (m0355085.ppops.net [127.0.0.1]) by mx0a-00823401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 43GGGWxZ009522; Tue, 16 Apr 2024 17:12:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=motorola.com; h= message-id:in-reply-to:references:to:cc:date:from; s=DKIM202306; bh=1bbiwhjSp0cC1+TX0aaOYyIv3i00GGwNyol8tBO0SV4=; b=C+iV7Q9mbw7h YGsQ4mhJqMj4T7kcPYjqR9Do1I+i0n7eNigBjkxj1KgydV8CzlTn6UyN814hDyfc YoN7pLQlGXIC6JSCtDAKWiUhOhC8In+sJivmMWnymqm738iqf1WsZqb2krSQ8PKP 7k7Ja5QICZNm0T7eBueYhXLzSgvY3VurmVccuPC9igBxiLoO4reENJZ45ftL4nri Bdr8oybXa48Sw43iV6XobLDJvE3s7OwuvDaRYYg8LzDcDTKFATjXrDgFmSVJlyAr dwfbcpRgH4SB7m9IdYrDc1od6ofxvRmfMLv42DFdxWR0+pJmW6r2ujVQtLOqJU/O AXWW6LTIZg== Received: from va32lpfpp01.lenovo.com ([104.232.228.21]) by mx0a-00823401.pphosted.com (PPS) with ESMTPS id 3xhea8bud5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Apr 2024 17:12:42 +0000 (GMT) Received: from va32lmmrp01.lenovo.com (va32lmmrp01.mot.com [10.62.177.113]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by va32lpfpp01.lenovo.com (Postfix) with ESMTPS id 4VJrFj0dNBzhSZk; Tue, 16 Apr 2024 17:12:41 +0000 (UTC) Received: from ilclbld243.mot.com (ilclbld243.mot.com [100.64.22.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mbland) by va32lmmrp01.lenovo.com (Postfix) with ESMTPSA id 4VJrFj0Rndz2VZS4; Tue, 16 Apr 2024 17:12:41 +0000 (UTC) Message-Id: <20240416120827.062020959-2-mbland@motorola.com> In-Reply-To: <20240416120827.062020959-1-mbland@motorola.com> References: <20240416120827.062020959-1-mbland@motorola.com> To: linux-mm@kvack.org Cc: Maxwell Bland Date: Tue, 16 Apr 2024 17:12:41 +0000 (UTC) From: mbland@motorola.com X-Proofpoint-GUID: FNTUsujeMw8WJjwDOzm-qtYJAdDO7Wh2 X-Proofpoint-ORIG-GUID: FNTUsujeMw8WJjwDOzm-qtYJAdDO7Wh2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-04-16_14,2024-04-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 bulkscore=0 phishscore=0 adultscore=0 spamscore=0 lowpriorityscore=0 clxscore=1011 mlxlogscore=999 suspectscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2404010003 definitions=main-2404160106 X-Rspamd-Queue-Id: 50E1D180011 X-Stat-Signature: nt93kzcioeqpezcd3uw7uc55a3gd4g4b X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1713287572-869935 X-HE-Meta: U2FsdGVkX1/IQv5NTR1+Fd0kcHGtIeZziw7wh7bPk219f5OMZ0/r/a7o0AC36hEiRLCbuzYYhhRUhl2GC1A9xxKf2ANor5Od9c+CDQfmRJA1fn3Ll7+olwMfLffeRoHHUwChXLW7MlPRwYwkNIGE6M8juhKcZoScny6xaSqAP9NAvs6YxkXRXHqnUXYwsIzWpbrNyAYBWj7Y6uds2xPd2a4CkzuRJSDg7/XwqBDjIr7Wk6X+UUB0n1OGWGB4wNZSaC/sDBcOFGqAzJtnQKteIFF8gCY74uPlLkn+nEotf4nbjMAPhAu6PlIWV9/E5OfnQtHKR+sSA4UxSGCF0pnqZVLvF/vxFrEnZ8z8YgSXrrKfaL09WvfLGnx/iCHDkt3VEpK8j/tMvRsdz01+tuZgiq4UGDULRL0PexzdM4K8ca4lSuUn3huknJw+EHABsCnxx5Uh6ur3FvMkNqovWFWYpPxvJ4p3HWDR/C2Qy+S54Wjdf3537vq1WC5X3/Z7epB/S6Be4ouOT7lU+tB3kpqQQHqeRpdyBybNbhGZTz35xqhoHhb15hhOOzKMTtOwHshZk1YdbjaXEexV/42lpfCMtTcKTmAI5MCYQxCUeeFl2W0CMk8Wd+R9wDDp41esWtBiI3CKkKF7qVm+KGNeusgavj2tE79p7sXSwUAG0q+xMBcBEw5m987GsfgHDJyDFjPJtKXCZdXFVKqc+KfPZopoOVXkvDGwEHBVUX8LQ6hkGH7YLpvOtlfeAfgkpO54FJ7Gh07f1EC0Z41p4fbVeEhv3c0XkvZfP+Dx5yBp2FLiZ5wfrGaJouPwMnyBUTt4VIXUMSqvVPjrXX+dQaN8rZpwap2HYWJy8dz8aug6yRWbRC7rZVAh/v7o0IqzvhJcYdctYSCcE18O8/PG3gD//YJDry1PuG1xwLdBcclIqSw+2umVbRDpnj+usv8ywuRBRS0JxJk+D16fa31tuemu3qb 5Ok+vIBj zJHrSvsNBwLEqYzS9Y6tQeCA5zZ5NWA5/OMJismi374vetnwkzpB7ZZbBupkAM7YE2vZl1MfCGdUW5d6l6aGs6+Rp+dOQ/x45PRPpMNpAMfbAQkZUfDzh+H5b4Ex+1oqNscM6cA1EPFzBrjRW53Xq9zmpae9qGTpC6FIzFVhGvNH3DUzPZh/J5Maj87bU34poeGHUakzGzFFW4vSY4xX1DxqpKyB6+P7t7g3ehSq+Kipiavd1cwzoGue+FO7UVDaL7xyWKd+M7lqd0YcW+ACY5eWi6pYfmN22uCAY4EJjPAPF+/PlfAfjucroeoF85BvVSug0A+6tlAR3EjUgM03MgsCTutp/wWx8sUsE/GvS2U/STKWxM3/haKCmrVOWAwNm2Mf5FjAQd1yo5+y1CAWoSspAsDs6k+1FOznYxIzwuIvdgDPlkoBqNt8cWJ5m6E8M6i4G3d6L1W5WZv3hjKnZuAh9Zojnyv4+PY/pdyOw3muqdQPT9p7zZtimjeQWRcbefx4HWAnGjZLpDZSQep8hqMevHgTaPe+p90Rvi6PLNkbSOtL9JBGWC5Z4EA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: linux-kernel@vger.kernel.org, Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes From: Maxwell Bland Date: Tue, 2 Apr 2024 15:15:01 -0500 Subject: [PATCH 1/5] mm: allow arch refinement/skip for vmap alloc Makes red black tree allocation more flexible on a per-architecture basis by introducing an optional hooks to refine the red-black tree structuring and exposing vmalloc functions for clipping vmap areas, finding vmap areas, and inserting vmap areas. With this patch, the red-black vmap tree can be refined to account for architecture-specific memory management operations, most notably address space layout randomization, as these features conflict with generic management of a single vmalloc_start to vmalloc_end range as given by mm/vmalloc.c. For example, x86 is forced to restrict aslr to 1024 possible locations, which is a very, very small number, and arm64 breaks standard code/data partitioning altogether, which prevents the enforcement of performant immmutability on kernel page tables. Signed-off-by: Maxwell Bland --- include/linux/vmalloc.h | 24 ++++++++++++++++++++++++ mm/vmalloc.c | 16 ++++++++++------ 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 98ea90e90439..3c5ce7ee0bea 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -12,6 +12,7 @@ #include +struct kmem_cache; struct vm_area_struct; /* vma defining user mapping in mm_types.h */ struct notifier_block; /* in notifier.h */ struct iov_iter; /* in uio.h */ @@ -125,6 +126,21 @@ static inline pgprot_t arch_vmap_pgprot_tagged(pgprot_t prot) } #endif +#ifndef arch_skip_va +static inline bool arch_skip_va(struct vmap_area *va, unsigned long vstart) +{ + return false; +} +#endif + +#ifndef arch_refine_vmap_space +static inline void arch_refine_vmap_space(struct rb_root *root, + struct list_head *head, + struct kmem_cache *cachep) +{ +} +#endif + /* * Highlevel APIs for driver use */ @@ -214,6 +230,14 @@ extern struct vm_struct *__get_vm_area_caller(unsigned long size, void free_vm_area(struct vm_struct *area); extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); +extern void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, + struct rb_root *root, + struct list_head *head); +extern int va_clip(struct rb_root *root, struct list_head *head, + struct vmap_area *va, unsigned long nva_start_addr, + unsigned long size); +extern struct vmap_area *__find_vmap_area(unsigned long addr, + struct rb_root *root); struct vmap_area *find_vmap_area(unsigned long addr); static inline bool is_vm_area_hugepages(const void *addr) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 68fa001648cc..de4577a3708e 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -989,7 +989,7 @@ unsigned long vmalloc_nr_pages(void) return atomic_long_read(&nr_vmalloc_pages); } -static struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) +struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) { struct rb_node *n = root->rb_node; @@ -1322,7 +1322,7 @@ insert_vmap_area(struct vmap_area *va, link_va(va, root, parent, link, head); } -static void +void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, struct rb_root *root, struct list_head *head) @@ -1501,7 +1501,7 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, vstart < va->va_start) { node = node->rb_left; } else { - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && is_within_this_va(va, size, align, vstart)) return va; /* @@ -1522,7 +1522,8 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, */ while ((node = rb_parent(node))) { va = rb_entry(node, struct vmap_area, rb_node); - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && + is_within_this_va(va, size, align, vstart)) return va; if (get_subtree_max_size(node->rb_right) >= length && @@ -1554,7 +1555,7 @@ find_vmap_lowest_linear_match(struct list_head *head, unsigned long size, struct vmap_area *va; list_for_each_entry(va, head, list) { - if (!is_within_this_va(va, size, align, vstart)) + if (arch_skip_va(va, vstart) || !is_within_this_va(va, size, align, vstart)) continue; return va; @@ -1617,7 +1618,7 @@ classify_va_fit_type(struct vmap_area *va, return type; } -static __always_inline int +__always_inline int va_clip(struct rb_root *root, struct list_head *head, struct vmap_area *va, unsigned long nva_start_addr, unsigned long size) @@ -5129,4 +5130,7 @@ void __init vmalloc_init(void) vmap_node_shrinker->count_objects = vmap_node_shrink_count; vmap_node_shrinker->scan_objects = vmap_node_shrink_scan; shrinker_register(vmap_node_shrinker); + + arch_refine_vmap_space(&free_vmap_area_root, &free_vmap_area_list, + vmap_area_cachep); } -- 2.39.2