From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2DB2C4345F for ; Tue, 16 Apr 2024 17:25:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1947A6B0085; Tue, 16 Apr 2024 13:25:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1453F6B0088; Tue, 16 Apr 2024 13:25:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00BF56B0089; Tue, 16 Apr 2024 13:25:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D070B6B0085 for ; Tue, 16 Apr 2024 13:25:08 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6A8A91A0A9B for ; Tue, 16 Apr 2024 17:25:08 +0000 (UTC) X-FDA: 82016070696.04.F302806 Received: from mx0a-00823401.pphosted.com (mx0a-00823401.pphosted.com [148.163.148.104]) by imf16.hostedemail.com (Postfix) with ESMTP id 8237E18000C for ; Tue, 16 Apr 2024 17:25:06 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=motorola.com header.s=DKIM202306 header.b="U XNGIZC"; spf=pass (imf16.hostedemail.com: domain of mbland@motorola.com designates 148.163.148.104 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713288306; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=Ktj2knpZkxEzepUnXbFCAHnzLfkAa26Odp8kjW5qVXw=; b=U1FV9jeSqYERZYPNU2f7PJPQWTnozeB1ifl8SAtnIQifFtNdj+bmvBf/6cSPvFknWTRksk uRHh+e3cskJmwkig4/pF3Wm8YQptunyjCUBTZ1CCuQ9WR7lgu2GtQvVmG3QukIIAm2vzLk yjKSGAxq4rU5+9BaMnvb+RBAMk4PSmU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=motorola.com header.s=DKIM202306 header.b="U XNGIZC"; spf=pass (imf16.hostedemail.com: domain of mbland@motorola.com designates 148.163.148.104 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713288306; a=rsa-sha256; cv=none; b=OW0yr6BJamgIuKx3wVZQl0IzM1yLJPiNxqve/uOOhLUl+VtZm5Gl/BajL28P2a+WnhCOnz EyPI7Jx2v6+dpOtfIYSExLQ/b406SSsrtv6MyIrB3CI3tkX4rinPhDNvOhGLFpSwuDVjXH g8I2Ego9H4MOaiqMeaiJGmpt4SlOVwI= Received: from pps.filterd (m0355087.ppops.net [127.0.0.1]) by mx0a-00823401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 43GFPN5k006602; Tue, 16 Apr 2024 17:24:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=motorola.com; h= message-id:in-reply-to:references:to:cc:from:date:subject; s= DKIM202306; bh=Ktj2knpZkxEzepUnXbFCAHnzLfkAa26Odp8kjW5qVXw=; b=U XNGIZCS9YmDgBUeEu5cofHA9mvaV6V1cdLYUPyD490DtaRc+ylTIkjPFzxuWV+4T 21f6DuwRlCKVXuIbxpUAP7zIVfKAOnpoOVrxFs50JPTCkWAv031LnE5jmeiZi1aA R4XwQPSs9ieKI8NnJOZWkmkEZnXk98ir5fZoWx004yfu9JRghgvTKQx8l5r46UH2 3Gxac+BT+Y/DdxHClOLeMtkCkpMCot4eJMbl5uT1eqEPYTiWpENBGPURZC3kfUqR JGoVqQIu/+3pYXSYSfFrBnBWrgCOkuJqt8EReXdQEsqrYza/cDg2Dejomjcze8ru uOgM14JDYDWyjwAaePTTg== Received: from ilclpfpp01.lenovo.com ([144.188.128.67]) by mx0a-00823401.pphosted.com (PPS) with ESMTPS id 3xhfb73r8r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Apr 2024 17:24:58 +0000 (GMT) Received: from va32lmmrp01.lenovo.com (va32lmmrp01.mot.com [10.62.177.113]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ilclpfpp01.lenovo.com (Postfix) with ESMTPS id 4VJrWs1z69zfBZq; Tue, 16 Apr 2024 17:24:57 +0000 (UTC) Received: from ilclbld243.mot.com (ilclbld243.mot.com [100.64.22.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mbland) by va32lmmrp01.lenovo.com (Postfix) with ESMTPSA id 4VJrWs0bYbz2VZS4; Tue, 16 Apr 2024 17:24:57 +0000 (UTC) Message-Id: <20240416122254.868007168-2-mbland@motorola.com> In-Reply-To: <20240416122254.868007168-1-mbland@motorola.com> References: <20240416122254.868007168-1-mbland@motorola.com> To: linux-mm@kvack.org Cc: Maxwell Bland , linux-kernel@vger.kernel.org, Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes From: Maxwell Bland Date: Tue, 2 Apr 2024 15:15:01 -0500 Subject: [PATCH 1/5] mm: allow arch refinement/skip for vmap alloc X-Proofpoint-GUID: m5q-_kUrWpAiWSiKGKl9KqD-wJ45KyMZ X-Proofpoint-ORIG-GUID: m5q-_kUrWpAiWSiKGKl9KqD-wJ45KyMZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-04-16_14,2024-04-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 mlxscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1015 mlxlogscore=999 phishscore=0 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2404010003 definitions=main-2404160108 X-Stat-Signature: wc8dygh6t9n7ao87goasjpdimorc75af X-Rspamd-Queue-Id: 8237E18000C X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1713288306-341819 X-HE-Meta: U2FsdGVkX1/SpeSPJflTsUqqse1mZKpwkGfJEMlQj1AMy5t8CFsiM1oCUMy1U+QNvYSz4NzRm0tbSPyk4nYoGeknafy/wGe2b6je21zwv4dBmejBLcYjwue1yN1hPaZEKGL66RBWm0DhacZEVph47742EIqB0PlO9oqJ7staoYaOkVK8AFVzIfG8jNUCnscIlwhUB9oEeC6kShn0gZ4T9gbWn0HM04XjeKLH6RjHC6MlGvSkYReqT4400wKxJh9yOltT46TQJIO/jZE3zHzFjRDhVjbN6MYv4eX73ZKWCqvrEEX6gQh+ICIksRZbQajNtjX4Pd05GTOIHaaLHEG+VLyt8XfCNGfv53kGzibMg77m68qqeMbDOeYbNKvcduLlZDsutv3L5zePskRkL7pEjGHy2c7AaiLjiXIEhEfHRpb14o4iIJwweSfH6gY8EwjLRnogMpL9mjvKBNypXPhKti/nbmUvvhHQQV/msRKgG34CiSwOB8tl6WBYkHOBFCYlHdjD+m46E1Hokf2mapwUYWN7YmRcEoepuuy6d4D8CjYp9gl856OVzWstjF0BSPdNSh8OO++j7ViysLymEs8TJozqT2DQEcgrGE9Kl5OoFVJkCYRWBwWgxOkdcdxJc4/FzmWyfnU2kqWTAuJBHohJ+q8ttTlvB9CL4sTxOO0aefSWCFsXyz50uv/pG6eMPHbr5B1IRlTiyz8J9LDUxoI4Q1IhZvc/FkHbLwrAYngk1afQF8STTSVeJB6U5vL2VGHMURd4UMusnY0fWtHZuGF+x0XR97/eL1K10JIFxiMkUkzhZ0m9Lm/TcOTXWkySwiqt3XqTwUD5nfd+W0CIreiN4Nf0Ksb+v3mYrbPnFPfWJPvr5pbKYPyT0kgqGJGkO9flZbQ1AwhmXxmnaMl7gcCWlidT7dwlvp2XkztdHfARNQZ0x/1632/9uB1V821GyLY9FMnUjagcrJNPPA2Av3F cDWGNPca QDLrypHqR4oqXEMdJSQtxtBvQ/tgT7qdbNVOq36moCaAqk1UoVm8sCTiC2JdPYqbZA0NFLpWSJSoGmbw5Nae9v2uAAm5/uC5X6Evornhwq3LPJ2C5sK6rCDXUcLM1rYlsKSL+QtGyxOj+Xm2/CqswfEvtwqDf+tRQjAsau//h4A3zmZL7ba5hUqC1mPLXzTEs6L+QjayaxiGYuBTihWuG4tE3ZD1L/j/WDDZhL5w4XoGmAW+5ADNz4r9YsgH1mVyr/hgOHbY8TisN5HQCYZRNQXcbug== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Makes red black tree allocation more flexible on a per-architecture basis by introducing an optional hooks to refine the red-black tree structuring and exposing vmalloc functions for clipping vmap areas, finding vmap areas, and inserting vmap areas. With this patch, the red-black vmap tree can be refined to account for architecture-specific memory management operations, most notably address space layout randomization, as these features conflict with generic management of a single vmalloc_start to vmalloc_end range as given by mm/vmalloc.c. For example, x86 is forced to restrict aslr to 1024 possible locations, which is a very, very small number, and arm64 breaks standard code/data partitioning altogether, which prevents the enforcement of performant immmutability on kernel page tables. Signed-off-by: Maxwell Bland --- include/linux/vmalloc.h | 24 ++++++++++++++++++++++++ mm/vmalloc.c | 16 ++++++++++------ 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 98ea90e90439..3c5ce7ee0bea 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -12,6 +12,7 @@ #include +struct kmem_cache; struct vm_area_struct; /* vma defining user mapping in mm_types.h */ struct notifier_block; /* in notifier.h */ struct iov_iter; /* in uio.h */ @@ -125,6 +126,21 @@ static inline pgprot_t arch_vmap_pgprot_tagged(pgprot_t prot) } #endif +#ifndef arch_skip_va +static inline bool arch_skip_va(struct vmap_area *va, unsigned long vstart) +{ + return false; +} +#endif + +#ifndef arch_refine_vmap_space +static inline void arch_refine_vmap_space(struct rb_root *root, + struct list_head *head, + struct kmem_cache *cachep) +{ +} +#endif + /* * Highlevel APIs for driver use */ @@ -214,6 +230,14 @@ extern struct vm_struct *__get_vm_area_caller(unsigned long size, void free_vm_area(struct vm_struct *area); extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); +extern void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, + struct rb_root *root, + struct list_head *head); +extern int va_clip(struct rb_root *root, struct list_head *head, + struct vmap_area *va, unsigned long nva_start_addr, + unsigned long size); +extern struct vmap_area *__find_vmap_area(unsigned long addr, + struct rb_root *root); struct vmap_area *find_vmap_area(unsigned long addr); static inline bool is_vm_area_hugepages(const void *addr) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 68fa001648cc..de4577a3708e 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -989,7 +989,7 @@ unsigned long vmalloc_nr_pages(void) return atomic_long_read(&nr_vmalloc_pages); } -static struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) +struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) { struct rb_node *n = root->rb_node; @@ -1322,7 +1322,7 @@ insert_vmap_area(struct vmap_area *va, link_va(va, root, parent, link, head); } -static void +void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, struct rb_root *root, struct list_head *head) @@ -1501,7 +1501,7 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, vstart < va->va_start) { node = node->rb_left; } else { - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && is_within_this_va(va, size, align, vstart)) return va; /* @@ -1522,7 +1522,8 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, */ while ((node = rb_parent(node))) { va = rb_entry(node, struct vmap_area, rb_node); - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && + is_within_this_va(va, size, align, vstart)) return va; if (get_subtree_max_size(node->rb_right) >= length && @@ -1554,7 +1555,7 @@ find_vmap_lowest_linear_match(struct list_head *head, unsigned long size, struct vmap_area *va; list_for_each_entry(va, head, list) { - if (!is_within_this_va(va, size, align, vstart)) + if (arch_skip_va(va, vstart) || !is_within_this_va(va, size, align, vstart)) continue; return va; @@ -1617,7 +1618,7 @@ classify_va_fit_type(struct vmap_area *va, return type; } -static __always_inline int +__always_inline int va_clip(struct rb_root *root, struct list_head *head, struct vmap_area *va, unsigned long nva_start_addr, unsigned long size) @@ -5129,4 +5130,7 @@ void __init vmalloc_init(void) vmap_node_shrinker->count_objects = vmap_node_shrink_count; vmap_node_shrinker->scan_objects = vmap_node_shrink_scan; shrinker_register(vmap_node_shrinker); + + arch_refine_vmap_space(&free_vmap_area_root, &free_vmap_area_list, + vmap_area_cachep); } -- 2.39.2 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0CB4C4345F for ; Tue, 16 Apr 2024 19:19:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 214A36B0098; Tue, 16 Apr 2024 15:19:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C5266B0099; Tue, 16 Apr 2024 15:19:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08CE96B009A; Tue, 16 Apr 2024 15:19:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DD8EC6B0098 for ; Tue, 16 Apr 2024 15:19:36 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9888BA0B5D for ; Tue, 16 Apr 2024 19:19:36 +0000 (UTC) X-FDA: 82016359152.25.6B78209 Received: from mx0b-00823401.pphosted.com (mx0b-00823401.pphosted.com [148.163.152.46]) by imf02.hostedemail.com (Postfix) with ESMTP id BFEC580002 for ; Tue, 16 Apr 2024 19:19:34 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=motorola.com header.s=DKIM202306 header.b="t IvRH91"; spf=pass (imf02.hostedemail.com: domain of mbland@motorola.com designates 148.163.152.46 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713295174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=Ktj2knpZkxEzepUnXbFCAHnzLfkAa26Odp8kjW5qVXw=; b=e0O2we2QmtvRz4hz2dqTuzUsRA2M0d+ufvdRpVcz/NlxaIB7r+X8NeOG7LxectVIU/xWN9 L2bjFwT864xKh+DB/AMDZsrD6fUmHzJvlWHGsfOyac8HPGLbIptUrh1SIiVjeJkFAYOY/3 fGipue9cAoKu6ZfS3uEKQjsfzbPXZio= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713295174; a=rsa-sha256; cv=none; b=zUKTPTqv6Bk1UElBSv6g0IAnWZicfMF6PguHEW+DVvKnaMZnHT8dYHRXBfOqouxtTumhWN 6CS4b/s/BWLfuBCbq8IFgm+OQpCWnA6nk01b6XugMjy3RUS3HcPFplp0+Bxq7og7Qae8XU ZexldPWMC9KHD+LNcpirXiD1X1+NAEU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=motorola.com header.s=DKIM202306 header.b="t IvRH91"; spf=pass (imf02.hostedemail.com: domain of mbland@motorola.com designates 148.163.152.46 as permitted sender) smtp.mailfrom=mbland@motorola.com; dmarc=pass (policy=none) header.from=motorola.com Received: from pps.filterd (m0355092.ppops.net [127.0.0.1]) by mx0b-00823401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 43GJI4gT002353; Tue, 16 Apr 2024 19:19:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=motorola.com; h= message-id:in-reply-to:references:to:cc:from:subject:date; s= DKIM202306; bh=Ktj2knpZkxEzepUnXbFCAHnzLfkAa26Odp8kjW5qVXw=; b=t IvRH91cKGfnfqx35lOWohhZ8d7xbjQ803QvX1t1pf3wFaQw5yoIlX9PbNHe0Z56R eTT7pWo/NaTzmdpM8+4hPV7EjKn8kYDs3XsP+11id2YuLgBZVrSwhB6FDvunP4iC 8q9ZUzFmq0OYFZNGlZVPjc5pzmj4rmghPQDRDXl+tzX1VKjMpqqe22Icedg2EplE PyieGPN8XCb8iQv6PUYpTJVnMMCacNx5qnjXhB+wQ3fe4Rx9Nwur73z/k1xnuBYU 6aVcebndAoaDBvneC/9R7WPSx8qiGJYMMwAFNDWNNhwA6oHlkl6TIh+5uhK5oRIR qybI2G7ivrYFNn9r989vw== Received: from va32lpfpp02.lenovo.com ([104.232.228.22]) by mx0b-00823401.pphosted.com (PPS) with ESMTPS id 3xhyc7r01x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Apr 2024 19:19:22 +0000 (GMT) Received: from va32lmmrp02.lenovo.com (va32lmmrp02.mot.com [10.62.176.191]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by va32lpfpp02.lenovo.com (Postfix) with ESMTPS id 4VJv3t08HKz53xyX; Tue, 16 Apr 2024 19:19:22 +0000 (UTC) Received: from ilclbld243.mot.com (ilclbld243.mot.com [100.64.22.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mbland) by va32lmmrp02.lenovo.com (Postfix) with ESMTPSA id 4VJv3s75L2z2SlV8; Tue, 16 Apr 2024 19:19:21 +0000 (UTC) Message-ID: <20240416122254.868007168-2-mbland@motorola.com> In-Reply-To: <20240416122254.868007168-1-mbland@motorola.com> References: <20240416122254.868007168-1-mbland@motorola.com> To: linux-mm@kvack.org Cc: Maxwell Bland , linux-kernel@vger.kernel.org, Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Mark Rutland , Greg Kroah-Hartman , Christoph Hellwig , Christophe Leroy , David Hildenbrand , Conor Dooley , Lorenzo Stoakes From: Maxwell Bland Subject: [PATCH 1/5 RESEND] mm: allow arch refinement/skip for vmap alloc Date: Tue, 16 Apr 2024 14:18:15 -0500 X-Proofpoint-GUID: H86CUAODmaPSKsWi32l4bYEE5cNkOhPT X-Proofpoint-ORIG-GUID: H86CUAODmaPSKsWi32l4bYEE5cNkOhPT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-04-16_17,2024-04-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 suspectscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2404010003 definitions=main-2404160122 X-Stat-Signature: ougtdrki81onaksw6g54g3sbnik67yws X-Rspamd-Queue-Id: BFEC580002 X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1713295174-397596 X-HE-Meta: U2FsdGVkX19zUbqLBsHBWQGlPWFW20CVmHhFSumnpRjCRO7+7zagqZZ+6YO9lUJ2YbbP4+3qGYKq8Dv5h1usRhh2q19/M0D+VPeCxfFu38RpMtwW/2a7f6HRz6VmHcUZBrOShTNcB0Tcdc55+UPtqhah5S15D+kKmy48NYT4bdXOa8ctLFOqfJqJTtzmlWm2LD30vD2Rc5j7yhs5FQNa+mcErHunHVC+C3teyNzqMfNjQApO5pRaxYc4lVh8WS2bfnYNsPtw3iyzGlu0qLvt1sZGIQXiVgAo2xp9X649RPGhl3nP3tABsmlkFB15fJW8LtLFaV0BHF4roD03elQIHc3EJUlTrwx+6LKNswwVNUhM07tf0Io0Xp7nu9FwmjOvj3U+KJmwoEKGxPrD4o9mtNmfErusfXuXoE9i2IYlE49Mtw18blEfYkgDzvSxUQ1oL+VDsvsWVdgdN9VBG8Y9p6I5FEEaHj6DEtalBU4SGL+h0viA57+Equo5U6QjZlLA8QpkZXYJk5jD3KNbdg8Vuz7ZkabHlXLzxYAO2k9Cfm1yYgLFDr0LAAMqVItcR6K8EGCjMuvCQLlIFKowdk03Kbu7gbutMQFT915dUuH2TWj0waRGdBNJxv79ifRFAf9SMy6bhOid6G/ZkrGyV8hUqbYdfdmvifpqENURnGapOymyjHFKEfBcrqUpyjbYo2AOwS7ET3dVVKypyS1FVJ83qRKk5b02IZaL01YAw1PXpGmlJiNWl4Vxq9YWmysqswV/5s51xqGFVt6cpTq35yVHvD+aA8plqaswx0EbA/iqNJp+nVGjJQ2WIwmICJ14JuAXaKyGTKWHhm3Qp631wuv2jxuUjV+aPwAF0kiOtzmWLy5lTyXMjlizARDNFLzeepYkdaRh3AN2+E4ueP/SYWu8XBLNC4GCiji3XvVZpRSTynRzNhadUpsofzEO9hViY5ldmF6ogJ+NqvNVbwcT8wb LZSl/mZh R5DVvSfLs+uOgZiCWsDeyMyXcLgQ9aoYrusxPzUCt5ZffhnXdsSbrd4TMFx0Yu26w8PuxRyV0yIPTpzVo8o/APXfnwX6mykbvMeZcSwxD0q/AJZad09z4fOIKgeesRf28x7EOTlZRQdFoPwvCDiZ6hzDmmpJD4qTcMdXS8e6CUh6/6rXtW8yhu+/XY+aczuKT+q4ubiRZHj1p1iODCwK9O7qWtxZ9C7jsGG8HMs53H1jInMHiX+P0LKvOOA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Message-ID: <20240416191815.v6VenyHuprasWDsYfV9tmOFTvAFQnpwXPLayz_rTTxM@z> Makes red black tree allocation more flexible on a per-architecture basis by introducing an optional hooks to refine the red-black tree structuring and exposing vmalloc functions for clipping vmap areas, finding vmap areas, and inserting vmap areas. With this patch, the red-black vmap tree can be refined to account for architecture-specific memory management operations, most notably address space layout randomization, as these features conflict with generic management of a single vmalloc_start to vmalloc_end range as given by mm/vmalloc.c. For example, x86 is forced to restrict aslr to 1024 possible locations, which is a very, very small number, and arm64 breaks standard code/data partitioning altogether, which prevents the enforcement of performant immmutability on kernel page tables. Signed-off-by: Maxwell Bland --- include/linux/vmalloc.h | 24 ++++++++++++++++++++++++ mm/vmalloc.c | 16 ++++++++++------ 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 98ea90e90439..3c5ce7ee0bea 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -12,6 +12,7 @@ #include +struct kmem_cache; struct vm_area_struct; /* vma defining user mapping in mm_types.h */ struct notifier_block; /* in notifier.h */ struct iov_iter; /* in uio.h */ @@ -125,6 +126,21 @@ static inline pgprot_t arch_vmap_pgprot_tagged(pgprot_t prot) } #endif +#ifndef arch_skip_va +static inline bool arch_skip_va(struct vmap_area *va, unsigned long vstart) +{ + return false; +} +#endif + +#ifndef arch_refine_vmap_space +static inline void arch_refine_vmap_space(struct rb_root *root, + struct list_head *head, + struct kmem_cache *cachep) +{ +} +#endif + /* * Highlevel APIs for driver use */ @@ -214,6 +230,14 @@ extern struct vm_struct *__get_vm_area_caller(unsigned long size, void free_vm_area(struct vm_struct *area); extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); +extern void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, + struct rb_root *root, + struct list_head *head); +extern int va_clip(struct rb_root *root, struct list_head *head, + struct vmap_area *va, unsigned long nva_start_addr, + unsigned long size); +extern struct vmap_area *__find_vmap_area(unsigned long addr, + struct rb_root *root); struct vmap_area *find_vmap_area(unsigned long addr); static inline bool is_vm_area_hugepages(const void *addr) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 68fa001648cc..de4577a3708e 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -989,7 +989,7 @@ unsigned long vmalloc_nr_pages(void) return atomic_long_read(&nr_vmalloc_pages); } -static struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) +struct vmap_area *__find_vmap_area(unsigned long addr, struct rb_root *root) { struct rb_node *n = root->rb_node; @@ -1322,7 +1322,7 @@ insert_vmap_area(struct vmap_area *va, link_va(va, root, parent, link, head); } -static void +void insert_vmap_area_augment(struct vmap_area *va, struct rb_node *from, struct rb_root *root, struct list_head *head) @@ -1501,7 +1501,7 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, vstart < va->va_start) { node = node->rb_left; } else { - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && is_within_this_va(va, size, align, vstart)) return va; /* @@ -1522,7 +1522,8 @@ find_vmap_lowest_match(struct rb_root *root, unsigned long size, */ while ((node = rb_parent(node))) { va = rb_entry(node, struct vmap_area, rb_node); - if (is_within_this_va(va, size, align, vstart)) + if (!arch_skip_va(va, vstart) && + is_within_this_va(va, size, align, vstart)) return va; if (get_subtree_max_size(node->rb_right) >= length && @@ -1554,7 +1555,7 @@ find_vmap_lowest_linear_match(struct list_head *head, unsigned long size, struct vmap_area *va; list_for_each_entry(va, head, list) { - if (!is_within_this_va(va, size, align, vstart)) + if (arch_skip_va(va, vstart) || !is_within_this_va(va, size, align, vstart)) continue; return va; @@ -1617,7 +1618,7 @@ classify_va_fit_type(struct vmap_area *va, return type; } -static __always_inline int +__always_inline int va_clip(struct rb_root *root, struct list_head *head, struct vmap_area *va, unsigned long nva_start_addr, unsigned long size) @@ -5129,4 +5130,7 @@ void __init vmalloc_init(void) vmap_node_shrinker->count_objects = vmap_node_shrink_count; vmap_node_shrinker->scan_objects = vmap_node_shrink_scan; shrinker_register(vmap_node_shrinker); + + arch_refine_vmap_space(&free_vmap_area_root, &free_vmap_area_list, + vmap_area_cachep); } -- 2.39.2