From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D79CBC433FE for ; Mon, 10 Oct 2022 19:09:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 460586B0071; Mon, 10 Oct 2022 15:09:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40FFA6B0073; Mon, 10 Oct 2022 15:09:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 215528E0001; Mon, 10 Oct 2022 15:09:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id F25896B0071 for ; Mon, 10 Oct 2022 15:09:04 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C43B5A04B1 for ; Mon, 10 Oct 2022 19:09:04 +0000 (UTC) X-FDA: 80005977408.13.09AB320 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf17.hostedemail.com (Postfix) with ESMTP id 4708140032 for ; Mon, 10 Oct 2022 19:09:04 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29AIpEBu015816; Mon, 10 Oct 2022 12:08:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=facebook; bh=SXPDk7BbCbmsa1nSyXk45xokdnliM4Ysc43ElfvdpYk=; b=Ng/box1EBI87+qWHzOSHQAS9t9G301kkfRWIQsp/oa8ldWr43eyN0B6ZkL/uXCKw9wEc nZoXzB417dznKPLCR+vZPGWIfTjALZY2AREdnvfvnLABVYZ0gTuYijsPSPHBC7pBEzOB 2WfvRhu5p2S0Srosgv4sDHLfyzFyNw1hefk= Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2169.outbound.protection.outlook.com [104.47.56.169]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3k4n9rj7rn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 10 Oct 2022 12:08:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hJMv1GZ9cTzywjpG8+R7DMFWsjjOPMxK9/JQXcxYQPqtN64DPgURcHgjjsZy4sc+3Iwaj/OTp5xbOI1gqegPwkapSRdl+2SYa5GEc0gcuZucRa3AB7l8py1dwebPyL0zId4YpLu8Tty1/z0X2Xk9SU8GKVNupH2gzydV/R/3XByCsoo6W8WKo40wikOLfWWBSSJ3MTgpcKlYPlHwFtTB34/zjuf+agTmiux8ijid0KxteWvqG+wBoYIoLMOOxDoz3BxxwkOZoySD/yA+Fom9A9mYWzxTEbIOcJNHsWtsq4DbqdhNyG/rrZdT4eVocrBMuK1nLq/6qkF6Xa1JmkcjiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SXPDk7BbCbmsa1nSyXk45xokdnliM4Ysc43ElfvdpYk=; b=gY8zmdFRwjm1H43aymj3LvgqywOz7dJ7pM8pDn3L/x+cgWG/CUb+EKioRsGSFxprfvLpE32UqQwy0G91ADyLJevjL0tgQ5RuTrs3nFzlnZdrYpwCeK3GkgOqVtl6LfsIqO/+YJ+CLGMsa+THLDVzeKMRnHYYtZzRr8pRu1tPiRitVtPAY4lx9QNIgAR8GJB650qWIHk+fp2aTswydZQ/hBRDflEcFyWAdg2ujomwUB5ZD1uE0kPgKNBP7eeH/VrO6iDvN/8L0Tyf3kmJkoCuO65kvmIwxNgt7ydYDCG2Q/dnIMW+xZco7BRt/fMaDFi8gR09B6rYTlDeQGa1OI+W3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fb.com; dmarc=pass action=none header.from=fb.com; dkim=pass header.d=fb.com; arc=none Received: from SA1PR15MB5109.namprd15.prod.outlook.com (2603:10b6:806:1dc::10) by DM6PR15MB3610.namprd15.prod.outlook.com (2603:10b6:5:1f6::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.28; Mon, 10 Oct 2022 19:08:53 +0000 Received: from SA1PR15MB5109.namprd15.prod.outlook.com ([fe80::d70d:8cce:bb1:e537]) by SA1PR15MB5109.namprd15.prod.outlook.com ([fe80::d70d:8cce:bb1:e537%6]) with mapi id 15.20.5709.015; Mon, 10 Oct 2022 19:08:53 +0000 From: Song Liu To: "Edgecombe, Rick P" CC: "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "song@kernel.org" , "hch@lst.de" , Kernel Team , "peterz@infradead.org" , "urezki@gmail.com" , "akpm@linux-foundation.org" , "x86@kernel.org" , "Hansen, Dave" Subject: Re: [RFC v2 4/4] vmalloc_exec: share a huge page with kernel text Thread-Topic: [RFC v2 4/4] vmalloc_exec: share a huge page with kernel text Thread-Index: AQHY2qb4/DleVypvfEGw7i865KJ5wq4H9+4AgAAKPAA= Date: Mon, 10 Oct 2022 19:08:53 +0000 Message-ID: References: <20221007234315.2877365-1-song@kernel.org> <20221007234315.2877365-5-song@kernel.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3696.120.41.1.1) x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SA1PR15MB5109:EE_|DM6PR15MB3610:EE_ x-ms-office365-filtering-correlation-id: 03476899-759f-4bfc-d544-08daaaf2df51 x-fb-source: Internal x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vJNeDrbc6SLcdjwhc6Kmx4kNEX2ZJfMT+2gIvJzvrjWoFVvZuGe2P3GVr9uQKuM0I0womwi7cRczkTpQSaK3/UexXkFFcr/74TD8YaZFuOigHF6wNvrjMDCGAPwL4bavF8poucEuIkEEkiKm4sxYjWDFRJKIsCzxOiLfs7ntEfkhdHgBdJaKv9QHrw+NMnwiAawHc9MOKyoV2Wx6jKx4HB+8jUveVu/RQ2pFXoMc0i2yHDj2UjzulSeh9Dq8fEOQ5/gK5E+lmt3bYMYsiodqdPKHrPlDF2qdDxISxcdMTWwHzJZMyAmX9hlqnkkok4ZE3mbal5HZFYv0jrvAG9IJfidwGMwAuXf0+kK0rGuGwp2iLZjcTk/ddwTn2UJGUYJj2KAFv7BHPyBTamD7GHYs29IvdaZNeRQqOmPdA3ZT+8c1pciUx/FYzMko7pwzViGvqzgCs9aoOulkyekyEnWpUWk5xDbpkNfY/g2411cLEpzXL07RHjhguiX8Xjizd2IrEcjfPsmFqOercQQc4ai35xzySdRIEcszcrqSZXvt06md1OwKptAcQBdPNjhLNwtGFTZS5GHza/Z+PnrhZHnDpiw5PKRlNP66bFmLhz7ThNzuMmjfsWNsrli5krj/kjFPkGowf+S6ZBlQNapyk/8btSf2rkVDz5eyzvWkSmM6G1Ne8/ZpGz+UBLWOgSf0XvIXt84/1dAYCDfj19rf++KiVRvUURvPWjrnUaVtVS0inqbDoCPR/2s222dBKW+fozrfyvnLWYd/KG7g+3cqO4wJfAAcKbiXXB62Dlat6kmVB/8poD+6v1e3opLO/aSV346y x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA1PR15MB5109.namprd15.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(451199015)(4326008)(76116006)(66946007)(8676002)(66556008)(66446008)(64756008)(5660300002)(66476007)(41300700001)(86362001)(6916009)(316002)(54906003)(91956017)(2906002)(33656002)(36756003)(8936002)(7416002)(122000001)(478600001)(2616005)(186003)(38100700002)(53546011)(83380400001)(6486002)(6512007)(38070700005)(6506007)(71200400001)(14583001)(45980500001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ghiEkagg3noO7NJ/8xcziSCiLMD860oXDeWKI+T5smFQcKx8UBzosBkfJ6PT?= =?us-ascii?Q?22ut+9cv/BLVGIAXnC2x+UG+wFgZdSSnMwypoufFlWU+IqagsclvAyv+WzYA?= =?us-ascii?Q?rH5E+3peg/KYF4zEI5IJFNJD4GQaHebPZTx2mS4xTROhJ+4DOnsdBTYeHmyl?= =?us-ascii?Q?ahtie6xCXzaosraeey2747q3NqfeQ77hnmv2eWwMRJtL+NNYMSP6tk+Jiw24?= =?us-ascii?Q?zE6I2Otyyi1gfocMdWgabwNxQGNMBf0HM+RmJXSluQ4dXbsLk6rSvKb5NHj5?= =?us-ascii?Q?DrKR+oZertVi0+Oq2JFu0dHmaxmoAucbFVzf6RsIyTjha7Obd+lBIwsIndCh?= =?us-ascii?Q?vFxnht3/xBy2nVF8jOcXbO7wsiy+aVCVjctio1r7znbmlgBHr6atDxz0MSVp?= =?us-ascii?Q?V0tdtc9Rzal5N7bGRgGdQkeUx0PthYhdJItNahWrhMcKWWFnBXgk95UnZ9vh?= =?us-ascii?Q?viwaePXyNGNbUuEBt77y4eEyhx23PfnyWrIu+AkjTqZawyaIpWGZqHWEir1b?= =?us-ascii?Q?ecLyH+3OqqNWEuFDKOmklXrfrggC/paTfQUToXb4SFZ1eE4VAvWTFlyiQCYx?= =?us-ascii?Q?g4yKFxFBRrxeZlkTeztXqGpeMPvXg7oEOKsE3oAq37nj81NqvbPdeCjNe1T2?= =?us-ascii?Q?OTWGT0FJSFv9LP7a6gBi9+23l+SfamHUaV1SN0vLiFyRcvSo12hExO7ceKMy?= =?us-ascii?Q?A2+wPIZBwQ62TKJ1fgSfeS5KGxfm/eMiDp8FYTWYPXjPjapXn7oM9i0hCuM9?= =?us-ascii?Q?j+Loj3zNVRY4SX/S9GOsZD3KWQfxNDMjPVJfua8JdOQiXMgbD8gotUnRfiQS?= =?us-ascii?Q?Zb3fiYVCW4sue4vBY45fvmHqbm60b2wO7VcR5o9gQ3+HFi+RTo6ZSjl5O+HX?= =?us-ascii?Q?zHICtXCjA1WrekjkorSdePBnEniXb/8MWyMbcfXFeSQfKLlR559vvdkH1yJQ?= =?us-ascii?Q?wA7faVQnQyEli1HIaUY64YE9KHBy3yDKcI8md68LMFPJVH4Vtp3QrmqMDayS?= =?us-ascii?Q?e9Am9QDB9dB1hSkWvEZWTK+mlf6R+meyyrdaW7sfXKHkCd7uCI4T9ly3hueS?= =?us-ascii?Q?xSTx1nski+AsBKbc2wuI1vG/pAb6yQNvOjIDe1a9Lk1OkfSzpi5MHNwCq1eI?= =?us-ascii?Q?31oWKGx/tbzjY9LpLBc6qQyBv9YidtOHxE9iUnR1shs8/THN/gBmW6jNV43Q?= =?us-ascii?Q?tkOXTLHov4lTavos1p4bFyyIU8jfVhKti5H1F7NqeEohq/3usgZJP45Mmff9?= =?us-ascii?Q?x3EUGGD0q6x/i4o2yTIgSW2QD/ygy7VASYpsAAzVNAfB0m+rbn7NEnKk24Wl?= =?us-ascii?Q?WvXJR03BXa42TCFldad9HFhHCWDI9WZfSYsUHa4JuNWw9S4y7r2RxPYlq3nR?= =?us-ascii?Q?JJfqVOxg5alGRHbr+LEuxAStHrVNAvz7Q6rKHsfIFQ/gH4FBavj39ZIlVTy6?= =?us-ascii?Q?z0dIGTO1LJQ1junWIK4GKixEFh+3JgG8cIj3gWMBWClVSmq8azCNnTOd4wEA?= =?us-ascii?Q?ilrCz909pcJPZSmdvj6VdEdZtgKJQSy4o84fyjoJCeS9IUed+QNsDI2cTbUE?= =?us-ascii?Q?GSWWUHdV+M5bkGL/seAzVeDT8Z051AnbdNrduianvZvaYD9l2gT8AF3pJHSk?= =?us-ascii?Q?bRIFhMXzfG5cd9zjZXIL1uc=3D?= Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: fb.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SA1PR15MB5109.namprd15.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 03476899-759f-4bfc-d544-08daaaf2df51 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Oct 2022 19:08:53.1679 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: lbdc6vqeiUrgOYGsK2so7RQIP53LUGuEAeoIoODnu0PNIn1tnvAX317g6AycFCkO4TsKdQ9u6xhXCjmpmG3Gnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR15MB3610 X-Proofpoint-ORIG-GUID: 1QTTB9vA4hQYe8UgajSwQ33BQyX0Jtl6 X-Proofpoint-GUID: 1QTTB9vA4hQYe8UgajSwQ33BQyX0Jtl6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-10-10_12,2022-10-10_02,2022-06-22_01 ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1665428944; a=rsa-sha256; cv=pass; b=67cD1PrkHGU/IQFJgoZlhbl8wDujNZA2GqQLbQrkDWxaOSQL1snpIv5T2NQ/YsmhkwrwoN QFo4merp92XE60oMS9tkmF1kGbRSdK1fSYn98hfUb8zlpyF2dSsoW8pKxAxCpX0sZdnCaG I2H1bKC64mu8Tefmttrpc8kZ1W8LAGs= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="Ng/box1E"; dmarc=pass (policy=reject) header.from=fb.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf17.hostedemail.com: domain of "prvs=1282a047fa=songliubraving@fb.com" designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1282a047fa=songliubraving@fb.com" ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665428944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SXPDk7BbCbmsa1nSyXk45xokdnliM4Ysc43ElfvdpYk=; b=Efv139V9BH3Cn9H/3Kr6CvA1NUzAAuFi0kDPmAGKXg9atcdzs03N7/z1ZKGommWBgrarnc /9MEKNIXgrJg9LPvE9yCmotu8yJkcVouRwG02pImnOxz/7mhIJnuJlD6JBvknlOVUJVuPg 0V0DgwR/pxVSjL5qC72hnHg3j6VrW2A= X-Stat-Signature: qmw8nxuh5g1bc9if6cy3yagdzji7cuxz X-Rspamd-Queue-Id: 4708140032 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="Ng/box1E"; dmarc=pass (policy=reject) header.from=fb.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf17.hostedemail.com: domain of "prvs=1282a047fa=songliubraving@fb.com" designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1282a047fa=songliubraving@fb.com" X-Rspam-User: X-Rspamd-Server: rspam03 X-HE-Tag: 1665428944-336852 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Oct 10, 2022, at 11:32 AM, Edgecombe, Rick P wrote: >=20 > On Fri, 2022-10-07 at 16:43 -0700, Song Liu wrote: >> On x86 kernel, we allocate 2MB pages for kernel text up to >> round_down(_etext, 2MB). Therefore, some of the kernel text is still >> on 4kB pages. With vmalloc_exec, we can allocate 2MB pages up to >> round_up(_etext, 2MB), and use the rest of the page for modules and >> BPF programs. >>=20 >> Here is an example: >>=20 >> [root@eth50-1 ~]# grep _etext /proc/kallsyms >> ffffffff82202a08 T _etext >>=20 >> [root@eth50-1 ~]# grep bpf_prog_ /proc/kallsyms | tail -n 3 >> ffffffff8220f920 t >> bpf_prog_cc61a5364ac11d93_handle__sched_wakeup [bpf] >> ffffffff8220fa28 t >> bpf_prog_cc61a5364ac11d93_handle__sched_wakeup_new [bpf] >> ffffffff8220fad4 t >> bpf_prog_3bf73fa16f5e3d92_handle__sched_switch [bpf] >>=20 >> [root@eth50-1 ~]# grep 0xffffffff82200000 >> /sys/kernel/debug/page_tables/kernel >> 0xffffffff82200000- >> 0xffffffff82400000 2M ro PSE x pmd >>=20 >> [root@eth50-1 ~]# grep xfs_flush_inodes /proc/kallsyms >> ffffffff822ba910 t xfs_flush_inodes_worker [xfs] >> ffffffff822bc580 t xfs_flush_inodes [xfs] >>=20 >> ffffffff82200000-ffffffff82400000 is a 2MB page, serving kernel text, >> xfs >> module, and bpf programs. >=20 > Can this memory range be freed as part of a vfree_exec() call then? > Does vmalloc actually try to unmap it? If so, it could get complicated > with PTI. >=20 > It probably should be a special case that never gets fully freed. Right, this is never freed.=20 >=20 >>=20 >> Signed-off-by: Song Liu >> --- >> arch/x86/mm/init_64.c | 3 ++- >> mm/vmalloc.c | 24 ++++++++++++++++++++++++ >> 2 files changed, 26 insertions(+), 1 deletion(-) >>=20 >> diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c >> index 0fe690ebc269..d94f196c541a 100644 >> --- a/arch/x86/mm/init_64.c >> +++ b/arch/x86/mm/init_64.c >> @@ -1367,12 +1367,13 @@ int __init >> deferred_page_init_max_threads(const struct cpumask *node_cpumask) >>=20 >> int kernel_set_to_readonly; >>=20 >> +#define PMD_ALIGN(x) (((unsigned long)(x) + (PMD_SIZE - 1)) & >> PMD_MASK) >> void mark_rodata_ro(void) >> { >> unsigned long start =3D PFN_ALIGN(_text); >> unsigned long rodata_start =3D PFN_ALIGN(__start_rodata); >> unsigned long end =3D (unsigned long)__end_rodata_hpage_align; >> - unsigned long text_end =3D PFN_ALIGN(_etext); >> + unsigned long text_end =3D PMD_ALIGN(_etext); >=20 > This should probably have more logic and adjustments. If etext is PMD > aligned, some of the stuff outside the diff won't do anything. Hmm.. I don't quite follow this comment. If the etext is PMD aligned,=20 we can still use vmalloc_exec to allocate memory. So it shouldn't=20 matter, no? >=20 > Also, if a kernel doesn't have modules or BPF JIT it would be a waste > of memory. I guess we can add a command line argument for these corner cases?=20 Thanks, Song >=20 >> unsigned long rodata_end =3D PFN_ALIGN(__end_rodata); >> unsigned long all_end; >>=20 >> diff --git a/mm/vmalloc.c b/mm/vmalloc.c >> index 9212ff96b871..41509bbec583 100644 >> --- a/mm/vmalloc.c >> +++ b/mm/vmalloc.c >> @@ -75,6 +75,9 @@ static const bool vmap_allow_huge =3D false; >> #define PMD_ALIGN(addr) ALIGN(addr, PMD_SIZE) >> #define PMD_ALIGN_DOWN(addr) ALIGN_DOWN(addr, PMD_SIZE) >>=20 >> +static struct vm_struct text_tail_vm; >> +static struct vmap_area text_tail_va; >> + >> bool is_vmalloc_addr(const void *x) >> { >> unsigned long addr =3D (unsigned long)kasan_reset_tag(x); >> @@ -637,6 +640,8 @@ int is_vmalloc_or_module_addr(const void *x) >> unsigned long addr =3D (unsigned long)kasan_reset_tag(x); >> if (addr >=3D MODULES_VADDR && addr < MODULES_END) >> return 1; >> + if (addr >=3D text_tail_va.va_start && addr < >> text_tail_va.va_end) >> + return 1; >> #endif >> return is_vmalloc_addr(x); >> } >> @@ -2422,6 +2427,24 @@ static void vmap_init_free_space(void) >> } >> } >>=20 >> +static void register_text_tail_vm(void) >> +{ >> + unsigned long start =3D PFN_ALIGN((unsigned long)_etext); >> + unsigned long end =3D PMD_ALIGN((unsigned long)_etext); >> + struct vmap_area *va; >> + >> + va =3D kmem_cache_zalloc(vmap_area_cachep, GFP_NOWAIT); >> + if (WARN_ON_ONCE(!va)) >> + return; >> + text_tail_vm.addr =3D (void *)start; >> + text_tail_vm.size =3D end - start; >> + text_tail_va.va_start =3D start; >> + text_tail_va.va_end =3D end; >> + text_tail_va.vm =3D &text_tail_vm; >> + memcpy(va, &text_tail_va, sizeof(*va)); >> + insert_vmap_area_augment(va, NULL, &free_text_area_root, >> &free_text_area_list); >> +} >> + >> void __init vmalloc_init(void) >> { >> struct vmap_area *va; >> @@ -2432,6 +2455,7 @@ void __init vmalloc_init(void) >> * Create the cache for vmap_area objects. >> */ >> vmap_area_cachep =3D KMEM_CACHE(vmap_area, SLAB_PANIC); >> + register_text_tail_vm(); >>=20 >> for_each_possible_cpu(i) { >> struct vmap_block_queue *vbq;