From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BA74C4332F for ; Fri, 14 Oct 2022 18:26:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B4A5A6B0072; Fri, 14 Oct 2022 14:26:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF9C06B0075; Fri, 14 Oct 2022 14:26:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FE2A6B0078; Fri, 14 Oct 2022 14:26:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 787926B0072 for ; Fri, 14 Oct 2022 14:26:20 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1A2EB160D5B for ; Fri, 14 Oct 2022 18:26:20 +0000 (UTC) X-FDA: 80020384920.01.061B17A Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf25.hostedemail.com (Postfix) with ESMTP id AAE6FA0027 for ; Fri, 14 Oct 2022 18:26:19 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 29EI5qtS002659; Fri, 14 Oct 2022 11:26:07 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=s2048-2021-q4; bh=FjRQAHNLLHPxa78R9NZtij8WlyIp8yq21VqD1/ChSKo=; b=Wp+3UuLP/4fY6Qd08SkMxBj5apvxNGnGiowjKUBUXQx+km7Payra1uVZXYTW66LfcmW+ ktsalTRp7wgxHpCxBjgwbYKuGFk2Tx3WA04+4X4+wtcIoDcprDDKFFdN+B4rt6mvjKkQ JFxTPEVEuUzc0N08syxarCIEZUR0nqJaHOKSuA8dEJT+tqoN2NO29a0zkxf9HUqRQvSZ 4+0rhQBvyqHjxH6ynGzN7S1lpo0FyVnbUwUuPaiJHrbmc3YMtRzzV3azCIUmS6YOBqeI hqCXos65LfDMXqsuU32v/nL7kktqoKCg/gStaQ2WOGUorOjgn3zjhJYgMrUJsz6PYkIO Kw== Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2174.outbound.protection.outlook.com [104.47.57.174]) by m0089730.ppops.net (PPS) with ESMTPS id 3k70cue1qn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Oct 2022 11:26:07 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ihgWxqo6qdsQSmF3Dy2PXK5z+ayBUvTS/7KHijo5IAetqnj9/wiUqVqz+4HYI8MBIHBv7Y45yj5pDMICfEYrv4RKC8mgVjicNabXVnBQv0X3Iu5KQ9Prn28tGDgQCxK3SJ5eRuARNTxQjU4gW0KRKLkc73SGM0KVNJPX3C8qfoZqmSeNLzZmroeV+UdHopK4/4g6f3qgBJn4yJ0MHtPLCmWuMP/qX8c4Nqa9T9rIhZNrywmC0bol6JPb6S+BMAZGYeGtuaPF68I9VY8IE+a4DCaTgjPQUWwFRRbvt05hfLGQft3/l2BBZS9ZqQCtD1kOLZQIsI/Jn3AaJ3e7fEnuTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FjRQAHNLLHPxa78R9NZtij8WlyIp8yq21VqD1/ChSKo=; b=JuyVtsbdaq3odBkK2J38Y5ysxn2xg6X3Zmn/XSH6GCDq9485hlzitfYgd2PxcjGGHJ60l1h49O/QeP10H/PXBYTzTz+05Witu1zXL+OSvGWUIGZ6Ck3mLO6B3ixb5NszIJaXBxac6WheJF3TJ4LHdcYRarsNeST5JW/cXNKfkQtVVlb1Wq3LGhJRxuvCxMx7FR9MyzL3L+B6PiRwKcPxAuvNnXyFdZnAcOAYLVESfMAmBHQgbxPzqlM5CKdgJdN7fe9K0d3vzePMKPORJlaAKQAb1ZqHTQfvR9Q6udUjeq+ia89NMfnr7smjWYYIo415VFO36NVHd64cltFRoCMwDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=meta.com; dmarc=pass action=none header.from=meta.com; dkim=pass header.d=meta.com; arc=none Received: from SA1PR15MB5109.namprd15.prod.outlook.com (2603:10b6:806:1dc::10) by BN7PR15MB2497.namprd15.prod.outlook.com (2603:10b6:406:86::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.15; Fri, 14 Oct 2022 18:26:04 +0000 Received: from SA1PR15MB5109.namprd15.prod.outlook.com ([fe80::d70d:8cce:bb1:e537]) by SA1PR15MB5109.namprd15.prod.outlook.com ([fe80::d70d:8cce:bb1:e537%7]) with mapi id 15.20.5723.026; Fri, 14 Oct 2022 18:26:04 +0000 From: Song Liu To: "Edgecombe, Rick P" CC: "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "song@kernel.org" , "hch@lst.de" , Kernel Team , "peterz@infradead.org" , "urezki@gmail.com" , "akpm@linux-foundation.org" , "x86@kernel.org" , "Hansen, Dave" Subject: Re: [RFC v2 3/4] modules, x86: use vmalloc_exec for module core Thread-Topic: [RFC v2 3/4] modules, x86: use vmalloc_exec for module core Thread-Index: AQHY2qcNgpyftmWu6UK6OlojFXIGNa4OEceAgAAtwQA= Date: Fri, 14 Oct 2022 18:26:04 +0000 Message-ID: <7112B8B4-B593-45AA-A9AD-2ABEEE96223E@fb.com> References: <20221007234315.2877365-1-song@kernel.org> <20221007234315.2877365-4-song@kernel.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3696.120.41.1.1) x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SA1PR15MB5109:EE_|BN7PR15MB2497:EE_ x-ms-office365-filtering-correlation-id: 65dbec71-fcbf-49b2-d1ae-08daae118e1f x-fb-source: Internal x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Dq1Gqi8WAiNRHnvtitqprU5THQPISaCFcXMFNhgvwCHdvC9Xl38FI16BCA9bwDXTcIIF9+zWpGL0XfOLqGzKGRc8yY6QKkURX50hNeuCjNcagoajfYhhioQr+cgrxfK0TjL4S4oPSxYYWbKs2et6cx2MJGBr0sryOs7TgxKBaVANaEn0tUK9H+qBAXfVLxZ4rmPsww3jMSnCV57XmjN4Wo0zSz+HuXd14fTGHF9x4XEd1IvqWZLZ6bDbRfqqbvAFKOQNA3Uufhugwdl4Mtbugej/i/Rkkuaw0LueAEJ7f9f0C2LBaD45abGGJbiu4jvLiR3PIfb8Bkkp2/c12Jsqq7bLN53jPBnRIiKsqlkiyY5XZXYFsJPMKao23LBdIkDxD4r+OS26ysYWy6nSbe3J2ci+E8UbRWfFKusv02KWw1AhWaG4NfH7AZvYELiLlH7F49Ai0/EvbVTrM0z+uEycRxGrJ2LB2qKhypZ2fD9u0AENZwVxaBs+6Y1sLo/XpDoOmXwsYCK8M7mPrGERztEh2uM0Mpelenj5sL4VdKofEZeQxIAiU3X7Q0N+JxX9zgqB/VdTDR/XFtggL2dLI/i4YYh2P+g2FODL2cwq97K9XWYF6NlUFv6cBYZ8izQBe9eR3mV+tL+Fhle6bAPQLQlBhcU1jiwBRUx9TYvPQuuwisLdchh+K1fNuHVX7d0FZxTrbyG/Gd+EkpsSgpNrxormIC7Hs451Mu1MAVNTpllijKwkfQKa7ZwwPwXyw+0wfT08wFvtTnNBsA5okAC0f+plHw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA1PR15MB5109.namprd15.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(136003)(39860400002)(376002)(396003)(366004)(346002)(451199015)(6512007)(6916009)(54906003)(8936002)(9686003)(6486002)(478600001)(5660300002)(7416002)(66446008)(64756008)(316002)(91956017)(4326008)(66556008)(36756003)(8676002)(76116006)(66946007)(66476007)(186003)(71200400001)(33656002)(122000001)(41300700001)(38100700002)(53546011)(38070700005)(83380400001)(6506007)(86362001)(2906002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?kq/cshrpoSWvuLsBMoAOcBsNTl+bJlbPpiRbo+65pAYFiKcU15vtamu2RXVd?= =?us-ascii?Q?YNE/21sBuOUxi/RdtbW9cxusl9ChFwcXXwQ4a3mOM6DfNJqibP3PtsdxqGE8?= =?us-ascii?Q?hURX1MnJSH2MOqCc3b9yYqZHLUOJSi9I6FfMmLl9rR28PmVEJzP8TSCFb1an?= =?us-ascii?Q?xtp6TTYCqOD5Gj0PNznBmzKCyQawP1ZCxCrAT7PeAcmxFASj8gRwb4GpZnSb?= =?us-ascii?Q?Nai3UyLmPYSsAKv3/TBy+60/Q9N1kAC8rk/qVxMYg5VxvV5jNujYjeO/Ygds?= =?us-ascii?Q?Lxlqd4Jf01wB2i9FWFYFqnxjzmabk44mtwaiib1XU2AmT/JJFn7huXLhbmBw?= =?us-ascii?Q?I0PWVxNCT31ig1rWZjH4g47W6gb/AaY05noPxf+JXZr29bZ4sPXydQEaQ4op?= =?us-ascii?Q?04GfxPw/CHkpK5TASPwiuthZwugR8sz87UIpEd0eYqQnRH1Q4ncPOytvs8ob?= =?us-ascii?Q?w0y/yeb8rSUZd9XfuoKlDSUTbKkMgIdRJeJ9TvOC1VyCEq2HJTB0i/avX5DU?= =?us-ascii?Q?JQzc2dPXkPjJXGNmCoEc9Nm9CLS0im5VXPluByDfnXhTvne0gKjikMrEnJBR?= =?us-ascii?Q?3xQjGcGuGrUdgQCZIlhU+X/Vld4w1LQzhjf9PCbsehytW+WK3CcniorWIrP0?= =?us-ascii?Q?b5ua0twBzazRph1MPYGkgD1w47/eVVm2N+8WA+bNpBhy/YvFbqPw6V3hgLhC?= =?us-ascii?Q?IasTymbBcWOLo44ZX9upCly0xgz4cE9E4z9bRz8/AbYR5rIGyyoVyWQ5XFaM?= =?us-ascii?Q?gsEjrqDryCYahLAX6/mfo1PiKPKK8atdXpjPLjP+RxKQ7zTCnXORaUsBejru?= =?us-ascii?Q?i5rBXmDmj6s+aNRnqBfAebIFHXwMZQdzz5yYz5QHvWGqZLHjgWckGkbnYjCg?= =?us-ascii?Q?z/kIfH4Zbltg22rgLnO8Kz+GBpwfWyFiL1OalCR+RwxFeVqs2lvkjD7eWnIL?= =?us-ascii?Q?XVfwRK8lyU6hIPibu6wDWoIfilwKXXWfpDClRY7/lbXYsKMSuIsEb1Kb/jto?= =?us-ascii?Q?mhGsM/GBcuC2qzIt/Hp6xPYPzDyhfyT65t6cxW/FDRsb7/AxUr2JvnlB3KMN?= =?us-ascii?Q?f3CCrHPlbYQJfHInDsrW/qErSo7buj5jw1ZP53w3XymzqNrdUIqPlEE6YX3I?= =?us-ascii?Q?36eGHGFsRK5fhyuoKnJbPKcDtL8AMxgF2V+WeO6ph2rYj50UEf1v7FnJgXNR?= =?us-ascii?Q?naal3SXT3nCVkvqyKFLJJ+FK5rtMyk9XEdnxW5DASzx0SRNQyJeHijllr0ic?= =?us-ascii?Q?Twya5jeXsPQlcN7a4RsLu4daeg5kfpb3sEEYdqMHtweuZWlW2gABJ8XVTUUG?= =?us-ascii?Q?L9vLBahBgvijax+K4qbF7v9au660zSKMIoZ63+RDZGIDCBSvgY9FV3XLnZwZ?= =?us-ascii?Q?DVlK2AIaAcWsTRmkO0UyBDgzYUWZZrbAa37k1fnZpEmezvhFE80hvQcdaLPr?= =?us-ascii?Q?znBwX5qbhOm+MEgQCuEPwdO7QfBO6v0Hgjaiz+2DkEarl6GkM8k7JpDg9H7Z?= =?us-ascii?Q?6sEALv5xS61vmtaUKQvaxJyS1z81xnJeJmPw93fau+1zz+mwhPeRPMPIZbRY?= =?us-ascii?Q?50wGK+oWIdMTXc26GlFKZUv+5P2dcS/ynjar379k6CBcrVjXw1Op6D4smIOb?= =?us-ascii?Q?L99RAWxVaRhQ7eRP6fS8f39aQkJDhxFe8cUCYzuhSBp+?= Content-Type: text/plain; charset="us-ascii" Content-ID: <989E48E96AE1A847882E6CE55A36BDB9@namprd15.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: meta.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SA1PR15MB5109.namprd15.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65dbec71-fcbf-49b2-d1ae-08daae118e1f X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Oct 2022 18:26:04.8004 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DJm+aYQrPkWgRa3A7BSVkjWlbUFcXoyQoYJynXj0/SnM3GsqlYLuYLQJwueN0hxST4dR9qcDPpIXAoYQ6PEMBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR15MB2497 X-Proofpoint-GUID: RNf8PEdE8ypJ2_UjCUyLpSE2rOZwFdW6 X-Proofpoint-ORIG-GUID: RNf8PEdE8ypJ2_UjCUyLpSE2rOZwFdW6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-14_09,2022-10-14_01,2022-06-22_01 ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1665771979; a=rsa-sha256; cv=pass; b=4dTFPdKiZ1PkuS5wqJ2LihZ7JpFGqMsTyCAx/aeltOVJ38GrX4ZBp3izjmFDjyXDlYpmHy hIP445oDK/WpUuod255VRFC73WmgDgySozHbZ843pCwiYqffjdSj4lLe4pQ6p97sf/D0kZ 8TgqB3xGyYwz7c5BfcOnMsyG+VhHJ+o= ARC-Authentication-Results: i=2; imf25.hostedemail.com; dkim=pass header.d=meta.com header.s=s2048-2021-q4 header.b=Wp+3UuLP; dmarc=pass (policy=reject) header.from=meta.com; spf=pass (imf25.hostedemail.com: domain of "prvs=1286a03188=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1286a03188=songliubraving@meta.com"; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665771979; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FjRQAHNLLHPxa78R9NZtij8WlyIp8yq21VqD1/ChSKo=; b=QIodTrjb6TIRmJ25RbSie/kkPXx2zMCbDFb8nyC2HniwRUQDgfq32Wz+d4CpCLDh/mcFxl DAFcDK6jmM+DRt4LyyecYQ8CzSttwzehh/G123HCGMgwqvRD8kpWqZv3GJ6kvNVZ2jSupL takr5iptHcvaaz5t/iUTdxCCZ6UvCNs= X-Stat-Signature: x1sya6sp8515n7sesmjctuzfdnw45ndt X-Rspamd-Queue-Id: AAE6FA0027 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=meta.com header.s=s2048-2021-q4 header.b=Wp+3UuLP; dmarc=pass (policy=reject) header.from=meta.com; spf=pass (imf25.hostedemail.com: domain of "prvs=1286a03188=songliubraving@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1286a03188=songliubraving@meta.com"; arc=pass ("microsoft.com:s=arcselector9901:i=1") X-Rspam-User: X-Rspamd-Server: rspam03 X-HE-Tag: 1665771979-837008 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Oct 14, 2022, at 8:42 AM, Edgecombe, Rick P wrote: >=20 > On Fri, 2022-10-07 at 16:43 -0700, Song Liu wrote: >> diff --git a/kernel/module/main.c b/kernel/module/main.c >> index a4e4d84b6f4e..b44806e31a56 100644 >> --- a/kernel/module/main.c >> +++ b/kernel/module/main.c >> @@ -53,6 +53,7 @@ >> #include >> #include >> #include >> +#include >> #include >> #include "internal.h" >>=20 >> @@ -1203,7 +1204,7 @@ static void free_module(struct module *mod) >> lockdep_free_key_range(mod->data_layout.base, mod- >>> data_layout.size); >>=20 >> /* Finally, free the core (containing the module structure) >> */ >> - module_memfree(mod->core_layout.base); >> + vfree_exec(mod->core_layout.base); >> #ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC >> vfree(mod->data_layout.base); >> #endif >> @@ -1321,7 +1322,8 @@ static int simplify_symbols(struct module *mod, >> const struct load_info *info) >> ksym =3D resolve_symbol_wait(mod, info, name); >> /* Ok if resolved. */ >> if (ksym && !IS_ERR(ksym)) { >> - sym[i].st_value =3D >> kernel_symbol_value(ksym); >> + unsigned long val =3D >> kernel_symbol_value(ksym); >> + bpf_arch_text_copy(&sym[i].st_value, >> &val, sizeof(val)); >=20 > Why bpf_arch_text_copy()? This of course won't work for other > architectures. So there needs to be fallback method. That RFC broke the > operation into two stages: Loading and finalized. When loading, on non- > x86 the writes would simply be to the allocation mapped as writable. > When it was finalized it changed it to it's final permission (RO, etc). > Then for x86 it does text_pokes() for the writes and has it RO from the > beginning. Yeah, this one (3/4) is really a prototype to show vmalloc_exec could=20 work for modules (with a lot more work of course). And something to replace bpf_arch_text_copy() is one of the issues we need to address in the future.=20 >=20 > I ended up needing a staging buffer for modules too, so that the code > could operate on it directly. I can't remember why that was, it might > be unneeded now since you moved data out of the core allocation. Both bpf_jit and bpf_dispather uses a staging buffer with bpf_prog_pack.=20 The benefit of this approach is that it minimizes the number of=20 text_poke/copy() calls. OTOH, it is quite a pain to make all the=20 relative calls correct, as the staging buffer has different address to=20 the final allocation.=20 I think we may not need the staging buffer for modules, as module=20 load/unload happens less often than BPF program JITs (so it is ok for=20 it to be slightly slower).=20 btw: I cannot take credit for split module data out of core allocation, Christophe Leroy did the work. :) Thanks, Song=