From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69E68CF11FD for ; Thu, 10 Oct 2024 15:18:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4C056B0082; Thu, 10 Oct 2024 11:18:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFBFD6B0083; Thu, 10 Oct 2024 11:18:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9C416B0085; Thu, 10 Oct 2024 11:18:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8B1EC6B0082 for ; Thu, 10 Oct 2024 11:18:21 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8F0D21604E7 for ; Thu, 10 Oct 2024 15:18:17 +0000 (UTC) X-FDA: 82658048802.25.31F81E6 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf08.hostedemail.com (Postfix) with ESMTP id 0E00516000D for ; Thu, 10 Oct 2024 15:18:17 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JJJWVitm; spf=pass (imf08.hostedemail.com: domain of maz@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=maz@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728573428; a=rsa-sha256; cv=none; b=Dfl7X8Z27DS0eq436nOgBTJNjiKKTEfBQu0unyfIsGbB+beVaFDBPDifKVt1+Izgyw2SSU EtzeFTo5qJcxjB1EiYS6qCoS5NFj/SXqOIrG0NPs/5fU2UoVCoa3Q7SohycY8o4JZOWUgj vIl7fAL4G7KPB2iTpdN5xadFGUXr/6k= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JJJWVitm; spf=pass (imf08.hostedemail.com: domain of maz@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=maz@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728573428; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=54MtzJlR+SnVq+d+No5FF71L1Wan5yhRMhXOi5964gE=; b=noxz7HJpmg9rVOG9hsCNCuOu7QaE4acrXrhXbqQb7FwQleXzDgB4pgrFehjoY038HoT+kc O4hpRX3jZcx2P6UVre+cwH3yQmHNbuaLHspqEaYcYI6j+Zfy0XI3FUCvEaBFVAMlGmBfeO MYRUp2XljWSM2SH4gnpFGObSRLnXAfM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 5677CA44A1E; Thu, 10 Oct 2024 15:18:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D004C4CEC5; Thu, 10 Oct 2024 15:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728573497; bh=6dvKTY2cy/sodJ8sieN7BO/TIIO+P82VgqdOgk5UcTA=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=JJJWVitmpvLvarMUPZUkuoB8gP29XOw0sXNcXNa5kByyDAx7zTpXCXE3HAPvwucIc nqKSjxxqVqe/grf/6yTnmqZ60aylmorsMhYnLl/p7veN8L+6Z6UpBle4GJuML4f7xb i4kReeFAE20WpHLaj8KfSAKT2u3eSPxMo5z1CNQJcQfr/ga40kbvqrkDBDHrIhHoXF Fx9jWKSkz74XO4oqeZAS85rWP/9HC/v2M1aUJUQw6PW1C+qyBOnY/aeVO+soSpZuga 63WABLS/bqHPf7/DMKi/vZmUVhEJ5IYzs9ZH3gGYScj5D1kIistrSeFfZCohaujqqN f91Lx8662bKcA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1syuve-002Eqs-J7; Thu, 10 Oct 2024 16:18:14 +0100 Date: Thu, 10 Oct 2024 16:18:13 +0100 Message-ID: <86msjc56mi.wl-maz@kernel.org> From: Marc Zyngier To: Nathan Chancellor Cc: Mark Brown , Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook , "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: Re: [PATCH v13 11/40] arm64/gcs: Provide basic EL2 setup to allow GCS usage at EL0 and EL1 In-Reply-To: <20241009204903.GA3353168@thelio-3990X> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> <20241001-arm64-gcs-v13-11-222b78d87eee@kernel.org> <20241009204903.GA3353168@thelio-3990X> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: nathan@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, will@kernel.org, corbet@lwn.net, akpm@linux-foundation.org, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, arnd@arndb.de, oleg@redhat.com, ebiederm@xmission.com, shuah@kernel.org, rick.p.edgecombe@intel.com, debug@rivosinc.com, ardb@kernel.org, Szabolcs.Nagy@arm.com, kees@kernel.org, hjl.tools@gmail.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, fweimer@redhat.com, brauner@kernel.org, thiago.bauermann@linaro.org, ross.burton@arm.com, david.spickett@arm.com, yury.khrustalev@arm.com, wilco.dijkstra@arm.com, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-Stat-Signature: h9o9z591z55ja9iswqgodeyjo8jyftux X-Rspamd-Queue-Id: 0E00516000D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1728573497-686471 X-HE-Meta: U2FsdGVkX1972bXGRPz3qQ6YdeHvtxGwy6Dfm0zPgQiXE/N9oBERdkqCssCQFHOSNmLuYyT9Y3Bca+yTZduEVzLH1aJPrCIjbVP8fmY5feYHpKkiZVY3tMraDiaUKWykpXnnot7NLuuy+fT8HopaRz6L5a1i3fV/2JFhJW8nboSdbhOoi3iyKS6FQMO2zJSdDxw+9/aI307ftC0wsMVeQLZSS7s7T6XqPwcx7O/om67aERHjncfLitxFNkFIYU7pyS8wvC8loTOfbFDz+NcpH8/RuEe+5rHuNmEJiKFHmiX3BvGn2v7XLf7F5MwqCpC4/YRIPHmXzHkQs23iqBB9GHssMsRkb4m/RRDq+95QAW+y6Vfqh0gXxw0Wrh6KcI0ooWbWrxyuW6oUqkPD87YkLdxDN52BJ3VmJXfXibFuUJWKNYDPDFzuDU/QAjFzJwUNRRu6DmT4dgpngAzmYjR0PLYMsySG1eY9uTQgHqabbIRIgQZeuPIA1NGpBUGRzsT7xMWtY3xpGkmCX+LLuYlPS42vf2sSA6tE6aL+2z3gnMJqkbe27Uz7WZ5pE4Bxuz54C8XsFKwoSoAgPT0QF46ieCgADIVT0WObUYwf9lTj4u2ynqe/m6TyObZDtWelelQzZQ0EEFnlHns/AXGkUxSdIxJIr2GxknuB0o4oOMoBVrrAGlQgVA0KTvfCVcAS5FfDh4YvGNkkc2FlLVNkF6aDKYG4r72RFvMWG78cIVp56ioOX5Pr3HQpLUktWdxgBGfdUvJQx57FOl93pwmlma2HSaFfYrT2c3NQT7OLPuJgcUDHwwj006uho/ltj6FiIQWlgl1N2fsg8OHCpoDBmBqRCyB3JERJzJJFhIiWPRWR/Uaq0DkyOEFb4HsNK4lrm3U8Bys0oXNa+hUCuZp0KlOX+46Whj6n/uFRlDQcJYzIK6AFDwRLNBWjaAi/s/1MVUPK1nTUnUKA8QBRqZl706B +lpqSvY0 4bvyRfHoiuz90T6q12CTjx1XHbBpyCBTQpH9tEre3xB7Xy+mb0/LCIAKvnTNTXEB+LfzBCJejd08OqbQKmFwEXJRa0krGNIZEAM66Nte2VM5Oi16qVR+KT7O406AmsCGAGlXUlbOZXDX5nen/X96gaUvOZrJarllxd4QehfUTbUIJoh6lcGNSUSOF9Zpg+lw/c7TDYesnjZLZNoFSF+wjcxM9gpQ8Ev/6JzszmVk7q7Bl/LMkcdZCE82kEH2lWJISrt0opifCpk0bnKJf+I1oSOZQdK79csBRsuBorO5f27GPA32Unt+b+IMcGcYOLESs5EvRtiJx1aOn7BGlPf/ikjBKNlIoYv5XZnmQVrRquu6U6lDYoe2BU9YvZoEdNfRXtcxVspwBFHzGBuV42qYnYbepbfaZGoVJt/Oo3RIIk3QOip44MjCPb5xKSw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 09 Oct 2024 21:49:03 +0100, Nathan Chancellor wrote: >=20 > Hi Mark, >=20 > On Tue, Oct 01, 2024 at 11:58:50PM +0100, Mark Brown wrote: > > There is a control HCRX_EL2.GCSEn which must be set to allow GCS > > features to take effect at lower ELs and also fine grained traps for GCS > > usage at EL0 and EL1. Configure all these to allow GCS usage by EL0 and > > EL1. > >=20 > > We also initialise GCSCR_EL1 and GCSCRE0_EL1 to ensure that we can > > execute function call instructions without faulting regardless of the > > state when the kernel is started. > >=20 > > Reviewed-by: Thiago Jung Bauermann > > Reviewed-by: Catalin Marinas > > Signed-off-by: Mark Brown >=20 > I just bisected a build failure from a failed linker script assertion > that I see with allmodconfig to this change in -next as commit > ff5181d8a2a8 ("arm64/gcs: Provide basic EL2 setup to allow GCS usage at > EL0 and EL1"): >=20 > $ make -skj"$(nproc)" ARCH=3Darm64 CROSS_COMPILE=3Daarch64-linux- mrpro= per allmodconfig vmlinux > aarch64-linux-ld: HYP init code too big > make[4]: *** [scripts/Makefile.vmlinux:34: vmlinux] Error 1 > ... >=20 > I see this with both GCC 14 and clang 19, in case toolchain version > matters. Bisect log included as well. Grmbl... 16 bytes too big. The hack below buys us about ~600 bytes by removing some duplication, but we're losing half of the space to the vectors. Anyway, this is very lightly tested and it may eat your box. Thanks, M. =46rom 20c98d2647c11db1e40768f92c5998ff5d764a3a Mon Sep 17 00:00:00 2001 From: Marc Zyngier Date: Thu, 10 Oct 2024 16:13:26 +0100 Subject: [PATCH] KVM: arm64: Shave a few bytes from the EL2 idmap code Our idmap is becoming too big, to the point where it doesn't fit in a 4kB page anymore. There are some low-hanging fruits though, such as the el2_init_state horror that is expanded 3 times in the kernel. Let's at least limit ourselves to two copies, which makes the kernel link again. At some point, we'll have to have a better way of doing this. Reported-by: Nathan Chancellor Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20241009204903.GA3353168@thelio-3990X --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/nvhe/hyp-init.S | 52 +++++++++++++++++------------- 3 files changed, 31 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_= asm.h index b36a3b6cc0116..67afac659231e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -178,6 +178,7 @@ struct kvm_nvhe_init_params { unsigned long hcr_el2; unsigned long vttbr; unsigned long vtcr; + unsigned long tmp; }; =20 /* diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offset= s.c index 27de1dddb0abe..b21dd24b8efc3 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -146,6 +146,7 @@ int main(void) DEFINE(NVHE_INIT_HCR_EL2, offsetof(struct kvm_nvhe_init_params, hcr_el2)= ); DEFINE(NVHE_INIT_VTTBR, offsetof(struct kvm_nvhe_init_params, vttbr)); DEFINE(NVHE_INIT_VTCR, offsetof(struct kvm_nvhe_init_params, vtcr)); + DEFINE(NVHE_INIT_TMP, offsetof(struct kvm_nvhe_init_params, tmp)); #endif #ifdef CONFIG_CPU_PM DEFINE(CPU_CTX_SP, offsetof(struct cpu_suspend_ctx, sp)); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/h= yp-init.S index 401af1835be6b..fc18662260676 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -24,28 +24,25 @@ .align 11 =20 SYM_CODE_START(__kvm_hyp_init) - ventry __invalid // Synchronous EL2t - ventry __invalid // IRQ EL2t - ventry __invalid // FIQ EL2t - ventry __invalid // Error EL2t + ventry . // Synchronous EL2t + ventry . // IRQ EL2t + ventry . // FIQ EL2t + ventry . // Error EL2t =20 - ventry __invalid // Synchronous EL2h - ventry __invalid // IRQ EL2h - ventry __invalid // FIQ EL2h - ventry __invalid // Error EL2h + ventry . // Synchronous EL2h + ventry . // IRQ EL2h + ventry . // FIQ EL2h + ventry . // Error EL2h =20 ventry __do_hyp_init // Synchronous 64-bit EL1 - ventry __invalid // IRQ 64-bit EL1 - ventry __invalid // FIQ 64-bit EL1 - ventry __invalid // Error 64-bit EL1 + ventry . // IRQ 64-bit EL1 + ventry . // FIQ 64-bit EL1 + ventry . // Error 64-bit EL1 =20 - ventry __invalid // Synchronous 32-bit EL1 - ventry __invalid // IRQ 32-bit EL1 - ventry __invalid // FIQ 32-bit EL1 - ventry __invalid // Error 32-bit EL1 - -__invalid: - b . + ventry . // Synchronous 32-bit EL1 + ventry . // IRQ 32-bit EL1 + ventry . // FIQ 32-bit EL1 + ventry . // Error 32-bit EL1 =20 /* * Only uses x0..x3 so as to not clobber callee-saved SMCCC registers. @@ -76,6 +73,13 @@ __do_hyp_init: eret SYM_CODE_END(__kvm_hyp_init) =20 +SYM_CODE_START_LOCAL(__kvm_init_el2_state) + /* Initialize EL2 CPU state to sane values. */ + init_el2_state // Clobbers x0..x2 + finalise_el2_state + ret +SYM_CODE_END(__kvm_init_el2_state) + /* * Initialize the hypervisor in EL2. * @@ -102,9 +106,12 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) // TPIDR_EL2 is used to preserve x0 across the macro maze... isb msr tpidr_el2, x0 - init_el2_state - finalise_el2_state + str lr, [x0, #NVHE_INIT_TMP] + + bl __kvm_init_el2_state + mrs x0, tpidr_el2 + ldr lr, [x0, #NVHE_INIT_TMP] =20 1: ldr x1, [x0, #NVHE_INIT_TPIDR_EL2] @@ -199,9 +206,8 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu) =20 2: msr SPsel, #1 // We want to use SP_EL{1,2} =20 - /* Initialize EL2 CPU state to sane values. */ - init_el2_state // Clobbers x0..x2 - finalise_el2_state + bl __kvm_init_el2_state + __init_el2_nvhe_prepare_eret =20 /* Enable MMU, set vectors and stack. */ --=20 2.39.2 --=20 Without deviation from the norm, progress is not possible.