From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07600C433ED for ; Sat, 24 Apr 2021 11:31:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 09DDE61584 for ; Sat, 24 Apr 2021 11:31:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09DDE61584 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 603A86B0036; Sat, 24 Apr 2021 07:31:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B32B6B006C; Sat, 24 Apr 2021 07:31:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 405F16B006E; Sat, 24 Apr 2021 07:31:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 20E656B0036 for ; Sat, 24 Apr 2021 07:31:23 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BD033180357EE for ; Sat, 24 Apr 2021 11:31:22 +0000 (UTC) X-FDA: 78067044804.28.6F2B694 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by imf10.hostedemail.com (Postfix) with ESMTP id EFE6140002E6 for ; Sat, 24 Apr 2021 11:30:12 +0000 (UTC) X-Originating-IP: 2.7.49.219 Received: from [192.168.1.12] (lfbn-lyo-1-457-219.w2-7.abo.wanadoo.fr [2.7.49.219]) (Authenticated sender: alex@ghiti.fr) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id D983D1BF208; Sat, 24 Apr 2021 11:30:17 +0000 (UTC) Subject: Re: [PATCH v8] RISC-V: enable XIP To: Vitaly Wool , Palmer Dabbelt Cc: Paul Walmsley , Albert Ou , linux-riscv , LKML , linux-arch@vger.kernel.org, Linux-MM References: <20210413063514.23105-1-alex@ghiti.fr> From: Alex Ghiti Message-ID: <5c3aa9e5-c074-5ac6-6ad1-2c8e0d9fd48f@ghiti.fr> Date: Sat, 24 Apr 2021 07:30:17 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: fr X-Rspamd-Queue-Id: EFE6140002E6 X-Stat-Signature: kwonh4mntzeohjg7o9c1o5615bkun59u X-Rspamd-Server: rspam02 Received-SPF: none (ghiti.fr>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=relay8-d.mail.gandi.net; client-ip=217.70.183.201 X-HE-DKIM-Result: none/none X-HE-Tag: 1619263812-949947 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Le 4/23/21 =C3=A0 5:27 PM, Vitaly Wool a =C3=A9crit=C2=A0: > Hi Palmer, >=20 > On Fri, Apr 23, 2021 at 6:22 AM Palmer Dabbelt wro= te: >> >> On Mon, 12 Apr 2021 23:35:14 PDT (-0700), alex@ghiti.fr wrote: >>> From: Vitaly Wool >>> >>> Introduce XIP (eXecute In Place) support for RISC-V platforms. >>> It allows code to be executed directly from non-volatile storage >>> directly addressable by the CPU, such as QSPI NOR flash which can >>> be found on many RISC-V platforms. This makes way for significant >>> optimization of RAM footprint. The XIP kernel is not compressed >>> since it has to run directly from flash, so it will occupy more >>> space on the non-volatile storage. The physical flash address used >>> to link the kernel object files and for storing it has to be known >>> at compile time and is represented by a Kconfig option. >>> >>> XIP on RISC-V will for the time being only work on MMU-enabled >>> kernels. >>> >>> Signed-off-by: Alexandre Ghiti [ Rebase on top of "Mo= ve >>> kernel mapping outside the linear mapping" ] >>> Signed-off-by: Vitaly Wool >=20 > I'm currently looking at this patch in for-next > [https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/commit= /?h=3Dfor-next&id=3D2fd9503070f7e52cc8397845beb27ece2bcfe410] > and see unresolved rejects, i. e. in '@@ -451,8 +547,13 @@ asmlinkage > void __init setup_vm(uintptr_t dtb_pa)' chunk. Am I looking at the > wrong place? I thought I'd flag it anyway just in case. The patches I have done for fixing 32b kernel messed things up, so I'm=20 not surprised Palmer struggled to merge everything. @Palmer: FWIW, I fixed all those things here, hope it helps! https://github.com/AlexGhiti/riscv-linux/tree/int/alex%2Ffor_next_fix_5.1= 3 >=20 > Best regards, > Vitaly >=20 >>> --- >>> arch/riscv/Kconfig | 55 +++++++++++- >>> arch/riscv/Makefile | 8 +- >>> arch/riscv/boot/Makefile | 13 +++ >>> arch/riscv/include/asm/page.h | 21 +++++ >>> arch/riscv/include/asm/pgtable.h | 25 +++++- >>> arch/riscv/kernel/head.S | 46 +++++++++- >>> arch/riscv/kernel/head.h | 3 + >>> arch/riscv/kernel/setup.c | 10 ++- >>> arch/riscv/kernel/vmlinux-xip.lds.S | 133 +++++++++++++++++++++++++= +++ >>> arch/riscv/kernel/vmlinux.lds.S | 6 ++ >>> arch/riscv/mm/init.c | 115 ++++++++++++++++++++++-- >>> 11 files changed, 418 insertions(+), 17 deletions(-) >>> create mode 100644 arch/riscv/kernel/vmlinux-xip.lds.S >>> >>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >>> index 8ea60a0a19ae..7c7efdd67a10 100644 >>> --- a/arch/riscv/Kconfig >>> +++ b/arch/riscv/Kconfig >>> @@ -28,7 +28,7 @@ config RISCV >>> select ARCH_HAS_PTE_SPECIAL >>> select ARCH_HAS_SET_DIRECT_MAP >>> select ARCH_HAS_SET_MEMORY >>> - select ARCH_HAS_STRICT_KERNEL_RWX if MMU >>> + select ARCH_HAS_STRICT_KERNEL_RWX if MMU && !XIP_KERNEL >> >> We now also have STRICT_MODULE_RWX, which causes a merge conflict. I'= ve >> added both as !XIP_KERNEL, though TBH I'm not even sure thot modules >> make any sense with XIP. >> >>> select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAS= T >>> select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX >>> select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT >>> @@ -441,7 +441,7 @@ config EFI_STUB >>> >>> config EFI >>> bool "UEFI runtime support" >>> - depends on OF >>> + depends on OF && !XIP_KERNEL >>> select LIBFDT >>> select UCS2_STRING >>> select EFI_PARAMS_FROM_FDT >>> @@ -465,11 +465,60 @@ config STACKPROTECTOR_PER_TASK >>> def_bool y >>> depends on STACKPROTECTOR && CC_HAVE_STACKPROTECTOR_TLS >>> >>> +config PHYS_RAM_BASE_FIXED >>> + bool "Explicitly specified physical RAM address" >>> + default n >>> + >>> +config PHYS_RAM_BASE >>> + hex "Platform Physical RAM address" >>> + depends on PHYS_RAM_BASE_FIXED >>> + default "0x80000000" >>> + help >>> + This is the physical address of RAM in the system. It has to = be >>> + explicitly specified to run early relocations of read-write d= ata >>> + from flash to RAM. >>> + >>> +config XIP_KERNEL >>> + bool "Kernel Execute-In-Place from ROM" >>> + depends on MMU && SPARSEMEM >>> + select PHYS_RAM_BASE_FIXED >>> + help >>> + Execute-In-Place allows the kernel to run from non-volatile s= torage >>> + directly addressable by the CPU, such as NOR flash. This save= s RAM >>> + space since the text section of the kernel is not loaded from= flash >>> + to RAM. Read-write sections, such as the data section and st= ack, >>> + are still copied to RAM. The XIP kernel is not compressed si= nce >>> + it has to run directly from flash, so it will take more space= to >>> + store it. The flash address used to link the kernel object f= iles, >>> + and for storing it, is configuration dependent. Therefore, if= you >>> + say Y here, you must know the proper physical address where t= o >>> + store the kernel image depending on your own flash memory usa= ge. >>> + >>> + Also note that the make target becomes "make xipImage" rather= than >>> + "make zImage" or "make Image". The final kernel binary to pu= t in >>> + ROM memory will be arch/riscv/boot/xipImage. >>> + >>> + SPARSEMEM is required because the kernel text and rodata that= are >>> + flash resident are not backed by memmap, then any attempt to = get >>> + a struct page on those regions will trigger a fault. >>> + >>> + If unsure, say N. >>> + >>> +config XIP_PHYS_ADDR >>> + hex "XIP Kernel Physical Location" >>> + depends on XIP_KERNEL >>> + default "0x21000000" >>> + help >>> + This is the physical address in your flash memory the kernel = will >>> + be linked for and stored to. This address is dependent on yo= ur >>> + own flash usage. >>> + >>> endmenu >>> >>> config BUILTIN_DTB >>> - def_bool n >>> + bool >>> depends on OF >>> + default y if XIP_KERNEL >>> >>> menu "Power management options" >>> >>> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile >>> index 1368d943f1f3..8fcbec03974d 100644 >>> --- a/arch/riscv/Makefile >>> +++ b/arch/riscv/Makefile >>> @@ -82,7 +82,11 @@ CHECKFLAGS +=3D -D__riscv -D__riscv_xlen=3D$(BITS) >>> >>> # Default target when executing plain make >>> boot :=3D arch/riscv/boot >>> +ifeq ($(CONFIG_XIP_KERNEL),y) >>> +KBUILD_IMAGE :=3D $(boot)/xipImage >>> +else >>> KBUILD_IMAGE :=3D $(boot)/Image.gz >>> +endif >>> >>> head-y :=3D arch/riscv/kernel/head.o >>> >>> @@ -95,12 +99,14 @@ PHONY +=3D vdso_install >>> vdso_install: >>> $(Q)$(MAKE) $(build)=3Darch/riscv/kernel/vdso $@ >>> >>> +ifneq ($(CONFIG_XIP_KERNEL),y) >>> ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_SOC_CANAAN),yy) >>> KBUILD_IMAGE :=3D $(boot)/loader.bin >>> else >>> KBUILD_IMAGE :=3D $(boot)/Image.gz >>> endif >>> -BOOT_TARGETS :=3D Image Image.gz loader loader.bin >>> +endif >>> +BOOT_TARGETS :=3D Image Image.gz loader loader.bin xipImage >>> >>> all: $(notdir $(KBUILD_IMAGE)) >>> >>> diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile >>> index 03404c84f971..6bf299f70c27 100644 >>> --- a/arch/riscv/boot/Makefile >>> +++ b/arch/riscv/boot/Makefile >>> @@ -17,8 +17,21 @@ >>> KCOV_INSTRUMENT :=3D n >>> >>> OBJCOPYFLAGS_Image :=3D-O binary -R .note -R .note.gnu.build-id -R = .comment -S >>> +OBJCOPYFLAGS_xipImage :=3D-O binary -R .note -R .note.gnu.build-id -= R .comment -S >>> >>> targets :=3D Image Image.* loader loader.o loader.lds loader.bin >>> +targets :=3D Image Image.* loader loader.o loader.lds loader.bin xip= Image >>> + >>> +ifeq ($(CONFIG_XIP_KERNEL),y) >>> + >>> +quiet_cmd_mkxip =3D $(quiet_cmd_objcopy) >>> +cmd_mkxip =3D $(cmd_objcopy) >>> + >>> +$(obj)/xipImage: vmlinux FORCE >>> + $(call if_changed,mkxip) >>> + @$(kecho) ' Physical Address of xipImage: $(CONFIG_XIP_PHYS_AD= DR)' >>> + >>> +endif >>> >>> $(obj)/Image: vmlinux FORCE >>> $(call if_changed,objcopy) >>> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/p= age.h >>> index 22cfb2be60dc..635c6b486806 100644 >>> --- a/arch/riscv/include/asm/page.h >>> +++ b/arch/riscv/include/asm/page.h >>> @@ -91,6 +91,9 @@ typedef struct page *pgtable_t; >>> #ifdef CONFIG_MMU >>> extern unsigned long va_pa_offset; >>> extern unsigned long va_kernel_pa_offset; >>> +#ifdef CONFIG_XIP_KERNEL >>> +extern unsigned long va_kernel_xip_pa_offset; >>> +#endif >>> extern unsigned long pfn_base; >>> #define ARCH_PFN_OFFSET (pfn_base) >>> #else >>> @@ -102,11 +105,29 @@ extern unsigned long pfn_base; >>> extern unsigned long kernel_virt_addr; >>> >>> #define linear_mapping_pa_to_va(x) ((void *)((unsigned long)(x) += va_pa_offset)) >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define kernel_mapping_pa_to_va(y) ({ = \ >>> + unsigned long _y =3D y; = \ >>> + (_y >=3D CONFIG_PHYS_RAM_BASE) ? = \ >>> + (void *)((unsigned long)(_y) + va_kernel_pa_offset + XI= P_OFFSET) : \ >>> + (void *)((unsigned long)(_y) + va_kernel_xip_pa_offset)= ; \ >>> + }) >>> +#else >>> #define kernel_mapping_pa_to_va(x) ((void *)((unsigned long)(x) += va_kernel_pa_offset)) >>> +#endif >>> #define __pa_to_va_nodebug(x) linear_mapping_pa_to_v= a(x) >>> >>> #define linear_mapping_va_to_pa(x) ((unsigned long)(x) - va_pa_of= fset) >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define kernel_mapping_va_to_pa(y) ({ = \ >>> + unsigned long _y =3D y; = \ >>> + (_y < kernel_virt_addr + XIP_OFFSET) ? = \ >>> + ((unsigned long)(_y) - va_kernel_xip_pa_offset) : = \ >>> + ((unsigned long)(_y) - va_kernel_pa_offset - XIP_OFFSET= ); \ >>> + }) >>> +#else >>> #define kernel_mapping_va_to_pa(x) ((unsigned long)(x) - va_kerne= l_pa_offset) >>> +#endif >>> #define __va_to_pa_nodebug(x) ({ = \ >>> unsigned long _x =3D x; = \ >>> (_x < kernel_virt_addr) ? = \ >>> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/as= m/pgtable.h >>> index 80e63a93e903..c2dc4f83eed8 100644 >>> --- a/arch/riscv/include/asm/pgtable.h >>> +++ b/arch/riscv/include/asm/pgtable.h >>> @@ -64,6 +64,19 @@ >>> #define FIXADDR_SIZE PGDIR_SIZE >>> #endif >>> #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) >>> + >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define XIP_OFFSET SZ_8M >>> +#define XIP_FIXUP(addr) ({ = \ >>> + uintptr_t __a =3D (uintptr_t)(addr); = \ >>> + (__a >=3D CONFIG_XIP_PHYS_ADDR && __a < CONFIG_XIP_PHYS_ADDR + = SZ_16M) ? \ >>> + __a - CONFIG_XIP_PHYS_ADDR + CONFIG_PHYS_RAM_BASE - XIP= _OFFSET :\ >>> + __a; = \ >>> + }) >>> +#else >>> +#define XIP_FIXUP(addr) (addr) >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> #endif >>> >>> #ifndef __ASSEMBLY__ >>> @@ -499,8 +512,16 @@ static inline int ptep_clear_flush_young(struct = vm_area_struct *vma, >>> #define kern_addr_valid(addr) (1) /* FIXME */ >>> >>> extern char _start[]; >>> -extern void *dtb_early_va; >>> -extern uintptr_t dtb_early_pa; >>> +extern void *_dtb_early_va; >>> +extern uintptr_t _dtb_early_pa; >>> +#if defined(CONFIG_XIP_KERNEL) && defined(CONFIG_MMU) >>> +#define dtb_early_va (*(void **)XIP_FIXUP(&_dtb_early_va)) >>> +#define dtb_early_pa (*(uintptr_t *)XIP_FIXUP(&_dtb_early_pa)) >>> +#else >>> +#define dtb_early_va _dtb_early_va >>> +#define dtb_early_pa _dtb_early_pa >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> void setup_bootmem(void); >>> void paging_init(void); >>> void misc_mem_init(void); >>> diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S >>> index 6cb05f22e52a..89cc58ab52b4 100644 >>> --- a/arch/riscv/kernel/head.S >>> +++ b/arch/riscv/kernel/head.S >>> @@ -9,11 +9,23 @@ >>> #include >>> #include >>> #include >>> +#include >>> #include >>> #include >>> #include >>> #include "efi-header.S" >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> +.macro XIP_FIXUP_OFFSET reg >>> + REG_L t0, _xip_fixup >>> + add \reg, \reg, t0 >>> +.endm >>> +_xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP= _OFFSET >>> +#else >>> +.macro XIP_FIXUP_OFFSET reg >>> +.endm >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> __HEAD >>> ENTRY(_start) >>> /* >>> @@ -70,6 +82,7 @@ pe_head_start: >>> relocate: >>> /* Relocate return address */ >>> la a1, kernel_virt_addr >>> + XIP_FIXUP_OFFSET a1 >>> REG_L a1, 0(a1) >>> la a2, _start >>> sub a1, a1, a2 >>> @@ -92,6 +105,7 @@ relocate: >>> * to ensure the new translations are in use. >>> */ >>> la a0, trampoline_pg_dir >>> + XIP_FIXUP_OFFSET a0 >>> srl a0, a0, PAGE_SHIFT >>> or a0, a0, a1 >>> sfence.vma >>> @@ -145,7 +159,9 @@ secondary_start_sbi: >>> >>> slli a3, a0, LGREG >>> la a4, __cpu_up_stack_pointer >>> + XIP_FIXUP_OFFSET a4 >>> la a5, __cpu_up_task_pointer >>> + XIP_FIXUP_OFFSET a5 >>> add a4, a3, a4 >>> add a5, a3, a5 >>> REG_L sp, (a4) >>> @@ -157,6 +173,7 @@ secondary_start_common: >>> #ifdef CONFIG_MMU >>> /* Enable virtual memory and relocate to virtual address */ >>> la a0, swapper_pg_dir >>> + XIP_FIXUP_OFFSET a0 >>> call relocate >>> #endif >>> call setup_trap_vector >>> @@ -237,12 +254,33 @@ pmp_done: >>> .Lgood_cores: >>> #endif >>> >>> +#ifndef CONFIG_XIP_KERNEL >>> /* Pick one hart to run the main boot sequence */ >>> la a3, hart_lottery >>> li a2, 1 >>> amoadd.w a3, a2, (a3) >>> bnez a3, .Lsecondary_start >>> >>> +#else >>> + /* hart_lottery in flash contains a magic number */ >>> + la a3, hart_lottery >>> + mv a2, a3 >>> + XIP_FIXUP_OFFSET a2 >>> + lw t1, (a3) >>> + amoswap.w t0, t1, (a2) >>> + /* first time here if hart_lottery in RAM is not set */ >>> + beq t0, t1, .Lsecondary_start >>> + >>> + la sp, _end + THREAD_SIZE >>> + XIP_FIXUP_OFFSET sp >>> + mv s0, a0 >>> + call __copy_data >>> + >>> + /* Restore a0 copy */ >>> + mv a0, s0 >>> +#endif >>> + >>> +#ifndef CONFIG_XIP_KERNEL >>> /* Clear BSS for flat non-ELF images */ >>> la a3, __bss_start >>> la a4, __bss_stop >>> @@ -252,15 +290,18 @@ clear_bss: >>> add a3, a3, RISCV_SZPTR >>> blt a3, a4, clear_bss >>> clear_bss_done: >>> - >>> +#endif >>> /* Save hart ID and DTB physical address */ >>> mv s0, a0 >>> mv s1, a1 >>> + >>> la a2, boot_cpu_hartid >>> + XIP_FIXUP_OFFSET a2 >>> REG_S a0, (a2) >>> >>> /* Initialize page tables and relocate to virtual addresses */ >>> la sp, init_thread_union + THREAD_SIZE >>> + XIP_FIXUP_OFFSET sp >>> #ifdef CONFIG_BUILTIN_DTB >>> la a0, __dtb_start >>> #else >>> @@ -269,6 +310,7 @@ clear_bss_done: >>> call setup_vm >>> #ifdef CONFIG_MMU >>> la a0, early_pg_dir >>> + XIP_FIXUP_OFFSET a0 >>> call relocate >>> #endif /* CONFIG_MMU */ >>> >>> @@ -293,7 +335,9 @@ clear_bss_done: >>> >>> slli a3, a0, LGREG >>> la a1, __cpu_up_stack_pointer >>> + XIP_FIXUP_OFFSET a1 >>> la a2, __cpu_up_task_pointer >>> + XIP_FIXUP_OFFSET a2 >>> add a1, a3, a1 >>> add a2, a3, a2 >>> >>> diff --git a/arch/riscv/kernel/head.h b/arch/riscv/kernel/head.h >>> index b48dda3d04f6..aabbc3ac3e48 100644 >>> --- a/arch/riscv/kernel/head.h >>> +++ b/arch/riscv/kernel/head.h >>> @@ -12,6 +12,9 @@ extern atomic_t hart_lottery; >>> >>> asmlinkage void do_page_fault(struct pt_regs *regs); >>> asmlinkage void __init setup_vm(uintptr_t dtb_pa); >>> +#ifdef CONFIG_XIP_KERNEL >>> +asmlinkage void __init __copy_data(void); >>> +#endif >>> >>> extern void *__cpu_up_stack_pointer[]; >>> extern void *__cpu_up_task_pointer[]; >>> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c >>> index 30e4af0fd50c..2ddf654c72bb 100644 >>> --- a/arch/riscv/kernel/setup.c >>> +++ b/arch/riscv/kernel/setup.c >>> @@ -50,7 +50,11 @@ struct screen_info screen_info __section(".data") = =3D { >>> * This is used before the kernel initializes the BSS so it can't b= e in the >>> * BSS. >>> */ >>> -atomic_t hart_lottery __section(".sdata"); >>> +atomic_t hart_lottery __section(".sdata") >>> +#ifdef CONFIG_XIP_KERNEL >>> +=3D ATOMIC_INIT(0xC001BEEF) >>> +#endif >>> +; >>> unsigned long boot_cpu_hartid; >>> static DEFINE_PER_CPU(struct cpu, cpu_devices); >>> >>> @@ -254,7 +258,7 @@ void __init setup_arch(char **cmdline_p) >>> #if IS_ENABLED(CONFIG_BUILTIN_DTB) >>> unflatten_and_copy_device_tree(); >>> #else >>> - if (early_init_dt_verify(__va(dtb_early_pa))) >>> + if (early_init_dt_verify(__va(XIP_FIXUP(dtb_early_pa)))) >>> unflatten_device_tree(); >>> else >>> pr_err("No DTB found in kernel mappings\n"); >>> @@ -266,7 +270,7 @@ void __init setup_arch(char **cmdline_p) >>> if (IS_ENABLED(CONFIG_STRICT_KERNEL_RWX)) >>> protect_kernel_text_data(); >>> >>> -#if defined(CONFIG_64BIT) && defined(CONFIG_MMU) >>> +#if defined(CONFIG_64BIT) && defined(CONFIG_MMU) && !defined(CONFIG_= XIP_KERNEL) >>> protect_kernel_linear_mapping_text_rodata(); >>> #endif >> >> My tree doesn't look quite like this. Not sure if I just got the wron= g >> version of something, but I didn't have the CONFIG_MMU check. It turn= s >> out that, as I was merging this in, I saw that failing allnoconfig >> (which is !MMU). I went ahead and squashed in >> >> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c >> index ab394d173cd4..3e7930869ee7 100644 >> --- a/arch/riscv/kernel/setup.c >> +++ b/arch/riscv/kernel/setup.c >> @@ -266,7 +266,7 @@ void __init setup_arch(char **cmdline_p) >> >> if (IS_ENABLED(CONFIG_STRICT_KERNEL_RWX)) { >> protect_kernel_text_data(); >> -#ifdef CONFIG_64BIT >> +#if defined(CONFIG_64BIT) && defined(CONFIG_MMU) >> protect_kernel_linear_mapping_text_rodata(); >> #endif >> } >> >> to your memory map rearrangement patch, but I'm a bit worried that I m= anaged to >> miss something in the process. >> >>> >>> diff --git a/arch/riscv/kernel/vmlinux-xip.lds.S b/arch/riscv/kernel/= vmlinux-xip.lds.S >>> new file mode 100644 >>> index 000000000000..4b29b9917f99 >>> --- /dev/null >>> +++ b/arch/riscv/kernel/vmlinux-xip.lds.S >>> @@ -0,0 +1,133 @@ >>> +/* SPDX-License-Identifier: GPL-2.0-only */ >>> +/* >>> + * Copyright (C) 2012 Regents of the University of California >>> + * Copyright (C) 2017 SiFive >>> + * Copyright (C) 2020 Vitaly Wool, Konsulko AB >>> + */ >>> + >>> +#include >>> +#define LOAD_OFFSET KERNEL_LINK_ADDR >>> +/* No __ro_after_init data in the .rodata section - which will alway= s be ro */ >>> +#define RO_AFTER_INIT_DATA >>> + >>> +#include >>> +#include >>> +#include >>> +#include >>> +#include >>> + >>> +OUTPUT_ARCH(riscv) >>> +ENTRY(_start) >>> + >>> +jiffies =3D jiffies_64; >>> + >>> +SECTIONS >>> +{ >>> + /* Beginning of code and text segment */ >>> + . =3D LOAD_OFFSET; >>> + _xiprom =3D .; >>> + _start =3D .; >>> + HEAD_TEXT_SECTION >>> + INIT_TEXT_SECTION(PAGE_SIZE) >>> + /* we have to discard exit text and such at runtime, not link t= ime */ >>> + .exit.text : >>> + { >>> + EXIT_TEXT >>> + } >>> + >>> + .text : { >>> + _text =3D .; >>> + _stext =3D .; >>> + TEXT_TEXT >>> + SCHED_TEXT >>> + CPUIDLE_TEXT >>> + LOCK_TEXT >>> + KPROBES_TEXT >>> + ENTRY_TEXT >>> + IRQENTRY_TEXT >>> + SOFTIRQENTRY_TEXT >>> + *(.fixup) >>> + _etext =3D .; >>> + } >>> + RO_DATA(L1_CACHE_BYTES) >>> + .srodata : { >>> + *(.srodata*) >>> + } >>> + .init.rodata : { >>> + INIT_SETUP(16) >>> + INIT_CALLS >>> + CON_INITCALL >>> + INIT_RAM_FS >>> + } >>> + _exiprom =3D .; /* End of XIP ROM area */ >>> + >>> + >>> +/* >>> + * From this point, stuff is considered writable and will be copied = to RAM >>> + */ >>> + __data_loc =3D ALIGN(16); /* location in file */ >>> + . =3D LOAD_OFFSET + XIP_OFFSET; /* location in memory */ >>> + >>> + _sdata =3D .; /* Start of data section */ >>> + _data =3D .; >>> + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) >>> + _edata =3D .; >>> + __start_ro_after_init =3D .; >>> + .data.ro_after_init : AT(ADDR(.data.ro_after_init) - LOAD_OFFSE= T) { >>> + *(.data..ro_after_init) >>> + } >>> + __end_ro_after_init =3D .; >>> + >>> + . =3D ALIGN(PAGE_SIZE); >>> + __init_begin =3D .; >>> + .init.data : { >>> + INIT_DATA >>> + } >>> + .exit.data : { >>> + EXIT_DATA >>> + } >>> + . =3D ALIGN(8); >>> + __soc_early_init_table : { >>> + __soc_early_init_table_start =3D .; >>> + KEEP(*(__soc_early_init_table)) >>> + __soc_early_init_table_end =3D .; >>> + } >>> + __soc_builtin_dtb_table : { >>> + __soc_builtin_dtb_table_start =3D .; >>> + KEEP(*(__soc_builtin_dtb_table)) >>> + __soc_builtin_dtb_table_end =3D .; >>> + } >>> + PERCPU_SECTION(L1_CACHE_BYTES) >>> + >>> + . =3D ALIGN(PAGE_SIZE); >>> + __init_end =3D .; >>> + >>> + .sdata : { >>> + __global_pointer$ =3D . + 0x800; >>> + *(.sdata*) >>> + *(.sbss*) >>> + } >>> + >>> + BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0) >>> + EXCEPTION_TABLE(0x10) >>> + >>> + .rel.dyn : AT(ADDR(.rel.dyn) - LOAD_OFFSET) { >>> + *(.rel.dyn*) >>> + } >>> + >>> + /* >>> + * End of copied data. We need a dummy section to get its LMA. >>> + * Also located before final ALIGN() as trailing padding is not= stored >>> + * in the resulting binary file and useless to copy. >>> + */ >>> + .data.endmark : AT(ADDR(.data.endmark) - LOAD_OFFSET) { } >>> + _edata_loc =3D LOADADDR(.data.endmark); >>> + >>> + . =3D ALIGN(PAGE_SIZE); >>> + _end =3D .; >>> + >>> + STABS_DEBUG >>> + DWARF_DEBUG >>> + >>> + DISCARDS >>> +} >>> diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmli= nux.lds.S >>> index 0726c05e0336..0a59b65cf789 100644 >>> --- a/arch/riscv/kernel/vmlinux.lds.S >>> +++ b/arch/riscv/kernel/vmlinux.lds.S >>> @@ -4,8 +4,13 @@ >>> * Copyright (C) 2017 SiFive >>> */ >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> +#include "vmlinux-xip.lds.S" >>> +#else >>> + >>> #include >>> #define LOAD_OFFSET KERNEL_LINK_ADDR >>> + >>> #include >>> #include >>> #include >>> @@ -133,3 +138,4 @@ SECTIONS >>> >>> DISCARDS >>> } >>> +#endif /* CONFIG_XIP_KERNEL */ >>> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c >>> index 093f3a96ecfc..d03e5e43d2d0 100644 >>> --- a/arch/riscv/mm/init.c >>> +++ b/arch/riscv/mm/init.c >>> @@ -27,6 +27,9 @@ >>> >>> unsigned long kernel_virt_addr =3D KERNEL_LINK_ADDR; >>> EXPORT_SYMBOL(kernel_virt_addr); >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define kernel_virt_addr (*((unsigned long *)XIP_FIXUP(&kernel= _virt_addr))) >>> +#endif >>> >>> unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] >>> __page_aligned= _bss; >>> @@ -34,8 +37,8 @@ EXPORT_SYMBOL(empty_zero_page); >>> >>> extern char _start[]; >>> #define DTB_EARLY_BASE_VA PGDIR_SIZE >>> -void *dtb_early_va __initdata; >>> -uintptr_t dtb_early_pa __initdata; >>> +void *_dtb_early_va __initdata; >>> +uintptr_t _dtb_early_pa __initdata; >>> >>> struct pt_alloc_ops { >>> pte_t *(*get_pte_virt)(phys_addr_t pa); >>> @@ -118,6 +121,10 @@ void __init setup_bootmem(void) >>> phys_addr_t dram_end =3D memblock_end_of_DRAM(); >>> phys_addr_t max_mapped_addr =3D __pa(~(ulong)0); >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> + vmlinux_start =3D __pa_symbol(&_sdata); >>> +#endif >>> + >>> /* The maximal physical memory size is -PAGE_OFFSET. */ >>> memblock_enforce_memory_limit(-PAGE_OFFSET); >>> >>> @@ -159,15 +166,39 @@ void __init setup_bootmem(void) >>> memblock_allow_resize(); >>> } >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> + >>> +extern char _xiprom[], _exiprom[]; >>> +extern char _sdata[], _edata[]; >>> + >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> #ifdef CONFIG_MMU >>> -static struct pt_alloc_ops pt_ops; >>> +static struct pt_alloc_ops _pt_ops; >>> + >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&_pt_ops)) >>> +#else >>> +#define pt_ops _pt_ops >>> +#endif >>> >>> /* Offset between linear mapping virtual address and kernel load ad= dress */ >>> unsigned long va_pa_offset; >>> EXPORT_SYMBOL(va_pa_offset); >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define va_pa_offset (*((unsigned long *)XIP_FIXUP(&va_pa_offset))= ) >>> +#endif >>> /* Offset between kernel mapping virtual address and kernel load ad= dress */ >>> unsigned long va_kernel_pa_offset; >>> EXPORT_SYMBOL(va_kernel_pa_offset); >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define va_kernel_pa_offset (*((unsigned long *)XIP_FIXUP(&va_ker= nel_pa_offset))) >>> +#endif >>> +unsigned long va_kernel_xip_pa_offset; >>> +EXPORT_SYMBOL(va_kernel_xip_pa_offset); >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define va_kernel_xip_pa_offset (*((unsigned long *)XIP_FIXUP= (&va_kernel_xip_pa_offset))) >>> +#endif >>> unsigned long pfn_base; >>> EXPORT_SYMBOL(pfn_base); >>> >>> @@ -177,6 +208,12 @@ pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bs= s; >>> >>> pgd_t early_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE); >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir= )) >>> +#define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) >>> +#define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgpro= t_t prot) >>> { >>> unsigned long addr =3D __fix_to_virt(idx); >>> @@ -252,6 +289,12 @@ pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bs= s; >>> pmd_t early_pmd[PTRS_PER_PMD] __initdata __aligned(PAGE_SIZE); >>> pmd_t early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAGE_SIZE); >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define trampoline_pmd ((pmd_t *)XIP_FIXUP(trampoline_pmd)) >>> +#define fixmap_pmd ((pmd_t *)XIP_FIXUP(fixmap_pmd)) >>> +#define early_pmd ((pmd_t *)XIP_FIXUP(early_pmd)) >>> +#endif /* CONFIG_XIP_KERNEL */ >>> + >>> static pmd_t *__init get_pmd_virt_early(phys_addr_t pa) >>> { >>> /* Before MMU is enabled */ >>> @@ -368,6 +411,19 @@ static uintptr_t __init best_map_size(phys_addr_= t base, phys_addr_t size) >>> return PMD_SIZE; >>> } >>> >>> +#ifdef CONFIG_XIP_KERNEL >>> +/* called from head.S with MMU off */ >>> +asmlinkage void __init __copy_data(void) >>> +{ >>> + void *from =3D (void *)(&_sdata); >>> + void *end =3D (void *)(&_end); >>> + void *to =3D (void *)CONFIG_PHYS_RAM_BASE; >>> + size_t sz =3D (size_t)(end - from + 1); >>> + >>> + memcpy(to, from, sz); >>> +} >>> +#endif >>> + >>> /* >>> * setup_vm() is called from head.S with MMU-off. >>> * >>> @@ -387,7 +443,35 @@ static uintptr_t __init best_map_size(phys_addr_= t base, phys_addr_t size) >>> #endif >>> >>> uintptr_t load_pa, load_sz; >>> +#ifdef CONFIG_XIP_KERNEL >>> +#define load_pa (*((uintptr_t *)XIP_FIXUP(&load_pa))) >>> +#define load_sz (*((uintptr_t *)XIP_FIXUP(&load_sz))) >>> +#endif >>> + >>> +#ifdef CONFIG_XIP_KERNEL >>> +uintptr_t xiprom, xiprom_sz; >>> +#define xiprom_sz (*((uintptr_t *)XIP_FIXUP(&xiprom_sz))) >>> +#define xiprom (*((uintptr_t *)XIP_FIXUP(&xiprom))) >>> >>> +static void __init create_kernel_page_table(pgd_t *pgdir, uintptr_t = map_size) >>> +{ >>> + uintptr_t va, end_va; >>> + >>> + /* Map the flash resident part */ >>> + end_va =3D kernel_virt_addr + xiprom_sz; >>> + for (va =3D kernel_virt_addr; va < end_va; va +=3D map_size) >>> + create_pgd_mapping(pgdir, va, >>> + xiprom + (va - kernel_virt_addr), >>> + map_size, PAGE_KERNEL_EXEC); >>> + >>> + /* Map the data in RAM */ >>> + end_va =3D kernel_virt_addr + XIP_OFFSET + load_sz; >>> + for (va =3D kernel_virt_addr + XIP_OFFSET; va < end_va; va +=3D= map_size) >>> + create_pgd_mapping(pgdir, va, >>> + load_pa + (va - (kernel_virt_addr + = XIP_OFFSET)), >>> + map_size, PAGE_KERNEL); >>> +} >>> +#else >>> static void __init create_kernel_page_table(pgd_t *pgdir, uintptr_t= map_size) >>> { >>> uintptr_t va, end_va; >>> @@ -398,16 +482,28 @@ static void __init create_kernel_page_table(pgd= _t *pgdir, uintptr_t map_size) >>> load_pa + (va - kernel_virt_addr), >>> map_size, PAGE_KERNEL_EXEC); >>> } >>> +#endif >>> >>> asmlinkage void __init setup_vm(uintptr_t dtb_pa) >>> { >>> - uintptr_t pa; >>> + uintptr_t __maybe_unused pa; >>> uintptr_t map_size; >>> #ifndef __PAGETABLE_PMD_FOLDED >>> pmd_t fix_bmap_spmd, fix_bmap_epmd; >>> #endif >>> + >>> +#ifdef CONFIG_XIP_KERNEL >>> + xiprom =3D (uintptr_t)CONFIG_XIP_PHYS_ADDR; >>> + xiprom_sz =3D (uintptr_t)(&_exiprom) - (uintptr_t)(&_xiprom); >>> + >>> + load_pa =3D (uintptr_t)CONFIG_PHYS_RAM_BASE; >>> + load_sz =3D (uintptr_t)(&_end) - (uintptr_t)(&_sdata); >>> + >>> + va_kernel_xip_pa_offset =3D kernel_virt_addr - xiprom; >>> +#else >>> load_pa =3D (uintptr_t)(&_start); >>> load_sz =3D (uintptr_t)(&_end) - load_pa; >>> +#endif >>> >>> va_pa_offset =3D PAGE_OFFSET - load_pa; >>> va_kernel_pa_offset =3D kernel_virt_addr - load_pa; >>> @@ -441,8 +537,13 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa= ) >>> /* Setup trampoline PGD and PMD */ >>> create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, >>> (uintptr_t)trampoline_pmd, PGDIR_SIZE, PAGE= _TABLE); >>> +#ifdef CONFIG_XIP_KERNEL >>> + create_pmd_mapping(trampoline_pmd, kernel_virt_addr, >>> + xiprom, PMD_SIZE, PAGE_KERNEL_EXEC); >>> +#else >>> create_pmd_mapping(trampoline_pmd, kernel_virt_addr, >>> load_pa, PMD_SIZE, PAGE_KERNEL_EXEC); >>> +#endif >>> #else >>> /* Setup trampoline PGD */ >>> create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, >>> @@ -474,7 +575,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >>> * whereas dtb_early_va will be used before setup_vm_final ins= talls >>> * the linear mapping. >>> */ >>> - dtb_early_va =3D kernel_mapping_pa_to_va(dtb_pa); >>> + dtb_early_va =3D kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa)); >>> #endif /* CONFIG_BUILTIN_DTB */ >>> #else >>> #ifndef CONFIG_BUILTIN_DTB >>> @@ -486,7 +587,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >>> pa + PGDIR_SIZE, PGDIR_SIZE, PAGE_KERNEL); >>> dtb_early_va =3D (void *)DTB_EARLY_BASE_VA + (dtb_pa & (PGDIR_= SIZE - 1)); >>> #else /* CONFIG_BUILTIN_DTB */ >>> - dtb_early_va =3D kernel_mapping_pa_to_va(dtb_pa); >>> + dtb_early_va =3D kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa)); >>> #endif /* CONFIG_BUILTIN_DTB */ >>> #endif >>> dtb_early_pa =3D dtb_pa; >>> @@ -522,7 +623,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >>> #endif >>> } >>> >>> -#ifdef CONFIG_64BIT >>> +#if defined(CONFIG_64BIT) && !defined(CONFIG_XIP_KERNEL) >>> void protect_kernel_linear_mapping_text_rodata(void) >>> { >>> unsigned long text_start =3D (unsigned long)lm_alias(_start);