From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75473C433B4 for ; Fri, 9 Apr 2021 08:23:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF8F2611AB for ; Fri, 9 Apr 2021 08:23:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF8F2611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 552436B006C; Fri, 9 Apr 2021 04:23:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 502916B006E; Fri, 9 Apr 2021 04:23:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32E3A6B0070; Fri, 9 Apr 2021 04:23:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id 11F656B006C for ; Fri, 9 Apr 2021 04:23:14 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AC6C71802E6E6 for ; Fri, 9 Apr 2021 08:23:13 +0000 (UTC) X-FDA: 78012138666.07.DC0E7F8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf19.hostedemail.com (Postfix) with ESMTP id B617190009F1 for ; Fri, 9 Apr 2021 08:23:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1617956592; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cqrz76rzWIaRbgxVXwQLmavVfjNJYWdjy2pon2bya6E=; b=I+0mYcU0Q3Xiuvtbb3pGj3/j1v59u1V75EnmGCMXum57sLB0LMuSGf+czdjuGDEXrQ2IKN syV9n3v2EJ2Lq8ly1oVqrVk0pRh1jciwPtagYwP/0tsrQJ9F+FMIMyoveHUT8dIedrG5hh AiBprQnt3PobtVA9SHgT+i9NfJoCwsQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-Ih4agEZuODymKrg21ZEZoA-1; Fri, 09 Apr 2021 04:23:09 -0400 X-MC-Unique: Ih4agEZuODymKrg21ZEZoA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F01058026AC; Fri, 9 Apr 2021 08:23:07 +0000 (UTC) Received: from [10.36.115.11] (ovpn-115-11.ams2.redhat.com [10.36.115.11]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC9F810013D7; Fri, 9 Apr 2021 08:23:04 +0000 (UTC) To: Alex Ghiti , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Cc: Vitaly Wool , Mike Rapoport References: <20210409065115.11054-1-alex@ghiti.fr> <3500f3cb-b660-5bbc-ae8d-0c9770e4a573@ghiti.fr> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v7] RISC-V: enable XIP Message-ID: Date: Fri, 9 Apr 2021 10:23:03 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <3500f3cb-b660-5bbc-ae8d-0c9770e4a573@ghiti.fr> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B617190009F1 X-Stat-Signature: fmnmyp5tq8wsjiqow9hyqjnqj5oahuq1 Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617956582-17678 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 09.04.21 09:14, Alex Ghiti wrote: > Le 4/9/21 =C3=A0 2:51 AM, Alexandre Ghiti a =C3=A9crit=C2=A0: >> From: Vitaly Wool >> >> Introduce XIP (eXecute In Place) support for RISC-V platforms. >> It allows code to be executed directly from non-volatile storage >> directly addressable by the CPU, such as QSPI NOR flash which can >> be found on many RISC-V platforms. This makes way for significant >> optimization of RAM footprint. The XIP kernel is not compressed >> since it has to run directly from flash, so it will occupy more >> space on the non-volatile storage. The physical flash address used >> to link the kernel object files and for storing it has to be known >> at compile time and is represented by a Kconfig option. >> >> XIP on RISC-V will for the time being only work on MMU-enabled >> kernels. >> > I added linux-mm and linux-arch to get feedbacks because I noticed that > DEBUG_VM_PGTABLE fails for SPARSEMEM (it works for FLATMEM but I think > it does not do what is expected): the fact that we don't have any struc= t > page to back the text and rodata in flash is the problem but to which > extent ? Just wondering, why can't we create a memmap for that memory -- or is it=20 even desireable to not do that explicity? There might be some nasty side=20 effects when not having a memmap for text and rodata. I would assume stimply exposing the physical memory range to memblock as=20 RAM and marking it reserved would create a memmap that's fully=20 initialized like any bootmem (PG_reserved). Or is there a reason why we cannot do that? Also, will that memory properly be exposed in the resource tree as=20 System RAM (e.g., /proc/iomem) ? Otherwise some things (/proc/kcore)=20 won't work as expected - the kernel won't be included in a dump. >=20 > Thanks, >=20 > Alex >=20 >> Signed-off-by: Alexandre Ghiti [ Rebase on top of "Mov= e >> kernel mapping outside the linear mapping=C2=A0] >> Signed-off-by: Vitaly Wool >> --- >> >> Changes in v2: >> - dedicated macro for XIP address fixup when MMU is not enabled yet >> o both for 32-bit and 64-bit RISC-V >> - SP is explicitly set to a safe place in RAM before __copy_data call >> - removed redundant alignment requirements in vmlinux-xip.lds.S >> - changed long -> uintptr_t typecast in __XIP_FIXUP macro. >> Changes in v3: >> - rebased against latest for-next >> - XIP address fixup macro now takes an argument >> - SMP related fixes >> Changes in v4: >> - rebased against the current for-next >> - less #ifdef's in C/ASM code >> - dedicated XIP_FIXUP_OFFSET assembler macro in head.S >> - C-specific definitions moved into #ifndef __ASSEMBLY__ >> - Fixed multi-core boot >> Changes in v5: >> - fixed build error for non-XIP kernels >> Changes in v6: >> - XIP_PHYS_RAM_BASE config option renamed to PHYS_RAM_BASE >> - added PHYS_RAM_BASE_FIXED config flag to allow usage of >> PHYS_RAM_BASE in non-XIP configurations if needed >> - XIP_FIXUP macro rewritten with a tempoarary variable to avoid side >> effects >> - fixed crash for non-XIP kernels that don't use built-in DTB >> Changes in v7: >> - Fix pfn_base that required FIXUP >> - Fix copy_data which lacked + 1 in size to copy >> - Fix pfn_valid for FLATMEM >> - Rebased on top of "Move kernel mapping outside the linear mapping": >> this is the biggest change and affected mm/init.c, >> kernel/vmlinux-xip.lds.S and include/asm/pgtable.h: XIP kernel is = now >> mapped like 'normal' kernel at the end of the address space. >> >> arch/riscv/Kconfig | 51 ++++++++++- >> arch/riscv/Makefile | 8 +- >> arch/riscv/boot/Makefile | 13 +++ >> arch/riscv/include/asm/page.h | 28 ++++++ >> arch/riscv/include/asm/pgtable.h | 25 +++++- >> arch/riscv/kernel/head.S | 46 +++++++++- >> arch/riscv/kernel/head.h | 3 + >> arch/riscv/kernel/setup.c | 10 ++- >> arch/riscv/kernel/vmlinux-xip.lds.S | 133 +++++++++++++++++++++++++= +++ >> arch/riscv/kernel/vmlinux.lds.S | 6 ++ >> arch/riscv/mm/init.c | 118 ++++++++++++++++++++++-- >> 11 files changed, 424 insertions(+), 17 deletions(-) >> create mode 100644 arch/riscv/kernel/vmlinux-xip.lds.S >> >> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >> index 8ea60a0a19ae..4d0153805927 100644 >> --- a/arch/riscv/Kconfig >> +++ b/arch/riscv/Kconfig >> @@ -28,7 +28,7 @@ config RISCV >> select ARCH_HAS_PTE_SPECIAL >> select ARCH_HAS_SET_DIRECT_MAP >> select ARCH_HAS_SET_MEMORY >> - select ARCH_HAS_STRICT_KERNEL_RWX if MMU >> + select ARCH_HAS_STRICT_KERNEL_RWX if MMU && !XIP_KERNEL >> select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST >> select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX >> select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT >> @@ -441,7 +441,7 @@ config EFI_STUB >> =20 >> config EFI >> bool "UEFI runtime support" >> - depends on OF >> + depends on OF && !XIP_KERNEL >> select LIBFDT >> select UCS2_STRING >> select EFI_PARAMS_FROM_FDT >> @@ -465,11 +465,56 @@ config STACKPROTECTOR_PER_TASK >> def_bool y >> depends on STACKPROTECTOR && CC_HAVE_STACKPROTECTOR_TLS >> =20 >> +config PHYS_RAM_BASE_FIXED >> + bool "Explicitly specified physical RAM address" >> + default n >> + >> +config PHYS_RAM_BASE >> + hex "Platform Physical RAM address" >> + depends on PHYS_RAM_BASE_FIXED >> + default "0x80000000" >> + help >> + This is the physical address of RAM in the system. It has to be >> + explicitly specified to run early relocations of read-write data >> + from flash to RAM. >> + >> +config XIP_KERNEL >> + bool "Kernel Execute-In-Place from ROM" >> + depends on MMU >> + select PHYS_RAM_BASE_FIXED >> + help >> + Execute-In-Place allows the kernel to run from non-volatile storag= e >> + directly addressable by the CPU, such as NOR flash. This saves RAM >> + space since the text section of the kernel is not loaded from flas= h >> + to RAM. Read-write sections, such as the data section and stack, >> + are still copied to RAM. The XIP kernel is not compressed since >> + it has to run directly from flash, so it will take more space to >> + store it. The flash address used to link the kernel object files, >> + and for storing it, is configuration dependent. Therefore, if you >> + say Y here, you must know the proper physical address where to >> + store the kernel image depending on your own flash memory usage. >> + >> + Also note that the make target becomes "make xipImage" rather than >> + "make zImage" or "make Image". The final kernel binary to put in >> + ROM memory will be arch/riscv/boot/xipImage. >> + >> + If unsure, say N. >> + >> +config XIP_PHYS_ADDR >> + hex "XIP Kernel Physical Location" >> + depends on XIP_KERNEL >> + default "0x21000000" >> + help >> + This is the physical address in your flash memory the kernel will >> + be linked for and stored to. This address is dependent on your >> + own flash usage. >> + >> endmenu >> =20 >> config BUILTIN_DTB >> - def_bool n >> + bool >> depends on OF >> + default y if XIP_KERNEL >> =20 >> menu "Power management options" >> =20 >> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile >> index 1368d943f1f3..8fcbec03974d 100644 >> --- a/arch/riscv/Makefile >> +++ b/arch/riscv/Makefile >> @@ -82,7 +82,11 @@ CHECKFLAGS +=3D -D__riscv -D__riscv_xlen=3D$(BITS) >> =20 >> # Default target when executing plain make >> boot :=3D arch/riscv/boot >> +ifeq ($(CONFIG_XIP_KERNEL),y) >> +KBUILD_IMAGE :=3D $(boot)/xipImage >> +else >> KBUILD_IMAGE :=3D $(boot)/Image.gz >> +endif >> =20 >> head-y :=3D arch/riscv/kernel/head.o >> =20 >> @@ -95,12 +99,14 @@ PHONY +=3D vdso_install >> vdso_install: >> $(Q)$(MAKE) $(build)=3Darch/riscv/kernel/vdso $@ >> =20 >> +ifneq ($(CONFIG_XIP_KERNEL),y) >> ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_SOC_CANAAN),yy) >> KBUILD_IMAGE :=3D $(boot)/loader.bin >> else >> KBUILD_IMAGE :=3D $(boot)/Image.gz >> endif >> -BOOT_TARGETS :=3D Image Image.gz loader loader.bin >> +endif >> +BOOT_TARGETS :=3D Image Image.gz loader loader.bin xipImage >> =20 >> all: $(notdir $(KBUILD_IMAGE)) >> =20 >> diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile >> index 03404c84f971..6bf299f70c27 100644 >> --- a/arch/riscv/boot/Makefile >> +++ b/arch/riscv/boot/Makefile >> @@ -17,8 +17,21 @@ >> KCOV_INSTRUMENT :=3D n >> =20 >> OBJCOPYFLAGS_Image :=3D-O binary -R .note -R .note.gnu.build-id -R = .comment -S >> +OBJCOPYFLAGS_xipImage :=3D-O binary -R .note -R .note.gnu.build-id -R= .comment -S >> =20 >> targets :=3D Image Image.* loader loader.o loader.lds loader.bin >> +targets :=3D Image Image.* loader loader.o loader.lds loader.bin xipI= mage >> + >> +ifeq ($(CONFIG_XIP_KERNEL),y) >> + >> +quiet_cmd_mkxip =3D $(quiet_cmd_objcopy) >> +cmd_mkxip =3D $(cmd_objcopy) >> + >> +$(obj)/xipImage: vmlinux FORCE >> + $(call if_changed,mkxip) >> + @$(kecho) ' Physical Address of xipImage: $(CONFIG_XIP_PHYS_ADDR)' >> + >> +endif >> =20 >> $(obj)/Image: vmlinux FORCE >> $(call if_changed,objcopy) >> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/pa= ge.h >> index 22cfb2be60dc..6fe0ff8c8fa9 100644 >> --- a/arch/riscv/include/asm/page.h >> +++ b/arch/riscv/include/asm/page.h >> @@ -91,6 +91,9 @@ typedef struct page *pgtable_t; >> #ifdef CONFIG_MMU >> extern unsigned long va_pa_offset; >> extern unsigned long va_kernel_pa_offset; >> +#ifdef CONFIG_XIP_KERNEL >> +extern unsigned long va_kernel_xip_pa_offset; >> +#endif >> extern unsigned long pfn_base; >> #define ARCH_PFN_OFFSET (pfn_base) >> #else >> @@ -102,11 +105,29 @@ extern unsigned long pfn_base; >> extern unsigned long kernel_virt_addr; >> =20 >> #define linear_mapping_pa_to_va(x) ((void *)((unsigned long)(x) + v= a_pa_offset)) >> +#ifdef CONFIG_XIP_KERNEL >> +#define kernel_mapping_pa_to_va(y) ({ \ >> + unsigned long _y =3D y; \ >> + (_y >=3D CONFIG_PHYS_RAM_BASE) ? \ >> + (void *)((unsigned long)(_y) + va_kernel_pa_offset + XIP_OFFSET) : = \ >> + (void *)((unsigned long)(_y) + va_kernel_xip_pa_offset); \ >> + }) >> +#else >> #define kernel_mapping_pa_to_va(x) ((void *)((unsigned long)(x) + v= a_kernel_pa_offset)) >> +#endif >> #define __pa_to_va_nodebug(x) linear_mapping_pa_to_va(x) >> =20 >> #define linear_mapping_va_to_pa(x) ((unsigned long)(x) - va_pa_offs= et) >> +#ifdef CONFIG_XIP_KERNEL >> +#define kernel_mapping_va_to_pa(y) ({ \ >> + unsigned long _y =3D y; \ >> + (_y < kernel_virt_addr + XIP_OFFSET) ? \ >> + ((unsigned long)(_y) - va_kernel_xip_pa_offset) : \ >> + ((unsigned long)(_y) - va_kernel_pa_offset - XIP_OFFSET); \ >> + }) >> +#else >> #define kernel_mapping_va_to_pa(x) ((unsigned long)(x) - va_kernel_= pa_offset) >> +#endif >> #define __va_to_pa_nodebug(x) ({ \ >> unsigned long _x =3D x; \ >> (_x < kernel_virt_addr) ? \ >> @@ -139,9 +160,16 @@ extern phys_addr_t __phys_addr_symbol(unsigned lo= ng x); >> #define phys_to_page(paddr) (pfn_to_page(phys_to_pfn(paddr))) >> =20 >> #ifdef CONFIG_FLATMEM >> +#ifdef CONFIG_XIP_KERNEL >> +#define pfn_valid(pfn) \ >> + ((((pfn) >=3D ARCH_PFN_OFFSET) && (((pfn) - ARCH_PFN_OFFSET) < max_m= apnr)) || \ >> + ((pfn) >=3D PFN_DOWN(CONFIG_XIP_PHYS_ADDR) && \ >> + (((pfn) - PFN_DOWN(CONFIG_XIP_PHYS_ADDR)) < XIP_OFFSET))) >> +#else >> #define pfn_valid(pfn) \ >> (((pfn) >=3D ARCH_PFN_OFFSET) && (((pfn) - ARCH_PFN_OFFSET) < max_= mapnr)) >> #endif >> +#endif >> =20 >> #endif /* __ASSEMBLY__ */ >> =20 >> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm= /pgtable.h >> index 80e63a93e903..c2dc4f83eed8 100644 >> --- a/arch/riscv/include/asm/pgtable.h >> +++ b/arch/riscv/include/asm/pgtable.h >> @@ -64,6 +64,19 @@ >> #define FIXADDR_SIZE PGDIR_SIZE >> #endif >> #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) >> + >> +#ifdef CONFIG_XIP_KERNEL >> +#define XIP_OFFSET SZ_8M >> +#define XIP_FIXUP(addr) ({ \ >> + uintptr_t __a =3D (uintptr_t)(addr); \ >> + (__a >=3D CONFIG_XIP_PHYS_ADDR && __a < CONFIG_XIP_PHYS_ADDR + SZ_16= M) ? \ >> + __a - CONFIG_XIP_PHYS_ADDR + CONFIG_PHYS_RAM_BASE - XIP_OFFSET :\ >> + __a; \ >> + }) >> +#else >> +#define XIP_FIXUP(addr) (addr) >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> #endif >> =20 >> #ifndef __ASSEMBLY__ >> @@ -499,8 +512,16 @@ static inline int ptep_clear_flush_young(struct v= m_area_struct *vma, >> #define kern_addr_valid(addr) (1) /* FIXME */ >> =20 >> extern char _start[]; >> -extern void *dtb_early_va; >> -extern uintptr_t dtb_early_pa; >> +extern void *_dtb_early_va; >> +extern uintptr_t _dtb_early_pa; >> +#if defined(CONFIG_XIP_KERNEL) && defined(CONFIG_MMU) >> +#define dtb_early_va (*(void **)XIP_FIXUP(&_dtb_early_va)) >> +#define dtb_early_pa (*(uintptr_t *)XIP_FIXUP(&_dtb_early_pa)) >> +#else >> +#define dtb_early_va _dtb_early_va >> +#define dtb_early_pa _dtb_early_pa >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> void setup_bootmem(void); >> void paging_init(void); >> void misc_mem_init(void); >> diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S >> index 6cb05f22e52a..89cc58ab52b4 100644 >> --- a/arch/riscv/kernel/head.S >> +++ b/arch/riscv/kernel/head.S >> @@ -9,11 +9,23 @@ >> #include >> #include >> #include >> +#include >> #include >> #include >> #include >> #include "efi-header.S" >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> +.macro XIP_FIXUP_OFFSET reg >> + REG_L t0, _xip_fixup >> + add \reg, \reg, t0 >> +.endm >> +_xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_= OFFSET >> +#else >> +.macro XIP_FIXUP_OFFSET reg >> +.endm >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> __HEAD >> ENTRY(_start) >> /* >> @@ -70,6 +82,7 @@ pe_head_start: >> relocate: >> /* Relocate return address */ >> la a1, kernel_virt_addr >> + XIP_FIXUP_OFFSET a1 >> REG_L a1, 0(a1) >> la a2, _start >> sub a1, a1, a2 >> @@ -92,6 +105,7 @@ relocate: >> * to ensure the new translations are in use. >> */ >> la a0, trampoline_pg_dir >> + XIP_FIXUP_OFFSET a0 >> srl a0, a0, PAGE_SHIFT >> or a0, a0, a1 >> sfence.vma >> @@ -145,7 +159,9 @@ secondary_start_sbi: >> =20 >> slli a3, a0, LGREG >> la a4, __cpu_up_stack_pointer >> + XIP_FIXUP_OFFSET a4 >> la a5, __cpu_up_task_pointer >> + XIP_FIXUP_OFFSET a5 >> add a4, a3, a4 >> add a5, a3, a5 >> REG_L sp, (a4) >> @@ -157,6 +173,7 @@ secondary_start_common: >> #ifdef CONFIG_MMU >> /* Enable virtual memory and relocate to virtual address */ >> la a0, swapper_pg_dir >> + XIP_FIXUP_OFFSET a0 >> call relocate >> #endif >> call setup_trap_vector >> @@ -237,12 +254,33 @@ pmp_done: >> .Lgood_cores: >> #endif >> =20 >> +#ifndef CONFIG_XIP_KERNEL >> /* Pick one hart to run the main boot sequence */ >> la a3, hart_lottery >> li a2, 1 >> amoadd.w a3, a2, (a3) >> bnez a3, .Lsecondary_start >> =20 >> +#else >> + /* hart_lottery in flash contains a magic number */ >> + la a3, hart_lottery >> + mv a2, a3 >> + XIP_FIXUP_OFFSET a2 >> + lw t1, (a3) >> + amoswap.w t0, t1, (a2) >> + /* first time here if hart_lottery in RAM is not set */ >> + beq t0, t1, .Lsecondary_start >> + >> + la sp, _end + THREAD_SIZE >> + XIP_FIXUP_OFFSET sp >> + mv s0, a0 >> + call __copy_data >> + >> + /* Restore a0 copy */ >> + mv a0, s0 >> +#endif >> + >> +#ifndef CONFIG_XIP_KERNEL >> /* Clear BSS for flat non-ELF images */ >> la a3, __bss_start >> la a4, __bss_stop >> @@ -252,15 +290,18 @@ clear_bss: >> add a3, a3, RISCV_SZPTR >> blt a3, a4, clear_bss >> clear_bss_done: >> - >> +#endif >> /* Save hart ID and DTB physical address */ >> mv s0, a0 >> mv s1, a1 >> + >> la a2, boot_cpu_hartid >> + XIP_FIXUP_OFFSET a2 >> REG_S a0, (a2) >> =20 >> /* Initialize page tables and relocate to virtual addresses */ >> la sp, init_thread_union + THREAD_SIZE >> + XIP_FIXUP_OFFSET sp >> #ifdef CONFIG_BUILTIN_DTB >> la a0, __dtb_start >> #else >> @@ -269,6 +310,7 @@ clear_bss_done: >> call setup_vm >> #ifdef CONFIG_MMU >> la a0, early_pg_dir >> + XIP_FIXUP_OFFSET a0 >> call relocate >> #endif /* CONFIG_MMU */ >> =20 >> @@ -293,7 +335,9 @@ clear_bss_done: >> =20 >> slli a3, a0, LGREG >> la a1, __cpu_up_stack_pointer >> + XIP_FIXUP_OFFSET a1 >> la a2, __cpu_up_task_pointer >> + XIP_FIXUP_OFFSET a2 >> add a1, a3, a1 >> add a2, a3, a2 >> =20 >> diff --git a/arch/riscv/kernel/head.h b/arch/riscv/kernel/head.h >> index b48dda3d04f6..aabbc3ac3e48 100644 >> --- a/arch/riscv/kernel/head.h >> +++ b/arch/riscv/kernel/head.h >> @@ -12,6 +12,9 @@ extern atomic_t hart_lottery; >> =20 >> asmlinkage void do_page_fault(struct pt_regs *regs); >> asmlinkage void __init setup_vm(uintptr_t dtb_pa); >> +#ifdef CONFIG_XIP_KERNEL >> +asmlinkage void __init __copy_data(void); >> +#endif >> =20 >> extern void *__cpu_up_stack_pointer[]; >> extern void *__cpu_up_task_pointer[]; >> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c >> index 30e4af0fd50c..2ddf654c72bb 100644 >> --- a/arch/riscv/kernel/setup.c >> +++ b/arch/riscv/kernel/setup.c >> @@ -50,7 +50,11 @@ struct screen_info screen_info __section(".data") =3D= { >> * This is used before the kernel initializes the BSS so it can't b= e in the >> * BSS. >> */ >> -atomic_t hart_lottery __section(".sdata"); >> +atomic_t hart_lottery __section(".sdata") >> +#ifdef CONFIG_XIP_KERNEL >> +=3D ATOMIC_INIT(0xC001BEEF) >> +#endif >> +; >> unsigned long boot_cpu_hartid; >> static DEFINE_PER_CPU(struct cpu, cpu_devices); >> =20 >> @@ -254,7 +258,7 @@ void __init setup_arch(char **cmdline_p) >> #if IS_ENABLED(CONFIG_BUILTIN_DTB) >> unflatten_and_copy_device_tree(); >> #else >> - if (early_init_dt_verify(__va(dtb_early_pa))) >> + if (early_init_dt_verify(__va(XIP_FIXUP(dtb_early_pa)))) >> unflatten_device_tree(); >> else >> pr_err("No DTB found in kernel mappings\n"); >> @@ -266,7 +270,7 @@ void __init setup_arch(char **cmdline_p) >> if (IS_ENABLED(CONFIG_STRICT_KERNEL_RWX)) >> protect_kernel_text_data(); >> =20 >> -#if defined(CONFIG_64BIT) && defined(CONFIG_MMU) >> +#if defined(CONFIG_64BIT) && defined(CONFIG_MMU) && !defined(CONFIG_X= IP_KERNEL) >> protect_kernel_linear_mapping_text_rodata(); >> #endif >> =20 >> diff --git a/arch/riscv/kernel/vmlinux-xip.lds.S b/arch/riscv/kernel/v= mlinux-xip.lds.S >> new file mode 100644 >> index 000000000000..4b29b9917f99 >> --- /dev/null >> +++ b/arch/riscv/kernel/vmlinux-xip.lds.S >> @@ -0,0 +1,133 @@ >> +/* SPDX-License-Identifier: GPL-2.0-only */ >> +/* >> + * Copyright (C) 2012 Regents of the University of California >> + * Copyright (C) 2017 SiFive >> + * Copyright (C) 2020 Vitaly Wool, Konsulko AB >> + */ >> + >> +#include >> +#define LOAD_OFFSET KERNEL_LINK_ADDR >> +/* No __ro_after_init data in the .rodata section - which will always= be ro */ >> +#define RO_AFTER_INIT_DATA >> + >> +#include >> +#include >> +#include >> +#include >> +#include >> + >> +OUTPUT_ARCH(riscv) >> +ENTRY(_start) >> + >> +jiffies =3D jiffies_64; >> + >> +SECTIONS >> +{ >> + /* Beginning of code and text segment */ >> + . =3D LOAD_OFFSET; >> + _xiprom =3D .; >> + _start =3D .; >> + HEAD_TEXT_SECTION >> + INIT_TEXT_SECTION(PAGE_SIZE) >> + /* we have to discard exit text and such at runtime, not link time *= / >> + .exit.text : >> + { >> + EXIT_TEXT >> + } >> + >> + .text : { >> + _text =3D .; >> + _stext =3D .; >> + TEXT_TEXT >> + SCHED_TEXT >> + CPUIDLE_TEXT >> + LOCK_TEXT >> + KPROBES_TEXT >> + ENTRY_TEXT >> + IRQENTRY_TEXT >> + SOFTIRQENTRY_TEXT >> + *(.fixup) >> + _etext =3D .; >> + } >> + RO_DATA(L1_CACHE_BYTES) >> + .srodata : { >> + *(.srodata*) >> + } >> + .init.rodata : { >> + INIT_SETUP(16) >> + INIT_CALLS >> + CON_INITCALL >> + INIT_RAM_FS >> + } >> + _exiprom =3D .; /* End of XIP ROM area */ >> + >> + >> +/* >> + * From this point, stuff is considered writable and will be copied t= o RAM >> + */ >> + __data_loc =3D ALIGN(16); /* location in file */ >> + . =3D LOAD_OFFSET + XIP_OFFSET; /* location in memory */ >> + >> + _sdata =3D .; /* Start of data section */ >> + _data =3D .; >> + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) >> + _edata =3D .; >> + __start_ro_after_init =3D .; >> + .data.ro_after_init : AT(ADDR(.data.ro_after_init) - LOAD_OFFSET) { >> + *(.data..ro_after_init) >> + } >> + __end_ro_after_init =3D .; >> + >> + . =3D ALIGN(PAGE_SIZE); >> + __init_begin =3D .; >> + .init.data : { >> + INIT_DATA >> + } >> + .exit.data : { >> + EXIT_DATA >> + } >> + . =3D ALIGN(8); >> + __soc_early_init_table : { >> + __soc_early_init_table_start =3D .; >> + KEEP(*(__soc_early_init_table)) >> + __soc_early_init_table_end =3D .; >> + } >> + __soc_builtin_dtb_table : { >> + __soc_builtin_dtb_table_start =3D .; >> + KEEP(*(__soc_builtin_dtb_table)) >> + __soc_builtin_dtb_table_end =3D .; >> + } >> + PERCPU_SECTION(L1_CACHE_BYTES) >> + >> + . =3D ALIGN(PAGE_SIZE); >> + __init_end =3D .; >> + >> + .sdata : { >> + __global_pointer$ =3D . + 0x800; >> + *(.sdata*) >> + *(.sbss*) >> + } >> + >> + BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0) >> + EXCEPTION_TABLE(0x10) >> + >> + .rel.dyn : AT(ADDR(.rel.dyn) - LOAD_OFFSET) { >> + *(.rel.dyn*) >> + } >> + >> + /* >> + * End of copied data. We need a dummy section to get its LMA. >> + * Also located before final ALIGN() as trailing padding is not stor= ed >> + * in the resulting binary file and useless to copy. >> + */ >> + .data.endmark : AT(ADDR(.data.endmark) - LOAD_OFFSET) { } >> + _edata_loc =3D LOADADDR(.data.endmark); >> + >> + . =3D ALIGN(PAGE_SIZE); >> + _end =3D .; >> + >> + STABS_DEBUG >> + DWARF_DEBUG >> + >> + DISCARDS >> +} >> diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlin= ux.lds.S >> index 0726c05e0336..0a59b65cf789 100644 >> --- a/arch/riscv/kernel/vmlinux.lds.S >> +++ b/arch/riscv/kernel/vmlinux.lds.S >> @@ -4,8 +4,13 @@ >> * Copyright (C) 2017 SiFive >> */ >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> +#include "vmlinux-xip.lds.S" >> +#else >> + >> #include >> #define LOAD_OFFSET KERNEL_LINK_ADDR >> + >> #include >> #include >> #include >> @@ -133,3 +138,4 @@ SECTIONS >> =20 >> DISCARDS >> } >> +#endif /* CONFIG_XIP_KERNEL */ >> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c >> index 093f3a96ecfc..9961573f9a55 100644 >> --- a/arch/riscv/mm/init.c >> +++ b/arch/riscv/mm/init.c >> @@ -27,6 +27,9 @@ >> =20 >> unsigned long kernel_virt_addr =3D KERNEL_LINK_ADDR; >> EXPORT_SYMBOL(kernel_virt_addr); >> +#ifdef CONFIG_XIP_KERNEL >> +#define kernel_virt_addr (*((unsigned long *)XIP_FIXUP(&kernel_= virt_addr))) >> +#endif >> =20 >> unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] >> __page_aligned_bss; >> @@ -34,8 +37,8 @@ EXPORT_SYMBOL(empty_zero_page); >> =20 >> extern char _start[]; >> #define DTB_EARLY_BASE_VA PGDIR_SIZE >> -void *dtb_early_va __initdata; >> -uintptr_t dtb_early_pa __initdata; >> +void *_dtb_early_va __initdata; >> +uintptr_t _dtb_early_pa __initdata; >> =20 >> struct pt_alloc_ops { >> pte_t *(*get_pte_virt)(phys_addr_t pa); >> @@ -118,6 +121,10 @@ void __init setup_bootmem(void) >> phys_addr_t dram_end =3D memblock_end_of_DRAM(); >> phys_addr_t max_mapped_addr =3D __pa(~(ulong)0); >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> + vmlinux_start =3D __pa_symbol(&_sdata); >> +#endif >> + >> /* The maximal physical memory size is -PAGE_OFFSET. */ >> memblock_enforce_memory_limit(-PAGE_OFFSET); >> =20 >> @@ -159,17 +166,44 @@ void __init setup_bootmem(void) >> memblock_allow_resize(); >> } >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> + >> +extern char _xiprom[], _exiprom[]; >> +extern char _sdata[], _edata[]; >> + >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> #ifdef CONFIG_MMU >> -static struct pt_alloc_ops pt_ops; >> +static struct pt_alloc_ops _pt_ops; >> + >> +#ifdef CONFIG_XIP_KERNEL >> +#define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&_pt_ops)) >> +#else >> +#define pt_ops _pt_ops >> +#endif >> =20 >> /* Offset between linear mapping virtual address and kernel load ad= dress */ >> unsigned long va_pa_offset; >> EXPORT_SYMBOL(va_pa_offset); >> +#ifdef CONFIG_XIP_KERNEL >> +#define va_pa_offset (*((unsigned long *)XIP_FIXUP(&va_pa_offset))) >> +#endif >> /* Offset between kernel mapping virtual address and kernel load ad= dress */ >> unsigned long va_kernel_pa_offset; >> EXPORT_SYMBOL(va_kernel_pa_offset); >> +#ifdef CONFIG_XIP_KERNEL >> +#define va_kernel_pa_offset (*((unsigned long *)XIP_FIXUP(&va_kern= el_pa_offset))) >> +#endif >> +unsigned long va_kernel_xip_pa_offset; >> +EXPORT_SYMBOL(va_kernel_xip_pa_offset); >> +#ifdef CONFIG_XIP_KERNEL >> +#define va_kernel_xip_pa_offset (*((unsigned long *)XIP_FIXUP(= &va_kernel_xip_pa_offset))) >> +#endif >> unsigned long pfn_base; >> EXPORT_SYMBOL(pfn_base); >> +#ifdef CONFIG_XIP_KERNEL >> +#define pfn_base (*((unsigned long *)XIP_FIXUP(&pfn_base))) >> +#endif >> =20 >> pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss; >> pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss; >> @@ -177,6 +211,12 @@ pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss= ; >> =20 >> pgd_t early_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE); >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> +#define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)= ) >> +#define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) >> +#define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgpro= t_t prot) >> { >> unsigned long addr =3D __fix_to_virt(idx); >> @@ -252,6 +292,12 @@ pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss= ; >> pmd_t early_pmd[PTRS_PER_PMD] __initdata __aligned(PAGE_SIZE); >> pmd_t early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAGE_SIZE); >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> +#define trampoline_pmd ((pmd_t *)XIP_FIXUP(trampoline_pmd)) >> +#define fixmap_pmd ((pmd_t *)XIP_FIXUP(fixmap_pmd)) >> +#define early_pmd ((pmd_t *)XIP_FIXUP(early_pmd)) >> +#endif /* CONFIG_XIP_KERNEL */ >> + >> static pmd_t *__init get_pmd_virt_early(phys_addr_t pa) >> { >> /* Before MMU is enabled */ >> @@ -368,6 +414,19 @@ static uintptr_t __init best_map_size(phys_addr_t= base, phys_addr_t size) >> return PMD_SIZE; >> } >> =20 >> +#ifdef CONFIG_XIP_KERNEL >> +/* called from head.S with MMU off */ >> +asmlinkage void __init __copy_data(void) >> +{ >> + void *from =3D (void *)(&_sdata); >> + void *end =3D (void *)(&_end); >> + void *to =3D (void *)CONFIG_PHYS_RAM_BASE; >> + size_t sz =3D (size_t)(end - from + 1); >> + >> + memcpy(to, from, sz); >> +} >> +#endif >> + >> /* >> * setup_vm() is called from head.S with MMU-off. >> * >> @@ -387,7 +446,35 @@ static uintptr_t __init best_map_size(phys_addr_t= base, phys_addr_t size) >> #endif >> =20 >> uintptr_t load_pa, load_sz; >> +#ifdef CONFIG_XIP_KERNEL >> +#define load_pa (*((uintptr_t *)XIP_FIXUP(&load_pa))) >> +#define load_sz (*((uintptr_t *)XIP_FIXUP(&load_sz))) >> +#endif >> + >> +#ifdef CONFIG_XIP_KERNEL >> +uintptr_t xiprom, xiprom_sz; >> +#define xiprom_sz (*((uintptr_t *)XIP_FIXUP(&xiprom_sz))) >> +#define xiprom (*((uintptr_t *)XIP_FIXUP(&xiprom))) >> =20 >> +static void __init create_kernel_page_table(pgd_t *pgdir, uintptr_t m= ap_size) >> +{ >> + uintptr_t va, end_va; >> + >> + /* Map the flash resident part */ >> + end_va =3D kernel_virt_addr + xiprom_sz; >> + for (va =3D kernel_virt_addr; va < end_va; va +=3D map_size) >> + create_pgd_mapping(pgdir, va, >> + xiprom + (va - kernel_virt_addr), >> + map_size, PAGE_KERNEL_EXEC); >> + >> + /* Map the data in RAM */ >> + end_va =3D kernel_virt_addr + XIP_OFFSET + load_sz; >> + for (va =3D kernel_virt_addr + XIP_OFFSET; va < end_va; va +=3D map_= size) >> + create_pgd_mapping(pgdir, va, >> + load_pa + (va - (kernel_virt_addr + XIP_OFFSET)), >> + map_size, PAGE_KERNEL); >> +} >> +#else >> static void __init create_kernel_page_table(pgd_t *pgdir, uintptr_t= map_size) >> { >> uintptr_t va, end_va; >> @@ -398,16 +485,28 @@ static void __init create_kernel_page_table(pgd_= t *pgdir, uintptr_t map_size) >> load_pa + (va - kernel_virt_addr), >> map_size, PAGE_KERNEL_EXEC); >> } >> +#endif >> =20 >> asmlinkage void __init setup_vm(uintptr_t dtb_pa) >> { >> - uintptr_t pa; >> + uintptr_t __maybe_unused pa; >> uintptr_t map_size; >> #ifndef __PAGETABLE_PMD_FOLDED >> pmd_t fix_bmap_spmd, fix_bmap_epmd; >> #endif >> + >> +#ifdef CONFIG_XIP_KERNEL >> + xiprom =3D (uintptr_t)CONFIG_XIP_PHYS_ADDR; >> + xiprom_sz =3D (uintptr_t)(&_exiprom) - (uintptr_t)(&_xiprom); >> + >> + load_pa =3D (uintptr_t)CONFIG_PHYS_RAM_BASE; >> + load_sz =3D (uintptr_t)(&_end) - (uintptr_t)(&_sdata); >> + >> + va_kernel_xip_pa_offset =3D kernel_virt_addr - xiprom; >> +#else >> load_pa =3D (uintptr_t)(&_start); >> load_sz =3D (uintptr_t)(&_end) - load_pa; >> +#endif >> =20 >> va_pa_offset =3D PAGE_OFFSET - load_pa; >> va_kernel_pa_offset =3D kernel_virt_addr - load_pa; >> @@ -441,8 +540,13 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >> /* Setup trampoline PGD and PMD */ >> create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, >> (uintptr_t)trampoline_pmd, PGDIR_SIZE, PAGE_TABLE); >> +#ifdef CONFIG_XIP_KERNEL >> + create_pmd_mapping(trampoline_pmd, kernel_virt_addr, >> + xiprom, PMD_SIZE, PAGE_KERNEL_EXEC); >> +#else >> create_pmd_mapping(trampoline_pmd, kernel_virt_addr, >> load_pa, PMD_SIZE, PAGE_KERNEL_EXEC); >> +#endif >> #else >> /* Setup trampoline PGD */ >> create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, >> @@ -474,7 +578,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >> * whereas dtb_early_va will be used before setup_vm_final install= s >> * the linear mapping. >> */ >> - dtb_early_va =3D kernel_mapping_pa_to_va(dtb_pa); >> + dtb_early_va =3D kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa)); >> #endif /* CONFIG_BUILTIN_DTB */ >> #else >> #ifndef CONFIG_BUILTIN_DTB >> @@ -486,7 +590,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >> pa + PGDIR_SIZE, PGDIR_SIZE, PAGE_KERNEL); >> dtb_early_va =3D (void *)DTB_EARLY_BASE_VA + (dtb_pa & (PGDIR_SIZE= - 1)); >> #else /* CONFIG_BUILTIN_DTB */ >> - dtb_early_va =3D kernel_mapping_pa_to_va(dtb_pa); >> + dtb_early_va =3D kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa)); >> #endif /* CONFIG_BUILTIN_DTB */ >> #endif >> dtb_early_pa =3D dtb_pa; >> @@ -522,7 +626,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) >> #endif >> } >> =20 >> -#ifdef CONFIG_64BIT >> +#if defined(CONFIG_64BIT) && !defined(CONFIG_XIP_KERNEL) >> void protect_kernel_linear_mapping_text_rodata(void) >> { >> unsigned long text_start =3D (unsigned long)lm_alias(_start); >> >=20 --=20 Thanks, David / dhildenb