From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E4A1C43603 for ; Wed, 4 Dec 2019 16:00:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 45B2D2073C for ; Wed, 4 Dec 2019 16:00:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="YKxssQdB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 45B2D2073C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E552D6B0B77; Wed, 4 Dec 2019 11:00:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E29EF6B0B78; Wed, 4 Dec 2019 11:00:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D19156B0B79; Wed, 4 Dec 2019 11:00:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0108.hostedemail.com [216.40.44.108]) by kanga.kvack.org (Postfix) with ESMTP id BCFDA6B0B77 for ; Wed, 4 Dec 2019 11:00:16 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 7F363127C for ; Wed, 4 Dec 2019 16:00:16 +0000 (UTC) X-FDA: 76227920832.25.grain96_1f66233975f25 X-HE-Tag: grain96_1f66233975f25 X-Filterd-Recvd-Size: 10053 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Wed, 4 Dec 2019 16:00:15 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id z22so219244qto.7 for ; Wed, 04 Dec 2019 08:00:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=aEpoNSBZELpHiRBNE1QFjkWf+vscZ+r++2h/PQNGFz0=; b=YKxssQdB2nkSnRmDmbfn5VCUiH5kqdrBHhbh+V3b4+nr+z646g3JHqhL6bblZaZaLH Rolew7pKXT73uPrvNQPOfjbj5yFybvuIL+TGTxyv/SHYnwPFptcBj2Gfmz/yiObNOB3L d3ZOegLclFJOcKXZieLuGA/BK//Tvs3rEwrY/kExmgnSkmmoUukhtI9DSuJb9bLdZ2LC QvjS6/CRT8KGswjL0iNrFCpWp/G7OC2M+db/Cqjgb7Yuxl1VlJFI1X72s31ILNWj0aC8 TETaKaSfO2mkYxI+ENwonI0JyKT5GYCW4FBjVz0YLjw3JQdvtJ7S9X8uVUIbFPjW1zrz ZFhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aEpoNSBZELpHiRBNE1QFjkWf+vscZ+r++2h/PQNGFz0=; b=s3IC81l/IvOIXez7gKiECybrKUF48hPa/BW4KUXp+Zu5esecxOTahiTYA1zNGB3F0O DflyvxuLUfIyvfmWxDuitM0Ddg/XaYZ9fq+0Jk3uF4AvxNJrHdJHXG6/Vtua7eCNFE18 jckgp8WbG/0HchmxGAQNbm4FuRZUSMjrvwc5Wq7pK9en6EbIN13CWh0TMnSFyeCRrzME Br6lQfPgrXq7UkG8Se0jFe5lNkp8qAv8X0BhuYVZuipaXZ4PoopO/GcnBBlwKpQdeTwj sSw5k2v/Nt1PE0PIv0VaVvNCsA7RpCOB95FtJLZ0D/TuL7Hu/ledF4jRkTAaVO7mxryS FJuw== X-Gm-Message-State: APjAAAUVIEcim+4OGaoeYCRryCSR5GGhh/WbKk5mAd37dSgizHracReY wdf0NXiQrHfc3CZLfVdO8JenPQ== X-Google-Smtp-Source: APXvYqx6afAt2ZT0wnIMzmN9acj0tBCpZ5+3kIS/dhv4aZ1jIEtj4FSFJYdsTEufDSKLyhfBgevqsg== X-Received: by 2002:ac8:4509:: with SMTP id q9mr3224598qtn.214.1575475215043; Wed, 04 Dec 2019 08:00:15 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id w21sm4177585qth.17.2019.12.04.08.00.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2019 08:00:14 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de Subject: [PATCH v8 23/25] arm64: kexec: configure trans_pgd page table for kexec Date: Wed, 4 Dec 2019 10:59:36 -0500 Message-Id: <20191204155938.2279686-24-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191204155938.2279686-1-pasha.tatashin@soleen.com> References: <20191204155938.2279686-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Configure a page table located in kexec-safe memory that has the following mappings: 1. identity mapping for text of relocation function with executable permission. 2. identity mapping for argument for relocation function. 3. linear mappings for all source ranges 4. linear mappings for all destination ranges. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 14 ++++ arch/arm64/kernel/asm-offsets.c | 5 ++ arch/arm64/kernel/machine_kexec.c | 104 +++++++++++++++++++++++++++++- 3 files changed, 122 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexe= c.h index 414a0a41a60a..df911a4aa8ce 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -98,6 +98,10 @@ extern const unsigned long kexec_kern_reloc_offset; extern const unsigned long kexec_el2_vectors_offset; #endif =20 +#define KEXEC_SRC_START PAGE_OFFSET +#define KEXEC_DST_START (PAGE_OFFSET + \ + ((UL(0xffffffffffffffff) - PAGE_OFFSET) >> 1) + 1) + /* * kern_reloc_arg is passed to kernel relocation function as an argument= . * head kimage->head, allows to traverse through relocation segments. @@ -108,6 +112,11 @@ extern const unsigned long kexec_el2_vectors_offset; * el2_vector If present means that relocation routine will go to EL1 * from EL2 to do the copy, and then back to EL2 to do the jump * to new world. + * trans_ttbr0 idmap for relocation function and its argument + * trans_ttbr1 linear map for source/destination addresses. + * src_addr linear map for source pages. + * dst_addr linear map for destination pages. + * copy_len Number of bytes that need to be copied */ struct kern_reloc_arg { phys_addr_t head; @@ -117,6 +126,11 @@ struct kern_reloc_arg { phys_addr_t kern_arg2; phys_addr_t kern_arg3; phys_addr_t el2_vector; + phys_addr_t trans_ttbr0; + phys_addr_t trans_ttbr1; + void *src_addr; + void *dst_addr; + unsigned long copy_len; }; =20 #define ARCH_HAS_KIMAGE_ARCH diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offs= ets.c index ff974b648347..63060ea51727 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -137,6 +137,11 @@ int main(void) DEFINE(KEXEC_KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_ar= g2)); DEFINE(KEXEC_KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_ar= g3)); DEFINE(KEXEC_KRELOC_EL2_VECTOR, offsetof(struct kern_reloc_arg, el2_ve= ctor)); + DEFINE(KEXEC_KRELOC_TRANS_TTBR0, offsetof(struct kern_reloc_arg, trans= _ttbr0)); + DEFINE(KEXEC_KRELOC_TRANS_TTBR1, offsetof(struct kern_reloc_arg, trans= _ttbr1)); + DEFINE(KEXEC_KRELOC_SRC_ADDR, offsetof(struct kern_reloc_arg, src_addr= )); + DEFINE(KEXEC_KRELOC_DST_ADDR, offsetof(struct kern_reloc_arg, dst_addr= )); + DEFINE(KEXEC_KRELOC_COPY_LEN, offsetof(struct kern_reloc_arg, copy_len= )); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machin= e_kexec.c index ac6ade7c96ff..8edcc4be0b15 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 #include "cpu-reset.h" =20 @@ -70,10 +71,102 @@ static void *kexec_page_alloc(void *arg) return page_address(page); } =20 +/* + * Map source segments starting from KEXEC_SRC_START, and map destinatio= n + * segments starting from KEXEC_DST_START, and return size of copy in + * *copy_len argument. + * Relocation function essentially needs to do: + * memcpy(KEXEC_DST_START, KEXEC_SRC_START, copy_len); + */ +static int map_segments(struct kimage *kimage, pgd_t *pgdp, + struct trans_pgd_info *info, + unsigned long *copy_len) +{ + unsigned long *ptr =3D 0; + unsigned long dest =3D 0; + unsigned long src_va =3D KEXEC_SRC_START; + unsigned long dst_va =3D KEXEC_DST_START; + unsigned long len =3D 0; + unsigned long entry, addr; + int rc; + + for (entry =3D kimage->head; !(entry & IND_DONE); entry =3D *ptr++) { + addr =3D entry & PAGE_MASK; + + switch (entry & IND_FLAGS) { + case IND_DESTINATION: + dest =3D addr; + break; + case IND_INDIRECTION: + ptr =3D __va(addr); + if (rc) + return rc; + break; + case IND_SOURCE: + rc =3D trans_pgd_map_page(info, pgdp, __va(addr), + src_va, PAGE_KERNEL); + if (rc) + return rc; + rc =3D trans_pgd_map_page(info, pgdp, __va(dest), + dst_va, PAGE_KERNEL); + if (rc) + return rc; + dest +=3D PAGE_SIZE; + src_va +=3D PAGE_SIZE; + dst_va +=3D PAGE_SIZE; + len +=3D PAGE_SIZE; + } + } + *copy_len =3D len; + + return 0; +} + +static int mmu_relocate_setup(struct kimage *kimage, void *reloc_code, + struct kern_reloc_arg *kern_reloc_arg) +{ + struct trans_pgd_info info =3D { + .trans_alloc_page =3D kexec_page_alloc, + .trans_alloc_arg =3D kimage, + }; + + pgd_t *trans_ttbr0 =3D kexec_page_alloc(kimage); + pgd_t *trans_ttbr1 =3D kexec_page_alloc(kimage); + int rc; + + if (!trans_ttbr0 || !trans_ttbr1) + return -ENOMEM; + + rc =3D map_segments(kimage, trans_ttbr1, &info, + &kern_reloc_arg->copy_len); + if (rc) + return rc; + + /* Map relocation function va =3D=3D pa */ + rc =3D trans_pgd_map_page(&info, trans_ttbr0, reloc_code, + __pa(reloc_code), PAGE_KERNEL_EXEC); + if (rc) + return rc; + + /* Map relocation function argument va =3D=3D pa */ + rc =3D trans_pgd_map_page(&info, trans_ttbr0, kern_reloc_arg, + __pa(kern_reloc_arg), PAGE_KERNEL); + if (rc) + return rc; + + kern_reloc_arg->trans_ttbr0 =3D phys_to_ttbr(__pa(trans_ttbr0)); + kern_reloc_arg->trans_ttbr1 =3D phys_to_ttbr(__pa(trans_ttbr1)); + kern_reloc_arg->src_addr =3D (void *)KEXEC_SRC_START; + kern_reloc_arg->dst_addr =3D (void *)KEXEC_DST_START; + + return 0; +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code =3D page_to_virt(kimage->control_code_page); struct kern_reloc_arg *kern_reloc_arg =3D kexec_page_alloc(kimage); + int rc =3D 0; =20 if (!kern_reloc_arg) return -ENOMEM; @@ -89,9 +182,18 @@ int machine_kexec_post_load(struct kimage *kimage) kern_reloc_arg->el2_vector =3D __pa(reloc_code) + kexec_el2_vectors_offset; } + + /* + * If relocation is not needed, we do not need to enable MMU in + * relocation routine, therefore do not create page tables for + * scenarios such as crash kernel + */ + if (!(kimage->head & IND_DONE)) + rc =3D mmu_relocate_setup(kimage, reloc_code, kern_reloc_arg); + kexec_image_info(kimage); =20 - return 0; + return rc; } =20 =20 --=20 2.24.0