From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC41BC4332F for ; Thu, 14 Dec 2023 22:36:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59BC98D00F3; Thu, 14 Dec 2023 17:36:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 54C5E8D00C7; Thu, 14 Dec 2023 17:36:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43B0B8D00F3; Thu, 14 Dec 2023 17:36:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 328158D00C7 for ; Thu, 14 Dec 2023 17:36:10 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E337640457 for ; Thu, 14 Dec 2023 22:36:09 +0000 (UTC) X-FDA: 81566883258.18.DD9A762 Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by imf16.hostedemail.com (Postfix) with ESMTP id F2BC5180016 for ; Thu, 14 Dec 2023 22:36:07 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none); spf=pass (imf16.hostedemail.com: domain of robherring2@gmail.com designates 209.85.210.54 as permitted sender) smtp.mailfrom=robherring2@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702593368; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TPGpS3N5cf7hiGlwsGmwzPf9DamC+4nzAyalFlTf6vc=; b=SY5DDIcNIeFc6sTmC80GlMxTMJRLOei91NcbltdLg26hWvwM/qc94uRPGjNdWuPMvg1vOL zRr9P0KMNbToqmjivDGVvu0/cL4O9tx2/ByX10272uEX+9Rky0gZI5DFVZI29QB8Ye+z27 IrMRAEe8+z4bzbdJ5PpgoqA+VXSbRkQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none); spf=pass (imf16.hostedemail.com: domain of robherring2@gmail.com designates 209.85.210.54 as permitted sender) smtp.mailfrom=robherring2@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702593368; a=rsa-sha256; cv=none; b=vK5OEUThQg4HNQZlY2t7V2rjFAY5Tkdvs9h1uwJwZQ09PgYSQCCJ/cZZd9Tg966I6Vbgh9 D/a2jHPobP6+9FIN2NGD/ODfD0BUHECdDdm5Kc+1wiM/iyy+VF4czGG8OJXlFJloUW9Ug2 7iAMzTb40bsJBIBLv/yCVySOFffKSd4= Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-6d9d4193d94so61167a34.3 for ; Thu, 14 Dec 2023 14:36:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702593367; x=1703198167; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=TPGpS3N5cf7hiGlwsGmwzPf9DamC+4nzAyalFlTf6vc=; b=L/aLYKTecEE8JxUF74OocCtJFqGInk7dtgyhicvwh0oM43tCkyO8RQ5rtfBoRmlCF+ vRZOp6Zf7/T+AiU95pexM7OYOpSaJW1ZsJiubpCNXIamKtK6NCiNE29i0Gbps8YIZJGJ OgrBzPEpZKFXSw7dXzzPROQypGk9YIwmdyoe6lLufwVHYD7Uex+ru5pph5/1avGU54bO kpt1RwtvvG1txnJy57vb2lvN65uj6jjasexCbvGmoKOfxXKZ/u5sN/6sWrRMIUM6kJ3R Nh2FWeAkjBvg9phgXRszK8MXVVsCR3n28HzWOcf1Xmx44M+3TRoi6BFEk4qOoW2w7ydM FzMg== X-Gm-Message-State: AOJu0YyG/PD5wRwO4MlsGlu62jLk/Wg2iV4KOAt1+20mDYx0sYOwRbM8 hkR6uUfqkRcI2pPGiAOZ+A== X-Google-Smtp-Source: AGHT+IH/ntYBrk6W3zFr9Ig4vAkaNVNC1PcAuUD1j3FxDQR5iGtwTMlXCspq02WUlgURzzmxe383HA== X-Received: by 2002:a9d:7f8a:0:b0:6da:3f3e:7b30 with SMTP id t10-20020a9d7f8a000000b006da3f3e7b30mr2248166otp.39.1702593367061; Thu, 14 Dec 2023 14:36:07 -0800 (PST) Received: from herring.priv (66-90-144-107.dyn.grandenetworks.net. [66.90.144.107]) by smtp.gmail.com with ESMTPSA id k13-20020a056830150d00b006d81e704023sm3338450otp.2.2023.12.14.14.36.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Dec 2023 14:36:06 -0800 (PST) Received: (nullmailer pid 1058742 invoked by uid 1000); Thu, 14 Dec 2023 22:36:04 -0000 Date: Thu, 14 Dec 2023 16:36:04 -0600 From: Rob Herring To: Alexander Graf Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kexec@lists.infradead.org, linux-doc@vger.kernel.org, x86@kernel.org, Eric Biederman , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Steven Rostedt , Andrew Morton , Mark Rutland , Tom Lendacky , Ashish Kalra , James Gowans , Stanislav Kinsburskii , arnd@arndb.de, pbonzini@redhat.com, madvenka@linux.microsoft.com, Anthony Yznaga , Usama Arif , David Woodhouse , Benjamin Herrenschmidt Subject: Re: [PATCH 06/15] arm64: Add KHO support Message-ID: <20231214223604.GA1045434-robh@kernel.org> References: <20231213000452.88295-1-graf@amazon.com> <20231213000452.88295-7-graf@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231213000452.88295-7-graf@amazon.com> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F2BC5180016 X-Stat-Signature: iazy1wzdkizyfteqag3fj85rtzyix13j X-HE-Tag: 1702593367-870265 X-HE-Meta: U2FsdGVkX19f2IlPEAl/KCI5cxUvOETSsH4SpM7+NsHHyZ7CyKLAFTgmyVk5YHZsm0p03k/wzvVcTkPXZCiaSZt5X5ufAsItbwX7Pf6xot2YdKskAahERnYyiqtgacQkynFFjQQT8Ib71b22QdfUi//YikSUmlvJy+yzN2Irx8F5UybiralWc5BP2F4ily0cX/bdtnmLl4Arw7x7dPdNPGMzvrjPwzXGJq5kUOkOpTAKz0coh29YRAH32iTP1pQ/4Jo1mja1BTPX1a3R5thdWrX8ZeaFL9iiS2kMKsOvtnqtBqW28Gk8m4OSMWIaZvLtb14IBI0LFNMeEtmZ2lEtfrL+17+G/Sfk+adQHJqOUJDcpngwb30hYTXhCxEIG9nvuTJycUvV0wweBctaxbFnk9ZNgqxLd6FnZo9l0H1mN9RWFhWv8uJEakJTxR9j14cg9VoZ//LT4ED3WU0QZB56jIapGeoWy1HdR8O3aYmKocajPEP/FrmewgsJlR/ltmldkMKT9cAwVcoCgFVAq2wTU6h17xwDsF0H0NC1d9ioyDNSJhZCfcEafvjkmWgDlad4Nk9MOQHlbZCaWKtEMQlBdtu9OmpnaYmvhwSl41uovg+qH6SJksWP+pThLU/GnnBErWnBkrxyV1rAiI0ousb5gfV5+N/gktKSTyN7uZpaKVlCdNf8no3TvXe7ESMZv9/2d67Vyt3L97RoKXc4xPFmpIr/2WFt18CCgWt6/VVVLbfqPiFi8XLKvkRUabO7CLydJWk0rAbF25LmmLxVcEhhCE1GA3716hKPF4PX56kDNLHbD3IF6ZvoJ1SAIf9JHEjUTkiggzzsz6cAVf/o2dxAueX4H1ywGK15vcriX1gqeuStWF15mda+t2RSzCakq/abcS8PaWI9wxw3kPAGDAql7yl17u0iSR6siq2boaqhD4v48P9FouFAw5tiOAyOAgpPjWKtetusGJlxHAHj8uE wtDCyzWZ +oTQj8v2qOV1sVSvLETXCWv5A4Or+A4R4hVhePahl5k1UZX9MYbTF9VJX4q5Yv7cw0wGOjB8rtHPQv3n2/mRSINhRVwKPhUtpTuPFTn4RtCY75i2aOaHR3FwQDA4f0xHf2dhkESqzn/XQRGIJWg1WDvrjXCMTKpLjE790UqHXpGAfxyky5KqLrH7gAH6gpnRO21XINpo/8oBgiZMLmWL2oHjT2zyQkf7DIInd0+dX+hHp2Xgq26lSIMxKUn55vI5Yg+lvpB7b0wNWc4YgL2jwbQqIM67yF4utocdVPel+40lUSm00cig1ZPoDxoFAStMcJi+naUmrCmDC6AWc+x5k75lZZP3N0iddBau3RZcv3kew/4nJzuGjFAUjtv/5aAfHa7MCmnW/YSDQtPBlmTrIS3/KA0C6v6NiYibZRs5hwnicq+FJ0pApCGVQpwsVjEZCwiCpJCk3dFXeP768SD+us+tN7MmpHYtMRuauxJUf9wFIE2hQf4uwDLQsq1Lk/fsUGq3RyD6XgLU5Gpc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Dec 13, 2023 at 12:04:43AM +0000, Alexander Graf wrote: > We now have all bits in place to support KHO kexecs. This patch adds > awareness of KHO in the kexec file as well as boot path for arm64 and > adds the respective kconfig option to the architecture so that it can > use KHO successfully. > > Signed-off-by: Alexander Graf > --- > arch/arm64/Kconfig | 12 ++++++++++++ > arch/arm64/kernel/setup.c | 2 ++ > arch/arm64/mm/init.c | 8 ++++++++ > drivers/of/fdt.c | 41 +++++++++++++++++++++++++++++++++++++++ > drivers/of/kexec.c | 36 ++++++++++++++++++++++++++++++++++ > 5 files changed, 99 insertions(+) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 7b071a00425d..1ba338ce7598 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -1501,6 +1501,18 @@ config ARCH_SUPPORTS_CRASH_DUMP > config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION > def_bool CRASH_CORE > > +config KEXEC_KHO > + bool "kexec handover" > + depends on KEXEC > + select MEMBLOCK_SCRATCH > + select LIBFDT > + select CMA > + help > + Allow kexec to hand over state across kernels by generating and > + passing additional metadata to the target kernel. This is useful > + to keep data or state alive across the kexec. For this to work, > + both source and target kernels need to have this option enabled. Why do we have the same kconfig entry twice? Here and x86. > + > config TRANS_TABLE > def_bool y > depends on HIBERNATION || KEXEC_CORE > diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c > index 417a8a86b2db..8035b673d96d 100644 > --- a/arch/arm64/kernel/setup.c > +++ b/arch/arm64/kernel/setup.c > @@ -346,6 +346,8 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) > > paging_init(); > > + kho_reserve_mem(); > + > acpi_table_upgrade(); > > /* Parse the ACPI tables for possible boot-time configuration */ > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index 74c1db8ce271..254d82f3383a 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -358,6 +358,8 @@ void __init bootmem_init(void) > */ > arch_reserve_crashkernel(); > > + kho_reserve(); > + reserve what? It is not obvious what the difference between kho_reserve_mem() and kho_reserve() are. > memblock_dump_all(); > } > > @@ -386,6 +388,12 @@ void __init mem_init(void) > /* this will put all unused low memory onto the freelists */ > memblock_free_all(); > > + /* > + * Now that all KHO pages are marked as reserved, let's flip them back > + * to normal pages with accurate refcount. > + */ > + kho_populate_refcount(); > + > /* > * Check boundaries twice: Some fundamental inconsistencies can be > * detected at build time already. > diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c > index bf502ba8da95..af95139351ed 100644 > --- a/drivers/of/fdt.c > +++ b/drivers/of/fdt.c > @@ -1006,6 +1006,44 @@ void __init early_init_dt_check_for_usable_mem_range(void) > memblock_add(rgn[i].base, rgn[i].size); > } > > +/** > + * early_init_dt_check_kho - Decode info required for kexec handover from DT > + */ > +void __init early_init_dt_check_kho(void) > +{ > +#ifdef CONFIG_KEXEC_KHO if (!IS_ENABLED(CONFIG_KEXEC_KHO)) return; You'll need a kho_populate() stub. > + unsigned long node = chosen_node_offset; > + u64 kho_start, scratch_start, scratch_size, mem_start, mem_size; > + const __be32 *p; > + int l; > + > + if ((long)node < 0) > + return; > + > + p = of_get_flat_dt_prop(node, "linux,kho-dt", &l); > + if (l != (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32)) > + return; > + > + kho_start = dt_mem_next_cell(dt_root_addr_cells, &p); > + > + p = of_get_flat_dt_prop(node, "linux,kho-scratch", &l); > + if (l != (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32)) > + return; > + > + scratch_start = dt_mem_next_cell(dt_root_addr_cells, &p); > + scratch_size = dt_mem_next_cell(dt_root_addr_cells, &p); > + > + p = of_get_flat_dt_prop(node, "linux,kho-mem", &l); > + if (l != (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32)) > + return; > + > + mem_start = dt_mem_next_cell(dt_root_addr_cells, &p); > + mem_size = dt_mem_next_cell(dt_root_addr_cells, &p); > + > + kho_populate(kho_start, scratch_start, scratch_size, mem_start, mem_size); > +#endif > +} > + > #ifdef CONFIG_SERIAL_EARLYCON > > int __init early_init_dt_scan_chosen_stdout(void) > @@ -1304,6 +1342,9 @@ void __init early_init_dt_scan_nodes(void) > > /* Handle linux,usable-memory-range property */ > early_init_dt_check_for_usable_mem_range(); > + > + /* Handle kexec handover */ > + early_init_dt_check_kho(); > } > > bool __init early_init_dt_scan(void *params) > diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c > index 68278340cecf..a612e6bb8c75 100644 > --- a/drivers/of/kexec.c > +++ b/drivers/of/kexec.c > @@ -264,6 +264,37 @@ static inline int setup_ima_buffer(const struct kimage *image, void *fdt, > } > #endif /* CONFIG_IMA_KEXEC */ > > +static int kho_add_chosen(const struct kimage *image, void *fdt, int chosen_node) > +{ > + int ret = 0; > + > +#ifdef CONFIG_KEXEC_KHO ditto Though perhaps image->kho is not defined? > + if (!image->kho.dt.buffer || !image->kho.mem_cache.buffer) > + goto out; > + > + pr_debug("Adding kho metadata to DT"); > + > + ret = fdt_appendprop_addrrange(fdt, 0, chosen_node, "linux,kho-dt", > + image->kho.dt.mem, image->kho.dt.memsz); > + if (ret) > + goto out; > + > + ret = fdt_appendprop_addrrange(fdt, 0, chosen_node, "linux,kho-scratch", > + kho_scratch_phys, kho_scratch_len); > + if (ret) > + goto out; > + > + ret = fdt_appendprop_addrrange(fdt, 0, chosen_node, "linux,kho-mem", > + image->kho.mem_cache.mem, > + image->kho.mem_cache.bufsz); > + if (ret) > + goto out; > + > +out: > +#endif > + return ret; > +} > + > /* > * of_kexec_alloc_and_setup_fdt - Alloc and setup a new Flattened Device Tree > * > @@ -412,6 +443,11 @@ void *of_kexec_alloc_and_setup_fdt(const struct kimage *image, > } > } > > + /* Add kho metadata if this is a KHO image */ > + ret = kho_add_chosen(image, fdt, chosen_node); > + if (ret) > + goto out; > + > /* add bootargs */ > if (cmdline) { > ret = fdt_setprop_string(fdt, chosen_node, "bootargs", cmdline); > -- > 2.40.1 > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > >