From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C37BC3DA6E for ; Wed, 3 Jan 2024 18:49:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A02DA6B03BD; Wed, 3 Jan 2024 13:49:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B31C6B03BE; Wed, 3 Jan 2024 13:49:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87AC46B03BF; Wed, 3 Jan 2024 13:49:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 786C96B03BD for ; Wed, 3 Jan 2024 13:49:22 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 49C871203A2 for ; Wed, 3 Jan 2024 18:49:22 +0000 (UTC) X-FDA: 81638887764.22.A6DBF8C Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf27.hostedemail.com (Postfix) with ESMTP id 8870A4000A for ; Wed, 3 Jan 2024 18:49:18 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="K5x/6q9e"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of robh+dt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=robh+dt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704307759; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XHTgKWDe92NCb/9OIN+O1wGnaeXvh8c60xvs5SAQn/M=; b=Bylk7MbYhuS14+udQDpjEoUDZzsXlM95j2XYR58ZwW/lFPLmbxgGbRixfFsC+zCR9oW0rx 6Nv/Bx1vS0B10vRzQRThR3WD4ZZDEJa8ChZWGsmfc4jpqkExSrKEmqwiwjMBuKJ0hgR/sN VDPNUCPgBNm2Kfq3b82LqbEB9F9uQt8= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="K5x/6q9e"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of robh+dt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=robh+dt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704307759; a=rsa-sha256; cv=none; b=pYBp0tbDpgp8uWYxFIipnoVFv/s/sgI5VTCK2m1+NuJ94hjuK2eZazkKVi+De53zBtrnPs PdhA+sv61FctZ6MAdMP4aHFpMzLZZUXOtbQN0l9agLWpFI+qQ9+yn+tvMCWvldJPd3SM0i en1aDxO4IQQ9VHEwHg1/Rf5INe398YE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2430BCE1807 for ; Wed, 3 Jan 2024 18:49:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DA29C433BC for ; Wed, 3 Jan 2024 18:49:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704307753; bh=RMZCoyni1dg/u5PWGXOjNR0UmnG/FzeuF8AtWgpE53o=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=K5x/6q9eylmzF5sAr17s6S0TLA04X/pKUJtUZ4DLRSRd/xvrn2X79COvg8hz2W9ys 58dGoQN1/FvEv8ceI3Ryh8hOh6Eizc/NzXe+eJdeK+/US5xP9/Rgjd/ZT55tfizZtO Qn6PGEuYH+j77hWtDiMErJqhVPAXvGdg/KEJV8VI+HwJHOrtB6sgyJktzW8mS4Mf+Y S7K8ADPaOzZtqZiiXIiszv3cuUl2vWNxaAb3YMRTf0/ipdFOg6zAK/6nPvLY5gCbrF e6Qc+p7ZzKuHF6tiJiA+thhHyXkj0qhWZQgSc92lhS89xikV3FtXrh9ik/222P9eS7 wjBaxRmlXRTQw== Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2ccb923c4d2so81328061fa.1 for ; Wed, 03 Jan 2024 10:49:12 -0800 (PST) X-Gm-Message-State: AOJu0YynKFpBmd27eDS4WtsGIHxrk/BaHPIDFHmW+LRLnwyicxa+z1S8 zA/0B7E1d9Cy65Vy2HLO9mIN4tfNoY84aYswpw== X-Google-Smtp-Source: AGHT+IFb3zTv3w6i/eFBsPe9r0iuXmmX/2JiP7xGNWZZFBabGYb5yb23iDV40OA8jh6BzQ1/gm/DPrOco/LCOyYOSFg= X-Received: by 2002:a05:651c:168f:b0:2cc:6dff:9383 with SMTP id bd15-20020a05651c168f00b002cc6dff9383mr5104245ljb.83.1704307750860; Wed, 03 Jan 2024 10:49:10 -0800 (PST) MIME-Version: 1.0 References: <20231222193607.15474-1-graf@amazon.com> <20231222195144.24532-1-graf@amazon.com> <20231222195144.24532-2-graf@amazon.com> In-Reply-To: <20231222195144.24532-2-graf@amazon.com> From: Rob Herring Date: Wed, 3 Jan 2024 11:48:58 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 07/17] kexec: Add documentation for KHO To: Alexander Graf Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kexec@lists.infradead.org, linux-doc@vger.kernel.org, x86@kernel.org, Eric Biederman , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Steven Rostedt , Andrew Morton , Mark Rutland , Tom Lendacky , Ashish Kalra , James Gowans , Stanislav Kinsburskii , arnd@arndb.de, pbonzini@redhat.com, madvenka@linux.microsoft.com, Anthony Yznaga , Usama Arif , David Woodhouse , Benjamin Herrenschmidt Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 8870A4000A X-Stat-Signature: dippzod19ko1hr8nwhpnimprfapmniky X-HE-Tag: 1704307758-101957 X-HE-Meta: U2FsdGVkX18Zlo+jYul6TNviqR8MG5bhiF6kLbDxusxq/oHgddvlKGmMctyF8I7VSEBR4JoN1fwd1ulOR2UVBlDjSt8wdngjzdDLx2xUltS/iqgyedFttPOfvFYl+weZVn/rBhvR9FbkeLr1dcN5i4olru38FwkB3An5JcjCsoTIOyhN+VPcbb6qcQgFNTwqfP+H5M9nWK/s6cl8dPuB2E6Yk4Bgf0aAkxkcXYCcPlI7dUmuvGRVoJKO1E6iqBGBzK33nExCb5gbfmQW1xhkjFCdnfekXhA9CXwkowoaAgPPzLMhH4jfN5Y98Ng24/oI4M0d50gmGVp/+TuVqK+VAEjvWBVOzRpeugjt6OHbWnJWTtwHNITc4Yw74TvBjvZ6g30H6STISE59iA6iAZgrkSi3SsLgTpwqxIFvZic8yfYJheGWTocoFkPNfKu/ouwZk9YCYDgTn8NoivwULA40hGLMWOewqvr7wJAzCznlrR/5MhlWfTe3OfSoS2vdV0zJl9dsdT9ifkbxQz9+M3cKNH9xTMxbejm/YkC1ATFB4Ws6+AU5K4kmqEF+eVwmXMe29rLs7MmMBAjQZwUFpAaRElrPrY73Yd09bhgHeaVmO956cWFHaI6nN+Dr2tORawrg4zL4U3B4Kf7yrmlC3MWYW7iiglfHgd/IWfwFOdiV7fjNtEKaapBj0vGxlCYWEt+87LWX8HuATvhwPyJtrFYrZG6aEcRYAzfnxd4J5emy/4Q3v7zPCaX021MkAd5kz4DZ9b3VU0VoplR0WQ5RJN2Sj7RrzbaZFelFkKIvrbb+xAm5dJv8+iTHXue46dP+9S6FVnAfS3SxZhnYbHZz5xNQk24eQ5JRG8n/7NRXdXk8GRh9R36CiXv6QKW6SMcfsH9stOuY4q90W6HELitN1hR1Auc2l24VIWIIF/UHSr4HxCUgJFouVUGNiC7uiR4RP1e/qSEZlR/bCC8fZ7AidIC SRYl8J7I z7LzAfNh0tH45tnsZSVSbyE3NFFoz/xqmSbAOTqG6pSzYmkhhztILxsLJolIt/xA6HDD1o6JJopYfT19H/1XrK1aeF5XwlZPECHj/rwM09L7+XvTI/ZN67Bk6/cj9wetPR8VS5Fdt0GSsjQAWAKJlt81gmGHXvfnZ3TfTrqEWkYJFXFYAifuo+oV3C6khvWw/3Qki4b6+nKIWH08c5mi5hC0Cqp5tkCX4yCP0wDpRAJbiHJDcXqhXb3IRsBl5ETjEV8zJJPECy7+7eUWMv2Vidg7Uvg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 22, 2023 at 12:52=E2=80=AFPM Alexander Graf w= rote: > > With KHO in place, let's add documentation that describes what it is and > how to use it. > > Signed-off-by: Alexander Graf > --- > Documentation/kho/concepts.rst | 88 ++++++++++++++++++++++++++++++++ > Documentation/kho/index.rst | 19 +++++++ > Documentation/kho/usage.rst | 57 +++++++++++++++++++++ > Documentation/subsystem-apis.rst | 1 + > 4 files changed, 165 insertions(+) > create mode 100644 Documentation/kho/concepts.rst > create mode 100644 Documentation/kho/index.rst > create mode 100644 Documentation/kho/usage.rst > > diff --git a/Documentation/kho/concepts.rst b/Documentation/kho/concepts.= rst > new file mode 100644 > index 000000000000..8e4fe8c57865 > --- /dev/null > +++ b/Documentation/kho/concepts.rst > @@ -0,0 +1,88 @@ > +.. SPDX-License-Identifier: GPL-2.0-or-later > + > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Kexec Handover Concepts > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +Kexec HandOver (KHO) is a mechanism that allows Linux to preserve state = - > +arbitrary properties as well as memory locations - across kexec. > + > +It introduces multiple concepts: > + > +KHO Device Tree > +--------------- > + > +Every KHO kexec carries a KHO specific flattened device tree blob that > +describes the state of the system. Device drivers can register to KHO to > +serialize their state before kexec. After KHO, device drivers can read > +the device tree and extract previous state. How does this work with kexec when there is also the FDT for the h/w? The h/w FDT has a /chosen property pointing to this FDT blob? > + > +KHO only uses the fdt container format and libfdt library, but does not > +adhere to the same property semantics that normal device trees do: Prope= rties > +are passed in native endianness and standardized properties like ``regs`= ` and > +``ranges`` do not exist, hence there are no ``#...-cells`` properties. I think native endianness is asking for trouble. libfdt would need different swap functions here than elsewhere in the kernel for example which wouldn't even work. So you are just crossing your fingers that you aren't using any libfdt functions that swap. And when I sync dtc/libfdt and that changes, I might break you. Also, if you want to dump the FDT and do a dtc DTB->DTS pass, it is not going to be too readable given that outputs swapped 32-bit values for anything that's a 4 byte multiple. > + > +KHO introduces a new concept to its device tree: ``mem`` properties. A > +``mem`` property can inside any subnode in the device tree. When present= , > +it contains an array of physical memory ranges that the new kernel must = mark > +as reserved on boot. It is recommended, but not required, to make these = ranges > +as physically contiguous as possible to reduce the number of array eleme= nts :: > + > + struct kho_mem { > + __u64 addr; > + __u64 len; > + }; > + > +After boot, drivers can call the kho subsystem to transfer ownership of = memory > +that was reserved via a ``mem`` property to themselves to continue using= memory > +from the previous execution. > + > +The KHO device tree follows the in-Linux schema requirements. Any elemen= t in > +the device tree is documented via device tree schema yamls that explain = what > +data gets transferred. If this is all separate, then I think the schemas should be too. And then from my (DT maintainer) perspective, you can do whatever you want here (like FIT images). The dtschema tools are pretty much only geared for "normal" DTs. A couple of problems come to mind. You can't exclude or change standard properties. The decoding of the DTB to run validation assumes big endian. We could probably split things up a bit, but you may be better off just using jsonschema directly. I'm not even sure running validation here would that valuable. You have 1 source of code generating the DT and 1 consumer. Yes, there's different kernel versions to deal with, but it's not 100s of people creating 1000s of DTs with 100s of nodes. You might look at the netlink stuff which is using its own yaml syntax to generate code and jsonschema is used to validate the yaml. Rob