From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A496C4828D for ; Tue, 6 Feb 2024 14:40:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 99E766B0078; Tue, 6 Feb 2024 09:40:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 950106B007B; Tue, 6 Feb 2024 09:40:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 816766B007D; Tue, 6 Feb 2024 09:40:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6B23B6B0078 for ; Tue, 6 Feb 2024 09:40:33 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3BC1E14063A for ; Tue, 6 Feb 2024 14:40:33 +0000 (UTC) X-FDA: 81761639946.29.30CDCC6 Received: from metis.whiteo.stw.pengutronix.de (metis.whiteo.stw.pengutronix.de [185.203.201.7]) by imf25.hostedemail.com (Postfix) with ESMTP id EBD9CA0016 for ; Tue, 6 Feb 2024 14:40:29 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of ore@pengutronix.de designates 185.203.201.7 as permitted sender) smtp.mailfrom=ore@pengutronix.de; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707230430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rCDJSTSzYANm5ykSW4ll7DMglD5TcS+mhDZp6YQRzjk=; b=35rfYfY9kP+QfRDTfMMPFJaufBAAFLlBIQtamWnMTIhf2bOBZhR93aV0k0dIsafRkF2thw mUMuirxkqpP+5nFXUIYrV4lZzAxTo+kxO9DptaNrELKWu3lD65p95k3ZdW8uEAFday1cIW 4b+9kN2CRXsupYbP4Sp1Q1vqpH5F2Sw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707230430; a=rsa-sha256; cv=none; b=vcvKkTsjnhGKbyGIM68RDG16KMNqrSHkLhT6bPhnp88orWEZcI1jCPuFU0VXNQr/8hYbkM c/9Qw5GXSwebDLaAGojXecGl9fyUX0YtbnvY1IEJa8oXuoxHAk+E6UNIoWb2c7FaRhxcaE Lw5KiUtCEaqrtHfpsRH2SyThsZOjLsY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of ore@pengutronix.de designates 185.203.201.7 as permitted sender) smtp.mailfrom=ore@pengutronix.de; dmarc=none Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2]) by metis.whiteo.stw.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rXMcd-0003xc-Ql; Tue, 06 Feb 2024 15:40:27 +0100 Received: from [2a0a:edc0:2:b01:1d::c5] (helo=pty.whiteo.stw.pengutronix.de) by drehscheibe.grey.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rXMcY-004qRv-6e; Tue, 06 Feb 2024 15:40:22 +0100 Received: from ore by pty.whiteo.stw.pengutronix.de with local (Exim 4.96) (envelope-from ) id 1rXMcY-00EFqp-0E; Tue, 06 Feb 2024 15:40:22 +0100 Date: Tue, 6 Feb 2024 15:40:22 +0100 From: Oleksij Rempel To: Alexander Graf Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kexec@lists.infradead.org, linux-doc@vger.kernel.org, x86@kernel.org, Eric Biederman , "H . Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Steven Rostedt , Andrew Morton , Mark Rutland , Tom Lendacky , Ashish Kalra , James Gowans , Stanislav Kinsburskii , arnd@arndb.de, pbonzini@redhat.com, madvenka@linux.microsoft.com, Anthony Yznaga , Usama Arif , David Woodhouse , Benjamin Herrenschmidt , Rob Herring , Krzysztof Kozlowski Subject: Re: [PATCH v3 00/17] kexec: Allow preservation of ftrace buffers Message-ID: References: <20240117144704.602-1-graf@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Sent-From: Pengutronix Hildesheim X-URL: http://www.pengutronix.de/ X-Accept-Language: de,en X-Accept-Content-Type: text/plain X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2 X-SA-Exim-Mail-From: ore@pengutronix.de X-SA-Exim-Scanned: No (on metis.whiteo.stw.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-mm@kvack.org X-Stat-Signature: gudt51obbmxyrtwt53a7mktjprcjo4i6 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: EBD9CA0016 X-Rspam-User: X-HE-Tag: 1707230429-377697 X-HE-Meta: U2FsdGVkX19fMzpbYlxo4yuWc61AehzsRQPdZo4F9Ms2LGhb6V2juyzrVPr3TbZgDI9jTfEGy6GzPon8vc1CimYJ/M6KYLgARFMm4tUEYhE/3n+EUEv8ZAtAB4HcOue+MQfb6knR2Bhi/BL8+1YQU3c/v5hLyNJ2HH4dkKhrCtCcD2k4Ep9km65oM97BmeSvS/aQsET5HIND4qy0NSO+Dmz63GpbMQDhOcTrL/K3myXalMNxxJadcCmUeT9X4RSCG0jTRDqvC/A1m5eE7TxWePk3dcx3OjLP58r5ZoAOXFg9r0F7ZOVHrDM8U207iSM52ROcRqC8tmdyvCFbf2MHcmgtfkxpJognVuvyorHkDy8LyAOTjoxdi/nHRfjeKYHlhJSR8bDB7UUeuoZvnAVQog2jF9J8keI90xQM6J+HpB00Lv6WPtgwPuWNegSbqg1dnhAklbmn+8V9ndvX9UpejYXTvIRalCL8rZYa3ox0NNJ9R5aF5W9eDEMoRTKK5HbGIwug4z+ls+N/pz49wdgZloqPm7957wlq9M5J1NxIQliUYlNyAQeb/B9wnNHw8LPY19q5KbwPlBld4QL+Vlw4Qu7Zs+2mfTGz4VN5KvsLzC9afmcvGWlrIoa+C5PzzFCiqtjtne8z5vx4eNSWAZ3WD5dhtX4nSu1kfVzXR0bMIjtcBvolYzMDajp6MsYsslPfw89BVvU5ofIHApJqpd76rhXViuEZQkbNWgPSHv9zBfTDgT3sVbnDvikjE+MeOL3O0dkuyB3xla+E34u7/+8orEgn+pVQ0o/up3dmiTz9psbVTAG79iO2vvvEvuvaxCCFjPEKdK7vTah6+0DDNHffMRmciljcnqeiRieqBz7yCFFaBu+3EdcNm80yvXRh/whsZTlT3Ylsfnst3W3Es5fpKFXc0nzA5I/cffzQ8OQODub5mAdQtmsdOGNtb/N3madG5Yllb5cOt2x3UkqQvyO LK1IRZd0 3YZet/BpXiFIOkR6WrMnjaNweAwnOh02W6+/1TpVHXbfGyq0ROWCVl3qgHquywjdIrY+LiTHlTL76bP1FqRr9CvOXyxrCYHOVxhN8PCEHgJ9DvGbxCa9gPNgzyzbkPGZLvap7c99bn+/wRLtdtw9Q69QaHkls/gRu1JWXV2r5MlPsAKSxskO3QotzjSkw+eAoTiuoLXTICwIekQJDj6Y5OvZ6et+BsQRWKlkDoEiLWDlBlF8zjs9galiCxd9H8NrmCSXAAgbY4xTl3WQVAs3BP8/dQ3nYWB8pBeShsauE7XKhU7SMReATJzyylwDHl/pNKBY+gDqreFmwXUqeorfAsL5swZXoSuYHoHbm8/4z5nLA+qbj9n+BH1XjYlUOnkqQX67XN9MM8Oo+jRCabrpGMvSsYK6MlhNfLk28fH19BgU0Ud55yDdqFmMAHUQK9KpURuS24VWqakwWUhqNORg7/7fnVdr7qfs+yewJyYqC11SAAtdYN7jYniLaKQdZcVs3l6iV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 06, 2024 at 02:43:15PM +0100, Alexander Graf wrote: > Hey Oleksij! > > On 06.02.24 09:17, Oleksij Rempel wrote: > > Hi Alexander, > > > > Nice work! > > > > On Wed, Jan 17, 2024 at 02:46:47PM +0000, Alexander Graf wrote: > > > Make sure to fill ftrace with contents that you want to observe after > > > kexec. Then, before you invoke file based "kexec -l", activate KHO: > > > > > > # echo 1 > /sys/kernel/kho/active > > > # kexec -l Image --initrd=initrd -s > > > # kexec -e > > > > > > The new kernel will boot up and contain the previous kernel's trace > > > buffers in /sys/kernel/debug/tracing/trace. > > Assuming: > > - we wont to start tracing as early as possible, before rootfs > > or initrd would be able to configure it. > > - traces are stored on a different device, not RAM. For example NVMEM. > > - Location of NVMEM is different for different board types, but > > bootloader is able to give the right configuration to the kernel. > > > Let me try to really understand what you're tracing here. Are we talking > about exposing boot loader traces into Linux [1]? In that case, I think a > mechanism like [2] is what you're looking for. > > Or do you want to transfer genuine Linux ftrace traces? In that case, why > would you want to store them outside of RAM? The high level object of what i need is to find how embedded systems in fields do break. Since this devices should be always on, there are different situations where system may reboot. For example, voltage related issues, temperature, scheduled system updates, HW or SW errors. To get better understand on what is going on, information should be collected. But there are some limitations: - voltage drops can be recorder only with prepared HW: https://www.spinics.net/lists/devicetree/msg644030.html - In case of voltage drops RAM or block devices can't be used. Instead, some variant of NVMEM should be used. In my case, NVMEM has 8 bits of storage :) So, only one entry of the "trace" is compressed to this storage. https://lore.kernel.org/all/20240124122204.730370-1-o.rempel@pengutronix.de The reset reason information is provide by kernel and used by firmware and kernel on next reboot The implementation is not a big deal. The problematic part is the way how the system should get information about existence of recorder and where the recorder should stored things, for example NVMEM cell. In my initial implementation I used devicetree to configure the software based recorder and linked it with NVMEM cell. But it is against the DT purpose to describe only HW and it makes this recorder unusable for not DT basd systems. Krzysztof is suggesting to configure it from initrd. This has own limitations as well: - record can't be used before initrd. - we have multiple configuration point of board specific information - firmware (bootloader) and initrd. - initrd take place and reduce boot time for device which do not needed it before. Other variants like kernel command-line and/or module parameters seems to be not acceptable depending maintainer. So, I'm still seeking proper, acceptable, portable way to hand over not HW specific information to the kernel. > > What would be the best, acceptable for mainline, way to provide this > > kind of configuration? At least part of this information do not > > describes devices or device states, this would not fit in to devicetree > > universe. Amount of possible information would not fit in to bootconfig > > too. > > > We have precedence for configuration in device tree: You can use device tree > to describe partitions on a NAND device, you can use it to specify MAC > address overrides of devices attached to USB, etc etc. At the end of the day > when people say they don't want configuration in device tree, what they mean > is that device tree should be a hand over data structure from firmware to > kernel, not from OS integrator to kernel :). If your firmware is the place > that knows about offsets and you need to pass those offsets, IMHO DT is a > good fit. Yes, the layout of the NVMEM can be described in the DT. How can I tell the system that this NVMEM cell should be used by some recorder or tracer? Before sysfs is available any how. @Krzysztof ? > > Other more or less overlapping use case I have in mind is a netbootable > > embedded system with a requirement to boot as fast as possible. Since > > bootloader already established a link and got all needed ip > > configuration, it would be able to hand over etherent controller and ip > > configuration states. Wille be the KHO the way to go for this use case? > > > That's an interesting one too. I would lean towards "try with normal device > tree first" here as well. It's again a very clear case of "firmware wants to > tell OS about things it knows, but the OS doesn't know" to me. That means > device tree should be fine to describe it. I can imagine description of PHY and MAC state. But IP configuration state of the firmware seems to be out of DT scope? Regards, Oleksij -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |