From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 329C7C02188 for ; Mon, 27 Jan 2025 20:02:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB0A62801A2; Mon, 27 Jan 2025 15:02:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B60FC280191; Mon, 27 Jan 2025 15:02:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4FCC2801A2; Mon, 27 Jan 2025 15:02:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 839AF280191 for ; Mon, 27 Jan 2025 15:02:01 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 458311A0665 for ; Mon, 27 Jan 2025 20:02:01 +0000 (UTC) X-FDA: 83054302842.05.E83ADC6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id 7945B40007 for ; Mon, 27 Jan 2025 20:01:59 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of "SRS0=3h4l=UT=goodmis.org=rostedt@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=3h4l=UT=goodmis.org=rostedt@kernel.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738008119; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=CwgG7Gi6IvnlXhUzgjFu4FqgJyv+uaY91MujByC93iw=; b=h1ENxnHteShOO9sHttoO95oBuTm4ZDYr8SLFhL6X/h7O7jALZpZrDhu99setr5Drog60x8 xLfp4XTr/9bsI1SAcxmal3sK68hm4tDBzivOUSnmKPBjNHnxBg/8lrGjR7Zq/4UIdO1HRM LPlDV4TbqaoLaXbP5g5eaIiPBcVZtqQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of "SRS0=3h4l=UT=goodmis.org=rostedt@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=3h4l=UT=goodmis.org=rostedt@kernel.org"; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738008119; a=rsa-sha256; cv=none; b=tag2Pc+k+cgPv2OUmFx5E3/nlnZ8ysD3vWQcrJZDV+3AfSXEUinWOwd9fegkjEq65nUVMm Y8kGoIMZU0j3XzljBAt2IjrX7EVJbFb9T1PCrahmN3cPviRCfeejGJVpMQERseFJYme68W 6Levk8Lxh5gQsB6RIE8a4dJEAvrtJOc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 44B975C54F4; Mon, 27 Jan 2025 20:01:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8503DC4CED2; Mon, 27 Jan 2025 20:01:57 +0000 (UTC) Date: Mon, 27 Jan 2025 15:02:14 -0500 From: Steven Rostedt To: lsf-pc@lists.linux-foundation.org, Mike Rapoport , Alexander Graf Cc: linux-mm@kvack.org Subject: [LSF/MM/BPF TOPIC] persistent memory, tracing, KASLR, Oh my! Message-ID: <20250127150214.4d715cad@gandalf.local.home> X-Mailer: Claws Mail 3.20.0git84 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 7945B40007 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 18x95gcmaidkq5qpbiqpcng77636augj X-HE-Tag: 1738008119-207176 X-HE-Meta: U2FsdGVkX1+EhLoWH82qqwlbo6Mjtt5EfKIfB/hr8DUf4WFtUuBlYiskwVGByzYII+NT5zM0HxJiEwASdVPLmxNN3j3QAea4Z4sa4k/FBMg8wNDTRw+u3n/Yu3k3VjiVlKG+e2PtWVEKLakHW+29xHH8q4H9rbKJs2rO1sds5jEMefkMfavXoHfZ2i3NRAqQsjJF2grHXq2om07ccBM98ozoqsfqy++Vte4KWC/fWRJD0ir0Ht33FyRLYbubVdQCikAhtWtWikTR3Ow76BzzESyVGPGM0j8eqgAsoi0NJwmc3JvtqVdOSiifwTg+lfH2iYaI/c5nKLtWsHxmQJJ8Rrs5jxlH73I9/UNHZ057KkzdlEmlBFFDR+Drqycu+STXvGQ4hk7opKvGoNZTMDv8JLoRE8BqQdR51Vkadza1UeYmql8pgO5Y+ZtN1apc9D8+Jc3FSjFGtrwbF65ON8Q7DBP0LM3UtNEHVc0qna3a7lRz7HDmUv+dM3sqshLSI0e2kOkStVLp/i5QG3z6uvsSQwIuvCzcI+/0NC+M1TZkH58XV0IA4OuAPv/pt8Pi46JMunWg1Jw9a9wq2M1wHyT8pSWPPsHcQhVqR/zbIAinWgMwDzt5RBvv5Tshg5yFF7l9nxUwz4iavOHuvf7U0mJKvlFWbbnG/SvKMzt42OJ9rneQUL9bZmDsoTmyGGBTXXxlSBSd5K184Mqdd3AbA0XOwTC3zmZUnk6umBqNa3Zt3jQ2YXtVv5+gukqinoIJ8MGWBvG/1uzV5LjFoFCzCS3DmPvi2y8ZyywsmvPVAdp8DSwYeofDI5/iedgoLywnjtgzcF8OFjunVZQckFOZTFkgyYvq1clcFWjW01UNEhBVsN12tSel1qUEPUxRqWY5vKIsLGBLoe0N1QbMnDK7IAHc6pSEz++jWJqig5hPs/SYQ50FZ2wKpykK7KOiGJy+X1TW7bMeGKAvmrSMm7r0IZX 4RcPaZ4C ZRhNpgcBOpi1lvl6/tM2TLF/psfYSrvO1jQlI9iXQaArJENdj96L70oJ1AuAlNSC4ctlpiUabNEnYXSQcl7gAoIldSK5L8uzuvNPkURgaHrBkqTFHXOlNRZMui8gW1qfmQAX0BQ2QJzCQqS2SLyMVy40VKdFuSNG0WLr7Sg+Hp8QD8ggSBfDCSvLFMvO5W22p5mxTV8Ziid92Z4CzDxcZHK8h/a9HNG7ezCU4q/+ZDjZ5i/zpdJRIdHbnpr3iyYGQ3nsRUV+sk0WtNcaWT0ijBqW9yLeq4IoXAcb5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is different than Mike's "memory persistence over kexec" topic which has to do with Kexec Hand Over (KHO). This is about tracing and generic pstore itself. Here's the problem statement: To enable tracing over persistent memory in the field we add to the kernel command line: reserve_mem=20M:12M:trace trace_instance=boot_mapped^traceoff@trace The "reserve_mem=20M:12M:trace" will reserve 20 megabytes of memory that is aligned by 12 megabytes and label it "trace". The "trace_instance=boot_mapped^traceoff@trace" will look for memory from the "reserve_mem" that has the label "trace", assign it to the boot_mapped instance (found in /sys/kernel/tracing/instances/boot_mapped) and it will be have tracing disabled by default on boot. (Note that pstore can also get its persistent memory from the "reserve_mem" option as well). On boot up, various events will be started in the "boot_mapped" trace instance. If there's a crash, on the next boot, the "boot_mapped" trace instance will be read which will contain all the events that led up to the crash (this can also trace reboots too). Obviously this depends on the machine not clearing memory and only doing a soft reset where the memory is still persistent from the last boot. This also only works if the "reserve_mem=20M:12M:trace" reserves the memory in the same location as it did from the previous boot. Hence the problem I have here. Namely KASLR. If I disable KASLR (nokaslr on the kernel command line) then I found that the memory reserved is consistent between boots and I have not hit any issues while debugging in the lab, or on my own devices, as I can safely add "nokaslr" on test machines. But for productions machines and for machines in the field, that is not an option. KASLR can move the placement of the kernel where it overlaps the previous saved buffer, or what I found is even more common, it causes something that was reserved before the "reserve_mem" was called to be allocated in a different location which caused the memory reserved by reserve_mem to move. I also found (at least on a few machines I played with), adding a large alignment (like 12M) helps with that from happening. That is, things that were allocated before "reserve_mem" was parsed can move around and the memory reserved stays consistent. That helps, but does not prevent it. My question: Would it be possible to have some communication between KASLR and the kernel to know about this reserved memory and work to keep it from moving? Since movement of allocations that happen before "reserve_mem" is parsed can affect where the reserved memory is allocated, I'm not sure just keeping KASLR away from the reserved memory is enough. Perhaps, after a successful allocation, there can be some memory range that can communicate back to KASLR on the next boot to actually reserve that memory location again, and then communicate back to the booting kernel that this memory was reserved there and the "reserve_mem" will just used that memory? Really, I'm looking for any ideas that can help keep "reserve_mem" allocating memory at the same location after each boot. Thoughts? -- Steve