From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24F2CC3ABBC for ; Fri, 9 May 2025 07:47:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16C556B00C1; Fri, 9 May 2025 03:47:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A3A66B00C2; Fri, 9 May 2025 03:47:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D750C6B00C3; Fri, 9 May 2025 03:47:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AF9A06B00C1 for ; Fri, 9 May 2025 03:47:08 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id F29D616129F for ; Fri, 9 May 2025 07:47:09 +0000 (UTC) X-FDA: 83422588578.16.8CAA9FB Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf21.hostedemail.com (Postfix) with ESMTP id 3595A1C0003 for ; Fri, 9 May 2025 07:47:07 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZsnbJoO1; spf=pass (imf21.hostedemail.com: domain of 3-rIdaAoKCB87C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3-rIdaAoKCB87C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746776828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rJPeIElnNt/I3/MaK5uh0cldn9yJJ+y5OnW6ddziJAo=; b=MRpFpP0glCXl4Uy6YvfJjJogAHfrZxzBaMchGVRWUM+/suL3Q8Qxvbs6R9h1gpBgOs0QXO eCp76XM/F+sXdFEHvuAy6LVua3nph+W1rIxXFvkto3V7/gjF/AHQ4nuJ6/WiOSoqM6JHeK 8QWl8xMBEJ24pZoJCurfbiLoiq4czQ8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZsnbJoO1; spf=pass (imf21.hostedemail.com: domain of 3-rIdaAoKCB87C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3-rIdaAoKCB87C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746776828; a=rsa-sha256; cv=none; b=0iG/0F+PNDwgaQrYs6qts9N6oEGtPaATHALwkRkoVyK8Vj7Gawd0YBhRIbNoFeU8zo9u77 dDMpZJdp6ZSk0YEL5bcWqGs+XVR6/rAnLwGFvVNF5Y8e9+VWFyfo6k1EMxs3vCJsdqbivO nx03bTs2FSZ+t1vQ3/B5V6jzoabI0aY= Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7394772635dso1514737b3a.0 for ; Fri, 09 May 2025 00:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746776827; x=1747381627; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rJPeIElnNt/I3/MaK5uh0cldn9yJJ+y5OnW6ddziJAo=; b=ZsnbJoO1T6bKJruSQrrGrBm/qpWf552C/ca+y8gXuuTtjzpFQ7eOCANaDBKYnlWUoN AN8+RR//BolTPY063WR99yrj8eVdVcqlkaxaE7U9PmWJgXsM4DF4gwc5SNpmg7cHsIqy KNks2rG+S5wLYYv5tP6+dHJTp5YwP84UDPJiAg4IMY14j5IzWnfZpLeQ6zoD0vAwmb09 UxT4VrX0Ki0yFAlQXRZc62xVXcrQJoYfff5cctqJj+o8uflutiSQFwkqbVFPrultnMBG vX6THNbeYRteb+u3Oi4DpBgDuJc1nUSeN3WM4PJuLhhCQn5KyfVIGyRIo2ntn1iu1hse oN+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746776827; x=1747381627; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rJPeIElnNt/I3/MaK5uh0cldn9yJJ+y5OnW6ddziJAo=; b=rWrmU7MzNakzcEeFEBlLpzRR2tZ8JhLNPuxW2z5Wh4U5lpyqT4uuws1TFTg9KjBdfX Jd1OKygvUkINkDT7JzLzPfcXTrOebRPxdDJwSOGW1X9p+Smm76fZxwECF6zsLdD+Wm+3 vOcQaWm8U9ZNvI5ta2ogCByOmmWPDCJ0Eh+XR6vvGy7jjUs0/f+LRSf8J3V0Mn/WXW28 9OGZEYSDxatWmFbU56rCjYVE/GHFqc3xcDGffPuJE62B25MliUCY9noZ8TxEfZE3T7Ky FfTSskKO2fr6XgaN+8uNTFRHfsgjrBHfhqDBYY8pASq4dJy5dMBN79iXDqtfU16LAaTO RuvA== X-Forwarded-Encrypted: i=1; AJvYcCW7Za2DcrrZS5KKtPT4KWt5XrlbZODSYfSNI+4eqQPowV/8PELre3K0TnSs8C5IH6E1fcT53wUQMQ==@kvack.org X-Gm-Message-State: AOJu0Ywtp9+IYlz4lgbITilOQYfp7LWG7G2AkpAHrqcgkMUK1nyBYihu Y0cfrfjnPqVzRuCW7g7fF3zmFCMkNu0bTJjC5oHFeBpMhRsmMY9YQX+6+UPQjgk4pEo6fIDP/vU xgpIrpRG3H/FO+PlaDg== X-Google-Smtp-Source: AGHT+IEK1/WlmP38MYP4hEes8GqKaqWx3jiFruF23mNJt24+iXk5dMi5uOZdu7gymuTIRK+YGmcQLoBz9uRHRSQG X-Received: from pga10.prod.google.com ([2002:a05:6a02:4f8a:b0:b20:c9:a993]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:1582:b0:1f5:79c4:5da0 with SMTP id adf61e73a8af0-215abc16c45mr3639959637.31.1746776826867; Fri, 09 May 2025 00:47:06 -0700 (PDT) Date: Fri, 9 May 2025 00:46:25 -0700 In-Reply-To: <20250509074635.3187114-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250509074635.3187114-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250509074635.3187114-8-changyuanl@google.com> Subject: [PATCH v8 07/17] kexec: add KHO support to kexec file loads From: Changyuan Lyu To: akpm@linux-foundation.org, linux-kernel@vger.kernel.org Cc: anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, corbet@lwn.net, dave.hansen@linux.intel.com, devicetree@vger.kernel.org, dwmw2@infradead.org, ebiederm@xmission.com, graf@amazon.com, hpa@zytor.com, jgowans@amazon.com, kexec@lists.infradead.org, krzk@kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, luto@kernel.org, mark.rutland@arm.com, mingo@redhat.com, pasha.tatashin@soleen.com, pbonzini@redhat.com, peterz@infradead.org, ptyadav@amazon.de, robh@kernel.org, rostedt@goodmis.org, rppt@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, tglx@linutronix.de, thomas.lendacky@amd.com, will@kernel.org, x86@kernel.org, Changyuan Lyu Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3595A1C0003 X-Stat-Signature: i8f3k7xasbt55gyci4xm3yt6bwfow417 X-HE-Tag: 1746776827-153924 X-HE-Meta: U2FsdGVkX1/18nOB77G7oCa8QShlvnL3V4fdAZ9XQhAvyAO6s0V3yxNEYs9mhPnqP4wWuyaN4hbCaFWxQ8PDlr5dfRfNCF33fpqlWmLaIg8jz3KM9fDfS0LG4Kbp62czqonf42T0I/lcJKlGf2fK2w2v6NXbu5vVOKFUDTBGmr+8nnsaE69qX0l0RU6eTbHp+HrXc//7sOMVvk/ZZjR4okhc9Ap79zW6zFd3II4l8S84uuee48dNc19pr/VRE+YYQgR/RlHaQUTtmsJaR9+X6oZGot1aTdAz0VBu8stzOGcK1LR+UL4NE8M67ZvxSZPiZlfU3O2+ULJMamOHei4aA0IRC9CGwL45MkD+H8je5zHy90ZVXZwlSv/tLk028rFcb7IbHTCMdH8mjJr2D4GpeKpysgmr7pbn5amIfJGcXF3Q158DOGF76YgAMoj6QspCJjcVVSdVSCqMmPExFH1EoeAK7CTCuJhx7CqvJMrk0G6VaPryHnio56MjOyPOYUAIt7TtOlkxRvzHH7GRCT6EenQiQ+DfH/6O3d/zK9UrGdGWr//BTgrXYXt3WrbRElR8ZLN5HzFL+W3NTZjloW+jPwm7OZdLsXqd+7KtcZywZHOJdgFtwgmqZqWOjQcvhkRi1VGTRSlIfOABgajxL7K9aSSVOc5w3W/OfpF8+pKRpJz00BHE50nJ8BsL5hBqrH6pFs9dHY8cHUXdBfO4Hst/hSk/W2gYuS8Zl4G9yY2+DBfjJGTd4pKPQcxDK71xxQ+Qj/GOukyNFr5BfCpCZNyJK9TNPn6TvwnnO4AFKtJUqLK3KCUOH2e63qxzCh4TgdYlM9fjM71cpvql6sTP9V9vfsXqI64xlri96x9pX+ZqZcaNfPs7KIuTcscEHrfGVT0YsZO1CFX81IWa1h8M80HmapzFbwwUDRcCX6l2jiQ1OHcMTPGQbDKUWJ6+Fqw5wXHedI8JJXrPf/R9xxJJHkz S82rL3qt WqIE+L/d2bHCijRDQkegAJpqpxeoXwqT3/q3lglKCw1oA9/HJAFRFgOQULVahy1SwNS+A7KdUXOxrKQUK0osu5sspaiAUAROO5efx5RjCDkn2Cj6z3HL8HTt2WLMEfPkdFbLf0Q1Vx7aq5+V2sEfylH8DdTM0QOmH5Xpdcr5l2e3cvXd4Mo/CoApqSZq2WAoh3qo2d6h/iUGnFi5d6xW4Ca2TwNUjwN3Z1IaVGah9ozY8N7kuj6gFdIAt2766hoj5Um8c+B/jx2Z/6dE9H9qhlKHGjolW9b9Ds7H9/dMJMjyPQcjiWNR1+e9YuoAWEkTc5rDzLp3IHKCK9uZJepRXRkEbPyxAr7XN0w/jctfCGPrNLcw/RQUme4Qsj9Vkl9AMuZsH0PrN/1cgznpiszK4918oH9SGsqMwrm3v4X31kZIDD+oIOCpLhtLzLTObOXDIq5NEyK1pihqFmBlgTY0GCu716ktVd/f6RLepHpsELv2ZNWY3D8HsybjvRGv4uyC9gjTPUk2dLMvYlGrb0nTnXABOvBSy3UeOaMf8M9peXG3S1rAMgtTMCfWaGtgSi10aizrEpalJ9sQeuONsU1U60mnUq9oJ2fYV+4Wz+0ALiBX6aja9XQEfYhlNTw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf Kexec has 2 modes: A user space driven mode and a kernel driven mode. For the kernel driven mode, kernel code determines the physical addresses of all target buffers that the payload gets copied into. With KHO, we can only safely copy payloads into the "scratch area". Teach the kexec file loader about it, so it only allocates for that area. In addition, enlighten it with support to ask the KHO subsystem for its respective payloads to copy into target memory. Also teach the KHO subsystem how to fill the images for file loads. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec.h | 5 +++ kernel/kexec_file.c | 13 ++++++++ kernel/kexec_handover.c | 67 +++++++++++++++++++++++++++++++++++++++++ kernel/kexec_internal.h | 16 ++++++++++ 4 files changed, 101 insertions(+) diff --git a/include/linux/kexec.h b/include/linux/kexec.h index c8971861521a5..075255de81543 100644 --- a/include/linux/kexec.h +++ b/include/linux/kexec.h @@ -371,6 +371,11 @@ struct kimage { size_t ima_buffer_size; #endif + struct { + struct kexec_segment *scratch; + phys_addr_t fdt; + } kho; + /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_sz; diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index fba686487e3b5..77758c5331229 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -253,6 +253,11 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd, /* IMA needs to pass the measurement list to the next kernel. */ ima_add_kexec_buffer(image); + /* If KHO is active, add its images to the list */ + ret = kho_fill_kimage(image); + if (ret) + goto out; + /* Call image load handler */ ldata = kexec_image_load_default(image); @@ -648,6 +653,14 @@ int kexec_locate_mem_hole(struct kexec_buf *kbuf) if (kbuf->mem != KEXEC_BUF_MEM_UNKNOWN) return 0; + /* + * If KHO is active, only use KHO scratch memory. All other memory + * could potentially be handed over. + */ + ret = kho_locate_mem_hole(kbuf, locate_mem_hole_callback); + if (ret <= 0) + return ret; + if (!IS_ENABLED(CONFIG_ARCH_KEEP_MEMBLOCK)) ret = kexec_walk_resources(kbuf, locate_mem_hole_callback); else diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 9cc818cefd151..69b953551677c 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -26,6 +26,7 @@ * internal APIs. */ #include "../mm/internal.h" +#include "kexec_internal.h" #define KHO_FDT_COMPATIBLE "kho-v1" #define PROP_PRESERVED_MEMORY_MAP "preserved-memory-map" @@ -1197,3 +1198,69 @@ void __init kho_populate(phys_addr_t fdt_phys, u64 fdt_len, if (err) pr_warn("disabling KHO revival: %d\n", err); } + +/* Helper functions for kexec_file_load */ + +int kho_fill_kimage(struct kimage *image) +{ + ssize_t scratch_size; + int err = 0; + struct kexec_buf scratch; + + if (!kho_enable) + return 0; + + image->kho.fdt = page_to_phys(kho_out.ser.fdt); + + scratch_size = sizeof(*kho_scratch) * kho_scratch_cnt; + scratch = (struct kexec_buf){ + .image = image, + .buffer = kho_scratch, + .bufsz = scratch_size, + .mem = KEXEC_BUF_MEM_UNKNOWN, + .memsz = scratch_size, + .buf_align = SZ_64K, /* Makes it easier to map */ + .buf_max = ULONG_MAX, + .top_down = true, + }; + err = kexec_add_buffer(&scratch); + if (err) + return err; + image->kho.scratch = &image->segment[image->nr_segments - 1]; + + return 0; +} + +static int kho_walk_scratch(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret = 0; + int i; + + for (i = 0; i < kho_scratch_cnt; i++) { + struct resource res = { + .start = kho_scratch[i].addr, + .end = kho_scratch[i].addr + kho_scratch[i].size - 1, + }; + + /* Try to fit the kimage into our KHO scratch region */ + ret = func(&res, kbuf); + if (ret) + break; + } + + return ret; +} + +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret; + + if (!kho_enable || kbuf->image->type == KEXEC_TYPE_CRASH) + return 1; + + ret = kho_walk_scratch(kbuf, func); + + return ret == 1 ? 0 : -EADDRNOTAVAIL; +} diff --git a/kernel/kexec_internal.h b/kernel/kexec_internal.h index d35d9792402d1..30a733a55a67b 100644 --- a/kernel/kexec_internal.h +++ b/kernel/kexec_internal.h @@ -39,4 +39,20 @@ extern size_t kexec_purgatory_size; #else /* CONFIG_KEXEC_FILE */ static inline void kimage_file_post_load_cleanup(struct kimage *image) { } #endif /* CONFIG_KEXEC_FILE */ + +struct kexec_buf; + +#ifdef CONFIG_KEXEC_HANDOVER +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)); +int kho_fill_kimage(struct kimage *image); +#else +static inline int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + return 1; +} + +static inline int kho_fill_kimage(struct kimage *image) { return 0; } +#endif /* CONFIG_KEXEC_HANDOVER */ #endif /* LINUX_KEXEC_INTERNAL_H */ -- 2.49.0.1015.ga840276032-goog