From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA176C7EE2A for ; Wed, 25 Jun 2025 23:19:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B62C6B00A0; Wed, 25 Jun 2025 19:18:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 916906B00A1; Wed, 25 Jun 2025 19:18:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DECF6B00BE; Wed, 25 Jun 2025 19:18:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6DF3E6B00A0 for ; Wed, 25 Jun 2025 19:18:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4D2E1102B1B for ; Wed, 25 Jun 2025 23:18:58 +0000 (UTC) X-FDA: 83595490356.14.C7A85CD Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by imf14.hostedemail.com (Postfix) with ESMTP id 64FE710000B for ; Wed, 25 Jun 2025 23:18:56 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=r5nqSpaL; spf=pass (imf14.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750893536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SZDu7RJqPXAY2ruR1j/0614GYveU9RxRfC4eQg5qP6Q=; b=PoIkvxk5RbZ9CfVfyLfCa7XghHZfwoW1QWh5D4akFjblkJojdRUihRRWth2kpXKg0wQ7AT QxH7Bkm6NvuPhPQzBfHmod9X2jPAyOfJpRCj0P8SOaVy/UpmqY6SevPlazq+CDz1v7AhmG bqVOHdt9BcgOP+n8IswiuHMgI2DBxKY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750893536; a=rsa-sha256; cv=none; b=yw0ljpNWQGUrnVWHFZ2xUoiiFmZt/XMEuNPXEfnqHiJQbQmYYu0N2TwrQJe0ScDRn3UTMP hW8B6j1H1KgmM/0fEb+M+5ffVq4Rk8DPaRyu6TecUIvX7msIGzs5euRhq0rVfEyJZnw+zG OzfWa1YMBBbpopumFWrsPImN5s9urvU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=r5nqSpaL; spf=pass (imf14.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-e733cd55f9eso318358276.1 for ; Wed, 25 Jun 2025 16:18:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen-com.20230601.gappssmtp.com; s=20230601; t=1750893535; x=1751498335; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=SZDu7RJqPXAY2ruR1j/0614GYveU9RxRfC4eQg5qP6Q=; b=r5nqSpaLP7FQDK7/jl5PIzPZ6b5sgzFPrHH0fMkWtLfWY/rOdWfiCTkCnenMjpTGMY 0qnNyVu+EudSKZ/DsGEqHONS28ejgmJ5ArfV2Tby7yHdPOQgbPDllwDsQaaPL3Vki2XJ eb3jmCU/XSlfRsC+cM5B2Kj3YXp0eIDlHmmlo7PgghueGDF9YgBVmZyJCHREgitHzJh2 SanNSm3t9YKUDcQlzE4i8VI2Ik0akZSx1lAbIQHwlCjM2q+0uVG1ZWnW+5IihQ8Y5loW 55o12fmm4BfGsNqz6KTspOWmcqb/tYsYEmW84g+/JK2fhiDVerPRNh1vNZywbRz5UNgP P+rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750893535; x=1751498335; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SZDu7RJqPXAY2ruR1j/0614GYveU9RxRfC4eQg5qP6Q=; b=kv5bk9p+Nyp+qDrx4YDi+jV4yU205C859TBIz7q7Q3ib0jLoBwfvODW7nBiaarv1EO 8T65PUJlVqG8H3RCZ5J9Lli8jUm+pfYiucx9fTquLUoJzGXLf2sbr4M5lZ4Jxe0bjG/F aZuE1nSAKLMwsxBfDivAUiNRJjm1n0TPrFdSSDlevSrV0cQC6uC2GC3/nvFBXojemnHL 5/sur32YNrGtn7Y+cegKLwktDCmycz3UvVqPyT72VjwASmkA5Km0jcei9pPilYTTseEi UoVyWKnpkFAnAqYW1DpKfwhTM6asb0AQo3dN0T5MgHeOPqMFDgdVoBUCgEOIO3PjX5bd fovw== X-Forwarded-Encrypted: i=1; AJvYcCXxgYQO/qcKPRbf/CQgr+1cVJ3pF0Xz1zytR6j6h/tMnQYUvw5OV6sZCifEU+CVab6AjKP0yk7Ozg==@kvack.org X-Gm-Message-State: AOJu0YxdETK6z7mP+ZAbcZA5YlTxFBw6zD0RJn51Uy+PCqAtgddAcSGZ IzdKDJNp2QJ/h7k0P19A3Xn5xhPwX3DGGQYT8JHfPQ+YAEY/kTzeCrgFBShnNxTkbbA= X-Gm-Gg: ASbGnctiObkE6rzAkgyDCTg+pWgXw0/lwZLMyKmF9Z5PlvSgwYyE8TVIhzM9/P4tP2i 3oYmKUTS/xLEGR1ZhQzZIuf/IhgGZI470nFHxrfcOn8c2p5IAfEhhoikdNADIzgHOB0FQWALf+N 686PYaDsIhmirxdAy2MFkyjeuXgUcnN7J0Yk1rCdy8W6jhGGEhhcAPp08V8VfqKBt2LSBrIzPCD fXX1mJHHEBDILWqC1R6palfnLmj1CvGoi+lMlFh4wdlcLYn6Yc06lhb2mWJMsYqMKpHZXdhp4BH qC0+2MzQwmt0xfUS3OURl/3to7WN7ETUhLbmG6rp2WXfW/7Ln31RupddP+DuA2pzQPw1GjMa1DM eeBWX8USKFaoyEmE1Et6GeSErnT1NoyHAqvPSPJonIwk2D1BbVrdUX4z4sF6PQUI= X-Google-Smtp-Source: AGHT+IF6d+wQdJueaL+2XDR+0IAu1dQMzIzIZ7VLaIY39ujjcBBrFSOUIoGsCEOL7tdAla+chMnDaQ== X-Received: by 2002:a05:6902:2182:b0:e84:46f2:8233 with SMTP id 3f1490d57ef6-e879b98cebdmr2790163276.45.1750893535364; Wed, 25 Jun 2025 16:18:55 -0700 (PDT) Received: from soleen.c.googlers.com.com (64.167.245.35.bc.googleusercontent.com. [35.245.167.64]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e842ac5c538sm3942684276.33.2025.06.25.16.18.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jun 2025 16:18:54 -0700 (PDT) From: Pasha Tatashin To: pratyush@kernel.org, jasonmiu@google.com, graf@amazon.com, changyuanl@google.com, pasha.tatashin@soleen.com, rppt@kernel.org, dmatlack@google.com, rientjes@google.com, corbet@lwn.net, rdunlap@infradead.org, ilpo.jarvinen@linux.intel.com, kanie@linux.alibaba.com, ojeda@kernel.org, aliceryhl@google.com, masahiroy@kernel.org, akpm@linux-foundation.org, tj@kernel.org, yoann.congal@smile.fr, mmaurer@google.com, roman.gushchin@linux.dev, chenridong@huawei.com, axboe@kernel.dk, mark.rutland@arm.com, jannh@google.com, vincent.guittot@linaro.org, hannes@cmpxchg.org, dan.j.williams@intel.com, david@redhat.com, joel.granados@kernel.org, rostedt@goodmis.org, anna.schumaker@oracle.com, song@kernel.org, zhangguopeng@kylinos.cn, linux@weissschuh.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, gregkh@linuxfoundation.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, rafael@kernel.org, dakr@kernel.org, bartosz.golaszewski@linaro.org, cw00.choi@samsung.com, myungjoo.ham@samsung.com, yesanishhere@gmail.com, Jonathan.Cameron@huawei.com, quic_zijuhu@quicinc.com, aleksander.lobakin@intel.com, ira.weiny@intel.com, andriy.shevchenko@linux.intel.com, leon@kernel.org, lukas@wunner.de, bhelgaas@google.com, wagi@kernel.org, djeffery@redhat.com, stuart.w.hayes@gmail.com, ptyadav@amazon.de, lennart@poettering.net, brauner@kernel.org, linux-api@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v1 07/32] kho: add interfaces to unpreserve folios and physical memory ranges Date: Wed, 25 Jun 2025 23:17:54 +0000 Message-ID: <20250625231838.1897085-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog In-Reply-To: <20250625231838.1897085-1-pasha.tatashin@soleen.com> References: <20250625231838.1897085-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Stat-Signature: 1i7oxrfqbecqriywhjyo1mk8d7wfqbwo X-Rspamd-Queue-Id: 64FE710000B X-Rspamd-Server: rspam08 X-HE-Tag: 1750893536-973051 X-HE-Meta: U2FsdGVkX1/22yOylEZ87znkiirLWd6qTxFexHlr/o3Om93EXyv3KJxr2XGrAn5K+nwKQjPrdegIXpdmsGgfJBJpvL4Gzqttd+RbZ157zfqIQGvvpxMTvGlERv8iyTiq7q3xY0KiRMhmmXmZxxPUy/6eCAismhns8wOx8Bas+yZurdFPlMWaMcmgXLXQB2gSemnRmaUOaw7EqeJfHn72IcHsqh0Gwbv/xMwklYO5iTUHIi87nhLktWRTIT7VmYNyxblcxUC+Hqway+X7MLpjYRQpZMWZ2iQ6Wq96c8721Qzfg+G3rFLtIHX4s9MJoNDVhX2hgh06F/510ce18HZO3wTJBkfmbke99T++TXNopk39+1Ymt0o8hSSNa/QR6J5GI1JHIHQ3IQ7Y71okUIQgd1Obt/VDDqxWokkge3M31fwauMXjAO2llp4HjecQLI+vhAUx5/zJMTpKJ/hXSeeBT7jO6AtELUX3d4J5c0pte7lDFKZD4P3h8eBSLBZWUT0zz1fZRWWbOtHjYiYeBi/DhOlz6m3kKuOH25kGgR0wfV+q7idChx7ICY8ccTRtsTxHhGifOkm+ZNBUmYaVTJJFN2YQeMwo1bho/egbhKvvyqRh/j+58yz9VA2J3+vEZuCs6SrNcj2YZhIdsjIf72fVfa+DgDFApq1P4Mavt/htp4TexdCV5QyU4Cn9qRskMoIBW2myflc4EzahaOdw4ZbPzhSC/RjVquX7DGmhDd4vhT638S4dGi8+YEkZDthqmYL11INoTkz2UEFVP03AVRgN8u/Jq2v7XIrFo2UYmR7RxRzbS37gQk3JvZe6Hb0kg3GgAYyNz4/+2tRrr1j1IpgghbWbOggMpGTbWJ1ssyl99qJmebJcO/mpC/59BXmVnTMVlA1Ki+giyeQIIBF9QGmV24wKNMvZewPxpFPeF+IhGmm3uHbqNg6fLYEQ4Stu+NP33V5cO8VPZ3xswYxhy+s X8Az9aFT +KAtkKS/eglNyR1QmRwt+1NANm7PEtwXjyinXPo78Eu4aCgYrsT6lv4Exr8m1C0x8jU9r3VSoprctLDJgGgri+5ELJ6g4qljYbidG9QADJMT8cXGxj3TMUNgy8tlmLv2umuHlx7oiOhaAwDVXGNVCoBgZFJu8PKucTzlPB0NWdgvOuOwowkffipFkN4uQs/D2I9E7j0FCzTh30TUgQaL/i8o/ipciWZYwtnxxTZrSvRMkncAT+B8fiWaMlTO+yoxcD2AeRH0L5UCFGbfY2fT1BmtbuzNRuel+8ERabxNvyJ0P9TBBZJgM4V/CFISsvWP1W5Wl3nvAnvIGP+L4BbZ5RbP6b5/jKdZBw8APgxuBawiuWt1sQBT/iOPTFnsw5dVTtIZruOMYZuJPzn1QyLPr+Kh29YiOzcmHYZYBUv3OvtweCWnGWSnD3JeYS6xUyoGLg3fbogsAlOzBvWgSe3Mpap9DMzUqGb3QeaRoJiX22Qi5RBuxuTObJnBii+L5GSOPahnmWTcGO0Qe0ZY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Changyuan Lyu Allow users of KHO to cancel the previous preservation by adding the necessary interfaces to unpreserve folio. Signed-off-by: Changyuan Lyu Co-developed-by: Pasha Tatashin Signed-off-by: Pasha Tatashin --- include/linux/kexec_handover.h | 12 +++++ kernel/kexec_handover.c | 90 +++++++++++++++++++++++++++++----- 2 files changed, 89 insertions(+), 13 deletions(-) diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h index cabdff5f50a2..383e9460edb9 100644 --- a/include/linux/kexec_handover.h +++ b/include/linux/kexec_handover.h @@ -33,7 +33,9 @@ struct folio; bool kho_is_enabled(void); int kho_preserve_folio(struct folio *folio); +int kho_unpreserve_folio(struct folio *folio); int kho_preserve_phys(phys_addr_t phys, size_t size); +int kho_unpreserve_phys(phys_addr_t phys, size_t size); struct folio *kho_restore_folio(phys_addr_t phys); int kho_add_subtree(const char *name, void *fdt); void kho_remove_subtree(void *fdt); @@ -58,11 +60,21 @@ static inline int kho_preserve_folio(struct folio *folio) return -EOPNOTSUPP; } +static inline int kho_unpreserve_folio(struct folio *folio) +{ + return -EOPNOTSUPP; +} + static inline int kho_preserve_phys(phys_addr_t phys, size_t size) { return -EOPNOTSUPP; } +static inline int kho_unpreserve_phys(phys_addr_t phys, size_t size) +{ + return -EOPNOTSUPP; +} + static inline struct folio *kho_restore_folio(phys_addr_t phys) { return NULL; diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index e1f0b7a9f5e5..d9e947eac041 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -136,26 +136,33 @@ static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size_t sz) return elm; } -static void __kho_unpreserve(struct kho_mem_track *track, unsigned long pfn, - unsigned long end_pfn) +static void __kho_unpreserve_order(struct kho_mem_track *track, unsigned long pfn, + unsigned int order) { struct kho_mem_phys_bits *bits; struct kho_mem_phys *physxa; + const unsigned long pfn_high = pfn >> order; - while (pfn < end_pfn) { - const unsigned int order = - min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn)); - const unsigned long pfn_high = pfn >> order; + physxa = xa_load(&track->orders, order); + if (!physxa) + return; - physxa = xa_load(&track->orders, order); - if (!physxa) - continue; + bits = xa_load(&physxa->phys_bits, pfn_high / PRESERVE_BITS); + if (!bits) + return; - bits = xa_load(&physxa->phys_bits, pfn_high / PRESERVE_BITS); - if (!bits) - continue; + clear_bit(pfn_high % PRESERVE_BITS, bits->preserve); +} - clear_bit(pfn_high % PRESERVE_BITS, bits->preserve); +static void __kho_unpreserve(struct kho_mem_track *track, unsigned long pfn, + unsigned long end_pfn) +{ + unsigned int order; + + while (pfn < end_pfn) { + order = min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn)); + + __kho_unpreserve_order(track, pfn, order); pfn += 1 << order; } @@ -667,6 +674,30 @@ int kho_preserve_folio(struct folio *folio) } EXPORT_SYMBOL_GPL(kho_preserve_folio); +/** + * kho_unpreserve_folio - unpreserve a folio. + * @folio: folio to unpreserve. + * + * Instructs KHO to unpreserve a folio that was preserved by + * kho_preserve_folio() before. The provided @folio (pfn and order) + * must exactly match a previously preserved folio. + * + * Return: 0 on success, error code on failure + */ +int kho_unpreserve_folio(struct folio *folio) +{ + const unsigned long pfn = folio_pfn(folio); + const unsigned int order = folio_order(folio); + struct kho_mem_track *track = &kho_out.track; + + if (kho_out.finalized) + return -EBUSY; + + __kho_unpreserve_order(track, pfn, order); + return 0; +} +EXPORT_SYMBOL_GPL(kho_unpreserve_folio); + /** * kho_preserve_phys - preserve a physically contiguous range across kexec. * @phys: physical address of the range. @@ -712,6 +743,39 @@ int kho_preserve_phys(phys_addr_t phys, size_t size) } EXPORT_SYMBOL_GPL(kho_preserve_phys); +/** + * kho_unpreserve_phys - unpreserve a physically contiguous range. + * @phys: physical address of the range. + * @size: size of the range. + * + * Instructs KHO to unpreserve the memory range from @phys to @phys + @size. + * The @phys address must be aligned to @size, and @size must be a + * power-of-2 multiple of PAGE_SIZE. + * This call must exactly match a granularity at which memory was originally + * preserved (either by a `kho_preserve_phys` call with the same `phys` and + * `size`). Unpreserving arbitrary sub-ranges of larger preserved blocks is not + * supported. + * + * Return: 0 on success, error code on failure + */ +int kho_unpreserve_phys(phys_addr_t phys, size_t size) +{ + struct kho_mem_track *track = &kho_out.track; + unsigned long pfn = PHYS_PFN(phys); + unsigned long end_pfn = PHYS_PFN(phys + size); + + if (kho_out.finalized) + return -EBUSY; + + if (!PAGE_ALIGNED(phys) || !PAGE_ALIGNED(size)) + return -EINVAL; + + __kho_unpreserve(track, pfn, end_pfn); + + return 0; +} +EXPORT_SYMBOL_GPL(kho_unpreserve_phys); + static int __kho_abort(void) { int err = 0; -- 2.50.0.727.gbf7dc18ff4-goog