From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEB92C282C6 for ; Fri, 28 Feb 2025 20:20:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30331280001; Fri, 28 Feb 2025 15:20:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B4FC280002; Fri, 28 Feb 2025 15:20:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ACF9280001; Fri, 28 Feb 2025 15:20:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id F26B8280001 for ; Fri, 28 Feb 2025 15:20:48 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id AE2D51A1215 for ; Fri, 28 Feb 2025 20:20:48 +0000 (UTC) X-FDA: 83170471776.14.B294591 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id E79B680011 for ; Fri, 28 Feb 2025 20:20:46 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kfFIn91K; spf=pass (imf02.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740774047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oMsvCa0RZBbh1a14dfeYFwcIWgC4rKUWn90abl03P5w=; b=b+LaaiNwSK2jemUyrk/bKevYltmAz/I4buYnPYndgDhMOefhKI1a3Fs70W1snGLq1Wfbxg BcQYPf+tVog1fw7xQQLn4WnVEBePa5MNJqP4aVlN8zMGUhdIU1cLULcYRf9PSC03BwLMo9 LK/iRuLgOlae71mXYcIyDMEvijUETmY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kfFIn91K; spf=pass (imf02.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740774047; a=rsa-sha256; cv=none; b=XIJ4S0iVho+AfXCFEWTEJSB8FM82HmcSIWRCk7EBT4zbIXm49fgRwDghSKBaPAsX1GpMbZ R9ruCz/O947XmCFPJmRXIc4IkjFzYaoY58KUoIDCofNaxDiLS5/bidJis+laHg/Ii5bPJK pokMuTOfHsSE3K3JoeDDxbXVBuIh/IY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9F51D5C62F8; Fri, 28 Feb 2025 20:18:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5A39C4CED6; Fri, 28 Feb 2025 20:20:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740774045; bh=bbzE7bwrYKBUNI91akgimXhe+iLOXIRvjnh9wKEiBG0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=kfFIn91KSNHhn07yKT9l2oOU8euWhSQeLxxD1+CP5Lmkt8N8i6O/Zn9pEs4NGiCJj qJEjrvhzqZMOIYECg3O0McklsQMWurW5bICaO6Eqlily54ppx+yXiPrIfcDSkbmmEy mgA+RTO662SeKhJ/BeytWpw2yGPCMjmkdX2P8FuJVAKh+rj8Ay73HcAZ8F0mxABP/0 t/Etv53LQUPDT8LDrOxXxcp5ZTSXxggaTWVIPD4REQMGOIcj+m3X88aLIXEmmDQWmj 830FksBD61RtpQPNDjonn0bpZ2XqD0baA3+sDkYSi8hDY+Sf6kIPnRC3EZcEYLP+NU hO0vT2n2HTtbg== Date: Fri, 28 Feb 2025 22:20:23 +0200 From: Mike Rapoport To: Pratyush Yadav Cc: linux-kernel@vger.kernel.org, Alexander Graf , Andrew Morton , Andy Lutomirski , Anthony Yznaga , Arnd Bergmann , Ashish Kalra , Benjamin Herrenschmidt , Borislav Petkov , Catalin Marinas , Dave Hansen , David Woodhouse , Eric Biederman , Ingo Molnar , James Gowans , Jonathan Corbet , Krzysztof Kozlowski , Mark Rutland , Paolo Bonzini , Pasha Tatashin , "H. Peter Anvin" , Peter Zijlstra , Rob Herring , Rob Herring , Saravana Kannan , Stanislav Kinsburskii , Steven Rostedt , Thomas Gleixner , Tom Lendacky , Usama Arif , Will Deacon , devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Subject: Re: [PATCH v4 00/14] kexec: introduce Kexec HandOver (KHO) Message-ID: References: <20250206132754.2596694-1-rppt@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Queue-Id: E79B680011 X-Rspamd-Server: rspam09 X-Stat-Signature: 6yjfp5m78jbbab6tx6twrgmq17wi9r8c X-HE-Tag: 1740774046-460883 X-HE-Meta: U2FsdGVkX1+5r5QblXBtitTnUidkPGgLfx7YYDXZdwi2EfaJwU9TpsZQmkhUnGoKZP8IAQEDZvvMROxkXJdi3hcud9PHvc46gixtYoTZxVr8VdDfkkpxmyyu3dt04bD4DNrCt8K2R2PH/us9DftIRiTqA7gHrX84EEpWbFHjfL/hN820fczZhfRO56FJwbQjh0e+XJQwViavZI3HfR3b532KqeNRWGucv/iHnit/34ffUfq+2jwUfjtvhoQehOXElj2xN6lYmEuZxSm5YSVOYy9N2pLbO47DAWzO6tfayxDqwJbJuDohYDALfN4A5ZU2NfrqehoiPHY35aIx7klFms99KR3Vesmum+eypRWZJZqv9xtYCbOR5hsA7Frcp+vVdCDOAeePE8Oouq4vSP/2mQDSPML1hHN6LybdOFGqCDNppHp8JMZmpTIjuMNYPDK/IzJy1+/JPwoPTXP6t1kmrd7MSR8XfVF8Kh9Qc5b2vs0yBttFsZMGkleaxuvULDdFGrTuRO4NZpXvvzRwkSQOFDmAH9hQ58e9LepJsYZKKpdpBVZHrVaHg5Rk+J/OgwzkgSmZReoeiTRQxRfhSlUQYsu2DMBQrpgHA7tqgBDBwWkqamCOt7CPPPsT1zFsGixbER6CNEXSDzMAMp0/cijhTgWigRhB4hULBKg3BniX6XFpUy5ccqDhMWY01XZEmU5cEIQ0EjmeQg5I1ULabBVbUiMybik0UpzFUtzRoV5ccv3GACaY7gxmq7NcOSNAIV8fGQRzKabxCrgBGalvod79VgXinwkt1pH+UZ9KhIWBaPosGaNuzKDqrqLxwJFdzIpk2F62jFbff5MXdrlTRhpHTmtffQbaMnpxyccmhf8oJJu1n+EkudaXqTf/Hl74UnZASpNWGOqBlvweHFjRMIACnvLQmIo4EYst97d5hH4QTCdOzjOWKLe0CTcbZRsSYskwd1iEcZZLHcd9AvTaCfK DD/frMOS 3fRc6A3BglbwSRDyjR0I29q1iO7eGbSiDJ6bqlo2hJ5yZ+6TqXTGr4fT0HpKdZDxrzI0cKtv5LMwZZhWv9TKn5wFz3PFimreOmBAec5j4P/iE4QkFdcmj6K8M/OASEYIvpt08thUNP9XiUI+H6VuTb9WuGPibXgi9x+m6zPek/iS+r99WNCVPZSpm9OnRo/kdHilMmPoIoSeNU30yuhmgrfe0zMfivjwm2yypYlpJtydrvRpyWU65GunmlTFC8Tt+zsNAYZEK9oWdcYfNxTsxVG7zAY+THJMofeSvuh3wJFldcWVNu3oGF4PIXwJR2l2EbzBorESojLVcun1z/jsxksjXA5R5DNFs9iBK9hL3R6bn/vC33n6u4wnmkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Pratyush, On Wed, Feb 26, 2025 at 08:08:27PM +0000, Pratyush Yadav wrote: > Hi Mike, > > On Thu, Feb 06 2025, Mike Rapoport wrote: > > > From: "Mike Rapoport (Microsoft)" > > > > Hi, > > > > This a next version of Alex's "kexec: Allow preservation of ftrace buffers" > > series (https://lore.kernel.org/all/20240117144704.602-1-graf@amazon.com), > > just to make things simpler instead of ftrace we decided to preserve > > "reserve_mem" regions. > [...] > > I applied the patches on top of v6.14-rc1 and tried them out on an x86 > qemu machine . When I do a plain KHO activate and kexec, I get the below > errors on boot. This causes networking to fail on the VM. The errors are > consistent and happen every kexec-reboot, though fairly late in boot > after systemd tries to bring up network. The same setup has worked fine > with Alex's v3 of KHO patches. > > Do you see anything obvious that might cause this? I can try to debug > this tomorrow, but if it rings any loud bells it would be nice to know. Thanks for the report! It didn't ring any bells, but after I've found the issue and a fast-and-dirty fix. The scratch areas are allocated from high addresses and there is no scratch memory to satisfy memblock_alloc_low() in swiotb, so second kernel produces a couple of software IO TLB: swiotlb_memblock_alloc: Failed to allocate 67108864 bytes for tlb structure and without those buffers e1000 can't dma :( A quick fix would be to add another scratch area in the lower memory (below). I'll work on a better fix. diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index c26753d613cb..37bb54cdb130 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -623,13 +623,13 @@ static phys_addr_t __init scratch_size(int nid) static void kho_reserve_scratch(void) { phys_addr_t addr, size; - int nid, i = 1; + int nid, i = 2; if (!kho_enable) return; /* FIXME: deal with node hot-plug/remove */ - kho_scratch_cnt = num_online_nodes() + 1; + kho_scratch_cnt = num_online_nodes() + 2; size = kho_scratch_cnt * sizeof(*kho_scratch); kho_scratch = memblock_alloc(size, PAGE_SIZE); if (!kho_scratch) @@ -644,6 +644,15 @@ static void kho_reserve_scratch(void) kho_scratch[0].addr = addr; kho_scratch[0].size = size; + addr = memblock_phys_alloc_range(size, CMA_MIN_ALIGNMENT_BYTES, + MEMBLOCK_LOW_LIMIT, + ARCH_LOW_ADDRESS_LIMIT); + if (!addr) + goto err_free_scratch_areas; + + kho_scratch[1].addr = addr; + kho_scratch[1].size = size; + for_each_online_node(nid) { size = scratch_size(nid); addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, > -- > Regards, > Pratyush Yadav > -- Sincerely yours, Mike.