From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4C30C36011 for ; Wed, 26 Mar 2025 16:26:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 333FE280098; Wed, 26 Mar 2025 12:26:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BB2728008D; Wed, 26 Mar 2025 12:26:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10DE8280098; Wed, 26 Mar 2025 12:26:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E0DFD28008D for ; Wed, 26 Mar 2025 12:26:17 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 05DD0ABD27 for ; Wed, 26 Mar 2025 16:26:17 +0000 (UTC) X-FDA: 83264229636.02.EA9DC6A Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf21.hostedemail.com (Postfix) with ESMTP id 196C01C0012 for ; Wed, 26 Mar 2025 16:26:12 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=m8LQcG2R; spf=pass (imf21.hostedemail.com: domain of fvdl@google.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=fvdl@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743006373; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TP3pQO3RlTwxLWj//K05AHCYcZl/96NtoRgWC1Sg7VM=; b=o9RtaESEuGxb/Ar4OTGwAie6UMsFK8rWfAUmd5M0MFMjEAo73oS+WQlkgS3PxJoSHJoxlW JnBz+kAsJpwkTlD9OuT3qgfcMqFq57bMYFhyYP564zIoqClGwzMMjSyWOuV1Lbct8+ZxkF RnNVlrryK5TWldO03P+wA2mCW7OMRno= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=m8LQcG2R; spf=pass (imf21.hostedemail.com: domain of fvdl@google.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=fvdl@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743006373; a=rsa-sha256; cv=none; b=Gg6TOXqmx519pZ0fxt2iQXgFzDMJ69W/Ao/TXWpujtZimux3k2Ff5SdM+pD0aMN6EcUNYg ZODrw3lydq5aI6Biy/iDw775mUyrUmM21IOCDWc+wojJGeBSahVRrCoX/G6EWMJHVA8bg3 BPzednlYlFzHk/+MV8cYrJRRQtuRjj4= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-4774611d40bso323541cf.0 for ; Wed, 26 Mar 2025 09:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743006371; x=1743611171; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TP3pQO3RlTwxLWj//K05AHCYcZl/96NtoRgWC1Sg7VM=; b=m8LQcG2RfWoaAER11esSsEpSfUb7MziQ0GZ6AKL1nLKy9FLNXPKN/n4pOnXz7ASD4j VnuGt5wnQ5MVhHDctrDIZN9eCWnY7qEFqooPb/CUrdjG/7lgFiVO7h1+3l4uN8vYxsYb lAGSH6I/tcj43BT2AIvxBg/0hYTV6JOTOWaq9QUmMPBdNs67FI6X6qVV7IFEceQW2YHZ Ppw6GhA/wHux9gZIzLnyhinaXzJg655ggZpsHiiJozEi7vqajjJHNNfGwQIFCy2A7p+G n0OQx/vmmJrd8STGvV7Ho1UEKFbKKzNkwuAT9HLfr6RTt1ZsMFCpFzZwcGeqMwz3PyEt q6sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743006371; x=1743611171; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TP3pQO3RlTwxLWj//K05AHCYcZl/96NtoRgWC1Sg7VM=; b=SLG8Sz5ZRgtoDT1Qc8xsBossJhM70jAqIi9wn24LAjrTYR257EOY9VT3l2+v8LHoUq F9hr2XjVSF+nwwZ+lScXinsKRJk66eaaa2HsNWSKEwznKI+APy+PyNoE1Q8lQ7CpzFvp mr2DM5R246YJeCuJRk2LZFSqOlC4kRqZMZAGZEWS2UcBOT2n0rewBXXDfqXpWHWmkXh5 ksa1QC06DUph0wR1tyFnFS7F1Nqpbf3ERxtxdwfQmTBYIYvwIWE4DfcCEYF4c915IAgm 62xwWgg4xaE43gek8qS6yhL8WxHoAy0omO9iivZWRhoT+pszdFmIz10d3M47QJFq8V8F 1W9Q== X-Forwarded-Encrypted: i=1; AJvYcCXBlOJJw48Bk9fkedN11dkwQq5wNCR6e7fsuN4hk5p911wGBv9P0BFcNV67x3Q2l/XQ73bdNa4mTw==@kvack.org X-Gm-Message-State: AOJu0Yzh069/c2se6hps6eWo+Wor0q5+WKTMurAdCzQLBsS+bNwQymFY 42AA33IGyEwzeNUWWfe3rikcTmJ3rQUkVoULnsXOnj5Vd9XXkKmcBU/lXyC65i6uSFuw6tfHHID 00PPZ8er9u/LYBoar3gV14OSQmH7ij6PnSP/m X-Gm-Gg: ASbGncs+VjODcOpWdyPvXbE+SnwFp5/nY5qRFulnJD8hBk3u+1cT4vlCraSIUlhDOn0 apoevKMjmGkrU8LzfUOpcuhK+dec0yl4oSboh1h55OoKrli+OURETeiMXAgoFo2sQuffPxfbC2L m+X5JROwj/b/zSZ9ub1xhl+1lnAnXcQhIlBA== X-Google-Smtp-Source: AGHT+IFOa5U4EkoXfdO0+LtQUSzYhAnBQIC8ww9fg/mAwgTBmcE9rzJd63FTHpnj7/g5eVnvx4SFKaBHLQRjG62h4KA= X-Received: by 2002:a05:622a:2515:b0:477:2c1e:d252 with SMTP id d75a77b69052e-47762eb2ac5mr4836571cf.20.1743006371331; Wed, 26 Mar 2025 09:26:11 -0700 (PDT) MIME-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> <20250320015551.2157511-8-changyuanl@google.com> In-Reply-To: From: Frank van der Linden Date: Wed, 26 Mar 2025 09:25:59 -0700 X-Gm-Features: AQ5f1JqROvSmDfkTDrNCkDrxnAL_58GWFP96LlXcUaVtPVLlGbzUuTMgIm7BDUs Message-ID: Subject: Re: [PATCH v5 07/16] kexec: add Kexec HandOver (KHO) generation helpers To: Mike Rapoport Cc: Changyuan Lyu , linux-kernel@vger.kernel.org, graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 196C01C0012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: rwag9xzhc5bx7bubb4iua13iu4a6ozae X-HE-Tag: 1743006372-194167 X-HE-Meta: U2FsdGVkX183y7jLBlR+TeWw1hY2ylihwhufB9dUy1f4QR2Q8MoK67isfJeiWW6k6WiXYnokZWZIlyETuLTTexvxfyYfUhj3atUOIdfKGrTIHpxdHCAhZj7yDxNzv/dm9NGb8p8YjOIlv/IR04P+lXcn3wM8lQR5brq06iRcNA6AwfOSKr0dhaJ7RGzewWTQh9Ee0uLM8MNONgnQSqOSBp0TMcHP/rIvrmXyYoxdTp3w/Gep3MNvu4y1/XG/A3cMjRTJ/lqfJt85sSkmScQP1SeGrP2QwwTbN3yt3it7gk8oCtACn+F6pfo4emcXdj0YNyr23abUpQyNVIYR2GAPx6Fg7Ll4ZaZ71bOUtOeYfsjhz5Pb9Yzucf7X99rWJSdYnqlQV1zNeRP0ZT0+vpSVpekhov7/QeYdCD40lUZW72zBSDDI8A3OaPIZcPkfWSAI8STJZbGjUbn2VNrs1hfXFGv+eMI/xSJTDySC96z2z0KrMJoVQ6FkcotqITBIfArPRWOLzKKlaTh8ITOuvBq6P+RVTXicdvbFKQgQi/s1WEz1jfGThiv8VkjhQqeNaDWwtMD4M5wI2wxh3+zV1sgkpEDEvUohO5FlaCKSaGw4e3pO0gZeuGEb5IOEyAqpqjjVtH/aRzjZoWcFtvUpCqNJEAkfZb6Dxxnk72NjfFnLI7Fs28AyqQfToVXIMd7gYCcaC8tp4Tr2x1Vr+S0uEBsrWENj6hQLftF2sZHwO7r3+dO3J4TccltedX6bbAf+fdYJfZ0WphPhogvoHJW30s1lTrVEA/jfvO1UDoWVB397xST8UIYZDUjZjiG2jM259CCH1t2o33GfUEpSJuWqw3r4qY4Yorc0nKvswLLTP0Vau/H4vNB/hiQoL9mWPBaHCcfZgqL7KH+vVOLuPGe+G8iHoARxHmbAMwNWFUEuDVcULyvY+BeLXLMLCLfnZoR83JVnFQXvtWekbKJjE7WBfeq hbUaJPEx xeEgChCpATa6anbGwGpjApEpxVIk79xUZQF7sWgYmh2aF6uOraatQWjCNtOHDBLsbR0KwqpcZzvXYO0009GzRKF/OlgF3pSIRh8YXR15SX5aoMdtu+r1xszwry6yrJG2PaU49Ny7bkHZE4uqDXVvge8aLZ85OVmJgGFgcMpvNhnoQ+3zQkaFmemIHPkmuHm9XIpPud5/9mnNQXcA/j+nNH05/EXvPtLcRrBQrfPd2eK1U44Qst4jPPh1r+vZTPCOGsRwMsm/wagCQ3Lk/HySWdxNXDEvhBXF1F6UgVznd0YRCs08= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 26, 2025 at 4:59=E2=80=AFAM Mike Rapoport wro= te: [...] > > There has, for example, been some talk about making hugetlbfs > > persistent. You could have hugetlb_cma active. The hugetlb CMA areas > > are set up quite early, quite some time before KHO restores memory. So > > that would have to be changed somehow if the location of the KHO init > > call would remain as close as possible to buddy init as possible. I > > suspect there may be other uses. > > I think we can address this when/if implementing preservation for hugetlb= fs > and it will be tricky. > If hugetlb in the first kernel uses a lot of memory, we just won't have > enough scratch space for early hugetlb reservations in the second kernel > regardless of hugetlb_cma. On the other hand, we already have the preserv= ed > hugetlbfs memory, so we'd probably need to reserve less memory in the > second kernel. > > But anyway, it's completely different discussion about how to preserve > hugetlbfs. Right, there would have to be a KHO interface way to carry over the early reserved memory and reinit it early too. > > > > > current requirement in the patch set seems to be "after sparse/page > > > > init", but I'm not sure why it needs to be as close as possibly to > > > > buddy init. > > > > > > Why would you say that sparse/page init would be a requirement here? > > > > At least in its current form, the KHO code expects vmemmap to be > > initialized, as it does its restore base on page structures, as > > deserialize_bitmap expects them. I think the use of the page->private > > field was discussed in a separate thread, I think. If that is done > > differently, it wouldn't rely on vmemmap being initialized. > > In the current form KHO does relies on vmemmap being allocated, but it do= es > not rely on it being initialized. Marking memblock ranges NOINT ensures > nothing touches the corresponding struct pages and KHO can use their fiel= ds > up to the point the memory is returned to KHO callers. > > > A few more things I've noticed (not sure if these were discussed before= ): > > > > * Should KHO depend on CONFIG_DEFERRED_STRUCT_PAGE_INIT? Essentially, > > marking memblock ranges as NOINIT doesn't work without > > DEFERRED_STRUCT_PAGE_INIT. Although, if the page->private use > > disappears, this wouldn't be an issue anymore. > > It does. > memmap_init_reserved_pages() is called always, no matter of > CONFIG_DEFERRED_STRUCT_PAGE_INIT is set or not and it skips initializatio= n > of NOINIT regions. Yeah, I see - the ordering makes this work out. MEMBLOCK_RSRV_NOINIT is a bit confusing in the sense that if you do a memblock allocation in the !CONFIG_DEFERRED_STRUCT_PAGE_INIT case, and that allocation is done before free_area_init(), the pages will always get initialized regardless, since memmap_init_range() will do it. But this is done before the KHO deserialize, so it works out. > > > * As a future extension, it could be nice to store vmemmap init > > information in the KHO FDT. Then you can use that to init ranges in an > > optimized way (HVO hugetlb or DAX-style persisted ranges) straight > > away. > > These days memmap contents is unstable because of the folio/memdesc > project, but in general carrying memory map data from kernel to kernel is > indeed something to consider. Yes, I think we might have a need for that, but we'll see. Thanks, - Frank