From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B466C47255 for ; Mon, 11 May 2020 17:09:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D863620714 for ; Mon, 11 May 2020 17:09:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D863620714 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=xmission.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 36F8890006E; Mon, 11 May 2020 13:09:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32067900036; Mon, 11 May 2020 13:09:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20F1290006E; Mon, 11 May 2020 13:09:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 07E84900036 for ; Mon, 11 May 2020 13:09:16 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CAFC88248068 for ; Mon, 11 May 2020 17:09:15 +0000 (UTC) X-FDA: 76805073870.04.voice01_8afbf1b0ac825 X-HE-Tag: voice01_8afbf1b0ac825 X-Filterd-Recvd-Size: 8752 Received: from out02.mta.xmission.com (out02.mta.xmission.com [166.70.13.232]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 17:09:14 +0000 (UTC) Received: from in02.mta.xmission.com ([166.70.13.52]) by out02.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jYBvV-0004ww-9J; Mon, 11 May 2020 11:09:13 -0600 Received: from ip68-227-160-95.om.om.cox.net ([68.227.160.95] helo=x220.xmission.com) by in02.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.87) (envelope-from ) id 1jYBvU-0004PL-El; Mon, 11 May 2020 11:09:13 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: David Hildenbrand Cc: James Morse , kexec@lists.infradead.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Dave Young , Baoquan He References: <20200501165701.24587-1-james.morse@arm.com> <40b07632-b044-d1cd-96a2-81eec3da93e7@redhat.com> <8736892l92.fsf@x220.int.ebiederm.org> Date: Mon, 11 May 2020 12:05:42 -0500 In-Reply-To: (David Hildenbrand's message of "Mon, 11 May 2020 10:19:46 +0200") Message-ID: <87h7wmtna1.fsf@x220.int.ebiederm.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1jYBvU-0004PL-El;;;mid=<87h7wmtna1.fsf@x220.int.ebiederm.org>;;;hst=in02.mta.xmission.com;;;ip=68.227.160.95;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX1/xJ6K+zbG22euq8Nn2E+Z81QjiTg4IXoc= X-SA-Exim-Connect-IP: 68.227.160.95 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: [PATCH] kexec: Discard loaded image on memory hotplug X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in02.mta.xmission.com) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > On 09.05.20 17:14, Eric W. Biederman wrote: >> David Hildenbrand writes: >> >>> On 01.05.20 18:57, James Morse wrote: >>>> On x86, the kexec payload contains a copy of the current memory map. >>>> If memory is added or removed, this copy of the memory map becomes >>>> stale. Getting this wrong may prevent the next kernel from booting. >>>> The first kernel may die if it tries to re-assemble the next kernel >>>> in memory that has been removed. >>>> >>>> Discard the loaded kexec image when the memory map changes, user-space >>>> should reload it. >>>> >>>> Kdump is unaffected, as it is placed within the crashkernel reserved >>>> memory area and only uses this memory. The stale memory map may affect >>>> generation of the vmcore, but the kdump kernel should be in a position >>>> to validate it. >>>> >>>> Signed-off-by: James Morse >>>> --- >>>> This patch obsoletes: >>>> * kexec/memory_hotplug: Prevent removal and accidental use >>>> https://lore.kernel.org/linux-arm-kernel/20200326180730.4754-1-james.morse@arm.com/ >>>> >>>> kernel/kexec_core.c | 40 ++++++++++++++++++++++++++++++++++++++++ >>>> 1 file changed, 40 insertions(+) >>>> >>>> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c >>>> index c19c0dad1ebe..e1901e5bd4b5 100644 >>>> --- a/kernel/kexec_core.c >>>> +++ b/kernel/kexec_core.c >>>> @@ -12,6 +12,7 @@ >>>> #include >>>> #include >>>> #include >>>> +#include >>>> #include >>>> #include >>>> #include >>>> @@ -22,10 +23,12 @@ >>>> #include >>>> #include >>>> #include >>>> +#include >>>> #include >>>> #include >>>> #include >>>> #include >>>> +#include >>>> #include >>>> #include >>>> #include >>>> @@ -1219,3 +1222,40 @@ void __weak arch_kexec_protect_crashkres(void) >>>> >>>> void __weak arch_kexec_unprotect_crashkres(void) >>>> {} >>>> + >>>> +/* >>>> + * If the memory layout changes, any loaded kexec image should be evicted >>>> + * as it may contain a copy of the (now stale) memory map. This also means >>>> + * we don't need to check the memory is still present when re-assembling the >>>> + * new kernel at machine_kexec() time. >>>> + */ >>> >>> Onlining/offlining is not a change of the memory map. >> >> Phrasing it that way is non-sense. What is important is memory >> available in the system. A memory map is just a reflection upon that, >> a memory map is not the definition of truth. >> >> So if this notifier reflects when memory is coming and going on the >> system this is a reasonable approach. >> >> Do these notifiers might fire for special kinds of memory that should >> only be used for very special purposes? >> >> This change with the addition of some filters say to limit taking action >> to MEM_ONLINE and MEM_OFFLINE looks reasonable to me. Probably also >> filtering out special kinds of memory that is not gernally useful. > > There are cases, where this notifier will not get called (e.g., hotplug > a DIMM and don't online it) or will get called, although nothing changed > (offline+re-online to a different zone triggered by user space). AFAIK, > nothing in kexec (*besides kdump) cares about online vs. offline memory. > This is why this feels wrong. So what precisely does offline and online of memory mean in this context? Is it turning the memory on and off? (which is the obvious meaning) Or is offline and online letting the ordinary kernel use a chunk of memory and not use a chunk of memory and the memory remains running the entire time? > add_memory()/try_remove_memory() is the place where: > - Memblocks are created/deleted (if the memblock allocator is still > alive) > - Memory resources are created/deleted (e.g., reflected in /proc/iomem) > - Firmware memmap entries are created/deleted (/sys/firmware/memmap) > > My idea would be to add something like > kexec_map_add()/kexec_map_remove() where we have > firmware_map_add_hotplug()/firmware_map_remove(). From there, we can > unload the kexec image like done in this patch. I don't see the connection with a firmware_map. Maybe that is how it is thought about in the code but in principle the firmware can not exist or completely ignore memory hotplug. > And these callbacks might come in handy for fixing up the kexec initial > memmap in case of kexec_file_load(). AFAIKS on x86_64: Maybe we have enough information to fixup the loaded kexec image in the kexec_file_load case, we certainly don't in the ordinary kexec_load case. For now I want to stick to the simplest thing we can do which is either blocking the memory hotplug operation (if that is possible) or dropping the loaded kexec image. If that actually becomes a problem in practice we can do something more. But for now let's just do the minimal we can do that will prevent incorrect operation. Eric