From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS,URIBL_DBL_ABUSE_MALW autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1500C2D0DB for ; Tue, 28 Jan 2020 13:50:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9676D24694 for ; Tue, 28 Jan 2020 13:50:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="M7R9ugzY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9676D24694 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 21AB76B026E; Tue, 28 Jan 2020 08:50:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 17B3D6B026F; Tue, 28 Jan 2020 08:50:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 044026B0270; Tue, 28 Jan 2020 08:49:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id DADAB6B026E for ; Tue, 28 Jan 2020 08:49:59 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 9596B8248D51 for ; Tue, 28 Jan 2020 13:49:59 +0000 (UTC) X-FDA: 76427176518.20.ants17_3a149d6a0a51e X-HE-Tag: ants17_3a149d6a0a51e X-Filterd-Recvd-Size: 13489 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 28 Jan 2020 13:49:58 +0000 (UTC) Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BDCBA2468F; Tue, 28 Jan 2020 13:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1580219398; bh=iebdCmOtPHRexywJ09GAy19F+JqZrSVQnrmBGl9nikI=; h=Subject:To:Cc:From:Date:In-Reply-To:From; b=M7R9ugzYwu8zQGXbjs6xRIt+LeLLyt0VbGVxtlEKV/YTOj3hqDizdOPyC2KZy4MpB 34wvi44R0znWGkw/LdR6cIa0mAWHIM8BS0jZnqIFSSk3753ujcAENFK5+oOLFL74HN ygx6TtQjKWD6ooopNa/SY/awgkFk2jk5ws2IGZG4= Subject: Patch "mm/memory_hotplug: fix try_offline_node()" has been added to the 4.19-stable tree To: 20191028105458.28320-1-david@redhat.com,20191102120221.7553-1-david@redhat.com,akpm@linux-foundation.org,aneesh.kumar@linux.ibm.com,bhe@redhat.com,dan.j.williams@intel.com,david@redhat.com,gregkh@linuxfoundation.org,jani.nikula@intel.com,jolsa@kernel.org,keith.busch@intel.com,linux-mm@kvack.org,m.mizuma@jp.fujitsu.com,mhocko@suse.com,nayna@linux.ibm.com,osalvador@suse.de,pasha.tatashin@soleen.com,peterz@infradead.org,rafael@kernel.org,richard.weiyang@gmail.com,sfr@canb.auug.org.au,tangchen@cn.fujitsu.com,torvalds@linux-foundation.org Cc: From: Date: Tue, 28 Jan 2020 14:49:40 +0100 In-Reply-To: <20200128095021.8076-24-david@redhat.com> Message-ID: <1580219380254222@kroah.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ANSI_X3.4-1968 X-stable: commit X-Patchwork-Hint: ignore Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a note to let you know that I've just added the patch titled mm/memory_hotplug: fix try_offline_node() to the 4.19-stable tree which can be found at: http://www.kernel.org/git/?p=3Dlinux/kernel/git/stable/stable-queue.g= it;a=3Dsummary The filename of the patch is: mm-memory_hotplug-fix-try_offline_node.patch and it can be found in the queue-4.19 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let know about it. >From foo@baz Tue 28 Jan 2020 02:32:10 PM CET From: David Hildenbrand Date: Tue, 28 Jan 2020 10:50:20 +0100 Subject: mm/memory_hotplug: fix try_offline_node() To: stable@vger.kernel.org Cc: linux-mm@kvack.org, Michal Hocko , Greg Kroah-Hartma= n , Andrew Morton = , "Aneesh Kumar K . V" , Baoquan He , Dan Williams , Oscar Salvador , Wei Yang , David Hildenbrand Message-ID: <20200128095021.8076-24-david@redhat.com> From: David Hildenbrand commit 2c91f8fc6c999fe10185d8ad99fda1759f662f70 upstream. -- snip -- Only contextual issues: - Unrelated check_and_unmap_cpu_on_node() changes are missing. - Unrelated walk_memory_blocks() has not been moved/refactored yet. -- snip -- try_offline_node() is pretty much broken right now: - The node span is updated when onlining memory, not when adding it. We ignore memory that was mever onlined. Bad. - We touch possible garbage memmaps. The pfn_to_nid(pfn) can easily trigger a kernel panic. Bad for memory that is offline but also bad for subsection hotadd with ZONE_DEVICE, whereby the memmap of the first PFN of a section might contain garbage. - Sections belonging to mixed nodes are not properly considered. As memory blocks might belong to multiple nodes, we would have to walk all pageblocks (or at least subsections) within present sections. However, we don't have a way to identify whether a memmap that is not online was initialized (relevant for ZONE_DEVICE). This makes things more complicated. Luckily, we can piggy pack on the node span and the nid stored in memory blocks. Currently, the node span is grown when calling move_pfn_range_to_zone() - e.g., when onlining memory, and shrunk when removing memory, before calling try_offline_node(). Sysfs links are created via link_mem_sections(), e.g., during boot or when adding memory. If the node still spans memory or if any memory block belongs to the nid, we don't set the node offline. As memory blocks that span multiple nodes cannot get offlined, the nid stored in memory blocks is reliable enough (for such online memory blocks, the node still spans the memory). Introduce for_each_memory_block() to efficiently walk all memory blocks. Note: We will soon stop shrinking the ZONE_DEVICE zone and the node span when removing ZONE_DEVICE memory to fix similar issues (access of garbage memmaps) - until we have a reliable way to identify whether these memmaps were properly initialized. This implies later, that once a node had ZONE_DEVICE memory, we won't be able to set a node offline - which should be acceptable. Since commit f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotadded memory to zones until online") memory that is added is not assoziated with a zone/node (memmap not initialized). The introducing commit 60a5a19e7419 ("memory-hotplug: remove sysfs file of node") already missed that we could have multiple nodes for a section and that the zone/node span is updated when onlining pages, not when adding them. I tested this by hotplugging two DIMMs to a memory-less and cpu-less NUMA node. The node is properly onlined when adding the DIMMs. When removing the DIMMs, the node is properly offlined. Masayoshi Mizuma reported: : Without this patch, memory hotplug fails as panic: : : BUG: kernel NULL pointer dereference, address: 0000000000000000 : ... : Call Trace: : remove_memory_block_devices+0x81/0xc0 : try_remove_memory+0xb4/0x130 : __remove_memory+0xa/0x20 : acpi_memory_device_remove+0x84/0x100 : acpi_bus_trim+0x57/0x90 : acpi_bus_trim+0x2e/0x90 : acpi_device_hotplug+0x2b2/0x4d0 : acpi_hotplug_work_fn+0x1a/0x30 : process_one_work+0x171/0x380 : worker_thread+0x49/0x3f0 : kthread+0xf8/0x130 : ret_from_fork+0x35/0x40 [david@redhat.com: v3] Link: http://lkml.kernel.org/r/20191102120221.7553-1-david@redhat.com Link: http://lkml.kernel.org/r/20191028105458.28320-1-david@redhat.com Fixes: 60a5a19e7419 ("memory-hotplug: remove sysfs file of node") Fixes: f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotadded memor= y to zones until online") # visiable after d0dc12e86b319 Signed-off-by: David Hildenbrand Tested-by: Masayoshi Mizuma Cc: Tang Chen Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Cc: Keith Busch Cc: Jiri Olsa Cc: "Peter Zijlstra (Intel)" Cc: Jani Nikula Cc: Nayna Jain Cc: Michal Hocko Cc: Oscar Salvador Cc: Stephen Rothwell Cc: Dan Williams Cc: Pavel Tatashin Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: David Hildenbrand Signed-off-by: Greg Kroah-Hartman --- drivers/base/memory.c | 36 ++++++++++++++++++++++++++++++++++++ include/linux/memory.h | 2 ++ mm/memory_hotplug.c | 47 +++++++++++++++++++++++++++++-------------= ----- 3 files changed, 67 insertions(+), 18 deletions(-) --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -862,3 +862,39 @@ out: printk(KERN_ERR "%s() failed: %d\n", __func__, ret); return ret; } + +struct for_each_memory_block_cb_data { + walk_memory_blocks_func_t func; + void *arg; +}; + +static int for_each_memory_block_cb(struct device *dev, void *data) +{ + struct memory_block *mem =3D to_memory_block(dev); + struct for_each_memory_block_cb_data *cb_data =3D data; + + return cb_data->func(mem, cb_data->arg); +} + +/** + * for_each_memory_block - walk through all present memory blocks + * + * @arg: argument passed to func + * @func: callback for each memory block walked + * + * This function walks through all present memory blocks, calling func o= n + * each memory block. + * + * In case func() returns an error, walking is aborted and the error is + * returned. + */ +int for_each_memory_block(void *arg, walk_memory_blocks_func_t func) +{ + struct for_each_memory_block_cb_data cb_data =3D { + .func =3D func, + .arg =3D arg, + }; + + return bus_for_each_dev(&memory_subsys, NULL, &cb_data, + for_each_memory_block_cb); +} --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -119,6 +119,8 @@ extern int memory_isolate_notify(unsigne extern struct memory_block *find_memory_block_hinted(struct mem_section = *, struct memory_block *); extern struct memory_block *find_memory_block(struct mem_section *); +typedef int (*walk_memory_blocks_func_t)(struct memory_block *, void *); +extern int for_each_memory_block(void *arg, walk_memory_blocks_func_t fu= nc); #define CONFIG_MEM_BLOCK_SIZE (PAGES_PER_SECTION<nid =3D=3D nid ? -EEXIST : 0; +} + /** * try_offline_node * @nid: the node ID @@ -1824,25 +1836,24 @@ static int check_and_unmap_cpu_on_node(p void try_offline_node(int nid) { pg_data_t *pgdat =3D NODE_DATA(nid); - unsigned long start_pfn =3D pgdat->node_start_pfn; - unsigned long end_pfn =3D start_pfn + pgdat->node_spanned_pages; - unsigned long pfn; - - for (pfn =3D start_pfn; pfn < end_pfn; pfn +=3D PAGES_PER_SECTION) { - unsigned long section_nr =3D pfn_to_section_nr(pfn); - - if (!present_section_nr(section_nr)) - continue; - - if (pfn_to_nid(pfn) !=3D nid) - continue; - - /* - * some memory sections of this node are not removed, and we - * can't offline node now. - */ + int rc; + + /* + * If the node still spans pages (especially ZONE_DEVICE), don't + * offline it. A node spans memory after move_pfn_range_to_zone(), + * e.g., after the memory block was onlined. + */ + if (pgdat->node_spanned_pages) + return; + + /* + * Especially offline memory blocks might not be spanned by the + * node. They will get spanned by the node once they get onlined. + * However, they link to the node in sysfs and can get onlined later. + */ + rc =3D for_each_memory_block(&nid, check_no_memblock_for_node_cb); + if (rc) return; - } =20 if (check_and_unmap_cpu_on_node(pgdat)) return; Patches currently in stable-queue which might be from david@redhat.com ar= e queue-4.19/mm-memory_hotplug-remove-memory-block-devices-before-arch_remo= ve_memory.patch queue-4.19/mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch queue-4.19/mm-memory_hotplug-create-memory-block-devices-after-arch_add_m= emory.patch queue-4.19/drivers-base-memory-pass-a-block_id-to-init_memory_block.patch queue-4.19/drivers-base-memory.c-clean-up-relics-in-function-parameters.p= atch queue-4.19/mm-memory_hotplug-update-a-comment-in-unregister_memory.patch queue-4.19/mm-memory_hotplug-shrink-zones-when-offlining-memory.patch queue-4.19/mm-memory_hotplug-make-unregister_memory_block_under_nodes-nev= er-fail.patch queue-4.19/mm-memunmap-don-t-access-uninitialized-memmap-in-memunmap_page= s.patch queue-4.19/mm-memory_hotplug-make-__remove_section-never-fail.patch queue-4.19/mm-sparse-drop-pgdat_resize_lock-in-sparse_add-remove_one_sect= ion.patch queue-4.19/mm-memory_hotplug-make-unregister_memory_section-never-fail.pa= tch queue-4.19/mm-memory_hotplug-release-memory-resource-after-arch_remove_me= mory.patch queue-4.19/mm-memory_hotplug-add-nid-parameter-to-arch_remove_memory.patc= h queue-4.19/powerpc-mm-fix-section-mismatch-warning.patch queue-4.19/mm-memory_hotplug-allow-arch_remove_memory-without-config_memo= ry_hotremove.patch queue-4.19/mm-memory_hotplug-make-remove_memory-take-the-device_hotplug_l= ock.patch queue-4.19/mm-memory_hotplug-make-__remove_pages-and-arch_remove_memory-n= ever-fail.patch queue-4.19/mm-memory_hotplug-fix-try_offline_node.patch queue-4.19/mm-memory_hotplug-remove-zone-parameter-from-sparse_remove_one= _section.patch queue-4.19/drivers-base-memory.c-remove-an-unnecessary-check-on-nr_mem_se= ctions.patch queue-4.19/drivers-base-node.c-simplify-unregister_memory_block_under_nod= es.patch queue-4.19/s390x-mm-implement-arch_remove_memory.patch queue-4.19/mm-sparse-pass-nid-instead-of-pgdat-to-sparse_add_one_section.= patch