From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41BF4C2D0DB for ; Tue, 28 Jan 2020 13:50:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD8C424695 for ; Tue, 28 Jan 2020 13:50:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="DwE/ilCE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD8C424695 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 240946B0271; Tue, 28 Jan 2020 08:50:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CA4A6B0272; Tue, 28 Jan 2020 08:50:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3AB06B0273; Tue, 28 Jan 2020 08:50:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id CE7126B0271 for ; Tue, 28 Jan 2020 08:50:06 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 9C307180AD801 for ; Tue, 28 Jan 2020 13:50:06 +0000 (UTC) X-FDA: 76427176812.22.cord40_3b1c1ef4f3f39 X-HE-Tag: cord40_3b1c1ef4f3f39 X-Filterd-Recvd-Size: 14087 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Tue, 28 Jan 2020 13:50:06 +0000 (UTC) Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DCD562468A; Tue, 28 Jan 2020 13:50:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1580219405; bh=kePvq7CyFh5Tnd6QDxW6j/AYclAEz97YcZLrFdVP6ew=; h=Subject:To:Cc:From:Date:In-Reply-To:From; b=DwE/ilCEWBSCaOH+IhCkhbEoLcsyydxjI9yMg61plH4UCTiFDTodi35dtmKwNa3iO IOJRJt6bJv6tNIUkJ3Iv0yNXgCLcQLFsfKPdkL4kkp51S+LYiV/7RTkMAOcMeeCv+8 hOME5qT98eYBT/3moTP7hD54YaFQQN7c5zjjJGb8= Subject: Patch "mm/memory_hotplug: make remove_memory() take the device_hotplug_lock" has been added to the 4.19-stable tree To: 20180925091457.28651-2-david@redhat.com,akpm@linux-foundation.org,aneesh.kumar@linux.ibm.com,benh@kernel.crashing.org,bhe@redhat.com,boris.ostrovsky@oracle.com,bsingharora@gmail.com,corbet@lwn.net,dan.j.williams@intel.com,david@redhat.com,gregkh@linuxfoundation.org,haiyangz@microsoft.com,heiko.carstens@de.ibm.com,iamjoonsoo.kim@lge.com,jallen@linux.vnet.ibm.com,jgross@suse.com,kstewart@linuxfoundation.org,kys@microsoft.com,lenb@kernel.org,linux-mm@kvack.org,malat@debian.org,mhocko@suse.com,mikey@neuling.org,mpe@ellerman.id.au,nfont@linux.vnet.ibm.com,osalvador@suse.de,paulus@samba.org,pavel.tatashin@microsoft.com,pombredanne@nexb.com,rafael.j.wysocki@intel.com,rashmica.g@gmail.com,richard.weiyang@gmail.com,rjw@rjwysocki.net,schwidefsky@de.ibm.com,sthemmin@microsoft.com,tglx@linutronix.de,torvalds@linux-foundation.org,vbabka@suse.cz,yasu.isimatu@gmail.com Cc: From: Date: Tue, 28 Jan 2020 14:49:42 +0100 In-Reply-To: <20200128095021.8076-2-david@redhat.com> Message-ID: <158021938299109@kroah.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ANSI_X3.4-1968 X-stable: commit X-Patchwork-Hint: ignore Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a note to let you know that I've just added the patch titled mm/memory_hotplug: make remove_memory() take the device_hotplug_lock to the 4.19-stable tree which can be found at: http://www.kernel.org/git/?p=3Dlinux/kernel/git/stable/stable-queue.g= it;a=3Dsummary The filename of the patch is: mm-memory_hotplug-make-remove_memory-take-the-device_hotplug_lock.pa= tch and it can be found in the queue-4.19 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let know about it. >From foo@baz Tue 28 Jan 2020 02:32:10 PM CET From: David Hildenbrand Date: Tue, 28 Jan 2020 10:49:58 +0100 Subject: mm/memory_hotplug: make remove_memory() take the device_hotplug_= lock To: stable@vger.kernel.org Cc: linux-mm@kvack.org, Michal Hocko , Greg Kroah-Hartma= n , Andrew Morton = , "Aneesh Kumar K . V" , Baoquan He , Dan Williams , Oscar Salvador , Wei Yang , David Hildenbrand Message-ID: <20200128095021.8076-2-david@redhat.com> From: David Hildenbrand commit d15e59260f62bd5e0f625cf5f5240f6ffac78ab6 upstream. Patch series "mm: online/offline_pages called w.o. mem_hotplug_lock", v3. Reading through the code and studying how mem_hotplug_lock is to be used, I noticed that there are two places where we can end up calling device_online()/device_offline() - online_pages()/offline_pages() without the mem_hotplug_lock. And there are other places where we call device_online()/device_offline() without the device_hotplug_lock. While e.g. echo "online" > /sys/devices/system/memory/memory9/state is fine, e.g. echo 1 > /sys/devices/system/memory/memory9/online Will not take the mem_hotplug_lock. However the device_lock() and device_hotplug_lock. E.g. via memory_probe_store(), we can end up calling add_memory()->online_pages() without the device_hotplug_lock. So we can have concurrent callers in online_pages(). We e.g. touch in online_pages() basically unprotected zone->present_pages then. Looks like there is a longer history to that (see Patch #2 for details), and fixing it to work the way it was intended is not really possible. We would e.g. have to take the mem_hotplug_lock in device/base/core.c, whic= h sounds wrong. Summary: We had a lock inversion on mem_hotplug_lock and device_lock(). More details can be found in patch 3 and patch 6. I propose the general rules (documentation added in patch 6): 1. add_memory/add_memory_resource() must only be called with device_hotplug_lock. 2. remove_memory() must only be called with device_hotplug_lock. This is already documented and holds for all callers. 3. device_online()/device_offline() must only be called with device_hotplug_lock. This is already documented and true for now in co= re code. Other callers (related to memory hotplug) have to be fixed up. 4. mem_hotplug_lock is taken inside of add_memory/remove_memory/ online_pages/offline_pages. To me, this looks way cleaner than what we have right now (and easier to verify). And looking at the documentation of remove_memory, using lock_device_hotplug also for add_memory() feels natural. This patch (of 6): remove_memory() is exported right now but requires the device_hotplug_lock, which is not exported. So let's provide a variant that takes the lock and only export that one. The lock is already held in arch/powerpc/platforms/pseries/hotplug-memory.c drivers/acpi/acpi_memhotplug.c arch/powerpc/platforms/powernv/memtrace.c Apart from that, there are not other users in the tree. Link: http://lkml.kernel.org/r/20180925091457.28651-2-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Pavel Tatashin Reviewed-by: Rafael J. Wysocki Reviewed-by: Rashmica Gupta Reviewed-by: Oscar Salvador Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: Rashmica Gupta Cc: Michael Neuling Cc: Balbir Singh Cc: Nathan Fontenot Cc: John Allen Cc: Michal Hocko Cc: Dan Williams Cc: Joonsoo Kim Cc: Vlastimil Babka Cc: Greg Kroah-Hartman Cc: YASUAKI ISHIMATSU Cc: Mathieu Malaterre Cc: Boris Ostrovsky Cc: Haiyang Zhang Cc: Heiko Carstens Cc: Jonathan Corbet Cc: Juergen Gross Cc: Kate Stewart Cc: "K. Y. Srinivasan" Cc: Martin Schwidefsky Cc: Philippe Ombredanne Cc: Stephen Hemminger Cc: Thomas Gleixner Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: David Hildenbrand Signed-off-by: Greg Kroah-Hartman --- arch/powerpc/platforms/powernv/memtrace.c | 2 +- arch/powerpc/platforms/pseries/hotplug-memory.c | 6 +++--- drivers/acpi/acpi_memhotplug.c | 2 +- include/linux/memory_hotplug.h | 3 ++- mm/memory_hotplug.c | 9 ++++++++- 5 files changed, 15 insertions(+), 7 deletions(-) --- a/arch/powerpc/platforms/powernv/memtrace.c +++ b/arch/powerpc/platforms/powernv/memtrace.c @@ -122,7 +122,7 @@ static u64 memtrace_alloc_node(u32 nid, */ end_pfn =3D base_pfn + nr_pages; for (pfn =3D base_pfn; pfn < end_pfn; pfn +=3D bytes>> PAGE_SHIFT) { - remove_memory(nid, pfn << PAGE_SHIFT, bytes); + __remove_memory(nid, pfn << PAGE_SHIFT, bytes); } unlock_device_hotplug(); return base_pfn << PAGE_SHIFT; --- a/arch/powerpc/platforms/pseries/hotplug-memory.c +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c @@ -301,7 +301,7 @@ static int pseries_remove_memblock(unsig nid =3D memory_add_physaddr_to_nid(base); =20 for (i =3D 0; i < sections_per_block; i++) { - remove_memory(nid, base, MIN_MEMORY_BLOCK_SIZE); + __remove_memory(nid, base, MIN_MEMORY_BLOCK_SIZE); base +=3D MIN_MEMORY_BLOCK_SIZE; } =20 @@ -393,7 +393,7 @@ static int dlpar_remove_lmb(struct drmem block_sz =3D pseries_memory_block_size(); nid =3D memory_add_physaddr_to_nid(lmb->base_addr); =20 - remove_memory(nid, lmb->base_addr, block_sz); + __remove_memory(nid, lmb->base_addr, block_sz); =20 /* Update memory regions for memory remove */ memblock_remove(lmb->base_addr, block_sz); @@ -680,7 +680,7 @@ static int dlpar_add_lmb(struct drmem_lm =20 rc =3D dlpar_online_lmb(lmb); if (rc) { - remove_memory(nid, lmb->base_addr, block_sz); + __remove_memory(nid, lmb->base_addr, block_sz); invalidate_lmb_associativity_index(lmb); } else { lmb->flags |=3D DRCONF_MEM_ASSIGNED; --- a/drivers/acpi/acpi_memhotplug.c +++ b/drivers/acpi/acpi_memhotplug.c @@ -282,7 +282,7 @@ static void acpi_memory_remove_memory(st nid =3D memory_add_physaddr_to_nid(info->start_addr); =20 acpi_unbind_memory_blocks(info); - remove_memory(nid, info->start_addr, info->length); + __remove_memory(nid, info->start_addr, info->length); list_del(&info->list); kfree(info); } --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -303,6 +303,7 @@ extern bool is_mem_section_removable(uns extern void try_offline_node(int nid); extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages= ); extern void remove_memory(int nid, u64 start, u64 size); +extern void __remove_memory(int nid, u64 start, u64 size); =20 #else static inline bool is_mem_section_removable(unsigned long pfn, @@ -319,6 +320,7 @@ static inline int offline_pages(unsigned } =20 static inline void remove_memory(int nid, u64 start, u64 size) {} +static inline void __remove_memory(int nid, u64 start, u64 size) {} #endif /* CONFIG_MEMORY_HOTREMOVE */ =20 extern void __ref free_area_init_core_hotplug(int nid); @@ -333,7 +335,6 @@ extern void move_pfn_range_to_zone(struc unsigned long nr_pages, struct vmem_altmap *altmap); extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages= ); extern bool is_memblock_offlined(struct memory_block *mem); -extern void remove_memory(int nid, u64 start, u64 size); extern int sparse_add_one_section(struct pglist_data *pgdat, unsigned long start_pfn, struct vmem_altmap *altmap); extern void sparse_remove_one_section(struct zone *zone, struct mem_sect= ion *ms, --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1893,7 +1893,7 @@ EXPORT_SYMBOL(try_offline_node); * and online/offline operations before this call, as required by * try_offline_node(). */ -void __ref remove_memory(int nid, u64 start, u64 size) +void __ref __remove_memory(int nid, u64 start, u64 size) { int ret; =20 @@ -1922,5 +1922,12 @@ void __ref remove_memory(int nid, u64 st =20 mem_hotplug_done(); } + +void remove_memory(int nid, u64 start, u64 size) +{ + lock_device_hotplug(); + __remove_memory(nid, start, size); + unlock_device_hotplug(); +} EXPORT_SYMBOL_GPL(remove_memory); #endif /* CONFIG_MEMORY_HOTREMOVE */ Patches currently in stable-queue which might be from david@redhat.com ar= e queue-4.19/mm-memory_hotplug-remove-memory-block-devices-before-arch_remo= ve_memory.patch queue-4.19/mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch queue-4.19/mm-memory_hotplug-create-memory-block-devices-after-arch_add_m= emory.patch queue-4.19/drivers-base-memory-pass-a-block_id-to-init_memory_block.patch queue-4.19/drivers-base-memory.c-clean-up-relics-in-function-parameters.p= atch queue-4.19/mm-memory_hotplug-update-a-comment-in-unregister_memory.patch queue-4.19/mm-memory_hotplug-shrink-zones-when-offlining-memory.patch queue-4.19/mm-memory_hotplug-make-unregister_memory_block_under_nodes-nev= er-fail.patch queue-4.19/mm-memunmap-don-t-access-uninitialized-memmap-in-memunmap_page= s.patch queue-4.19/mm-memory_hotplug-make-__remove_section-never-fail.patch queue-4.19/mm-sparse-drop-pgdat_resize_lock-in-sparse_add-remove_one_sect= ion.patch queue-4.19/mm-memory_hotplug-make-unregister_memory_section-never-fail.pa= tch queue-4.19/mm-memory_hotplug-release-memory-resource-after-arch_remove_me= mory.patch queue-4.19/mm-memory_hotplug-add-nid-parameter-to-arch_remove_memory.patc= h queue-4.19/powerpc-mm-fix-section-mismatch-warning.patch queue-4.19/mm-memory_hotplug-allow-arch_remove_memory-without-config_memo= ry_hotremove.patch queue-4.19/mm-memory_hotplug-make-remove_memory-take-the-device_hotplug_l= ock.patch queue-4.19/mm-memory_hotplug-make-__remove_pages-and-arch_remove_memory-n= ever-fail.patch queue-4.19/mm-memory_hotplug-fix-try_offline_node.patch queue-4.19/mm-memory_hotplug-remove-zone-parameter-from-sparse_remove_one= _section.patch queue-4.19/drivers-base-memory.c-remove-an-unnecessary-check-on-nr_mem_se= ctions.patch queue-4.19/drivers-base-node.c-simplify-unregister_memory_block_under_nod= es.patch queue-4.19/s390x-mm-implement-arch_remove_memory.patch queue-4.19/mm-sparse-pass-nid-instead-of-pgdat-to-sparse_add_one_section.= patch