From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8183C433F5 for ; Wed, 30 Mar 2022 14:04:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 791318D0002; Wed, 30 Mar 2022 10:04:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F2978D0001; Wed, 30 Mar 2022 10:04:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BA8B8D0002; Wed, 30 Mar 2022 10:04:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 4CF6E8D0001 for ; Wed, 30 Mar 2022 10:04:36 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2041D21C23 for ; Wed, 30 Mar 2022 14:04:36 +0000 (UTC) X-FDA: 79301222952.09.738119D Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf24.hostedemail.com (Postfix) with ESMTP id 6E28E180021 for ; Wed, 30 Mar 2022 14:04:35 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6769C21613; Wed, 30 Mar 2022 14:04:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1648649074; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3HMcf6VpPdUtSHMVGDhdxTY9zIcILTXot1rIgQ5P1hw=; b=ef6F+hrxT7jsNU5wyGaDpWxZI/bcW+2pKvn+iRcd8z7sWGdhsLkagx60oPooXJve1pi+Y9 FPnRPPjwFAghwQpOSMuNv4+areI5A0klDnvvC2Mqlz2k6gttuEgRDihl0sDgfC1IO224Vu OdzOSjrvy29y0MNdWyihG3F6s/zHGcY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1648649074; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3HMcf6VpPdUtSHMVGDhdxTY9zIcILTXot1rIgQ5P1hw=; b=R4xgioPBIA28fZdQiFSKgD2Pu95fR2in5TKCC19Wxsz5AkvxXc3d9MYefMe6NeZTcEqCeJ zLS3RNxUBQRlRQDA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 09B9E13A60; Wed, 30 Mar 2022 14:04:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id kboWO3FjRGJyagAAMHmgww (envelope-from ); Wed, 30 Mar 2022 14:04:33 +0000 Date: Wed, 30 Mar 2022 16:04:32 +0200 From: Oscar Salvador To: Srikar Dronamraju Cc: Michael Ellerman , linuxppc-dev , linux-mm@kvack.org, Michal Hocko , Geetika Moolchandani Subject: Re: [PATCH] powerpc/numa: Handle partially initialized numa nodes Message-ID: References: <20220330135123.1868197-1-srikar@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220330135123.1868197-1-srikar@linux.vnet.ibm.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6E28E180021 X-Stat-Signature: 5ddwj7xcxxrwww8wui5stf65bepjwmfi X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ef6F+hrx; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=R4xgioPB; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf24.hostedemail.com: domain of osalvador@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=osalvador@suse.de X-HE-Tag: 1648649075-335741 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 30, 2022 at 07:21:23PM +0530, Srikar Dronamraju wrote: > With commit 09f49dca570a ("mm: handle uninitialized numa nodes > gracefully") NODE_DATA for even a memoryless/cpuless node is partially > initialized at boot time. > > Before onlining the node, current Powerpc code checks for NODE_DATA to > be NULL. However since NODE_DATA is partially initialized, this check > will end up always being false. > > This causes hotplugging a CPU to a memoryless/cpuless node to fail. > > Before adding CPUs > $ numactl -H > available: 1 nodes (4) > node 4 cpus: 0 1 2 3 4 5 6 7 > node 4 size: 97372 MB > node 4 free: 95545 MB > node distances: > node 4 > 4: 10 > > $ lparstat > System Configuration > type=Dedicated mode=Capped smt=8 lcpu=1 mem=99709440 kB cpus=0 ent=1.00 > > %user %sys %wait %idle physc %entc lbusy app vcsw phint > ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- > 2.66 2.67 0.16 94.51 0.00 0.00 5.33 0.00 67749 0 > > After hotplugging 32 cores > $ numactl -H > node 4 cpus: 0 1 2 3 4 5 6 7 120 121 122 123 124 125 126 127 128 129 130 > 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 > 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 > 167 168 169 170 171 172 173 174 175 > node 4 size: 97372 MB > node 4 free: 93636 MB > node distances: > node 4 > 4: 10 > > $ lparstat > System Configuration > type=Dedicated mode=Capped smt=8 lcpu=33 mem=99709440 kB cpus=0 ent=33.00 > > %user %sys %wait %idle physc %entc lbusy app vcsw phint > ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- > 0.04 0.02 0.00 99.94 0.00 0.00 0.06 0.00 1128751 3 > > As we can see numactl is listing only 8 cores while lparstat is showing > 33 cores. > > Also dmesg is showing messages like: > [ 2261.318350 ] BUG: arch topology borken > [ 2261.318357 ] the DIE domain not a subset of the NODE domain > > Fixes: 09f49dca570a ("mm: handle uninitialized numa nodes gracefully") > Cc: linuxppc-dev@lists.ozlabs.org > Cc: linux-mm@kvack.org > Cc: Michal Hocko > Cc: Michael Ellerman > Reported-by: Geetika Moolchandani > Signed-off-by: Srikar Dronamraju Acked-by: Oscar Salvador Thanks Srikar! > --- > arch/powerpc/mm/numa.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > index b9b7fefbb64b..13022d734951 100644 > --- a/arch/powerpc/mm/numa.c > +++ b/arch/powerpc/mm/numa.c > @@ -1436,7 +1436,7 @@ int find_and_online_cpu_nid(int cpu) > if (new_nid < 0 || !node_possible(new_nid)) > new_nid = first_online_node; > > - if (NODE_DATA(new_nid) == NULL) { > + if (!node_online(new_nid)) { > #ifdef CONFIG_MEMORY_HOTPLUG > /* > * Need to ensure that NODE_DATA is initialized for a node from > -- > 2.27.0 > > -- Oscar Salvador SUSE Labs