From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B291C433F5 for ; Mon, 11 Apr 2022 08:58:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C1806B0072; Mon, 11 Apr 2022 04:58:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 549FC6B0073; Mon, 11 Apr 2022 04:58:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C2D26B0074; Mon, 11 Apr 2022 04:58:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id 290446B0072 for ; Mon, 11 Apr 2022 04:58:22 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CBB55A5A88 for ; Mon, 11 Apr 2022 08:58:21 +0000 (UTC) X-FDA: 79343996802.30.A75C861 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf03.hostedemail.com (Postfix) with ESMTP id 0F1602000A for ; Mon, 11 Apr 2022 08:58:20 +0000 (UTC) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23B7Bu06005959; Mon, 11 Apr 2022 08:58:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc : subject : message-id : reply-to : references : content-type : in-reply-to : mime-version; s=pp1; bh=Sh4ProLqdUO6KdTRPfLnIRtU0OE9sHmn3EZtxlgOFcE=; b=kT6FdqCOoVthObne5m9V2Fig41yg0mPQZLkMkeHnfPrSUmFwxyI1a/p9dkHztwxkas35 QWrFdL8qF/HokaZFA1KBI3tP5FhJfPma30LVmZbBpzLWgvAleWN9RgHy1STXI4bC6kFf kXBxT3wk8RCQdrGRI4gnBohpRXVp1iGC5X//aM5o45vpJ4VhOrui8E7DgmSVFsVXYRLe KwZALeADbqTjqaIS3WaoRmCq+0W00XPlRvpncRb2S3cBINMhK4hgnl6fhKJBI59h2SbO K1LMYByPX6jp/NeamaPhAhvr3hpdpuXT7qaFsIZBENTHF6d/sQ+iFvzdBoKneDtG/Iui Pg== Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com [149.81.74.107]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fcfqp1xdm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Apr 2022 08:58:14 +0000 Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1]) by ppma03fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23B8w1kp000941; Mon, 11 Apr 2022 08:58:12 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma03fra.de.ibm.com with ESMTP id 3fb1s8j6eq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Apr 2022 08:58:12 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23B8jhNQ48300444 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Apr 2022 08:45:43 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BFD69A404D; Mon, 11 Apr 2022 08:58:09 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9A307A405E; Mon, 11 Apr 2022 08:58:08 +0000 (GMT) Received: from linux.vnet.ibm.com (unknown [9.40.195.195]) by d06av23.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 11 Apr 2022 08:58:08 +0000 (GMT) Date: Mon, 11 Apr 2022 14:28:08 +0530 From: Srikar Dronamraju To: Oscar Salvador Cc: Michael Ellerman , linuxppc-dev , Geetika Moolchandani , Michal Hocko , linux-mm@kvack.org Subject: Re: [PATCH] powerpc/numa: Associate numa node to its cpu earlier Message-ID: <20220411085808.GH568950@linux.vnet.ibm.com> Reply-To: Srikar Dronamraju References: <20220411074934.4632-1-osalvador@suse.de> Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20220411074934.4632-1-osalvador@suse.de> X-TM-AS-GCONF: 00 X-Proofpoint-GUID: TJf2HUrhuhphRva9ryOG2drSScOn9aUG X-Proofpoint-ORIG-GUID: TJf2HUrhuhphRva9ryOG2drSScOn9aUG X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-11_02,2022-04-08_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 impostorscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204110045 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=kT6FdqCO; spf=none (imf03.hostedemail.com: domain of srikar@linux.vnet.ibm.com has no SPF policy when checking 148.163.156.1) smtp.mailfrom=srikar@linux.vnet.ibm.com; dmarc=pass (policy=none) header.from=ibm.com X-Stat-Signature: tzdns1hd3s4oisetywsfbpfra61hm5ye X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0F1602000A X-HE-Tag: 1649667500-111338 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: * Oscar Salvador [2022-04-11 09:49:34]: > powerpc is the only platform that do not rely on > cpu_up()->try_online_node() to bring up a numa node, > and special cases it, instead, deep in its own machinery: > > dlpar_online_cpu > find_and_online_cpu_nid > try_online_node > > This should not be needed, but the thing is that the try_online_node() > from cpu_up() will not apply on the right node, because cpu_to_node() > will return the old mapping numa<->cpu that gets set on boot stage > for all possible cpus. > > That can be seen easily if we try to print out the numa node passed > to try_online_node() in cpu_up(). > > The thing is that the numa<->cpu mapping does not get updated till a much > later stage in start_secondary: > > start_secondary: > set_numa_node(numa_cpu_lookup_table[cpu]) > > But we do not really care, as we already now the > CPU <-> NUMA associativity back in find_and_online_cpu_nid(), > so let us make use of that and set the proper numa<->cpu mapping, > so cpu_to_node() in cpu_up() returns the right node and > try_online_node() can do its work. > > Signed-off-by: Oscar Salvador > Reviewed-by: Srikar Dronamraju > Tested-by: Geetika Moolchandani > --- > arch/powerpc/include/asm/topology.h | 8 ++----- > arch/powerpc/mm/numa.c | 31 +++++++--------------------- > arch/powerpc/platforms/pseries/hotplug-cpu.c | 2 +- > 3 files changed, 11 insertions(+), 30 deletions(-) > > diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h > index 36fcafb1fd6d..6ae1b2dce83e 100644 > --- a/arch/powerpc/include/asm/topology.h > +++ b/arch/powerpc/include/asm/topology.h > @@ -111,14 +111,10 @@ static inline void unmap_cpu_from_node(unsigned long cpu) {} > #endif /* CONFIG_NUMA */ > > #if defined(CONFIG_NUMA) && defined(CONFIG_PPC_SPLPAR) > -extern int find_and_online_cpu_nid(int cpu); > +extern void find_and_update_cpu_nid(int cpu); > extern int cpu_to_coregroup_id(int cpu); > #else > -static inline int find_and_online_cpu_nid(int cpu) > -{ > - return 0; > -} > - > +static inline void find_and_update_cpu_nid(int cpu) {} > static inline int cpu_to_coregroup_id(int cpu) > { > #ifdef CONFIG_SMP > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > index b9b7fefbb64b..b5bc8b1a833d 100644 > --- a/arch/powerpc/mm/numa.c > +++ b/arch/powerpc/mm/numa.c > @@ -1423,43 +1423,28 @@ static long vphn_get_associativity(unsigned long cpu, > return rc; > } > > -int find_and_online_cpu_nid(int cpu) > +void find_and_update_cpu_nid(int cpu) > { > __be32 associativity[VPHN_ASSOC_BUFSIZE] = {0}; > int new_nid; > > /* Use associativity from first thread for all siblings */ > if (vphn_get_associativity(cpu, associativity)) > - return cpu_to_node(cpu); > + return; > > + /* Do not have previous associativity, so find it now. */ > new_nid = associativity_to_nid(associativity); > + > if (new_nid < 0 || !node_possible(new_nid)) > new_nid = first_online_node; > - > - if (NODE_DATA(new_nid) == NULL) { Given that my patch got accepted into powerpc tree https://git.kernel.org/powerpc/c/e4ff77598a109bd36789ad5e80aba66fc53d0ffb is now part of Linus tree, this line may need a slight tweak. > -#ifdef CONFIG_MEMORY_HOTPLUG > - /* > - * Need to ensure that NODE_DATA is initialized for a node from > - * available memory (see memblock_alloc_try_nid). If unable to > - * init the node, then default to nearest node that has memory > - * installed. Skip onlining a node if the subsystems are not > - * yet initialized. > - */ > - if (!topology_inited || try_online_node(new_nid)) > - new_nid = first_online_node; > -#else > - /* > - * Default to using the nearest node that has memory installed. > - * Otherwise, it would be necessary to patch the kernel MM code > - * to deal with more memoryless-node error conditions. > + else > + /* Associate node <-> cpu, so cpu_up() calls > + * try_online_node() on the right node. > */ > - new_nid = first_online_node; > -#endif > - } > + set_cpu_numa_node(cpu, new_nid); > > pr_debug("%s:%d cpu %d nid %d\n", __FUNCTION__, __LINE__, > cpu, new_nid); > - return new_nid; > } > > int cpu_to_coregroup_id(int cpu) > diff --git a/arch/powerpc/platforms/pseries/hotplug-cpu.c b/arch/powerpc/platforms/pseries/hotplug-cpu.c > index b81fc846d99c..0f8cd8b06432 100644 > --- a/arch/powerpc/platforms/pseries/hotplug-cpu.c > +++ b/arch/powerpc/platforms/pseries/hotplug-cpu.c > @@ -398,7 +398,7 @@ static int dlpar_online_cpu(struct device_node *dn) > if (get_hard_smp_processor_id(cpu) != thread) > continue; > cpu_maps_update_done(); > - find_and_online_cpu_nid(cpu); > + find_and_update_cpu_nid(cpu); > rc = device_online(get_cpu_device(cpu)); > if (rc) { > dlpar_offline_cpu(dn); > -- > 2.16.4 >