From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04F41C369DC for ; Thu, 1 May 2025 13:56:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DAD9B6B0092; Thu, 1 May 2025 09:56:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D5EB16B0093; Thu, 1 May 2025 09:56:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C24016B0096; Thu, 1 May 2025 09:56:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A17706B0092 for ; Thu, 1 May 2025 09:56:06 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A8D6C80B84 for ; Thu, 1 May 2025 13:56:07 +0000 (UTC) X-FDA: 83394487974.07.44FAC01 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf09.hostedemail.com (Postfix) with ESMTP id 3D752140009 for ; Thu, 1 May 2025 13:56:05 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=isctNGub; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746107765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FdABLbGXU3xjbqcTCkxai/8AK3Ss9JYpvaFeHCIzyVM=; b=WvnWWL7O7AoGO39nJ6yfoSnizG3nnq8FGssI+O0iqMqWU5aC0Kkg/jdAeE4voIPTUUlilQ 9JxHdUV/3FOR60sQTkwPA1wPWScObHMDT2Lq2rVFluKxIJXr6gkxfvf0m4in88R8FoubwB bQvfNlyi8zlukzSwaA3wG65uC8NaKTw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746107765; a=rsa-sha256; cv=none; b=5GLk6/pRDhSRL3qlpfmb6xjQrWBQ7+E+A4wJHMjmbocv2cWN27lMGjZc2vzlJEh0ky6Xzs NDlD4Cy5V4vQnlPIBhbdIJJSVvJGtsqjUHp/WKmzjANMzFz8AUEuUBRGswoR7NKu1znmDL NY6NNpVYAjbui8st5FWzmCY+qfztwjU= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=isctNGub; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 541Bp0Co011326; Thu, 1 May 2025 13:55:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=FdABLb GXU3xjbqcTCkxai/8AK3Ss9JYpvaFeHCIzyVM=; b=isctNGub2RIa4/fRulxz4X ZUkoeQHvKXrP9zn7wvmIX8ZLlLGs8zIDxLx0pwG33WOLw5e6wduuhcceRG37zygj 2JszmqR5NGmsLoLLQ9g5jCfBC6KgRfEQm1qLF6jPw05mTB5GpQV9tKGubckdI9jN xupc5naLgXRSupi4JF56rVZVBG8HL+GZbkls8FhGGwSRZ1qMbJj+moX74LXEOCgB WK+YOCNCXeWJXj0Wm8ZjRlHuyGVyf8zTCg4+dAs6yMszXOrbRULtIjXLb0w0gmBD DgWAI1ilR+vovEvVQulT3fwM0B4i9wXmpqpsBqIwEzLpDpHb9zCeQv1xF9RvdtAg == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 46c7ds90aw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 May 2025 13:55:51 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 541Dnp4L023350; Thu, 1 May 2025 13:55:50 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 46c7ds90as-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 May 2025 13:55:50 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 541DbFin031662; Thu, 1 May 2025 13:55:49 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([172.16.1.7]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4699tudgkf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 May 2025 13:55:49 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 541DtniD30278344 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 1 May 2025 13:55:49 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6177058057; Thu, 1 May 2025 13:55:49 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D9C5358059; Thu, 1 May 2025 13:55:44 +0000 (GMT) Received: from [9.39.16.182] (unknown [9.39.16.182]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 1 May 2025 13:55:44 +0000 (GMT) Message-ID: <719c495f-d514-43f5-814d-e956e149f836@linux.ibm.com> Date: Thu, 1 May 2025 19:25:43 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] driver/base: Optimize memory block registration to reduce boot time To: Oscar Salvador Cc: Mike Rapoport , David Hildenbrand , Greg Kroah-Hartman , Andrew Morton , rafael@kernel.org, Danilo Krummrich , Ritesh Harjani , Jonathan Cameron , Alison Schofield , Yury Norov , Dave Jiang , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: Content-Language: en-US From: Donet Tom In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=afZhnQot c=1 sm=1 tr=0 ts=68137d67 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=IkcTkHD0fZMA:10 a=dt9VzEwgFbYA:10 a=w-oE209dy2xqcs4eIbgA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: DEm5KhvvTcQ4t8u_nclR-1t6-z4GG_AB X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNTAxMDEwMyBTYWx0ZWRfXzGNSYvEEJuHX RTsgRPQzK5bSc4xeV+2rl3JyqxqjMJlwPOWwS/C3FmLM4x1mupIAJ0h2jJn5x6Bv6b1jk1kSlbS aDMTgi96AhkKC663dyFRfO+yuwmhgS5ccagJ7ixin8drnDww5ZSqFBz0Dm1ebGDM+Ej3msIcXEK q6jcNwQIsinUT4wHk7bDBT7AwYNPnEUR/zuFaQBT/4CtC+RqGn+wvVKgUZUTCJLqS5PAXeEYRFk tPn1ec8KyFz+lYLjnmUDjGXJfaBNh2gwrTy6cpcN4mJeslj3OTFMTUf6UYu/Cpjzgg3lJ9XPE1d csbtCDw/sXjoVFR0NP6Rxg4aY7tDk5MBSzNl95rKkd0Y6cWgcJPsXrwAfbmenNTtDTMIg5PPsN5 xN88ia+NQzF4fCiE/aCoX6NNsIv00wp3Ui1ffm7C4EY2B8g1K/NWh3m0W/7f+v7HQnFN4QAA X-Proofpoint-GUID: EjgKDxAJQet6KACIEkPTCDye5aidXw1R X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-05-01_05,2025-04-24_02,2025-02-21_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 malwarescore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 clxscore=1015 phishscore=0 adultscore=0 mlxscore=0 mlxlogscore=978 suspectscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2504070000 definitions=main-2505010103 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3D752140009 X-Rspam-User: X-Stat-Signature: na9ys1oofwbqdtwggu4o494r1k7i973z X-HE-Tag: 1746107765-502158 X-HE-Meta: U2FsdGVkX18UW5syOWqZ8l3uF3MyE8ooHNkHMEa3rCMSodTECKJaGgU4tD1ky0CR5YrzU4J2cVgipCiXfxTmgPRAZlnRHrcnm/L/jSSqLHpx4DsND0mqvCfF6DyvLi0M+s2XIrVOLbgwtM+ObhXlXUe2LA4ND5Z8tFGHRAF/qQ2rAf0ODvqdVm9BHFgUREegWfEKZeGbCRiBjf3twfdncXjLDdSwhs6oHDfQmCKA5F+IfzDX2Vk3HyDcu1Rybnkzku5gpKfaH9Mdt3F6pfbe2Gs4QEnCTDkPURMA5HR4i8Bdksq+RrXsG+KPQ1a85JTBaPweo92g3CZn17nEd4Xlin3iB8gMfiFUmDD9+pCo/dYIDBOYT1tzznXsggArJrYz/bUO6Ob6RgkeTCLiNHN5+1YBejX1Me2kJGG/8VBVh1aW6+kuwZ6o0v2bfoOFIH+86CjsMy9oewy76WnfUCZoWuJoxHlpaO7RNOO3//duYYIZ1yUgn42mMG1QsIqqvO9FxCIUhQVFAZd6Q+cXDkObPn7fsmK4x/OOiTS/xHWPsyPRJn5DX56+JYmAOLRa2QICY42OsAFNRi7pAeAkxmUc/a+ylbl8xkA4VcHvET9uhZ/cFkzf04eS1Cz+XVVfLEoioZPnIAmSTD8tOpk/LKuV3Y54ljq61h7yDIySOhbcAaioJgRZvRTxSqCynUc1UNOwz1tH1Tc/c9QXmKnBnKPr8TDCS9gTmy1r3RvWb3066YkbWoOq4h5VUO8FN0mustdrviH+wGZTuENZz2ueN69RuBvhAFiknZorMvYuC2GzI8h7PTbPb2GGlpAc94M4fxJ2ocisMBm3aripGFZPYdnDNqpPJfN4WlztG2nqHC+pblsqU/h7SamseRErwo0moZXNbeyvYjHOc7vVzDu/jkZpPIeN8MT+yZbSNAG3qWUa0qCLuVH4qABgEgWa5nZ03V8Mxynk9VR7rIJHyxNha2t WjYeeSIZ FGgewXDsf9nwZzvXwovejqTPv/c7GbQVTMpsW/fpkSXH0EwHZbR5bi4mxmP9Hb/0ZIDnisciO+jt9jl2UFKSw5NKs2uDuIok2YF4ecdmx2YdBX+Bdx7Zz21OeDC57c7TZ6i7PA8qRvWMcGThKbRGTerWjQTARLA1KLpiULt5h9GEHmLMbZdK9CvmcDDfBPxYt8uESfZZjsgPew5ktcoZMyGE8hyh2Pf0/BKK2rhWBPGi0U1aB8RJDajVTMC1REm8LRTebLjKjoEoXUVE0P79eZoEIrCYj1mUKCpJvbK+iu0Q4cEc4sUwDa3EV7nZP4AJGBJJDben83j7H2P7FUpgr598IdDu9AuWg2KxDVjcmSlTLXZbNSDVOYgS7PIexcBPhosr9NYpBfbJdbnEAqVotsFuanJwPa/D/waW6oJn9MfNenjw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 4/30/25 1:08 PM, Oscar Salvador wrote: > On Mon, Apr 28, 2025 at 10:33:46PM +0530, Donet Tom wrote: >> During node device initialization, `memory blocks` are registered under >> each NUMA node. The `memory blocks` to be registered are identified using >> the node’s start and end PFNs, which are obtained from the node's pg_data >> > Hi Donet, > >> Test Results on My system with 32TB RAM >> ======================================= >> 1. Boot time with CONFIG_DEFERRED_STRUCT_PAGE_INIT enabled. >> >> Without this patch >> ------------------ >> Startup finished in 1min 16.528s (kernel) >> >> With this patch >> --------------- >> Startup finished in 17.236s (kernel) - 78% Improvement > That is pretty impressive. > >> +void register_memory_blocks_under_node_early(int nid) >> +{ >> + struct memblock_region *r; >> + unsigned long start_block_id; >> + unsigned long end_block_id; >> + struct memory_block *mem; >> + unsigned long block_id; >> + >> + for_each_mem_region(r) { >> + if (r->nid == nid) { >> + start_block_id = phys_to_block_id(r->base); >> + end_block_id = phys_to_block_id(r->base + r->size - 1); >> + >> + for (block_id = start_block_id; block_id <= end_block_id; block_id++) { >> + mem = find_memory_block_by_id(block_id); >> + if (!mem) >> + continue; > I would just mention what David already said here, reduce identation, > and maybe declare the variables where they are needed. It might be clearer. Sure, I will change it. >> + >> + do_register_memory_block_under_node(nid, mem, MEMINIT_EARLY); >> + put_device(&mem->dev); > I will comment on the "context" on patch#2. > >> +void register_memory_blocks_under_node_early(int nid); > static void ... ? Yes, We can make it as static. I will add it in next version. Thanks Donet > >