From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90153C3ABB0 for ; Mon, 5 May 2025 07:53:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4306F6B008C; Mon, 5 May 2025 03:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DE296B0092; Mon, 5 May 2025 03:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A5646B0093; Mon, 5 May 2025 03:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0DDCD6B008C for ; Mon, 5 May 2025 03:53:23 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5F7641A1B76 for ; Mon, 5 May 2025 07:53:24 +0000 (UTC) X-FDA: 83408089128.09.0909A5D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id B9931100003 for ; Mon, 5 May 2025 07:53:22 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uQPqRr2e; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746431602; a=rsa-sha256; cv=none; b=6OwsMCgvx5YlwvgumkjLqzgWcb+wgh/pomrBcVSvNUax9yIToj40NWY3QxKexpBdZHkpWy QTpSWFM4GhGkrLujvEw9E4AI94PDHZWV863xBI9mkFGYNI8+KPfzzIIwTD+CUhSHJ3gy5q 3K4c95rwpMm3A1dCbD7CoFEhEdrZfHM= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uQPqRr2e; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746431602; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zcHgBHpcAemlmq5cGPqHUmJEG1WRStamUmSbo34+EN8=; b=bDM2WaCg0XiqKvnt7mM1+fuNz731JjXIjLs5XklMtSfUnYxiId7hXH/U6SidHSGEsP3LqS oF32IU5tWt8IrGIoGH3m0DCuC9SR4UuuWc7EefzOKNLYuRoQWfPtPUYUncbWefeg2AORK3 FfRjHUXMsVkrIMii0ZJjKXA2B6Jd/Dw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 62F3D5C4D09; Mon, 5 May 2025 07:51:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3BAAC4CEE4; Mon, 5 May 2025 07:53:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746431601; bh=oCus7xWIklrFnQP8oc322gPIGxyJvrdIfB7ko5gsij8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=uQPqRr2ePG1wZNxpQSyhcSY/10SkN7w/4ybwSPA6oIjpYegvPo8OZ15M22Vm8IZXK QWTpLqFRCGPwMomDXZALJ1FW4U/65OjsGJrukiRcBx2FSlRMPE5bgj6m1iXM1qy0rs OgytwrDfyslbR/Hx0JvR97Q7IuJdnvVs3eL6vqe2/0FnEi22RxH3K0RuRzvcoQkigb 9srKU6dAtxrkb8ubvMRCxfkgcujrMU8rQ6xt0MzyMIRkf6amiEb3rh9JR8K4ynYYDY f2cv2Ty8GZQbNMYYN/iOzCdi+8VZ6l69M1n8kp8SXg2kH5X3Qj+TYGUE1KDnEUm/M9 7K8nQjkNUwlwQ== Date: Mon, 5 May 2025 10:53:09 +0300 From: Mike Rapoport To: David Hildenbrand Cc: Oscar Salvador , Donet Tom , Zi Yan , Greg Kroah-Hartman , Andrew Morton , rafael@kernel.org, Danilo Krummrich , Ritesh Harjani , Jonathan Cameron , Alison Schofield , Yury Norov , Dave Jiang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 1/3] driver/base: Optimize memory block registration to reduce boot time Message-ID: References: <188fbfba-afb4-4db7-bbba-7689a96be931@redhat.com> <74c500dd-8d1c-4177-96c7-ddd51ca77306@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <74c500dd-8d1c-4177-96c7-ddd51ca77306@redhat.com> X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: B9931100003 X-Stat-Signature: fmb4iype5b843nsckickzbhwr8z44oj3 X-Rspam-User: X-HE-Tag: 1746431602-343363 X-HE-Meta: U2FsdGVkX19wzoUUblstvw1hqH2XEYBAJ8jg3GMvw7d+CiKpA1grUKt7lzmaX6VP9l8BnWD0ZsTFSEjP8w25WKrnvUI72Ph35ZdDnTxd63sEqtpPRlVjRmk2IcfRf2PZkIS+j1UXhNpw1wjYBVYjAafJwBv0ZrYUojmmnp0sot/4swFq9I+Fr//8WdVyILwz0VFdEx/1tdtcq/GiqsUpq4IdcNfB8Swb0X7syu2IBYvoBfGbZQkh/ITTAvTsxeEtMkAfBumBHDvwNKwlaVR42PSG1C4XMHiZStvPH5Wom9DJ0NPoRLL2Ax1wyyFQZhRtCZtgLMjSYt2i5Mn1RB/3+Akmv9GTcWzXwECweW4BGmLFxuZduDBXX//fdEC8xm3U2mfQSsRwancLGUtTG4j1+GHJvCkL7vn0VlA6pjw2K8o6TITWjG1/YDg9Y4vtplgmrmdyUEVryEPb+zoH6zpiVx1pxGELo1yg97S0yIMmUC6KK2JzymM9ZE/RcvaJfI9rEkKBF3X0GCX3IylNxZW/D1T1UfEHdnslTSZNz8ARHIFtEvvFAf3tdDLRdgSCEXZsD2qOV5TgGUPPfwdh9l5mWbSq0hz3JQPipC2cji9MCRY8Dk8pwqComznvwcEvBuJkNJ4k/VXw/DfmGS6v2gLxrtuIpveTl3djR7elGR0er+OzX50eWs2GqrPd/NQWdlaCK6SqpBQ+6TIh9sfyAGLSKurAK4KrSSSy3K3nyrKVoWaLKtHVIGRaWhxMhHjCbvISop2wKFO2QrnDijgnbNV+Vl5xSFRfi0kDxp7MohelPa0almvh+vEvhrwmG+w4M3uFgla97pQBU7F1HwSX6nXoC7yvWldtXW858IB35UvUsiexj3h91Pn2y0RcQcF8z4g+Tj3Qm3nBTH1nZwuOIo06vxBQwTBZIaKxt2iPv1qETNJYWkYDIaS1E8FT/BYar/bxsD2hOHUC18P6Snl9xpy HJfOf6bG Lg9OhK3KH10H0pCJPIG3I6Pjr6WVOhImfD6BNNZFsuWcuWLDvSuZdDgxX4yFDlK9LIt4G29832Xo4vWJjYsolmUNHvEIaGsNLidh9W1omcxmDbVXyVSOo9oSjJe++pOnPvnu4xn97h5wMNACCg/SKEaxqRrhtTu/GXho2rnpfb6So+SPKlqOImZM8Zah5jide9zH6IPTYY42h3BfN8jg+uSQmP1lih8Vm583/WgPuZauiHj9y+z5Dvl4zc2NUbQ0Tr1XwhowTEc9aFZSJwTY/7rOMVpi3Fy8mgfaBCbjwvp7Nw722s/Xt5cTiKw1x1BS1EBS/H/JKNOuFWnk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, May 05, 2025 at 09:38:43AM +0200, David Hildenbrand wrote: > On 05.05.25 09:28, Oscar Salvador wrote: > > On Mon, May 05, 2025 at 09:16:48AM +0200, David Hildenbrand wrote: > > > memory hotplug code never calls register_one_node(), unless I am missing > > > something. > > > > > > During add_memory_resource(), we call __try_online_node(nid, false), meaning > > > we skip register_one_node(). > > > > > > The only caller of __try_online_node(nid, true) is try_online_node(), called > > > from CPU hotplug code, and I *guess* that is not required. > > > > Well, I guess this is because we need to link the cpus to the node. > > register_one_node() has two jobs: 1) register cpus belonging to the node > > and 2) register memory-blocks belonging to the node (if any). > > Ah, via __register_one_node() ... > > I would assume that an offline node > > (1) has no memory > (2) has no CPUs > > When we *hotplug* either memory or CPUs, and we first online the node, there > is nothing to register. Because if there would be something, the node would > already be online. > > In particular, try_offline_node() will only offline a node if > > (A) No present pages: No pages are spanned anymore. This includes > offline memory blocks. > (B) No present CPUs. > > But maybe there is some case that I am missing ... I actually hoped you and Oscar know how that stuff works :) I tried to figure what is going on there and it all looks really convoluted. So, on boot we have cpu_up() -> try_online_node() -> bails out because all nodes are online (at least on x86 AFAIU, see 1ca75fa7f19d ("arch/x86/mm/numa: Do not initialize nodes twice")) node_dev_init()i -> register_one_node() -> this one can use __register_one_node() and loop over memblock regions. And for the hotplug/unplug path, it seems that register_memory_blocks_under_node(MEMINIT_EARLY) is superfluous, because if a node had memory it wouldn't get offlined, and if we are hotplugging an node with memory and cpus, memory hotplug anyway calls register_memory_blocks_under_node_hotplug(). So, IMHO, register_one_node() should not call register_memory_blocks_under_node() at all, but again, I might have missed something :) > -- > Cheers, > > David / dhildenb > -- Sincerely yours, Mike.