From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EFB5C3ABB6 for ; Mon, 5 May 2025 09:36:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 650966B0088; Mon, 5 May 2025 05:36:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5FE7A6B0089; Mon, 5 May 2025 05:36:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A20B6B008A; Mon, 5 May 2025 05:36:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2B7546B0088 for ; Mon, 5 May 2025 05:36:55 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 238D1801BC for ; Mon, 5 May 2025 09:36:56 +0000 (UTC) X-FDA: 83408350032.10.CC7D98E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf22.hostedemail.com (Postfix) with ESMTP id EA3EBC0006 for ; Mon, 5 May 2025 09:36:53 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=iZHawpJe; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=xRQWHzpb; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=iZHawpJe; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=xRQWHzpb; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf22.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746437814; a=rsa-sha256; cv=none; b=Rk/STV6EGcR+oboGlwp/5EnF4kqV1koMqVyaO3MMQTSeOlX6qjH1tTrKJggVfg2BkfhpxR 5Yh2C8Wj5XvarOfRonSXqLH05GfSgj7ELDD0NGhG6f5jrP9y33Uoj4wSvBIX5RxblQrjJ+ AWHsTBEMPyDiaKUB+mTPiiqn3holwO0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=iZHawpJe; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=xRQWHzpb; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=iZHawpJe; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=xRQWHzpb; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf22.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746437814; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iqZVfig/8xyFjf3cLBUVBTTUuoxBEC4ZbopMJ9V8slI=; b=ppaMVL7EFp3NZmFfA3ON4p8yo/urU1CPTMgm60pX9Yd/bwE4YMtSVsqHUNDOawK+o/Ne7v t3AYeFKWSEIxz6KceRHkTZjPTHaEwEL+amiYo6nRCg5Cu4joYi8AYSGNBYiW/Lk5EGbQqW YjLaiitmvzS3O4n/kKWR/7BO12oTcpY= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2AC5521751; Mon, 5 May 2025 09:36:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1746437812; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iqZVfig/8xyFjf3cLBUVBTTUuoxBEC4ZbopMJ9V8slI=; b=iZHawpJeTkZ5WQKGQ9B7dID4kMuT89/8pB+Vm7CZ+W0MtLr1CxEOkBFpaI6kFRwOWrguXr +Og03cyYPnLh7s0YtBb4WgXPAzKeStNlgpNZu4ODzX9x6BPJN74xYkITeDUOpNONZSfClQ opn5+u7cVWgFiMm/qbmR877wBOvb7IM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1746437812; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iqZVfig/8xyFjf3cLBUVBTTUuoxBEC4ZbopMJ9V8slI=; b=xRQWHzpbaBRs5jZ0vrvtApqWUOd5cFt00HYLC9wszuqlKR/smkh/ZXu6osRqRUdnKAeHEo YvmlEIdIKgHJGsBQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1746437812; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iqZVfig/8xyFjf3cLBUVBTTUuoxBEC4ZbopMJ9V8slI=; b=iZHawpJeTkZ5WQKGQ9B7dID4kMuT89/8pB+Vm7CZ+W0MtLr1CxEOkBFpaI6kFRwOWrguXr +Og03cyYPnLh7s0YtBb4WgXPAzKeStNlgpNZu4ODzX9x6BPJN74xYkITeDUOpNONZSfClQ opn5+u7cVWgFiMm/qbmR877wBOvb7IM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1746437812; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iqZVfig/8xyFjf3cLBUVBTTUuoxBEC4ZbopMJ9V8slI=; b=xRQWHzpbaBRs5jZ0vrvtApqWUOd5cFt00HYLC9wszuqlKR/smkh/ZXu6osRqRUdnKAeHEo YvmlEIdIKgHJGsBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4038213883; Mon, 5 May 2025 09:36:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ZrbQDLOGGGjubgAAD6G6ig (envelope-from ); Mon, 05 May 2025 09:36:51 +0000 Date: Mon, 5 May 2025 11:36:42 +0200 From: Oscar Salvador To: David Hildenbrand Cc: Donet Tom , Mike Rapoport , Zi Yan , Greg Kroah-Hartman , Andrew Morton , rafael@kernel.org, Danilo Krummrich , Ritesh Harjani , Jonathan Cameron , Alison Schofield , Yury Norov , Dave Jiang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 1/3] driver/base: Optimize memory block registration to reduce boot time Message-ID: References: <188fbfba-afb4-4db7-bbba-7689a96be931@redhat.com> <74c500dd-8d1c-4177-96c7-ddd51ca77306@redhat.com> <0e568e33-34fa-40f6-a20d-ebf653de123d@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0e568e33-34fa-40f6-a20d-ebf653de123d@redhat.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EA3EBC0006 X-Stat-Signature: 74wk64yfcbgrj4ku4aeaqhzfxcrf6qr6 X-HE-Tag: 1746437813-795953 X-HE-Meta: U2FsdGVkX18N62omSH8gQ6sMEUh1xuZ2eWT+0h3DitwwkcB4asRIcDMIu0hPHqaiDJ11DrZywpwzJqgfWjtj5r0Pj+wJAzzBx5f3y16aHUlJvwB2osXtK+k+JCM3J7VzcWhY/BTakPvDYpkzEEHROtnyFjJrriUXxQlVoflnDSf93OYz2ggNKNR8SGRFPuvGFq23E5skC9p5lrLgFlNuN+zDJMgl06XNGvhOYdpPgwSFa6gKHLFqpm5+ZP1xEhxvBp5NxXylKiPRxNoQ1I6ybj5mRCCUgINkW0cQeMdLPPBO6Lf4rxV6o9ewqVhN1vUShO9XFu/oPKC/BlRRX2V8pyB0lw9xWTSOXROiMQ0CqPFlaWfhc19Q5i/sPJk0PZcIcoPm4O1uK2sgtniThbeLnjNsawfp8d3aI0qulsx1bO6kWLsnF/AXZlMYQFoJuIE8J9r5o7sxMZtZlUI2SHnhB1KyPoZMdwf8tdsDSCId5CVSLhWqByOn57GrXgnO7TTLWtqKtNoqYddizCPCEqL6ji9UlrF5+CxJecFN/+ACcMzG37ZcGF4KdUMlhRTcr1YKq0f2MXTSxrbtBq50UO2DQaz5kQ5X48L+pIkuOgvRYNduhcGUvixfnwEOJrAeKGORnDLd5mIPwAg7m3HQrjT83BhWQsHd0VkePdy8Flmu3NSyydlxxeYUNnz17/+gOT0WIaS/nhVYU6VhioyZsYIno5zZKVwV5GZxO0VSFbaQjFJhz1/I2ksmvP4bYO3EjWaqGWxLPNSbZTW+/Qco5tDuzYlWKRjHTILCplWSz34W/hHL4JAAKVzPSpfBK6KI6cF6bpbgtwU8RJdfc2Y0UwBv5oJiQOBA3oeYVEWP+RctiC0SB80mKqj48h1y7oX1xqDx277CUhwX3MsKftrwc9e9Gz3JajnszLRfPtmU01HbZ2+wTBYzexsL0BMP57wfo5s+m+3+5Yq0wrA/QbJAMSd USso8VXu jmaVx0J3RBNA6Ibs8qlRH9IlHdrKjGj9mDnW3PLL9M4i6+cfu+Ne8fscu1dQSJoMioOGW4kOiLTZeqKWa9EhMlSsSf7KCqrQ06VXEIVnfH+OolNs6okHpcwuxT/3PUQ/TzdkJa7J31FT9zxHu9GPn9XI8CJEHoRt2R6e8PBdGPvoIjsz8CfS3BSpeRI6aOQ5SJz4YMICWfLJ3LcKA6/JKFNRMKt9Mm8LVtXs/ATfr7yTFdCPvFJGS6Leka/s48n+G61d8A7/QkJkSU40mrzJ5oynKb3IQtYfyG7kcjrw3JjzWfiNufAHlNGCaTuTF4r8rYdEJLgfDkstOTrc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, May 05, 2025 at 10:12:48AM +0200, David Hildenbrand wrote: > Assume you hotplug the second CPU. The node is already registered/online, so > who does the register_cpu_under_node() call? > > It's register_cpu() I guess? But no idea in which order that is called with > node onlining. > > The code has to be cleaned up such that onlining a node does not traverse > any cpus / memory. > > Whoever adds a CPU / memory *after onlining the node* must register the > device manually under the *now online* node. So, I think this is the sequence of events: - hotplug cpu: acpi_processor_hotadd_init register_cpu register_cpu_under_node online_store device_online()->dev_bus_online() cpu_subsys->online() cpu_subsys_online cpu_device_up cpu_up try_online_node <- brings node online ... register_one_node <- registers cpu under node _cpu_up The first time we hotplug a cpu to the node, note that register_cpu()->register_cpu_under_node() will bail out as node is still offline, so only cpu's sysfs will be created but they will not be linked to the node. Later, online_store()->...->cpu_subsys_online()->..->cpu_up() will take care of 1) onlining the node and 2) register the cpu to the node (so, link the sysfs). The second time we hotplug a cpu, register_cpu()->register_cpu_under_node() will do its job as the node is already onlined. And we will not be calling register_one_node() from __try_online_node() because of the same reason. The thing that bothers me is having register_cpu_under_node() spread around. I think that ideally, we should only be calling register_cpu_under_node() from register_cpu(), but we have this kinda of (sort of weird?) relation that even if we hotplug the cpu, but we do not online it, the numa node will remain online, and so we cannot do the linking part (cpu <-> node), so we could not really only have register_cpu_under_node() in register_cpu(), which is the hot-add part, but we also need it in the cpu_up()->try_online_node() which is the online part. And we cannot also remove the register_cpu_under_node() from register_cpu() because it is used in other paths (e.g: at boot time ). I have to confess that yes, this is a bit tangled. Maybe there is room for improvement here, I will have to think about this some more and see if I can come up with something that makes sense. -- Oscar Salvador SUSE Labs