Re: [PATCH v3 1/3] driver/base: Optimize memory block registration to reduce boot time

[Oscar Salvador <osalvador@suse.de>]

On Mon, May 05, 2025 at 12:36:21PM +0200, David Hildenbrand wrote:
> My thinking was, whether we can simply move the register_cpu_under_node()
> after the try_online_node(). See below regarding early.
> 
> And then, remove the !node_online check from register_cpu_under_node().
> 
> But it's all complicated, because for memory, we link a memory block to the
> node (+set the node online) when it gets added, not when it gets onlined.
> 
> For CPUs, we seem to be creating the link + set the node online when the CPU
> gets onlined.

Yes, that is one think we need to align on.

So, add_memory_resource(), which is part of the hot-add stage, will mark the node
online if it needs to.
I guess that this is done this way because further down the road we rely
on the node to be online for some reason (e.g: online_memory_block() being
called from add_memory_resource()).

Although from the conceptual point of view, it does not make sense, does
it?
I mean, if a node does not have any resources onlined, why should it be
onlined?

But let us assume that that is the way to go, then we could in fact
tweak the cpu-hotplug code to do the same, and online the node whenever
a cpu gets hot-added, and not really waiting for it to be onlined.

I can do some experiments about it and see how it turns out.

> > The first time we hotplug a cpu to the node, note that
> > register_cpu()->register_cpu_under_node() will bail out as node is still
> > offline, so only cpu's sysfs will be created but they will not be linked
> > to the node.
> > Later, online_store()->...->cpu_subsys_online()->..->cpu_up() will take>
> care of 1) onlining the node and 2) register the cpu to the node (so,
> > link the sysfs).
> 
> 
> And only if it actually gets onlined I assume.

Yes, we only register it if we managed to online the node.

> > I think that ideally, we should only be calling register_cpu_under_node()
> > from register_cpu(), but we have this kinda of (sort of weird?) relation
> > that even if we hotplug the cpu, but we do not online it, the numa node
> > will remain online, and so we cannot do the linking part (cpu <-> node),
> > so we could not really only have register_cpu_under_node() in
> > register_cpu(), which is the hot-add part, but we also need it in the
> > cpu_up()->try_online_node() which is the online part.
> 
> Maybe one could handle CPUs similar to how we handle it with memory: node
> gets onlined + link created as soon as we add the CPU, not when we online
> it.

Yes, indeed, as I stated above I think this is the way to go, to have it
paired.

> But likely there is a reason why we do it like that today ...

I cannot think of any (Tm), but it is a possibility.

I will take a break from hugetlb stuff and play a little bit with this.


-- 
Oscar Salvador
SUSE Labs