Re: [PATCH v3 1/3] driver/base: Optimize memory block registration to reduce boot time

[Oscar Salvador <osalvador@suse.de>]

On Mon, May 05, 2025 at 10:12:48AM +0200, David Hildenbrand wrote:
> Assume you hotplug the second CPU. The node is already registered/online, so
> who does the register_cpu_under_node() call?
> 
> It's register_cpu() I guess? But no idea in which order that is called with
> node onlining.
> 
> The code has to be cleaned up such that onlining a node does not traverse
> any cpus / memory.
> 
> Whoever adds a CPU / memory *after onlining the node* must register the
> device manually under the *now online* node.

So, I think this is the sequence of events:

- hotplug cpu:
  acpi_processor_hotadd_init
   register_cpu
    register_cpu_under_node

  online_store
   device_online()->dev_bus_online()
    cpu_subsys->online()
     cpu_subsys_online
      cpu_device_up
       cpu_up
        try_online_node  <- brings node online
         ...
         register_one_node <- registers cpu under node
        _cpu_up

The first time we hotplug a cpu to the node, note that
register_cpu()->register_cpu_under_node() will bail out as node is still
offline, so only cpu's sysfs will be created but they will not be linked
to the node.

Later, online_store()->...->cpu_subsys_online()->..->cpu_up() will take
care of 1) onlining the node and 2) register the cpu to the node (so,
link the sysfs).

The second time we hotplug a cpu,
register_cpu()->register_cpu_under_node() will do its job as the node is
already onlined.
And we will not be calling register_one_node() from __try_online_node()
because of the same reason.

The thing that bothers me is having register_cpu_under_node() spread
around.
I think that ideally, we should only be calling register_cpu_under_node()
from register_cpu(), but we have this kinda of (sort of weird?) relation
that even if we hotplug the cpu, but we do not online it, the numa node
will remain online, and so we cannot do the linking part (cpu <-> node),
so we could not really only have register_cpu_under_node() in
register_cpu(), which is the hot-add part, but we also need it in the
cpu_up()->try_online_node() which is the online part.

And we cannot also remove the register_cpu_under_node() from
register_cpu() because it is used in other paths (e.g: at boot time ).

I have to confess that yes, this is a bit tangled.

Maybe there is room for improvement here, I will have to think about
this some more and see if I can come up with something that makes sense.



-- 
Oscar Salvador
SUSE Labs