* [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
@ 2026-01-20 17:59 Evangelos Petrongonas
2026-01-20 18:37 ` Pratyush Yadav
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Evangelos Petrongonas @ 2026-01-20 17:59 UTC (permalink / raw)
To: Mike Rapoport
Cc: Evangelos Petrongonas, Pasha Tatashin, Pratyush Yadav,
Alexander Graf, Jason Miu, linux-kernel, kexec, linux-mm,
nh-open-source
kho_reserve_scratch() iterates over all online NUMA nodes to allocate
per-node scratch memory. On systems with memoryless NUMA nodes (nodes
that have CPUs but no memory), memblock_alloc_range_nid() fails because
there is no memory available on that node. This causes KHO initialization
to fail and kho_enable to be set to false.
Some ARM64 systems have NUMA topologies where certain nodes contain only
CPUs without any associated memory. These configurations are valid and
should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (N_MEMORY state) and
skip memoryless nodes in the per-node scratch allocation loop.
Signed-off-by: Evangelos Petrongonas <epetron@amazon.de>
---
v2:
- Removed kho_mem_nodes_count in favour of nodes_weight(nodes_state[N_MEMORY])
- Use for_each_node_state(nid, N_MEMORY) to loop over nodes that are both
online and have memory.
TIL: Nodes in N_MEMORY are a subset of those that are online. Thanks Mike :)
kernel/liveupdate/kexec_handover.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 9dc51fab604f..979ebaf015bf 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)
scratch_size_update();
/* FIXME: deal with node hot-plug/remove */
- kho_scratch_cnt = num_online_nodes() + 2;
+ kho_scratch_cnt = nodes_weight(node_states[N_MEMORY]) + 2;
size = kho_scratch_cnt * sizeof(*kho_scratch);
kho_scratch = memblock_alloc(size, PAGE_SIZE);
if (!kho_scratch)
@@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void)
kho_scratch[i].size = size;
i++;
- for_each_online_node(nid) {
+ /*
+ * Loop over nodes that have both memory and are online. Skip
+ * memoryless nodes, as we can not allocate scratch areas there.
+ */
+ for_each_node_state(nid, N_MEMORY) {
size = scratch_size_node(nid);
addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES,
0, MEMBLOCK_ALLOC_ACCESSIBLE,
--
2.43.0
Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
2026-01-20 17:59 [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas Evangelos Petrongonas
@ 2026-01-20 18:37 ` Pratyush Yadav
2026-01-20 19:00 ` Mike Rapoport
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Pratyush Yadav @ 2026-01-20 18:37 UTC (permalink / raw)
To: Evangelos Petrongonas
Cc: Mike Rapoport, Pasha Tatashin, Pratyush Yadav, Alexander Graf,
Jason Miu, linux-kernel, kexec, linux-mm, nh-open-source
On Tue, Jan 20 2026, Evangelos Petrongonas wrote:
> kho_reserve_scratch() iterates over all online NUMA nodes to allocate
> per-node scratch memory. On systems with memoryless NUMA nodes (nodes
> that have CPUs but no memory), memblock_alloc_range_nid() fails because
> there is no memory available on that node. This causes KHO initialization
> to fail and kho_enable to be set to false.
>
> Some ARM64 systems have NUMA topologies where certain nodes contain only
> CPUs without any associated memory. These configurations are valid and
> should not prevent KHO from functioning.
>
> Fix this by only counting nodes that have memory (N_MEMORY state) and
> skip memoryless nodes in the per-node scratch allocation loop.
>
> Signed-off-by: Evangelos Petrongonas <epetron@amazon.de>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
[...]
--
Regards,
Pratyush Yadav
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
2026-01-20 17:59 [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas Evangelos Petrongonas
2026-01-20 18:37 ` Pratyush Yadav
@ 2026-01-20 19:00 ` Mike Rapoport
2026-01-22 18:18 ` Pasha Tatashin
2026-01-22 23:21 ` Andrew Morton
3 siblings, 0 replies; 6+ messages in thread
From: Mike Rapoport @ 2026-01-20 19:00 UTC (permalink / raw)
To: Evangelos Petrongonas
Cc: Pasha Tatashin, Pratyush Yadav, Alexander Graf, Jason Miu,
linux-kernel, kexec, linux-mm, nh-open-source
On Tue, Jan 20, 2026 at 05:59:11PM +0000, Evangelos Petrongonas wrote:
> kho_reserve_scratch() iterates over all online NUMA nodes to allocate
> per-node scratch memory. On systems with memoryless NUMA nodes (nodes
> that have CPUs but no memory), memblock_alloc_range_nid() fails because
> there is no memory available on that node. This causes KHO initialization
> to fail and kho_enable to be set to false.
>
> Some ARM64 systems have NUMA topologies where certain nodes contain only
> CPUs without any associated memory. These configurations are valid and
> should not prevent KHO from functioning.
>
> Fix this by only counting nodes that have memory (N_MEMORY state) and
> skip memoryless nodes in the per-node scratch allocation loop.
>
> Signed-off-by: Evangelos Petrongonas <epetron@amazon.de>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> ---
> v2:
> - Removed kho_mem_nodes_count in favour of nodes_weight(nodes_state[N_MEMORY])
> - Use for_each_node_state(nid, N_MEMORY) to loop over nodes that are both
> online and have memory.
> TIL: Nodes in N_MEMORY are a subset of those that are online. Thanks Mike :)
>
> kernel/liveupdate/kexec_handover.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 9dc51fab604f..979ebaf015bf 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)
> scratch_size_update();
>
> /* FIXME: deal with node hot-plug/remove */
> - kho_scratch_cnt = num_online_nodes() + 2;
> + kho_scratch_cnt = nodes_weight(node_states[N_MEMORY]) + 2;
> size = kho_scratch_cnt * sizeof(*kho_scratch);
> kho_scratch = memblock_alloc(size, PAGE_SIZE);
> if (!kho_scratch)
> @@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void)
> kho_scratch[i].size = size;
> i++;
>
> - for_each_online_node(nid) {
> + /*
> + * Loop over nodes that have both memory and are online. Skip
> + * memoryless nodes, as we can not allocate scratch areas there.
> + */
> + for_each_node_state(nid, N_MEMORY) {
> size = scratch_size_node(nid);
> addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES,
> 0, MEMBLOCK_ALLOC_ACCESSIBLE,
> --
> 2.43.0
>
>
>
>
> Amazon Web Services Development Center Germany GmbH
> Tamara-Danz-Str. 13
> 10243 Berlin
> Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
> Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
> Sitz: Berlin
> Ust-ID: DE 365 538 597
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
2026-01-20 17:59 [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas Evangelos Petrongonas
2026-01-20 18:37 ` Pratyush Yadav
2026-01-20 19:00 ` Mike Rapoport
@ 2026-01-22 18:18 ` Pasha Tatashin
2026-01-22 23:21 ` Andrew Morton
3 siblings, 0 replies; 6+ messages in thread
From: Pasha Tatashin @ 2026-01-22 18:18 UTC (permalink / raw)
To: Evangelos Petrongonas
Cc: Mike Rapoport, Pratyush Yadav, Alexander Graf, Jason Miu,
linux-kernel, kexec, linux-mm, nh-open-source
On Tue, Jan 20, 2026 at 12:59 PM Evangelos Petrongonas
<epetron@amazon.de> wrote:
>
> kho_reserve_scratch() iterates over all online NUMA nodes to allocate
> per-node scratch memory. On systems with memoryless NUMA nodes (nodes
> that have CPUs but no memory), memblock_alloc_range_nid() fails because
> there is no memory available on that node. This causes KHO initialization
> to fail and kho_enable to be set to false.
>
> Some ARM64 systems have NUMA topologies where certain nodes contain only
> CPUs without any associated memory. These configurations are valid and
> should not prevent KHO from functioning.
>
> Fix this by only counting nodes that have memory (N_MEMORY state) and
> skip memoryless nodes in the per-node scratch allocation loop.
>
> Signed-off-by: Evangelos Petrongonas <epetron@amazon.de>
> ---
> v2:
> - Removed kho_mem_nodes_count in favour of nodes_weight(nodes_state[N_MEMORY])
> - Use for_each_node_state(nid, N_MEMORY) to loop over nodes that are both
> online and have memory.
> TIL: Nodes in N_MEMORY are a subset of those that are online. Thanks Mike :)
>
> kernel/liveupdate/kexec_handover.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 9dc51fab604f..979ebaf015bf 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)
> scratch_size_update();
>
> /* FIXME: deal with node hot-plug/remove */
> - kho_scratch_cnt = num_online_nodes() + 2;
> + kho_scratch_cnt = nodes_weight(node_states[N_MEMORY]) + 2;
> size = kho_scratch_cnt * sizeof(*kho_scratch);
> kho_scratch = memblock_alloc(size, PAGE_SIZE);
> if (!kho_scratch)
> @@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void)
> kho_scratch[i].size = size;
> i++;
>
> - for_each_online_node(nid) {
> + /*
> + * Loop over nodes that have both memory and are online. Skip
> + * memoryless nodes, as we can not allocate scratch areas there.
> + */
> + for_each_node_state(nid, N_MEMORY) {
> size = scratch_size_node(nid);
> addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES,
> 0, MEMBLOCK_ALLOC_ACCESSIBLE,
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
2026-01-20 17:59 [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas Evangelos Petrongonas
` (2 preceding siblings ...)
2026-01-22 18:18 ` Pasha Tatashin
@ 2026-01-22 23:21 ` Andrew Morton
2026-01-23 8:53 ` Mike Rapoport
3 siblings, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2026-01-22 23:21 UTC (permalink / raw)
To: Evangelos Petrongonas
Cc: Mike Rapoport, Pasha Tatashin, Pratyush Yadav, Alexander Graf,
Jason Miu, linux-kernel, kexec, linux-mm, nh-open-source
On Tue, 20 Jan 2026 17:59:11 +0000 Evangelos Petrongonas <epetron@amazon.de> wrote:
> kho_reserve_scratch() iterates over all online NUMA nodes to allocate
> per-node scratch memory. On systems with memoryless NUMA nodes (nodes
> that have CPUs but no memory), memblock_alloc_range_nid() fails because
> there is no memory available on that node. This causes KHO initialization
> to fail and kho_enable to be set to false.
>
> Some ARM64 systems have NUMA topologies where certain nodes contain only
> CPUs without any associated memory. These configurations are valid and
> should not prevent KHO from functioning.
>
> Fix this by only counting nodes that have memory (N_MEMORY state) and
> skip memoryless nodes in the per-node scratch allocation loop.
>
So kho is unusable on such machines.
Should we backport this? I'm thinking
Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers").
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas
2026-01-22 23:21 ` Andrew Morton
@ 2026-01-23 8:53 ` Mike Rapoport
0 siblings, 0 replies; 6+ messages in thread
From: Mike Rapoport @ 2026-01-23 8:53 UTC (permalink / raw)
To: Andrew Morton
Cc: Evangelos Petrongonas, Pasha Tatashin, Pratyush Yadav,
Alexander Graf, Jason Miu, linux-kernel, kexec, linux-mm,
nh-open-source
On Thu, Jan 22, 2026 at 03:21:12PM -0800, Andrew Morton wrote:
> On Tue, 20 Jan 2026 17:59:11 +0000 Evangelos Petrongonas <epetron@amazon.de> wrote:
>
> > kho_reserve_scratch() iterates over all online NUMA nodes to allocate
> > per-node scratch memory. On systems with memoryless NUMA nodes (nodes
> > that have CPUs but no memory), memblock_alloc_range_nid() fails because
> > there is no memory available on that node. This causes KHO initialization
> > to fail and kho_enable to be set to false.
> >
> > Some ARM64 systems have NUMA topologies where certain nodes contain only
> > CPUs without any associated memory. These configurations are valid and
> > should not prevent KHO from functioning.
> >
> > Fix this by only counting nodes that have memory (N_MEMORY state) and
> > skip memoryless nodes in the per-node scratch allocation loop.
> >
>
> So kho is unusable on such machines.
>
> Should we backport this? I'm thinking
>
> Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers").
It's only for v6.18, but sure, why not.
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-01-23 8:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-01-20 17:59 [PATCH v2] kho: skip memoryless NUMA nodes when reserving scratch areas Evangelos Petrongonas
2026-01-20 18:37 ` Pratyush Yadav
2026-01-20 19:00 ` Mike Rapoport
2026-01-22 18:18 ` Pasha Tatashin
2026-01-22 23:21 ` Andrew Morton
2026-01-23 8:53 ` Mike Rapoport
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox