[PATCH v1 0/5] MM_CID and HPCC mm_struct static init fixes

[PATCH v1 0/5] MM_CID and HPCC mm_struct static init fixes

[Mathieu Desnoyers]

Hi Andrew,

Mark Brown reported a regression [1] on linux next due to the
hierarchical percpu counters (HPCC). You mentioned they were only in
mm-new (and therefore not pulled into -next) [2], but it looks like they
got more exposure that we expected. :)

This bug hunting got me to fix static initialization issues in both
MM_CID (for upstream) and HPCC (mm-new). Mark tested my series and
confirmed that it fixes his issues.

Please consider the HPCC fixes for mm-new, and the MM_CID fixes for
upstream.

This series is based on mm-new
commit 287373d0b6ee ("selftests/mm: fix comment for check_test_requirements")

Thanks,

Mathieu

Link: https://lore.kernel.org/lkml/c8acf650-ef96-4449-ba85-557dedba0ffc@sirena.org.uk/ # [1]
Link: https://lore.kernel.org/lkml/20251122111336.56b060c86ff6ba41fe76bfd4@linux-foundation.org/ [2]
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: linux-mm@kvack.org

Mathieu Desnoyers (5):
  mm: Add missing static initializer for init_mm::mm_cid.lock
  mm: Rename cpu_bitmap field to flexible_array
  mm: Take into account mm_cid size for mm_struct static definitions
  mm: Take into account hierarchical percpu tree items for static
    mm_struct definitions
  tsacct: Skip all kernel threads

 drivers/firmware/efi/efi.c          |  2 +-
 include/linux/mm.h                  |  2 +-
 include/linux/mm_types.h            | 18 +++++++---
 include/linux/percpu_counter_tree.h | 51 +++++++++++++++++++++++++++++
 kernel/tsacct.c                     |  2 +-
 mm/init-mm.c                        |  5 ++-
 6 files changed, 71 insertions(+), 9 deletions(-)

-- 
2.39.5

[PATCH v1 1/5] mm: Add missing static initializer for init_mm::mm_cid.lock

[Mathieu Desnoyers]

Initialize the mm_cid.lock struct member of init_mm.

Fixes: 8cea569ca785 ("sched/mmcid: Use proper data structures")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Cc: linux-mm@kvack.org
---
 mm/init-mm.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/init-mm.c b/mm/init-mm.c
index 4600e7605cab..a514f8ce47e3 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -44,6 +44,9 @@ struct mm_struct init_mm = {
 	.mm_lock_seq	= SEQCNT_ZERO(init_mm.mm_lock_seq),
 #endif
 	.user_ns	= &init_user_ns,
+#ifdef CONFIG_SCHED_MM_CID
+	.mm_cid.lock = __RAW_SPIN_LOCK_UNLOCKED(init_mm.mm_cid.lock),
+#endif
 	.cpu_bitmap	= CPU_BITS_NONE,
 	INIT_MM_CONTEXT(init_mm)
 };
-- 
2.39.5

[PATCH v1 2/5] mm: Rename cpu_bitmap field to flexible_array

[Mathieu Desnoyers]

The cpu_bitmap flexible array now contains more than just the
cpu_bitmap. In preparation for changing the static mm_struct
definitions to cover for the additional space required, change the
cpu_bitmap type from "unsigned long" to "char", require an unsigned long
alignment of the flexible array, and rename the field from "cpu_bitmap"
to "flexible_array".

Introduce the MM_STRUCT_FLEXIBLE_ARRAY_INIT macro to statically
initialize the flexible array. This covers the init_mm and efi_mm
static definitions.

This is a preparation step for fixing the missing mm_cid size for static
mm_struct definitions.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: linux-mm@kvack.org
---
 drivers/firmware/efi/efi.c |  2 +-
 include/linux/mm.h         |  2 +-
 include/linux/mm_types.h   | 13 +++++++++----
 mm/init-mm.c               |  2 +-
 4 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index a9070d00b833..3f5c2ae50024 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -73,10 +73,10 @@ struct mm_struct efi_mm = {
 	MMAP_LOCK_INITIALIZER(efi_mm)
 	.page_table_lock	= __SPIN_LOCK_UNLOCKED(efi_mm.page_table_lock),
 	.mmlist			= LIST_HEAD_INIT(efi_mm.mmlist),
-	.cpu_bitmap		= { [BITS_TO_LONGS(NR_CPUS)] = 0},
 #ifdef CONFIG_SCHED_MM_CID
 	.mm_cid.lock		= __RAW_SPIN_LOCK_UNLOCKED(efi_mm.mm_cid.lock),
 #endif
+	.flexible_array		= MM_STRUCT_FLEXIBLE_ARRAY_INIT,
 };
 
 struct workqueue_struct *efi_rts_wq;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 890dab720f75..8d9e3239d2cc 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2853,7 +2853,7 @@ static inline struct percpu_counter_tree_level_item *get_rss_stat_items(struct m
 {
 	unsigned long ptr = (unsigned long)mm;
 
-	ptr += offsetof(struct mm_struct, cpu_bitmap);
+	ptr += offsetof(struct mm_struct, flexible_array);
 	/* Skip cpu_bitmap */
 	ptr += cpumask_size();
 	/* Skip mm_cidmask */
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index a6287d07efb7..1531df8cda52 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1329,7 +1329,7 @@ struct mm_struct {
 	 * The mm_cpumask needs to be at the end of mm_struct, because it
 	 * is dynamically sized based on nr_cpu_ids.
 	 */
-	unsigned long cpu_bitmap[];
+	char flexible_array[] __aligned(__alignof__(unsigned long));
 };
 
 /* Copy value to the first system word of mm flags, non-atomically. */
@@ -1366,19 +1366,24 @@ static inline void __mm_flags_set_mask_bits_word(struct mm_struct *mm,
 			 MT_FLAGS_USE_RCU)
 extern struct mm_struct init_mm;
 
+#define MM_STRUCT_FLEXIBLE_ARRAY_INIT				\
+{								\
+	[0 ... sizeof(cpumask_t)-1] = 0				\
+}
+
 /* Pointer magic because the dynamic array size confuses some compilers. */
 static inline void mm_init_cpumask(struct mm_struct *mm)
 {
 	unsigned long cpu_bitmap = (unsigned long)mm;
 
-	cpu_bitmap += offsetof(struct mm_struct, cpu_bitmap);
+	cpu_bitmap += offsetof(struct mm_struct, flexible_array);
 	cpumask_clear((struct cpumask *)cpu_bitmap);
 }
 
 /* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
 static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
 {
-	return (struct cpumask *)&mm->cpu_bitmap;
+	return (struct cpumask *)&mm->flexible_array;
 }
 
 #ifdef CONFIG_LRU_GEN
@@ -1469,7 +1474,7 @@ static inline cpumask_t *mm_cpus_allowed(struct mm_struct *mm)
 {
 	unsigned long bitmap = (unsigned long)mm;
 
-	bitmap += offsetof(struct mm_struct, cpu_bitmap);
+	bitmap += offsetof(struct mm_struct, flexible_array);
 	/* Skip cpu_bitmap */
 	bitmap += cpumask_size();
 	return (struct cpumask *)bitmap;
diff --git a/mm/init-mm.c b/mm/init-mm.c
index a514f8ce47e3..c5556bb9d5f0 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -47,7 +47,7 @@ struct mm_struct init_mm = {
 #ifdef CONFIG_SCHED_MM_CID
 	.mm_cid.lock = __RAW_SPIN_LOCK_UNLOCKED(init_mm.mm_cid.lock),
 #endif
-	.cpu_bitmap	= CPU_BITS_NONE,
+	.flexible_array	= MM_STRUCT_FLEXIBLE_ARRAY_INIT,
 	INIT_MM_CONTEXT(init_mm)
 };
 
-- 
2.39.5

[PATCH v1 3/5] mm: Take into account mm_cid size for mm_struct static definitions

[Mathieu Desnoyers]

Both init_mm and efi_mm static definitions need to make room for the
2 mm_cid cpumasks.

This fixes possible out-of-bounds accesses to init_mm and efi_mm.

Fixes: af7f588d8f73 ("sched: Introduce per-memory-map concurrency ID")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Brown <broonie@kernel.org>
Cc: stable@vger.kernel.org
Cc: linux-mm@kvack.org
---
 include/linux/mm_types.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 1531df8cda52..aefa64db3499 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1368,7 +1368,7 @@ extern struct mm_struct init_mm;
 
 #define MM_STRUCT_FLEXIBLE_ARRAY_INIT				\
 {								\
-	[0 ... sizeof(cpumask_t)-1] = 0				\
+	[0 ... sizeof(cpumask_t) + MM_CID_STATIC_SIZE - 1] = 0	\
 }
 
 /* Pointer magic because the dynamic array size confuses some compilers. */
@@ -1500,7 +1500,7 @@ static inline int mm_alloc_cid_noprof(struct mm_struct *mm, struct task_struct *
 	mm_init_cid(mm, p);
 	return 0;
 }
-#define mm_alloc_cid(...)	alloc_hooks(mm_alloc_cid_noprof(__VA_ARGS__))
+# define mm_alloc_cid(...)	alloc_hooks(mm_alloc_cid_noprof(__VA_ARGS__))
 
 static inline void mm_destroy_cid(struct mm_struct *mm)
 {
@@ -1514,6 +1514,8 @@ static inline unsigned int mm_cid_size(void)
 	return cpumask_size() + bitmap_size(num_possible_cpus());
 }
 
+/* Use NR_CPUS as worse case for static allocation. */
+# define MM_CID_STATIC_SIZE	(2 * sizeof(cpumask_t))
 #else /* CONFIG_SCHED_MM_CID */
 static inline void mm_init_cid(struct mm_struct *mm, struct task_struct *p) { }
 static inline int mm_alloc_cid(struct mm_struct *mm, struct task_struct *p) { return 0; }
@@ -1522,6 +1524,7 @@ static inline unsigned int mm_cid_size(void)
 {
 	return 0;
 }
+# define MM_CID_STATIC_SIZE	0
 #endif /* CONFIG_SCHED_MM_CID */
 
 struct mmu_gather;
-- 
2.39.5

[PATCH v1 4/5] mm: Take into account hierarchical percpu tree items for static mm_struct definitions

[Mathieu Desnoyers]

Both init_mm and efi_mm static definitions need to make room for the
hierarchical percpu counters items.

This fixes possible out-of-bounds accesses to init_mm and efi_mm.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: linux-mm@kvack.org
---
 include/linux/mm_types.h            |  6 ++--
 include/linux/percpu_counter_tree.h | 51 +++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index aefa64db3499..234374c46b71 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1366,9 +1366,9 @@ static inline void __mm_flags_set_mask_bits_word(struct mm_struct *mm,
 			 MT_FLAGS_USE_RCU)
 extern struct mm_struct init_mm;
 
-#define MM_STRUCT_FLEXIBLE_ARRAY_INIT				\
-{								\
-	[0 ... sizeof(cpumask_t) + MM_CID_STATIC_SIZE - 1] = 0	\
+#define MM_STRUCT_FLEXIBLE_ARRAY_INIT									\
+{													\
+	[0 ... sizeof(cpumask_t) + MM_CID_STATIC_SIZE + PERCPU_COUNTER_TREE_ITEMS_STATIC_SIZE - 1] = 0	\
 }
 
 /* Pointer magic because the dynamic array size confuses some compilers. */
diff --git a/include/linux/percpu_counter_tree.h b/include/linux/percpu_counter_tree.h
index 0daf09e08111..2e8b1ce5cd13 100644
--- a/include/linux/percpu_counter_tree.h
+++ b/include/linux/percpu_counter_tree.h
@@ -10,6 +10,52 @@
 
 #ifdef CONFIG_SMP
 
+#if NR_CPUS == (1U << 0)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	0
+#elif NR_CPUS <= (1U << 1)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	1
+#elif NR_CPUS <= (1U << 2)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	3
+#elif NR_CPUS <= (1U << 3)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	7
+#elif NR_CPUS <= (1U << 4)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	7
+#elif NR_CPUS <= (1U << 5)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	11
+#elif NR_CPUS <= (1U << 6)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	21
+#elif NR_CPUS <= (1U << 7)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	21
+#elif NR_CPUS <= (1U << 8)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	37
+#elif NR_CPUS <= (1U << 9)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	73
+#elif NR_CPUS <= (1U << 10)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	149
+#elif NR_CPUS <= (1U << 11)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	293
+#elif NR_CPUS <= (1U << 12)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	585
+#elif NR_CPUS <= (1U << 13)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	1173
+#elif NR_CPUS <= (1U << 14)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	2341
+#elif NR_CPUS <= (1U << 15)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	4681
+#elif NR_CPUS <= (1U << 16)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	4681
+#elif NR_CPUS <= (1U << 17)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	8777
+#elif NR_CPUS <= (1U << 18)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	17481
+#elif NR_CPUS <= (1U << 19)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	34953
+#elif NR_CPUS <= (1U << 20)
+# define PERCPU_COUNTER_TREE_STATIC_NR_ITEMS	69905
+#else
+# error "Unsupported number of CPUs."
+#endif
+
 struct percpu_counter_tree_level_item {
 	atomic_t count;			/*
 					 * Count the number of carry fort this tree item.
@@ -18,6 +64,9 @@ struct percpu_counter_tree_level_item {
 					 */
 } ____cacheline_aligned_in_smp;
 
+#define PERCPU_COUNTER_TREE_ITEMS_STATIC_SIZE	\
+	(PERCPU_COUNTER_TREE_STATIC_NR_ITEMS * sizeof(struct percpu_counter_tree_level_item))
+
 struct percpu_counter_tree {
 	/* Fast-path fields. */
 	unsigned int __percpu *level0;	/* Pointer to per-CPU split counters (tree level 0). */
@@ -92,6 +141,8 @@ int percpu_counter_tree_approximate_sum(struct percpu_counter_tree *counter)
 
 #else	/* !CONFIG_SMP */
 
+#define PERCPU_COUNTER_TREE_ITEMS_STATIC_SIZE	0
+
 struct percpu_counter_tree_level_item;
 
 struct percpu_counter_tree {
-- 
2.39.5

[PATCH v1 5/5] tsacct: Skip all kernel threads

[Mathieu Desnoyers]

When we hit acct_account_cputime within a irq handler over a kthread
that happens to use a userspace mm, we end up summing up the mm's RSS
into the tsk acct_rss_mem1, which eventually decays.

I don't see a good rationale behind tracking the mm's rss in that way
when a kthread use a userspace mm temporarily through use_mm.

It causes issues with init_mm and efi_mm which only partially initialize
their mm_struct.

Skip all kernel threads in acct_account_cputime(), not just those that
happen to have a NULL mm.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: linux-mm@kvack.org
---
 kernel/tsacct.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/tsacct.c b/kernel/tsacct.c
index 6ea2f6363b90..3ef149b1245d 100644
--- a/kernel/tsacct.c
+++ b/kernel/tsacct.c
@@ -125,7 +125,7 @@ static void __acct_update_integrals(struct task_struct *tsk,
 {
 	u64 time, delta;
 
-	if (!likely(tsk->mm))
+	if (!tsk->mm || (tsk->flags & PF_KTHREAD))
 		return;
 
 	time = stime + utime;
-- 
2.39.5

Re: [PATCH v1 0/5] MM_CID and HPCC mm_struct static init fixes

[Andrew Morton]

On Sun, 21 Dec 2025 18:29:21 -0500 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:

> Mark Brown reported a regression [1] on linux next due to the
> hierarchical percpu counters (HPCC). You mentioned they were only in
> mm-new (and therefore not pulled into -next) [2], but it looks like they
> got more exposure that we expected. :)

We try to please ;)

Mark's report is why this series remains in mm-new and didn't get
promoted to mm-unstable (and hence linux-next).

Also, I anticipate a new version of this series thanks to
https://lkml.kernel.org/r/2fb78e41-5e56-425f-925f-a29524355d2c@efficios.com

> This bug hunting got me to fix static initialization issues in both
> MM_CID (for upstream) and HPCC (mm-new). Mark tested my series and
> confirmed that it fixes his issues.
> 
> Please consider the HPCC fixes for mm-new, and the MM_CID fixes for
> upstream.
> 
> This series is based on mm-new
> commit 287373d0b6ee ("selftests/mm: fix comment for check_test_requirements")

Well, simply appending this series to your "mm: Fix OOM killer
inaccuracy on large many-core systems, v10" will create an annoying
bisection hole.

Also, there are some cc:stable fixes in here which depend upon the
presence of "mm: Fix OOM killer inaccuracy on large many-core systems"
series, which is all backwards.


So for now I'll toss this series in there for testing but I'll ask for
a redo, please.

- All cc:stable patches prepared in a standalone fashion against
  latest mainline.

- Any non-cc:stable patches which fix things in current mm-new come
  next.  Perhaps view these as preparation for "mm: Fix OOM killer
  inaccuracy on large many-core systems".

- Then "mm: Fix OOM killer inaccuracy on large many-core systems,
  v11" which incorporates anything which didn't make it into the above
  two categories.

That could be as many as three patch series. Thanks.

Re: [PATCH v1 0/5] MM_CID and HPCC mm_struct static init fixes

[Mathieu Desnoyers]

On 2025-12-22 20:59, Andrew Morton wrote:
[...]
> Well, simply appending this series to your "mm: Fix OOM killer
> inaccuracy on large many-core systems, v10" will create an annoying
> bisection hole.
> 
> Also, there are some cc:stable fixes in here which depend upon the
> presence of "mm: Fix OOM killer inaccuracy on large many-core systems"
> series, which is all backwards.
> 
> 
> So for now I'll toss this series in there for testing but I'll ask for
> a redo, please.
> 
> - All cc:stable patches prepared in a standalone fashion against
>    latest mainline.
> 
> - Any non-cc:stable patches which fix things in current mm-new come
>    next.  Perhaps view these as preparation for "mm: Fix OOM killer
>    inaccuracy on large many-core systems".
> 
> - Then "mm: Fix OOM killer inaccuracy on large many-core systems,
>    v11" which incorporates anything which didn't make it into the above
>    two categories.
> 
> That could be as many as three patch series. Thanks.

Will do! It may take a few days due to holiday vacations here.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com