On Fri, Dec 05, 2025 at 12:01:06PM -0800, Shakeel Butt <shakeel.butt@linux.dev> wrote:
> On x86-64, this_cpu_cmpxchg() uses CMPXCHG without LOCK prefix which
> means it is only safe for the local CPU and not for multiple CPUs.
...
> The CMPXCNG without LOCK on CPU A is not safe and thus we need LOCK
> prefix.

Does it mean that this_cpu_cmpxchg() is generally useless? (It appears
so from your analysis.)

> Now concurrently CPU B is running the flusher and it calls
> llist_del_first_init() for CPU A and got rstatc_pcpu->lnode of cgroup C
> which was added by the IRQ/NMI updater.

Or it's rather the case where rstat code combines both this_cpu_* and
remote access from the flusher.

Documentation/core-api/this_cpu_ops.rst washes its hands with:
| Please note that accesses by remote processors to a per cpu area are
| exceptional situations and may impact performance and/or correctness
| (remote write operations) of local RMW operations via this_cpu_*.

I see there's currently only one other user of that in kernel/scs.c
(__scs_alloc() vs scs_cleanup() without even WRITE_ONCE, but the race
would involve CPU hotplug, so its impact may be limited(?)).

I think your learnt-the-hard-way discovery should not only be in
cgroup.c but also in this this_cpu_ops.rst document to be wary
especially with this_cpu_cmpxchg (when dealing with pointers and not
more tolerable counters).


> Consider this scenario: Updater for cgroup stat C on CPU A in process
> context is after llist_on_list() check and before this_cpu_cmpxchg() in
> css_rstat_updated() where it get interrupted by IRQ/NMI. In the IRQ/NMI
> context, a new updater calls css_rstat_updated() for same cgroup C and
> successfully inserts rstatc_pcpu->lnode.
> 
> Now imagine CPU B calling init_llist_node() on cgroup C's
> rstatc_pcpu->lnode of CPU A and on CPU A, the process context updater
> calling this_cpu_cmpxchg(rstatc_pcpu->lnode) concurrently.

Sounds feasible to me.

Thanks,
Michal