Re: [LSF/MM/BPF TOPIC] Improve this_cpu_ops performance for ARM64 (and potentially other architectures)

[Catalin Marinas <catalin.marinas@arm.com>]

More thoughts...

On Thu, Feb 12, 2026 at 05:54:19PM +0000, Catalin Marinas wrote:
> On Wed, Feb 11, 2026 at 03:58:50PM -0800, Yang Shi wrote:
> > So we just use the local address for this_cpu_add/sub/inc/dec and so
> > on, which just manipulate a scalar counter.
> 
> I wonder how much overhead is caused by calling into the scheduler on
> preempt_enable(). It would be good to get some numbers for something
> like the patch below

In case it wasn't obvious, the patch messes up the scheduling, so I
don't propose it as such, only to get some idea of where the bottleneck
is. Maybe it could be made to work with some need_resched() checks.

> (also removing the preempt disabling for
> this_cpu_read() as I don't think it matters - a thread cannot
> distinguish whether it was preempted between TPIDR read and variable
> read or immediately after the variable read; we can't do this for writes
> as other threads may notice unexpected updates).

There's a theoretical case where even this_cpu_read() needs preemption
disabling, e.g.:

thread0:
	preempt_disable();
	this_cpu_write(var, unique_val);
	// check that no-one has seen unique_value;
	this_cpu_write(var, other_val);
	preempt_enable();

thread1:
	this_cpu_read(var);

thread1 is not supposed to see the unique_val but it would if it was
preempted in the middle of the per-cpu op and migrated to another CPU.

> Another wild hack could be to read the kernel instruction at
> (current_pt_regs()->pc - 4) in arch_irqentry_exit_need_resched() and
> return false if it's a read from TPIDR_EL1/2, together with removing the
> preempt disabling.

This one also breaks the kernel scheduling just like using
preempt_enable_no_resched(). It might be possible but in combination
with additional need_resched() checks.

-- 
Catalin