Re: [PATCH v6 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes

[Peter Zijlstra <peterz@infradead.org>]

On Wed, Jan 22, 2025 at 08:13:03PM -0500, Rik van Riel wrote:
> On Wed, 2025-01-22 at 09:38 +0100, Peter Zijlstra wrote:
> > 
> > Looking at this more... I'm left wondering, did 'we' look at any
> > other
> > architecture code at all? 
> > 
> > For example, look at arch/arm64/mm/context.c and see how their reset
> > works. Notably, they are not at all limited to reclaiming free'd
> > ASIDs,
> > but will very aggressively take back all ASIDs except for the current
> > running ones.
> > 
> I did look at the ARM64 code, and while their reset
> is much nicer, it looks like that comes at a cost on
> each process at context switch time.
> 
> In new_context(), there is a call to check_update_reserved_asid(),
> which will iterate over all CPUs to check whether this
> process's ASID is part of the reserved list that got
> carried over during the rollover.
> 
> I don't know if that would scale well enough to work
> on systems with thousands of CPUs.

So assuming something like 1k CPUs and !PTI, we only have like 4 PCIDs
per CPU on average, and rollover could be frequent.

While an ARM64 with 1k CPUs and !PTI would have an average of 64 ASIDs
per CPU, and rollover would be far less frequent.

That is to say, their larger ASID space (16 bits, vs our 12) definitely
helps. But at some point yeah, this will become a problem.

Notably, I think think a 2 socket Epyc Turin with 192C is one of the
larger off-the-shelf systems atm, that gets you 768 CPUs and that is
already uncomfortably tight with our PCID space.