Re: [PATCH v6 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes

[Rik van Riel <riel@surriel.com>]

On Thu, 2025-01-23 at 10:07 +0100, Peter Zijlstra wrote:
> On Wed, Jan 22, 2025 at 08:13:03PM -0500, Rik van Riel wrote:
> > On Wed, 2025-01-22 at 09:38 +0100, Peter Zijlstra wrote:
> > > 
> > > Looking at this more... I'm left wondering, did 'we' look at any
> > > other
> > > architecture code at all? 
> > > 
> > > For example, look at arch/arm64/mm/context.c and see how their
> > > reset
> > > works. Notably, they are not at all limited to reclaiming free'd
> > > ASIDs,
> > > but will very aggressively take back all ASIDs except for the
> > > current
> > > running ones.
> > > 
> > I did look at the ARM64 code, and while their reset
> > is much nicer, it looks like that comes at a cost on
> > each process at context switch time.
> > 
> > In new_context(), there is a call to check_update_reserved_asid(),
> > which will iterate over all CPUs to check whether this
> > process's ASID is part of the reserved list that got
> > carried over during the rollover.
> > 
> > I don't know if that would scale well enough to work
> > on systems with thousands of CPUs.
> 
> So assuming something like 1k CPUs and !PTI, we only have like 4
> PCIDs
> per CPU on average, and rollover could be frequent.
> 
> While an ARM64 with 1k CPUs and !PTI would have an average of 64
> ASIDs
> per CPU, and rollover would be far less frequent.

Not necessarily. On ARM64, every short lived task will
get a global ASID, while on x86_64 only longer lived
processes that are simultaneously active on multiple
CPUs get a global ASID.

The situation could be fairly bad for both, which is
why I would like to solve the O(n^2) issues with the
rollover code before adding that in to our x86_64
side :)

I fully agree we should probably move in that direction,
but I would like to make the worst case in the rollover-reuse
cheaper.

> 
> That is to say, their larger ASID space (16 bits, vs our 12)
> definitely
> helps. But at some point yeah, this will become a problem.
> 
> Notably, I think think a 2 socket Epyc Turin with 192C is one of the
> larger off-the-shelf systems atm, that gets you 768 CPUs and that is
> already uncomfortably tight with our PCID space.
> 
> 
> 

-- 
All Rights Reversed.