Re: [PATCH v2 0/8] Cache coherency management subsystem

["H. Peter Anvin" <hpa@zytor.com>]

On June 25, 2025 1:52:04 AM PDT, Peter Zijlstra <peterz@infradead.org> wrote:
>On Tue, Jun 24, 2025 at 04:47:56PM +0100, Jonathan Cameron wrote:
>
>> On x86 there is the much loved WBINVD instruction that causes a write back
>> and invalidate of all caches in the system. It is expensive but it is
>
>Expensive is not the only problem. It actively interferes with things
>like Cache-Allocation-Technology (RDT-CAT for the intel folks). Doing
>WBINVD utterly destroys the cache subsystem for everybody on the
>machine.
>
>> necessary in a few corner cases. 
>
>Don't we have things like CLFLUSH/CLFLUSHOPT/CLWB exactly so that we can
>avoid doing dumb things like WBINVD ?!?
>
>> These are cases where the contents of
>> Physical Memory may change without any writes from the host. Whilst there
>> are a few reasons this might happen, the one I care about here is when
>> we are adding or removing mappings on CXL. So typically going from
>> there being actual memory at a host Physical Address to nothing there
>> (reads as zero, writes dropped) or visa-versa. 
>
>> The
>> thing that makes it very hard to handle with CPU flushes is that the
>> instructions are normally VA based and not guaranteed to reach beyond
>> the Point of Coherence or similar. You might be able to (ab)use
>> various flush operations intended to ensure persistence memory but
>> in general they don't work either.
>
>Urgh so this. Dan, Dave, are we getting new instructions to deal with
>this? I'm really not keen on having WBINVD in active use.
>

WBINVD is the nuclear weapon to use when you have lost all notion of where the problematic data can be, and amounts to a full reset of the cache system. 

WBINVD can block interrupts for many *milliseconds*, system wide, and so is really only useful for once-per-boot type events, like MTRR initialization.