Re: [PATCH 16/23] sched: Use lightweight hazard pointers to grab lazy mms

[Nadav Amit <nadav.amit@gmail.com>]

> On Jan 9, 2022, at 11:22 AM, Rik van Riel <riel@surriel.com> wrote:
> 
> On Sun, 2022-01-09 at 00:49 -0800, Nadav Amit wrote:
>> 
>> It is possible for instance to get rid of is_lazy, keep the CPU
>> on mm_cpumask when switching to kernel thread, and then if/when
>> an IPI is received remove it from cpumask to avoid further
>> unnecessary TLB shootdown IPIs.
>> 
>> I do not know whether it is a pure win, but there is a tradeoff.
> 
> That's not a win at all. It is what we used to have before
> the lazy mm stuff was re-introduced, and it led to quite a
> few unnecessary IPIs, which are especially slow on virtual
> machines, where the target CPU may not be running.

You make a good point about VMs.

IIUC Lazy-TLB serves several goals:

1. Avoid arch address-space switch to save switching time and
   TLB misses.
2. Prevent unnecessary IPIs while kernel threads run.
3. Avoid cache-contention on mm_cpumask when switching to a kernel
   thread.

Your concern is with (2), and this one is easy to keep regardless
of the rest.

I am not sure that (3) is actually helpful, since it might lead
to more cache activity than without lazy-TLB, but that is somewhat
orthogonal to everything else.

As for (1), which is the most fragile aspect, unless you use
shadow page-tables, I am not sure there is a significant benefit.