Re: [PATCH 0/4] Introduce QPW for per-cpu operations

[Michal Hocko <mhocko@suse.com>]

On Thu 19-02-26 12:27:23, Marcelo Tosatti wrote:
> Michal,
> 
> Again, i don't see how moving operations to happen at return to 
> kernel would help (assuming you are talking about 
> "context_tracking,x86: Defer some IPIs until a user->kernel transition").

Nope, I am not talking about IPIs, although those are an example of pcp
state as well. I am sorry I do not have a link handy, I am pretty sure
Frederic will have that. Another example, though, was vmstat flushes
that need to be pcp. There are many other examples. 

[...]

> You can't delay either kmalloc (removal of object from per-CPU freelist), 
> or kfree (return of object from per-CPU freelist), or kmem_cache_shrink 
> or kmem_cache_shrink to return to userspace.

Why?

> What i missing something here? (or do you have something on your mind
> which i can't see).

I am really sorry for being really vague here. Let me try to draw
a more abstract problem definition and let's see whether we are trying
to solve the same problem here. Maybe not...

I believe the main usecase of the interest here is uninterrupted
userspace execution and delayed pcp work that migh disturb such workload
after it has returned to the userspace. Right?
That is usually hauskeeping work that for, performance reasons, doesn't
happen in hot paths while the workload was executing in the kernel
space.

There are more ways to deal with that. You can either change the hot
path to not require deferred operation (tricky withtout introducing
regressions for most workloads) or you can define a more suitable place
to perform the housekeeping while still running in the kernel. 

Your QWP work relies on local_lock -> spin_lock transition and
performing the pcp work remotely so you do not need to disturb that
remote cpu. Correct?

Alternative approach is to define a moment when the housekeeping
operation is performed on that local cpu while still running in the
kernel space - e.g. when returning to the userspace. Delayed work is
then not necessary and userspace is not disrupted after returning to the
userspace.

Do I make more sense or does the above sound like a complete gibberish?
-- 
Michal Hocko
SUSE Labs