Re: [Ksummit-discuss] [BELATED CORE TOPIC] context tracking / nohz / RCU state

[Frederic Weisbecker <fweisbec@gmail.com>]

On Wed, Aug 12, 2015 at 09:03:42AM -0700, Paul E. McKenney wrote:
> On Wed, Aug 12, 2015 at 04:27:34PM +0200, Frederic Weisbecker wrote:
> > On Tue, Aug 11, 2015 at 08:42:58PM +0200, Luis R. Rodriguez wrote:
> > > On Tue, Aug 11, 2015 at 10:49:36AM -0700, Andy Lutomirski wrote:
> > > > This is a bit late, but here goes anyway.
> > > > 
> > > > Having played with the x86 context tracking hooks for awhile, I think
> > > > it would be nice if core code that needs to be aware of CPU context
> > > > (kernel, user, idle, guest, etc) could come up with single,
> > > > comprehensible, easily validated set of hooks that arch code is
> > > > supposed to call.
> > > > 
> > > > Currently we have:
> > > > 
> > > >  - RCU hooks, which come in a wide variety to notify about IRQs, NMIs, etc.
> > > > 
> > > >  - Context tracking hooks.  Only used by some arches.  Calling these
> > > > calls the RCU hooks for you in most cases.  They have weird
> > > > interactions with interrupts and they're slow.
> > > > 
> > > >  - vtime.  Beats the heck out of me.
> > > > 
> > > >  - Whatever deferred things Christoph keeps reminding us about.
> > > > 
> > > > Honestly, I don't fully understand what all these hooks are supposed
> > > > to do, nor do I care all that much.  From my perspective, the code
> > > > code should be able to do whatever it wants and rely on appropriate
> > > > notifications from arch code.  It would be great if we could come up
> > > > with something straightforward that covers everything.  For example:
> > > > 
> > > > user_mode_to_kernel_mode()
> > > > kernel_mode_to_user_mode()
> > > > kernel_mode_to_guest_mode()
> > > > in_a_periodic_tick()
> > > > starting_nmi()
> > > > ending_nmi()
> > > > may_i_turn_off_ticks_right_now()
> > > > or, better yet:
> > > > i_am_turning_off_ticks_right_now_and_register_your_own_darned_hrtimer_if_thats_a_problem()
> > > >
> > > > Some arches may need:
> > > > 
> > > > i_am_lame_and_forgot_my_previous_context()
> > > 
> > > Can all this information be generalized with some basic core hooks
> > > or could some of this contextual informatioin typically vary depending
> > > on the sequence we are in ? It sounds like its the later and that's
> > > the issue ?
> > 
> > That's what we do with context tracking. It tracks the context (user/kernel)
> > and stores these informations. And indeed the contextual informations can vary
> > depending for example if an exception triggered in userspace or kernelspace.
> 
> Another question of interest is "Can things be arranged so that RCU uses
> the context-tracking information directly in place of rcu_dynticks?"
> In theory, the answer is clearly "yes", but the reason that RCU's
> accounting is heavyweight is the need to get precise state readout on
> other CPUs.  So it is quite possible that making RCU directly use the
> context-tracking information will make that tracking slower and more
> complex, so that the overall effect will be zero net improvement.

Yeah, that's partly what I meant by "it's possible, but we might not be proud of
the result".

> But it does seem worth a look.

Sure.