Re: [PATCH 06/10] x86/entry: Move nmi entry/exit into common code

[Ira Weiny <ira.weiny@intel.com>]

On Fri, Oct 23, 2020 at 11:50:11PM +0200, Thomas Gleixner wrote:
> On Thu, Oct 22 2020 at 15:26, ira weiny wrote:
> 
> > From: Thomas Gleixner <tglx@linutronix.de>
> >
> > Lockdep state handling on NMI enter and exit is nothing specific to X86. It's
> > not any different on other architectures. Also the extra state type is not
> > necessary, irqentry_state_t can carry the necessary information as well.
> >
> > Move it to common code and extend irqentry_state_t to carry lockdep
> > state.
> 
> This lacks something like:
> 
>  [ Ira: Made the states a union as they are mutually exclusive and added
>         the missing kernel doc ]

Fair enough.  done.

> 
> Hrm.
>  
> >  #ifndef irqentry_state
> >  typedef struct irqentry_state {
> > -	bool	exit_rcu;
> > +	union {
> > +		bool	exit_rcu;
> > +		bool	lockdep;
> > +	};
> >  } irqentry_state_t;
> >  #endif
> 
>   -E_NO_KERNELDOC

Adding: Paul McKenney

I'm happy to write something but I'm very unfamiliar with this code.  So I'm
getting confused what exactly exit_rcu is flagging.

I can see that exit_rcu is a bad name for the state used in
irqentry_nmi_[enter|exit]().  Furthermore, I see why 'lockdep' is a better
name.  But similar lockdep handling is used in irqentry_exit() if exit_rcu is
true...


Given my limited knowledge; here is my proposed text:

/**
 * struct irqentry_state - Opaque object for exception state storage
 * @exit_rcu: Used exclusively in the irqentry_*() calls; tracks if the
 *            exception hit the idle task which requires special handling,
 *            including calling rcu_irq_exit(), when the exception exits.
 * @lockdep: Used exclusively in the irqentry_nmi_*() calls; ensures lockdep
 *           tracking is maintained if hardirqs were already enabled
 *
 * This opaque object is filled in by the irqentry_*_enter() functions and
 * should be passed back into the corresponding irqentry_*_exit() functions
 * when the exception is complete.
 *
 * Callers of irqentry_*_[enter|exit]() should consider this structure opaque
 * and all members private.  Descriptions of the members are provided to aid in
 * the maintenance of the irqentry_*() functions.
 */


Perhaps Paul can enlighten me on how exit_rcu is used beyond just flagging a
call to rcu_irq_exit()?

Why do we call lockdep_hardirqs_off() only when in the idle task?  That implies
that regs_irqs_disabled() can only be false if we were in the idle task to
match up the lockdep on/off calls.  This does not make sense to me because why
do we need the extra check for exit_rcu?  I'm still trying to understand when
regs_irqs_disabled() is false.


        } else if (!regs_irqs_disabled(regs)) {
...
        } else {
                /*
                 * IRQ flags state is correct already. Just tell RCU if it
                 * was not watching on entry.
                 */
                if (state.exit_rcu)
                        rcu_irq_exit();
        }

Also, the comment in irqentry_enter() refers to irq_enter_from_user_mode() which
does not seem to exist anymore.  So I'm not sure what careful sequence it is
referring to.

        /*
         * If RCU is not watching then the same careful
         * sequence vs. lockdep and tracing is required
         * as in irq_enter_from_user_mode().
         */

?

Ira