RE: [PATCH v2 0/5] mm/hwpoison: Fix regressions in memory failure handling

["Luck, Tony" <tony.luck@intel.com>]

> > For instr case: user process is killed by a SIGBUS signal
> >
> > Commit 046545a661af ("mm/hwpoison: fix error page recovered but reported "not
> > recovered"") introduced a bug that kill_accessing_process() return -EHWPOISON
> > for instr case, as result, kill_me_maybe() send a SIGBUS to user process.
>
> This makes my head hurt... a race between the CMCI reporting an uncorrected
> error... why does the CMCI report uncorrected errors? This sounds like some
> nasty confusion.

My head hurts too. The problem is the evolution and subsequent overloading of
limited signal options in Intel error reporting.

Prior to Icelake memory controllers reported patrol scrub events that detected
a previously unseen uncorrected error in memory by signaling a broadcast
machine check with an SRAO (Software Recoverable Action Optional) signature
in the machine check bank.

This was overkill. It's not an urgent problem. No core is on the verge of consuming
that bad data.

But the fix causes the confusion. The machine check bank signature was changed
to UCNA (Uncorrected, No Action required), and signal changed to #CMCI (since
that was the only option available in the toolbox :-(

That's how we ended up with *UN*corrected errors tied to *C*MCI.

Just to add to the confusion, Linux does take an action (in uc_decode_notifier())
to try to offline the page despite the UC*NA* signature name.

> And you've basically reused the format and wording of 046545a661af for your
> commit message and makes staring at those a PITA.
>
> Tony, what's going on with that CMCI and SRAR race?

Now the race ... having decided that CMCI/UCNA is the best action for patrol
scrub errors, the memory controller uses it for reads too. But the memory controller
is executing asynchronously from the core, and can't tell the difference between a
"real" read and a speculative read. So it will do CMCI/UCNA if an error is found in
any read.

Thus:

1) Core is clever and thinks address A is needed soon, issues a speculative read.
2) Core finds it is going to use address A soon after sending the read request
3) The CMCI from the memory controller is in a race with the core that will soon try to retire the load from address A.

Quite often (because speculation has got better) the CMCI from the memory controller
is delivered before the core is committed to the instruction reading address A, so the
interrupt is taken, and Linux offlines the page (marking it as poison).

When the interrupt returns, the core gets to the load instruction, and gets a #PF because
the offline process marked the page not-present and flushed the TLB.

Finally the #PF handler tries to fix the page fault, sees that page is marked as poison
so sends SIGBUS to the process.

Note, AMD might have a similar race with the MCE_DEFERRED_SEVERITY signal?
(but with less confusing naming).

-Tony