Re: Is it possible to corrupt disk when writeback page with undetected UE?

[Jane Chu <jane.chu@oracle.com>]

On 9/15/2022 5:30 PM, Yang Shi wrote:
> On Thu, Sep 15, 2022 at 5:27 PM Jane Chu <jane.chu@oracle.com> wrote:
>>
>> On 9/15/2022 3:50 PM, Luck, Tony wrote:
>>>> Suppose there is a UE in a DRAM page that is backed by a disk file.
>>>> The UE hasn't been reported to the kernel, but low level firmware
>>>> initiated scrubbing has already logged the UE.
>>>>
>>>> The page is then dirtied by a write, although the write clearly failed,
>>>> it didn't trigger an MCE.
>>>>
>>>> And without a subsequent read from the page, at some point, the page is
>>>> written back to the disk, leaving a PAGE_SIZE of zeros in the targeted
>>>> disk blocks.
>>>>
>>>> Is this mode of disk corruption possible?
>>>
>>> I didn't look at what was written to disk, but I have seen this. My test sequence
>>> was to compile and then immediately run an error injection test program that
>>> injected a memory UC error to an instruction.
>>>
>>> Because the program was freshly compiled, the executable file was in the
>>> page cache with all pages marked as modified. Later a sync (or memory
>>> pressure) wrote the dirty page with poison to filesystem.
>>>
>>> I did see an error reported by the disk controller.
>>
>> Thanks a lot for this information!
>>
>> Were you using madvise to inject an error to a mmap'ed address?
>> or a different tool?  Do you still have the test documented
>> somewhere?
>>
>> And, aside from verifying every write with a read prior to sync,
>> any suggestion to minimize the window of such corruption?
> 
> We discussed the topic at this year's LSFMM summit. Please refer to
> https://lwn.net/Articles/893565/

Thanks!  I'll take a look.

-jane

> 
>>
>> thanks!
>> -jane
>>
>>>
>>> -Tony
>>