Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64

[Ritesh Harjani (IBM) <ritesh.list@gmail.com>]

Christoph Hellwig <hch@lst.de> writes:

> On Fri, Mar 21, 2025 at 12:56:04AM -0400, Theodore Ts'o wrote:
>> As I recall, in the eary days Linux's safety for DIO and Bufered I/O
>> was best efforts, and other Unix system the recommendation to "don't
>> mix the streams" was far stronger.  Even if it works reliably for
>> Linux, it's still something I recommend that people avoid if at all
>> possible.
>
> It still is a best effort, just a much better effort now.  It's still
> pretty easy to break the coherent.


Thanks Ted & Christoph for the info. Do you think we should document
this recommendation, maybe somewhere in the kernel Documentation where
we can also lists the possible cases where the coherency could break?
(I am not too well aware of those cases though).

One case which I recently came across was where the application was not
setting --setbsz properly on a block device where system's pagesize is
64k. This if I understand correctly will install 1 buffer_head for a 64k
page for any buffered-io operation. Then, if someone mixes the 4k
buffered-io write, right next to 4k direct-io write, then well it
definitely ends up problematic. Because the 4k buffered-io write will
end up making a read-modify-write over a 64k page (1 buffer_head). This
means we now have the entire 64k dirty page, while there is also a
direct-io write operation in that region. This means both writes got
overlapped, hence causing coherency issues.

Such cases, I believe, are easy to miss. And now, with large folios
being used in block devices, I am not sure if there is much value in
applications mixing buffered I/O and direct I/O. Since direct I/O write
will just end up invalidating the entire large folio, that means it
could negate any benefits of using buffered I/O alongside it, on the
same block device.


-ritesh