Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64

["Theodore Ts'o" <tytso@mit.edu>]

On Thu, Mar 20, 2025 at 08:05:26PM -0700, Darrick J. Wong wrote:
> > So now applications need to be careful to not submit any direct-io &
> > buffered-io in parallel with such above patterns on a raw block device,
> > correct? That is what I would like to confirm.
> 
> I think that's correct, and kind of horrifying if true.  I wonder if
> ->invalidate_folio might be a reasonable way to clear the uptodate bits
> on the relevant parts of a large folio without having to split or remove
> it?

FWIW, I've always recommended not mixing DIO and buffered I/O, either
for filesystems or block device.

> > >> And IIUC, what Linux recommends is to never mix any kind of direct-io
> > >> and buffered-io when doing I/O on raw block devices, but I cannot find
> > >> this recommendation in any Documentation? So can someone please point me
> > >> one where we recommend this?
> > 
> > And this ^^^

From the open(2) man page, in the NOTES section:

    Applications should avoid mixing O_DIRECT and normal I/O to the
    same file, and especially to overlap‐ ping byte regions in the
    same file.  Even when the filesystem correctly handles the
    coherency issues in this situation, overall I/O throughput is
    likely to be slower than using either mode alone.  Likewise,
    applications should avoid mixing mmap(2) of files with direct I/O
    to the same files.

As I recall, in the eary days Linux's safety for DIO and Bufered I/O
was best efforts, and other Unix system the recommendation to "don't
mix the streams" was far stronger.  Even if it works reliably for
Linux, it's still something I recommend that people avoid if at all
possible.

						- Ted