Re: [LSF/MM/BPF TOPIC] Buffered atomic writes

[Andres Freund <andres@anarazel.de>]

Hi,

On 2026-02-18 18:37:45 +0100, Jan Kara wrote:
> On Tue 17-02-26 11:13:07, Andres Freund wrote:
> > Hm. It's somewhat painful to not know when we can write in what mode again -
> > with DIO that's not an issue. I guess we could use
> > sync_file_range(SYNC_FILE_RANGE_WAIT_BEFORE) if we really needed to know?
> > Although the semantics of the SFR flags aren't particularly clear, so maybe
> > not?
> 
> If you used RWF_WRITETHROUGH for your writes (so you are sure IO has
> already started) then sync_file_range(SYNC_FILE_RANGE_WAIT_BEFORE) would
> indeed be a safe way of waiting for that IO to complete (or just wait for
> the write(2) syscall itself to complete if we make RWF_WRITETHROUGH wait
> for IO completion as Dave suggests - but I guess writes may happen from
> multiple threads so that may be not very convenient and sync_file_range(2)
> might be actually easier).

For us a synchronously blocking RWF_WRITETHROUGH would actually be easier, I
think.

The issue with writes from multiple threads actually goes the other way for us
- without knowing when the IO actually completes, our buffer pool's state
cannot reflect whether there is ongoing IO for a buffer or not. So we would
always have to do sync_file_range(SYNC_FILE_RANGE_WAIT_BEFORE) before doing
further IO.

Not knowing how many writes are actually outstanding also makes it harder for
us to avoid overwhelming the storage (triggering e.g. poor commit latency).

Greetings,

Andres Freund