Re: [LSF/MM/BPF TOPIC] Buffered atomic writes

[Dave Chinner <dgc@kernel.org>]

On Wed, Feb 18, 2026 at 07:47:39AM +0100, Christoph Hellwig wrote:
> On Wed, Feb 18, 2026 at 12:04:43PM +1100, Dave Chinner wrote:
> > > > > I'd call it RWF_WRITETHROUGH but otherwise it makes sense.
> > > > > 
> > > > 
> > > > One naive question: semantically what will be the difference between
> > > > RWF_DSYNC and RWF_WRITETHROUGH?
> > 
> > None, except that RWF_DSYNC provides data integrity guarantees.
> 
> Which boils down to RWF_DSYNC still writing out the inode and flushing
> the cache.
> 
> > > Which
> > > wouldn't be needed for RWF_WRITETHROUGH, right?
> > 
> > Correct, there shouldn't be any data integrity guarantees associated
> > with plain RWF_WRITETHROUGH.
> 
> Which makes me curious if the plain RWF_WRITETHROUGH would be all
> that useful.

For modern SSDs, I think the answer is yes.

e.g. when you are doing lots of small writes to many files from many
threads, it bottlenecks on single threaded writeback. All of the IO
is submitted by background writeback which runs out of CPU fairly
quickly. We end up dirty throttling and topping out at ~100k random
4kB buffered writes IOPS regardless of how much submitter
concurrency we have.

If we switch that to RWF_WRITETHROUGH, we now have N submitting
threads that can all work in parallel, we get pretty much zero dirty
folio backlog (so no dirty throttling and more consistent IO
latency) and throughput can scales much higher because we have IO
submitter concurrency to spread the CPU load around.

I did a fsmark test of a write-though hack a couple of years back,
creating and writing 4kB data files concurrently in a directory per
thread.  With vanilla writeback, it topped out at about 80k 4kB file
creates/s from 4 threads and only wnet slower the more I increased
the userspace create concurrency.

Using writethrough submission, it topped out at about 400k 4kB file
creates/s from 32 threads and was largely limited in the fsmark
tasks by the CPU overhead for file creation, user data copying and
data extent space allocation.

I also did a multi-file, multi-process random 4kB write test with
fio, using files much larger than memory and long runtimes. Once the
normal background write path started dirty throttling, it ran at
about 100k 4kB write IOPS, again limited by the single threaded writeback
flusher using all it's CPU time for allocating blocks during
writeback.

Using writethrough, I saw about 900k IOPS being sustained right from
the start, largely limited by a combination of CPU usage and IO
latency in the fio task context. In comparison, the same workload
with DIO ran to the storage capability of 1.6M IOPS because it had
significantly lower CPU usage and IO latency.

I also did some kernel compile tests with writethrough for all
buffered write IO. On fast storage there was neglible
difference in performance between vanilla buffered writes and
submitter driver blocking write-through. This result made me
question the need for caching on modern SSDs at all :)

-Dave.
-- 
Dave Chinner
dgc@kernel.org