Re: [RFC] Bypass filesystems for reading cached pages

[Dave Chinner <david@fromorbit.com>]

On Fri, Jun 19, 2020 at 08:50:36AM -0700, Matthew Wilcox wrote:
> 
> This patch lifts the IOCB_CACHED idea expressed by Andreas to the VFS.
> The advantage of this patch is that we can avoid taking any filesystem
> lock, as long as the pages being accessed are in the cache (and we don't
> need to readahead any pages into the cache).  We also avoid an indirect
> function call in these cases.

What does this micro-optimisation actually gain us except for more
complexity in the IO path?

i.e. if a filesystem lock has such massive overhead that it slows
down the cached readahead path in production workloads, then that's
something the filesystem needs to address, not unconditionally
bypass the filesystem before the IO gets anywhere near it.

> This could go horribly wrong if filesystems rely on doing work in their
> ->read_iter implementation (eg checking i_size after acquiring their
> lock) instead of keeping the page cache uptodate.  On the other hand,
> the ->map_pages() method is already called without locks, so filesystems
> should already be prepared for this.

Oh, gawd, we have *yet another* unlocked page cache read path that
can race with invalidations during fallocate() operations?

/me goes and looks at filemap_map_pages()

Yup, filemap_map_pages() is only safe against invalidations beyond
EOF (i.e. truncate) and can still race with invalidations within
EOF. So, yes, I'm right in that this path is not safe to run without
filesystem locking to serialise the IO against fallocate()...

Darrick, it looks like we need to wrap filemap_map_pages() with the
XFS_MMAPLOCK_SHARED like we do for all the other page fault paths
that can call into the IO path.

> Arguably we could do something similar for writes.  I'm a little more
> scared of that patch since filesystems are more likely to want to do
> things to keep their fies in sync for writes.

Please, no.  We can have uptodate cached pages over holes, unwritten
extents, shared extents, etc but they all require filesystem level
serialisation and space/block allocation work *before* we copy data
into the page. i.e. if allocation/space reservation fails, we need
to abort before changing data.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com