Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Tue, 2025-01-14 at 09:08 +0530, Shyam Prasad N wrote:
> The Linux kernel does buffered reads and writes using the page cache
> layer, where the filesystem reads and writes are offloaded to the
> VM/MM layer. The VM layer does a predictive readahead of data by
> optionally asking the filesystem to read more data asynchronously
> than what was requested.
> 
> The VFS layer maintains a dentry cache which gets populated during
> access of dentries (either during readdir/getdents or during lookup).
> This dentries within a directory actually forms the address space for
> the directory, which is read sequentially during getdents. For
> network filesystems, the dentries are also looked up during
> revalidate.
> 
> During sequential getdents, it makes sense to perform a readahead
> similar to file reads. Even for revalidations and dentry lookups,
> there can be some heuristics that can be maintained to know if the
> lookups within the directory are sequential in nature. With this, the
> dentry cache can be pre-populated for a directory, even before the
> dentries are accessed, thereby boosting the performance. This could
> give even more benefits for network filesystems by avoiding costly
> round trips to the server.

If your theory were correct, especially the bit about using the dentry
cache to retain the readahead information, wouldn't a precursor
actually be populating the dentry cache on iterate_dir() which is the
engine for both the readdir() and getdents() syscalls?  It strikes me
the reason we don't do dentry population here is partly because the
lookup() on each name would slow everything down (iterate_dir is very
locking light weight because it needs to be fast) and partly because
whatever is doing the directory read may only be interested in a single
name.  The only userspace operation you can guarantee is going to do a
lookup() for every name is ls -l, but that doesn't seem to be a good
one to optimize for.

Regards,

James