Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries

[Shyam Prasad N <nspmangalore@gmail.com>]

Hi Paulo,

On Tue, Jan 14, 2025 at 8:31 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Benjamin Coddington <bcodding@redhat.com> writes:
>
> > On 14 Jan 2025, at 8:24, Amir Goldstein wrote:
> >
> >> On Tue, Jan 14, 2025 at 4:38 AM Shyam Prasad N <nspmangalore@gmail.com> wrote:
> >>>
> >>> The Linux kernel does buffered reads and writes using the page cache
> >>> layer, where the filesystem reads and writes are offloaded to the
> >>> VM/MM layer. The VM layer does a predictive readahead of data by
> >>> optionally asking the filesystem to read more data asynchronously than
> >>> what was requested.
> >>>
> >>> The VFS layer maintains a dentry cache which gets populated during
> >>> access of dentries (either during readdir/getdents or during lookup).
> >>> This dentries within a directory actually forms the address space for
> >>> the directory, which is read sequentially during getdents. For network
> >>> filesystems, the dentries are also looked up during revalidate.
> >>>
> >>> During sequential getdents, it makes sense to perform a readahead
> >>> similar to file reads. Even for revalidations and dentry lookups,
> >>> there can be some heuristics that can be maintained to know if the
> >>> lookups within the directory are sequential in nature. With this, the
> >>> dentry cache can be pre-populated for a directory, even before the
> >>> dentries are accessed, thereby boosting the performance. This could
> >>> give even more benefits for network filesystems by avoiding costly
> >>> round trips to the server.
> >>>
> >>
> >> I believe you are referring to READDIRPLUS, which is quite common
> >> for network protocols and also supported by FUSE.
> >>
> >> Unlike network protocols, FUSE decides by server configuration and
> >> heuristics whether to "fuse_use_readdirplus" - specifically in readdirplus_auto
> >> mode, FUSE starts with readdirplus, but if nothing calls lookup on the
> >> directory inode by the time the next getdents call, it stops with readdirplus.
> >>
> >> I personally ran into the problem that I would like to control from the
> >> application, which knows if it is doing "ls" or "ls -l" whether a specific
> >> getdents() will use FUSE readdirplus or not, because in some situations
> >> where "ls -l" is not needed that can avoid a lot of unneeded IO.
> >
> > Indeed, we often have folks wanting dramatically different behavior from
> > getdents() in NFS, and every time we've tried to improve our heuristics
> > someone else shouts "regression"!
>
> In CIFS, we already preload the dcache with the result of
> SMB2_QUERY_DIRECTORY, which I believe NFS does the same thing.
>
> Shyam, what's the problem with current approach?

We load the dentry cache with results of QueryDirectory. But what I'm
proposing here is a read ahead, even before the next readdir is done
by the application. i.e. the idea is that the data necessary to emit
dentries is already in the cache before it is even called. That should
speed up the overall directory reads.

-- 
Regards,
Shyam