Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries

[Paulo Alcantara <pc@manguebit.com>]

Benjamin Coddington <bcodding@redhat.com> writes:

> On 14 Jan 2025, at 8:24, Amir Goldstein wrote:
>
>> On Tue, Jan 14, 2025 at 4:38 AM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>>>
>>> The Linux kernel does buffered reads and writes using the page cache
>>> layer, where the filesystem reads and writes are offloaded to the
>>> VM/MM layer. The VM layer does a predictive readahead of data by
>>> optionally asking the filesystem to read more data asynchronously than
>>> what was requested.
>>>
>>> The VFS layer maintains a dentry cache which gets populated during
>>> access of dentries (either during readdir/getdents or during lookup).
>>> This dentries within a directory actually forms the address space for
>>> the directory, which is read sequentially during getdents. For network
>>> filesystems, the dentries are also looked up during revalidate.
>>>
>>> During sequential getdents, it makes sense to perform a readahead
>>> similar to file reads. Even for revalidations and dentry lookups,
>>> there can be some heuristics that can be maintained to know if the
>>> lookups within the directory are sequential in nature. With this, the
>>> dentry cache can be pre-populated for a directory, even before the
>>> dentries are accessed, thereby boosting the performance. This could
>>> give even more benefits for network filesystems by avoiding costly
>>> round trips to the server.
>>>
>>
>> I believe you are referring to READDIRPLUS, which is quite common
>> for network protocols and also supported by FUSE.
>>
>> Unlike network protocols, FUSE decides by server configuration and
>> heuristics whether to "fuse_use_readdirplus" - specifically in readdirplus_auto
>> mode, FUSE starts with readdirplus, but if nothing calls lookup on the
>> directory inode by the time the next getdents call, it stops with readdirplus.
>>
>> I personally ran into the problem that I would like to control from the
>> application, which knows if it is doing "ls" or "ls -l" whether a specific
>> getdents() will use FUSE readdirplus or not, because in some situations
>> where "ls -l" is not needed that can avoid a lot of unneeded IO.
>
> Indeed, we often have folks wanting dramatically different behavior from
> getdents() in NFS, and every time we've tried to improve our heuristics
> someone else shouts "regression"!

In CIFS, we already preload the dcache with the result of
SMB2_QUERY_DIRECTORY, which I believe NFS does the same thing.

Shyam, what's the problem with current approach?