From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F3D4C02180 for ; Wed, 15 Jan 2025 14:30:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97F1D6B0083; Wed, 15 Jan 2025 09:30:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 92FED6B0085; Wed, 15 Jan 2025 09:30:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A8EE6B0088; Wed, 15 Jan 2025 09:30:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5AAC36B0083 for ; Wed, 15 Jan 2025 09:30:34 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D7CA3813BB for ; Wed, 15 Jan 2025 14:30:33 +0000 (UTC) X-FDA: 83009921946.25.345BBA5 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by imf30.hostedemail.com (Postfix) with ESMTP id 5ACB580006 for ; Wed, 15 Jan 2025 14:30:31 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="nc/XGjFP"; spf=pass (imf30.hostedemail.com: domain of nspmangalore@gmail.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=nspmangalore@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736951431; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sCJC02eMOdA0o+qrOQIriyJQhWHEmCP/bNjox33kIpU=; b=pxrJT8EtglRC8Wl9df+YzM5yfglqfOez83QRPi3Z79pTBbhOdyk+WH6mALnsXk44h749NS fTV8YSWv/Of85WRHs04fDvmAnIPQR1Zc3prAlOBp/wWNEtTckrYz8LbgqJQUZtxWeBn9i+ M/0LBLApf3xKj17W2LYDI3st2VZqZfI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736951431; a=rsa-sha256; cv=none; b=2UiDu5ST38PoX0QtW6uFOOfVStpqeLY9ITw8wWquDM9PG54hTYgcnwgAvTd3j2yy7XPXDO OWICy3eNlcXwISk4yvN8Twg+G2gvpGL0dAwQf3u3SKF5UlQlHtD5GImLgxaKB8q2qhYtLC oU/eB8+S+1n2pTswrznzXTqzeVtJm4I= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="nc/XGjFP"; spf=pass (imf30.hostedemail.com: domain of nspmangalore@gmail.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=nspmangalore@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-5d3e9a88793so11390811a12.1 for ; Wed, 15 Jan 2025 06:30:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736951429; x=1737556229; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=sCJC02eMOdA0o+qrOQIriyJQhWHEmCP/bNjox33kIpU=; b=nc/XGjFP09gIh1b+ksc+RMrZLEHcwXlpKhF02jCMfFdHYpHSlN0A+lMlLORB3lTXBq aIOQGEEeC07psbvB6uHG4MACGDbAEqUwQo5Pp1Lj1VEkYO9XyDQ1p3mH9oggxKQkqTwd G28z5l6mhsqkV+QBux/Yr7JctK6rFXZELLo1nX4jJJOvd0s1zmQphBE2E3Lb8laYzD/B 4bF9HAW4Qep7j1VWk9kqk+4QLD3nWfvUVrulABa0/Elhi0i0o142COqtO81EFuqZaCkO YNA3hi8nTD+5VFqivlTT8KXUL9ia2aX9oOIhgSMjsm36sQlx7y/wdlnsds9IGZxKAvC+ EayQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736951429; x=1737556229; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sCJC02eMOdA0o+qrOQIriyJQhWHEmCP/bNjox33kIpU=; b=gdnYyx0ficeS9/kOlgBUwFu2zij7jSd3emyRC7NcZ+g47vhdzsMPfo6vrBCcQwtqpQ o0GLu0k7KxHD0RJ7ZBTU/ue329yMtIAk6lOslAZSMeMcerCAgayrvlzS0DjlsY1WuGUe nXRHL8GYfKJgrPo5d10Bz04XiWJi2t4tdvqv6QW/P0tEIkS5J45hQMxkO8Jw/y6+Gqcr b9G6TQVm9OzjkXvIs/Zky6S6ybMWOiXn9naIeh/VPRyCQkm9po4iSEgHM0opd4EmwLQR IghQYzf0mS1++2gLgiBtcsyR3q6jCrcH62fWrAA3ZXLsIYxZcdCOs2R3jDoLX2Zf2lns hBmg== X-Forwarded-Encrypted: i=1; AJvYcCVIE4MPyjWrBpY86zobh/j7AdAPyykIBFScwdxhuRjSfZdrCK5M/Md/8YuArNChxCWTj3w2GTCdaA==@kvack.org X-Gm-Message-State: AOJu0Yy9O63UlTopUg0TITrd+hzK589VFBjojChpYl6sfu8zQHc5XMlN /i55Aikg/x+A4k9INQbpZE0Fb+IrbFssO5U5HE4aOw//TV0sNxsEc/NTmVOFM3MbXGAfMSWwxcV H/OgR5OLZQMDzXiOc4PkSjdoiSYw= X-Gm-Gg: ASbGnctNMcIwq2PT9/zhyBlVVdKUGzoMTuAtQPlXLk01aJDZY2eYLE7q50wQgJZpOXe KbLD62W866VZVqol+FP8LI+sdX9DoFKp4d856Yu2gOLuuI+sVeYsIYQoljna/vi+0NezuSw== X-Google-Smtp-Source: AGHT+IHkYjkVBNjoJsdzJ0DyFJuugBprc54W+XoQWHG6fXpYKdRkvKlwNObdMmVbrn90prLvBAZj2q0yzNNV4Kh4KSo= X-Received: by 2002:a17:907:1c24:b0:aa6:85d0:1492 with SMTP id a640c23a62f3a-ab2abc6d423mr2838381466b.37.1736951429202; Wed, 15 Jan 2025 06:30:29 -0800 (PST) MIME-Version: 1.0 References: <460E352E-DDFA-4259-A017-CAE51C78EDFC@redhat.com> In-Reply-To: From: Shyam Prasad N Date: Wed, 15 Jan 2025 20:00:17 +0530 X-Gm-Features: AbW1kvZxXJF1cn4PgRgyinppCUqG_5upMAH8uxcLMF9S6w7BaIRYpL59BoIfeaM Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries To: Paulo Alcantara Cc: Benjamin Coddington , Amir Goldstein , lsf-pc@lists.linux-foundation.org, linux-fsdevel , linux-mm@kvack.org, brauner@kernel.org, Matthew Wilcox , David Howells , Jeff Layton , Steve French , trondmy@kernel.org, Shyam Prasad N Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 5ACB580006 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 8fsekzyq9nhkhprofux8znfi5gxpty6r X-HE-Tag: 1736951431-946748 X-HE-Meta: U2FsdGVkX1+AAg/ojBIrYkKCAaGifqiUIurgFcfLZ6mYFmyKQGsD/v1Xa1iix/8+CuM0c3dM9PruFaurT5Zm4vnbHYYoEEQlxBrpLzZR8vTh74v2keeiNyX+NmCDnT6EnC6kmhY4ne76xBMUP0zEi84MToLn0I6iS58q+KFHOtCAUwGzjOQ6aJATwbnyxSsFuMsca92mhBygYRqwYbmtZzkFGDTwCM65b69GXOLclOdj5f5/pwf+0OSxDMgW5p9SKwObbAo5TzjJO+u7qsjMTHk3jc08Q8HSE20VT21XaNeAc9iJcKRGh9b7OCpZ415QZn0+/GyvL4AHn+eOOG5V4G92fAKBekyusXL686whBbzWEkbQzxqZ+cZRWXa3Res2iqTHxKBxnQDpuzTs9xHNMcC7SnraraN9gh5VM16jsJxhgV5kfJ4JfXa1QwCOnNKPsiNJIfemQaoRdurH6jiSh8+s/fR/zFclHvQzSUcNWz4e71IkNgu1jKN18AbLtsRNsW8JyuCBCCQpyuQkCOwP45TLBtLnE6BVwu2B434+8UOXsIwusHs6s7iskgDIlAcfbBTojjEIqYui6x+DiuH/VJz93uAoyxLBBObS2dvcZ4vcG7cb0L0fmjm35N9IJlMYoUTP0Wm62pcdfzZUePFE8tjsYlBPfvXlPFvh0GsK+De46hjyTcdfORmmnOPTzlXzm6e1NRHh4Xi1iEW1+DcCB+glwRH3oGC4NJXbTbaW1vxgTSUlGV4rkXKBQ8HU+denQR+H1/a56ViGf/xuG+1HC9WVoskc1fGfDoEMt8azT3ZAcRlkednpGCioI1lxiX6RXUlobOjCxnVaoGm9U1pQWgoenBH8WCWftS4cbfNAaba3mS7Rqfc8vhs8noR2kaLKTiXohRluPp+IHe7q8SoViNo/W7xZ35bFsAGLm6ZX7WkCBafxzOEd+5XSLRf/kh6qxuGMQJs/SuYucXMe6Vp mAHMz35B hqGpcXcoXZ4O4rZM6QrcoeWcHrGu7KEGEGduhUDMAFhWgGgeFM03ddpslcTVhtMxxCp1s+GizrHqY2iNc382cpEqAYpvs8mWoTYMdx3RABEPck/FMP2ecrT/EO5ifcG/VN+KMLhJy+J7X5GXfYE/W6mFp0k6L/38WiBWYVrIl2NHx9ZtR00mlpz+8sh5lfwgLt1SEPVkeurvF0gE8ank0KSJYxLU0o/nqGq/3R+W08ljVuHLk1jT/dn2UMWZcSgYhFoUHbVGYMhhhK7iyh2T0E9SyizhBgxro1A7C/kkM0hb4D7BBUZNtNab6m7ZmW1Sx0Ov8M9zA29usj+Dhx2Jqwmlx5OHHEJnVLRJvhaPKH2cFXPcY/D+50kxoJNgtgln7p4ejEerBXYKaiUr4hHdEFYpAsFnQOV/Ut4G29KZ8fTTUT2A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000024, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Paulo, On Tue, Jan 14, 2025 at 8:31=E2=80=AFPM Paulo Alcantara = wrote: > > Benjamin Coddington writes: > > > On 14 Jan 2025, at 8:24, Amir Goldstein wrote: > > > >> On Tue, Jan 14, 2025 at 4:38=E2=80=AFAM Shyam Prasad N wrote: > >>> > >>> The Linux kernel does buffered reads and writes using the page cache > >>> layer, where the filesystem reads and writes are offloaded to the > >>> VM/MM layer. The VM layer does a predictive readahead of data by > >>> optionally asking the filesystem to read more data asynchronously tha= n > >>> what was requested. > >>> > >>> The VFS layer maintains a dentry cache which gets populated during > >>> access of dentries (either during readdir/getdents or during lookup). > >>> This dentries within a directory actually forms the address space for > >>> the directory, which is read sequentially during getdents. For networ= k > >>> filesystems, the dentries are also looked up during revalidate. > >>> > >>> During sequential getdents, it makes sense to perform a readahead > >>> similar to file reads. Even for revalidations and dentry lookups, > >>> there can be some heuristics that can be maintained to know if the > >>> lookups within the directory are sequential in nature. With this, the > >>> dentry cache can be pre-populated for a directory, even before the > >>> dentries are accessed, thereby boosting the performance. This could > >>> give even more benefits for network filesystems by avoiding costly > >>> round trips to the server. > >>> > >> > >> I believe you are referring to READDIRPLUS, which is quite common > >> for network protocols and also supported by FUSE. > >> > >> Unlike network protocols, FUSE decides by server configuration and > >> heuristics whether to "fuse_use_readdirplus" - specifically in readdir= plus_auto > >> mode, FUSE starts with readdirplus, but if nothing calls lookup on the > >> directory inode by the time the next getdents call, it stops with read= dirplus. > >> > >> I personally ran into the problem that I would like to control from th= e > >> application, which knows if it is doing "ls" or "ls -l" whether a spec= ific > >> getdents() will use FUSE readdirplus or not, because in some situation= s > >> where "ls -l" is not needed that can avoid a lot of unneeded IO. > > > > Indeed, we often have folks wanting dramatically different behavior fro= m > > getdents() in NFS, and every time we've tried to improve our heuristics > > someone else shouts "regression"! > > In CIFS, we already preload the dcache with the result of > SMB2_QUERY_DIRECTORY, which I believe NFS does the same thing. > > Shyam, what's the problem with current approach? We load the dentry cache with results of QueryDirectory. But what I'm proposing here is a read ahead, even before the next readdir is done by the application. i.e. the idea is that the data necessary to emit dentries is already in the cache before it is even called. That should speed up the overall directory reads. --=20 Regards, Shyam