From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8995E77188 for ; Thu, 16 Jan 2025 04:51:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02F0A6B007B; Wed, 15 Jan 2025 23:51:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F21C36B0082; Wed, 15 Jan 2025 23:51:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0F556B0085; Wed, 15 Jan 2025 23:51:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C2CA46B007B for ; Wed, 15 Jan 2025 23:51:00 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 23B3C140707 for ; Thu, 16 Jan 2025 04:51:00 +0000 (UTC) X-FDA: 83012090280.04.100AEB0 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf06.hostedemail.com (Postfix) with ESMTP id 37B8E18000B for ; Thu, 16 Jan 2025 04:50:58 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=lt8TVOrs; spf=none (imf06.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737003058; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VKZQZRWnBrBBSZjZNPjwgHgImhQxhVnbkp26zmIkpM8=; b=zIT3VUYTcoEyc41EooszLTF4wQhOoueq/nwt2FQPlouAnexcDRZhptJkAr7lDLq7/Xjae9 aTZvInElrvCtTyQp4f6uub6w2ZL1cbrNrllkgj20sOA7xS27/KY1hyfHSwhVgHIBC/7gfx 6qTDdNq4dkcrxtXCBD3J9KT5CJIldjg= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=lt8TVOrs; spf=none (imf06.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737003058; a=rsa-sha256; cv=none; b=KkRHtV1SJE20ftX5Iup0R/rmZCecvwpuzpfq6Lm6Qu4stz7MUnEis9fTDWAkdhPCzy6WGD PFCB17MXa9g8CMjuxMBHVGEEes8N6GgWKzzylccKc+palhExytzsP5x7LIzTKs2WtBV4Dv HGxLhHoHt0u7R3NFx3JZp+Z2TG8h5Wc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=VKZQZRWnBrBBSZjZNPjwgHgImhQxhVnbkp26zmIkpM8=; b=lt8TVOrsX6DH0/djBmkDqq+twV il+tDjuzV/zKS6nVYvA3TtvN8rJYUD3VWsXEO6m1JZKSqmOrZUnV8mws56qYk06GkyUZddafiHq6Z /2ejRjFs042SFQvn3lthrfsK4PjB4hH/hKOYzT+lcDb6xtBC2IcRrQC0/L79Ib9DhveoekhbFFFJY fI10P9ujvB9YzmHVqU2zrIbsQG0RCZIRqp36JBckQpEoEr0DBEEarDuRabfvHnk5FXe3+Ti4NGaM9 P4aJcMjHXHYIZt8buspR42LBfE/ERH85yhfjtjjr06C8svw2QxeZ7Jpyw4UW/7SRc9n/tF0etRXVH WzUVIYbQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tYHqI-000000020UF-0WGy; Thu, 16 Jan 2025 04:50:54 +0000 Date: Thu, 16 Jan 2025 04:50:54 +0000 From: Al Viro To: Shyam Prasad N Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel , linux-mm@kvack.org, brauner@kernel.org, Matthew Wilcox , David Howells , Jeff Layton , Steve French , trondmy@kernel.org, Shyam Prasad N Subject: Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries Message-ID: <20250116045054.GD1977892@ZenIV> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 37B8E18000B X-Stat-Signature: 9jefeosg7igtrzm4ok7j49p5rzppejum X-Rspam-User: X-HE-Tag: 1737003058-392562 X-HE-Meta: U2FsdGVkX18YdsQyg/13Igl8mimRZP6pHZnmzeypST+VOImqqR0kKbDadEQ2RAgcd/LiQ5lua4qlGrnIn1XHeUdzBlnniucFSoacrpoyCEnxUnHWMKlyxg9KTSzPyTmomcm01ewPLVFPSxXnBacZss3v8h287svERI4UB2oQ681NNza+B1OAJQli1+IvsGdBUDwkzXDEWJIKvN6NdRPhSmrDIcpbrfAVKZD3ebtH0W7666l7beW7MGvcpnxRDEK+mToyxak4/O5kG8zyh3cIuUEJnGqQHesIrOHpNKPL6IKmlNecZzq9syyyLAhCZ+w58iJTArMki5z6hb5FUkB7aOolfS3VpzzO85O1iz+ZpfzYlGxEKAKYVs9rtWC8h3EZ6Uh3QQ7k9UbduiM7Ta/y73H09H5nluoskNKNHk53E2VbbfP6VTWQhjcZJwcCgKVRArXWVIrSLpISh/m4H2V7tSQi20fS3VJeQPzqeLIWWgz6emZ1kmBeyZpCRQEDYg6JrYmcUZQtzV/Sab/cKGpMD8DmyKrW+CMSJtc5ppZiqipJ4h7rQCqpTDrNrWuxS3nsBT8kSvaO+MFqH04yiNZq7QR8yIQlox1ClctBUvQ3WBJPPqJd2flytQMHE39swZ/rxo8TZC6OJsAYqNjPwPMKmip4sY3gL2QGh08slW+uGFLb98X9pJzN25yp44DZdAKkcxt5bkMjeYNSbB+VsiIQ8FeWA/RfbWjl0m3psFT3udBhuhTqjLRrjDyGnTtNce2ukJWU8bP+JkiuzRURm1uC4EGjzvhg0YjYKe3X8XO1VqQkKApn216cePRxyTZCss344U3s002dKk7PNhNQyDfwOjJ6GOPWdgpqGBCx4GUaK8Ii2/+7ubeITeUfaXmmWLizjrpWfttzh0zXqJ4RvT+UUst3xo6THIRDcFZeusJuRQTZ5RK15jf9r0yXlhq/ZN47R/nMhQDdD7+UbY4E8L+ NW3nzC1Y Lbs20IIYMDjbe6AWFvHIhSdh+FDEBB2vb31XnmOgrgP8FLTfHRhf9EnZ+ZevT3s8lcPT++G+lmaKUdSZDpgA2gc1MGyn3D+nBq1t4IMaBAsxXiH/wuSiNhbk2U9pWievnyTShWHWBcNTBBwR3bNaGCj+sAy1nK/UbzOHTy46OZ3G1Szkph7trVaDkZdY4h+hAGmH9J309S3aVnvigFIj2DyGPXN5eOD4I+9aZ3Ixx8LS4fAgSgj+QItxNgDI3XW/vZ5rWPBnJzjZo5FJe45IQvQkUS2hRmXs4SchXpQiFbvSBtju8EKCg2GlsFvdeNPhg814oyx3RoNSFuwK8niGRgwhZQib7zEQ/VTnzYgepqKjAzaOdfKYav4C0gwiYL4eesjs4Sc9pSTVMAEpVOknEqGRnw7gOKJJ2vzq4Bsse43Awxmc4B+jmap/eFC0Bn8fOe/qA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 14, 2025 at 09:08:38AM +0530, Shyam Prasad N wrote: > The VFS layer maintains a dentry cache which gets populated during > access of dentries (either during readdir/getdents or during lookup). > This dentries within a directory actually forms the address space for > the directory, which is read sequentially during getdents. For network > filesystems, the dentries are also looked up during revalidate. > > During sequential getdents, it makes sense to perform a readahead > similar to file reads. Even for revalidations and dentry lookups, > there can be some heuristics that can be maintained to know if the > lookups within the directory are sequential in nature. With this, the > dentry cache can be pre-populated for a directory, even before the > dentries are accessed, thereby boosting the performance. This could > give even more benefits for network filesystems by avoiding costly > round trips to the server. > > NFS client already does a simplistic form of this readahead by > maintaining an address space for the directory inode and storing the > dentry records returned by the server in this space. However, this > dentry access mechanism is so generic that I feel that this can be a > part of the VFS/VM layer, similar to buffered reads of a file. Also, > VFS layer is better equipped to store heuristics about dentry access > patterns. You do realize that for local filesystems it'll actually hurt anything that does *not* stat() or open() everything it runs across, right? Directories do not contain inode metadata; on lookup you do want that - for given object. So you need to get the on-disk inode read, so that in-core inode could be set up. Adding that on readdir for every directory entry you run across can be thoroughly unpleasant. It should be up to filesystem. It's not just the access pattern. Imagine the joy of doing that on e.g. NFSv2; would you agree that "I'd have to send a bleeding GETATTR for every entry in READDIR response" is an important detail when deciding whether we want to do dcache prepopulation? Ideas regarding better infrastructure filesystems could use would be interesting, but decision whether to use that or not in any given case belongs in filesystem itself, *not* in upper layers.