From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10054E77188 for ; Tue, 14 Jan 2025 15:59:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 991ED6B007B; Tue, 14 Jan 2025 10:59:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 942556B0083; Tue, 14 Jan 2025 10:59:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8098D6B0088; Tue, 14 Jan 2025 10:59:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6241C6B007B for ; Tue, 14 Jan 2025 10:59:57 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0F2E9AFA00 for ; Tue, 14 Jan 2025 15:59:57 +0000 (UTC) X-FDA: 83006518434.25.56653AD Received: from bedivere.hansenpartnership.com (bedivere.hansenpartnership.com [104.223.66.194]) by imf22.hostedemail.com (Postfix) with ESMTP id 6ECB7C000A for ; Tue, 14 Jan 2025 15:59:53 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=hansenpartnership.com header.s=20151216 header.b="SJ9FN+Q/"; dkim=pass header.d=hansenpartnership.com header.s=20151216 header.b="SJ9FN+Q/"; spf=pass (imf22.hostedemail.com: domain of James.Bottomley@HansenPartnership.com designates 104.223.66.194 as permitted sender) smtp.mailfrom=James.Bottomley@HansenPartnership.com; dmarc=pass (policy=quarantine) header.from=hansenpartnership.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736870395; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MOgL5NcLQJTJIvAycB78AxBtPacSDcuXJ+r65QCRy8I=; b=mQtPXm9+mu6bZe+dxyi7E8s09jQ92xN1uFqVODTu0EkHdrCPoeVFJijl/QOLGCQd9JqUMq kMomKCL3h3XbG0fpkHKfvndn3LnmoMn44Sa+9qGe/DS9oQjiKWixv5NTbJMfXkoJzGKwhg 27O21u4qWdZY/+woVCjEr2gDs3Dj7Fw= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=hansenpartnership.com header.s=20151216 header.b="SJ9FN+Q/"; dkim=pass header.d=hansenpartnership.com header.s=20151216 header.b="SJ9FN+Q/"; spf=pass (imf22.hostedemail.com: domain of James.Bottomley@HansenPartnership.com designates 104.223.66.194 as permitted sender) smtp.mailfrom=James.Bottomley@HansenPartnership.com; dmarc=pass (policy=quarantine) header.from=hansenpartnership.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736870395; a=rsa-sha256; cv=none; b=18IPIX309S4wPXgg2QbkqxG4pzCGDgrig3vTaT2daXdvuOscYD8lc5kpoHH1fCdAamCgmA +FuKBhJ1JhYIjLwI7EmQwRb7fw9BKrHFGkq/Ujc9xI3XbOr1ZYrvNh33gjzv5Ef6KnCkuf anXk+ROnnAfqClmhh6ShAUlq86v9caM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hansenpartnership.com; s=20151216; t=1736870388; bh=aDMf0Oz6JrVFSS1OF87mdIRWcjKT4Mpu785y4e96QRA=; h=Message-ID:Subject:From:To:Date:In-Reply-To:References:From; b=SJ9FN+Q/LEex9sIUvLX/j2Lkuv6ou02LtfdY3g1Fd/KQUyDIfq9myy3cLADd3pT6P 1nMv6VnwQzcAQzzeeNqVUJhhJ+emqxJMfatPlOIuc/umx0xxjRvxljFoTg61SjzwxE 33Szp2l6X/CAF9LWkd2K0uEh62vfH71Gfk0YUt6U= Received: from localhost (localhost [127.0.0.1]) by bedivere.hansenpartnership.com (Postfix) with ESMTP id 6FEEA1287756; Tue, 14 Jan 2025 10:59:48 -0500 (EST) Received: from bedivere.hansenpartnership.com ([127.0.0.1]) by localhost (bedivere.hansenpartnership.com [127.0.0.1]) (amavis, port 10024) with ESMTP id Z2d66Q_SSJ9B; Tue, 14 Jan 2025 10:59:48 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hansenpartnership.com; s=20151216; t=1736870388; bh=aDMf0Oz6JrVFSS1OF87mdIRWcjKT4Mpu785y4e96QRA=; h=Message-ID:Subject:From:To:Date:In-Reply-To:References:From; b=SJ9FN+Q/LEex9sIUvLX/j2Lkuv6ou02LtfdY3g1Fd/KQUyDIfq9myy3cLADd3pT6P 1nMv6VnwQzcAQzzeeNqVUJhhJ+emqxJMfatPlOIuc/umx0xxjRvxljFoTg61SjzwxE 33Szp2l6X/CAF9LWkd2K0uEh62vfH71Gfk0YUt6U= Received: from lingrow.int.hansenpartnership.com (unknown [IPv6:2601:5c4:4302:c21::db7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (Client did not present a certificate) by bedivere.hansenpartnership.com (Postfix) with ESMTPSA id 5EE241287243; Tue, 14 Jan 2025 10:59:47 -0500 (EST) Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries From: James Bottomley To: Shyam Prasad N , lsf-pc@lists.linux-foundation.org, linux-fsdevel , linux-mm@kvack.org, brauner@kernel.org, Matthew Wilcox , David Howells , Jeff Layton , Steve French , trondmy@kernel.org Cc: Shyam Prasad N Date: Tue, 14 Jan 2025 10:59:46 -0500 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.4 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 6ECB7C000A X-Rspamd-Server: rspam12 X-Stat-Signature: 7mhfijz5ficizrcczbxdjn1376dq91xh X-Rspam-User: X-HE-Tag: 1736870393-749059 X-HE-Meta: U2FsdGVkX19n1LkA5KwHF2VYJYsb/QQh1wgUUgHz7fbTG1J5Ai+TvXEBTlIPhtOUvz47dqpcNttzkXRZ2Ui586mN5K9jCDIoWJo58PVqX+HrHLRlObCFF6IrwlPTG7BattIVNYXm+rdN2LqXGfMb97zVN1nZ4hK8SULc/SXugPZ7r50T8SI8/LTGnim5B3OgmGUAsdh4Dz4rGVUgtyDys8pa7AaLK+NKskt8tD2SFnAep1V4b1tqagCAZRBwYdB4Tg7CHmG8WCjV88Jh8H8+HyRHaEWleBBesE7EDaKZTJJt7lzc2ejcNIC7lyQLifpUKgablB+Gu9IHvvlkqdYi2Btnggxrc6797yu77BUY1n+3ZmnEUetK35o0Q+rxhs46sukUZIHB0/EwS4sjBxcy/2ZBLOyZhHOnzMncJEd6GrU6RARCzHcq5avWIleeT2vMlN1aOUsoy82wLebh6RnuPimhq/j+fHc6yJOOwYSZQLJJmOabSIMvaYRAsWxamIqFXIup9zEHMbqr8J3X1Tnbve9Cz43VnDqIgGQjIiU/w5jEU5EdRchekanM1PsQbo7JR42UV2hPaLVrw+T8dRnlvnpiuZGYpcBXD76fSxjcQM4P49jOWeaL+u78c4qazeBW9HtzEpKYD4qPKXL4HlFVehKDq8AOE2dKjM9qcNsJbSh4IG7uBU/+tbaaC8b+0l2SH+sAIX2woqVqA+uQpP+HxpZm2HpKYzj+ur/2fwXZ/eWXBvIQ1Q+2U1wic+rXjL2OwTTL5Gc1YfdrHFf2ka7vklyGxOPF7ExLQOlJ/UsX3DgH2pt8ASL7C79/JhVEW1MuU8lpExqKNMMH40I7uZerJ8+EBB8BGFWvz7c7gkQlMvQZemKYjO74tMObqEwlvkyvgVKlRa7JbnzjvyBFMQ2EdZOlz69ayDXIOm2o0HZ9uP0ZWtugrFGgvJykJZDHL5orOdQq0eCiphrpVbu4bIV xaYZ1PXD 70bDLRl4OjkX17WcRx/qa2liPBeBJcOx+39hEDhihNlEh7BrSdiYo5H6zzOl/XUfglZCsDdSgk751oIuRtKbasPuL49ynFjkVrBFwshWR48d/wHYhcDy8kEog8OJ5QRzNBJZVxDnRmNaMTqohcFylcD7thZy2gXNYd3XaPvJgYmhnnZNti4h9hl/nfI996A063bLLFI3NFN19t1JwMsXFVYKkYmXAz16BkMOB1sTcnZf1eCM9PCS0BuS9XLFX4q/obfPz0aad6UOHrp5/hr9XOl7USfx71noU4/voqWtvHtlDDjQlv5ATnzFHELUFvDgLldCW1SL4gfV9biDB/Gex96fwXme9iY+MuOjY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 2025-01-14 at 09:08 +0530, Shyam Prasad N wrote: > The Linux kernel does buffered reads and writes using the page cache > layer, where the filesystem reads and writes are offloaded to the > VM/MM layer. The VM layer does a predictive readahead of data by > optionally asking the filesystem to read more data asynchronously > than what was requested. > > The VFS layer maintains a dentry cache which gets populated during > access of dentries (either during readdir/getdents or during lookup). > This dentries within a directory actually forms the address space for > the directory, which is read sequentially during getdents. For > network filesystems, the dentries are also looked up during > revalidate. > > During sequential getdents, it makes sense to perform a readahead > similar to file reads. Even for revalidations and dentry lookups, > there can be some heuristics that can be maintained to know if the > lookups within the directory are sequential in nature. With this, the > dentry cache can be pre-populated for a directory, even before the > dentries are accessed, thereby boosting the performance. This could > give even more benefits for network filesystems by avoiding costly > round trips to the server. If your theory were correct, especially the bit about using the dentry cache to retain the readahead information, wouldn't a precursor actually be populating the dentry cache on iterate_dir() which is the engine for both the readdir() and getdents() syscalls? It strikes me the reason we don't do dentry population here is partly because the lookup() on each name would slow everything down (iterate_dir is very locking light weight because it needs to be fast) and partly because whatever is doing the directory read may only be interested in a single name. The only userspace operation you can guarantee is going to do a lookup() for every name is ls -l, but that doesn't seem to be a good one to optimize for. Regards, James