From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 375B1E77188 for ; Tue, 14 Jan 2025 12:39:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2FAC6B007B; Tue, 14 Jan 2025 07:39:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB8B46B0089; Tue, 14 Jan 2025 07:39:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A32BD6B008C; Tue, 14 Jan 2025 07:39:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 818056B007B for ; Tue, 14 Jan 2025 07:39:33 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4700140EDA for ; Tue, 14 Jan 2025 12:39:33 +0000 (UTC) X-FDA: 83006013426.28.CE6DBB8 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id E3EF420003 for ; Tue, 14 Jan 2025 12:39:30 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=T2poHIuS; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=88lQqFN9; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=T2poHIuS; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=88lQqFN9; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736858371; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bxZl4LR+WA11+D31Pi4AlAKojYOnGZrQ4tY0MJSgQ1U=; b=SN1nigf17CMLegCIkRGNK6z26JWWaEzGFnbbCjXWychVnCIQyWVMADkhknadK2gLY3XFhB bpKH8nyyDgqcl+FSnMXjNkoEndGjR80AWbQAi7z/fLEb0gE3OyCEAIAXT1Cpb1uEfevM1s H6yl/q6bYkZv29fiGWQuVPbgx9orm4o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736858371; a=rsa-sha256; cv=none; b=cZlNN7oVkggJbK4JwsAsO8gIdraGVqGjn16sCI+qGfS1Cf1NbMGwm3/Uc4eQ71Pxno23AD hcTcsAmkBnwFKDcSDCcz1x6VyeO08BbNDXMw/pT6WGF4iRO2gM4JbyHpyhemXG5C6GxwTa rZXyMVyRXHf4YsJXpLgWLUNa/YNNdrc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=T2poHIuS; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=88lQqFN9; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=T2poHIuS; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=88lQqFN9; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1A2C12115A; Tue, 14 Jan 2025 12:39:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1736858369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bxZl4LR+WA11+D31Pi4AlAKojYOnGZrQ4tY0MJSgQ1U=; b=T2poHIuS5KRb+nvjUNTbLSeuLfPv43LwNcw0UzZ22uych2PdrFkpowLbZEpRGtyzbnS4rg 4y+qSAHUmNxeIe/CiQhrMeZeiuyof8IPJdIPSC8QcH3PaFREjfGUc2Sgbb3vBzohEscrij lF4ZGbbjybG6TEYUqutjb5MNm/+jaFc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1736858369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bxZl4LR+WA11+D31Pi4AlAKojYOnGZrQ4tY0MJSgQ1U=; b=88lQqFN9OQ00BqcO9i7Fff1Sex3aGI/Y+QNEtoihgjv7aMnRci1Fq92RgPR1JjGkyRNpkR Nb2YlhA8iELEojAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1736858369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bxZl4LR+WA11+D31Pi4AlAKojYOnGZrQ4tY0MJSgQ1U=; b=T2poHIuS5KRb+nvjUNTbLSeuLfPv43LwNcw0UzZ22uych2PdrFkpowLbZEpRGtyzbnS4rg 4y+qSAHUmNxeIe/CiQhrMeZeiuyof8IPJdIPSC8QcH3PaFREjfGUc2Sgbb3vBzohEscrij lF4ZGbbjybG6TEYUqutjb5MNm/+jaFc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1736858369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bxZl4LR+WA11+D31Pi4AlAKojYOnGZrQ4tY0MJSgQ1U=; b=88lQqFN9OQ00BqcO9i7Fff1Sex3aGI/Y+QNEtoihgjv7aMnRci1Fq92RgPR1JjGkyRNpkR Nb2YlhA8iELEojAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0E3CF1384C; Tue, 14 Jan 2025 12:39:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 1JF2AwFbhmdANgAAD6G6ig (envelope-from ); Tue, 14 Jan 2025 12:39:29 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id B93E8A08CD; Tue, 14 Jan 2025 13:39:28 +0100 (CET) Date: Tue, 14 Jan 2025 13:39:28 +0100 From: Jan Kara To: Shyam Prasad N Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel , linux-mm@kvack.org, brauner@kernel.org, Matthew Wilcox , David Howells , Jeff Layton , Steve French , trondmy@kernel.org, Shyam Prasad N Subject: Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Predictive readahead of dentries Message-ID: <6wcmvyeuelngltuiohumo6pffwptgbgofqba453pdi45ahydkn@ern4qy4i2zoa> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Action: no action X-Stat-Signature: uummgwhbu3jjdt5znuj9xajx5fx8rp6u X-Rspamd-Queue-Id: E3EF420003 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1736858370-903439 X-HE-Meta: U2FsdGVkX19aPZbpA0hlTYpd6+hIaYy8uHAhNyXLqUIGci1k+C3hCaraYVrb47i8w7qRBit1qLBlDRqAqSCZUuipu0FZ+8WLhdHiOsIanhjJw0nu2ndvJ+PtHrateXLhl2jOs6wulgAzPUck5a/KbBeMSCNa62VEHn449B+6tMNud204YguVC6lvVE5r0JSWpJ5JP9y6ys24yME6fBdCsEMw7/agEnv2EwcNQ1+5mBM4Z2DrzSaEN5nes15KfEc1W9eFuWTE8MZrFYo6zlrJujSLqTBMDSJ0m4vAvU+kB5I+iX/sk83RUaaAucT51ZvoO1sSZU8DizlhT0yXzws7zBv7amL8CqTKsxtraU9E3Vf/04xPXpcvkgXXi7w1fqKUA26G/cejbfwVvjbTt2cuQfzXQGhjDwJIvpGHdN7hy3xRHENCWTKwkOiM4bcYCmX10FENYK+Gs2IIVoKcuRBZfUOxbEcPpM9FJwCHvedtLJD4yAlZQvlqMbM/WRIko68lExMXy30UXKdciiVEqFWjaxdOcpVmy+T4LvCdJeK40opSYs4oOzabYcdcm2G7EZQpUsI5T+gltK5bTkA4uPERQvkm5AqSRlIipXlPz1/0zE2pLMAFHjqemJw7OyNOeXrXBnmXG1tTb8mAMB+rxxZsTYdoKA3/nnEbeSduUyvLxrNc4XTBoQpWIslKnqpWmI8cE2J5c0RD92FLsT9ocJGPFLxE6gOJ2ub1oqq6rfcRb4ylXbpm+pOREndszlZDw4nYh8AC5E8sXsuarb5Fk8NFKNn1IWIbaHXP9rTiMkf/p53szFuxcvIUW/JKrTBy/lYqB42u8/KE4KLJoQjq/5H/PBiZ5PeVZd+GK+J2cc3gd0gGYytP4woSXrc5NgyaknbYW7xnI+dCl6Y3qyEU9HE7M1m60OZDdzvFTrQ0aTBcOFbmALT9q7LHd2yUQICbDqFbGfKFxuEupkYnEJ9Neei PiHpeamQ vfGhRrPD01025fRErAci7paydKgfKPNsJz/1ExBedqQTVG0KM4fhuGkxHT/g+2cXQfxSRp+NAaiNJMDvWV6f06rFXjivLKo43nAwS05AsnaD9JkhY3sygBhnDS+pTxGNV5uVL7/O0gx35b2rmW3DnitsFy9vmif+0ztTXeKqmRZmw8xH1o4OW1YOltDYGecg88wzfAzX0Xsya6ouKOHph9dc3e9ZQLvPP3bGTy0etyj7I2pStla3Ww2cF6b09PctHhmI+EVFsQw/kxIaIDZNFO7sVgV843Xs4MCmddnUj28MaNHH6rVhXe2RmuzXNwtYM4aQdsVM1Xz38EUZEkWRATVpKsg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello! On Tue 14-01-25 09:08:38, Shyam Prasad N wrote: > The Linux kernel does buffered reads and writes using the page cache > layer, where the filesystem reads and writes are offloaded to the > VM/MM layer. The VM layer does a predictive readahead of data by > optionally asking the filesystem to read more data asynchronously than > what was requested. > > The VFS layer maintains a dentry cache which gets populated during > access of dentries (either during readdir/getdents or during lookup). > This dentries within a directory actually forms the address space for > the directory, which is read sequentially during getdents. For network > filesystems, the dentries are also looked up during revalidate. > > During sequential getdents, it makes sense to perform a readahead > similar to file reads. Even for revalidations and dentry lookups, > there can be some heuristics that can be maintained to know if the > lookups within the directory are sequential in nature. With this, the > dentry cache can be pre-populated for a directory, even before the > dentries are accessed, thereby boosting the performance. This could > give even more benefits for network filesystems by avoiding costly > round trips to the server. > > NFS client already does a simplistic form of this readahead by > maintaining an address space for the directory inode and storing the > dentry records returned by the server in this space. However, this > dentry access mechanism is so generic that I feel that this can be a > part of the VFS/VM layer, similar to buffered reads of a file. Also, > VFS layer is better equipped to store heuristics about dentry access > patterns. Interesting idea. Note that individual filesystems actually do directory readahead on their own. They just don't readahead 'struct dentry' but rather issue readahead for metadata blocks to get into cache which is what takes most time. Readahead makes the most sense for readdir() (or getdents() as you call it) calls where the filesystem driver has all the information it needs (unlike VFS) for performing efficient readahead. So here I'm not sure there's much need for a change. I'm not against some form of readahead for ->lookup calls but we'd have to very carefully design the heuristics for detecting some kind of pattern of ->lookup calls so that we know which entry is going to be the next one looked up and evaluate whether it is actually an overall win or not. So for this the discussion would need a more concrete proposal to be useful I think. Honza -- Jan Kara SUSE Labs, CR