From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A954E77188 for ; Tue, 14 Jan 2025 13:25:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C79356B007B; Tue, 14 Jan 2025 08:25:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C2AFD6B0083; Tue, 14 Jan 2025 08:25:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF0C36B0085; Tue, 14 Jan 2025 08:25:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 926206B007B for ; Tue, 14 Jan 2025 08:25:41 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8B734A09BD for ; Tue, 14 Jan 2025 13:25:10 +0000 (UTC) X-FDA: 83006128380.11.33E45F8 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by imf04.hostedemail.com (Postfix) with ESMTP id A083640012 for ; Tue, 14 Jan 2025 13:25:08 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=lxox1Yua; spf=pass (imf04.hostedemail.com: domain of amir73il@gmail.com designates 209.85.208.48 as permitted sender) smtp.mailfrom=amir73il@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736861108; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E8AoZV5QSgm9bLiWhijrwsLuz1qoBYuWMl/ZwGLWBlA=; b=y1ZQ+r52b47HYR8Fjfsd6gd10eLEmvPw7NWz5srbw6/ANjlXuzx2qtV/QB1A9NOHd9AHGV Vg6TyC2/OshlboId8+PWIKutERZn20bRBpGOeJrWG9iavdGqaEvFfhanCbZBrMmgPl4XqD wuVJRHyUCFao8Uw5Ca5Co3D6a/PHa5w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736861108; a=rsa-sha256; cv=none; b=3ICNPnMlFRIHivgBIzCc1fgxZQPGgvs4Ufq7y80YdFLAt1XuMj61uNeH590GmK4SccGrIX hOQEZQfOOwkWea1fLKjIzM7CAJi7W//jknLDSBt7IekmUb/3R3Ct9Fw7JuoFgFrOT1MdKq pcsGkUgesPcE7GPMGaCoIEBJyRoaH2w= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=lxox1Yua; spf=pass (imf04.hostedemail.com: domain of amir73il@gmail.com designates 209.85.208.48 as permitted sender) smtp.mailfrom=amir73il@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-5d3f57582a2so1915383a12.1 for ; Tue, 14 Jan 2025 05:25:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736861107; x=1737465907; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=E8AoZV5QSgm9bLiWhijrwsLuz1qoBYuWMl/ZwGLWBlA=; b=lxox1Yuafte/aLoEM6E0Q22VmjLzOSZgBaMu1q/+LGwGumlczGNTsEVg2FL+PydqQB MGGljZgRWREdB6pyc4RQAq+17yqY5XoaITgV4EW5MlNJUSI2veUCVKbyBxd9gb4NZhhf GtOLujmHBUSzOEN60TJR6pbl1wzjHYe8KpvlU2r+8nVWOqVTScCV2/qjLwDe4y7Fuw+e Lp1FFB6l+uxlT1DCnld11kFN47Z3MBhLiS1jZ2wrgNEzI3To3YzM4eQhNY0lc2OUlf3o CS62ght6sXE3JsP46vcCOuix3ulBx9gJ9LTAQPWKkjkunW9M5JFW45idd6ylEG3j5tvm B4Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736861107; x=1737465907; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E8AoZV5QSgm9bLiWhijrwsLuz1qoBYuWMl/ZwGLWBlA=; b=YKRIJiHJ+Wfcean4jDYKcmW3+Azyd5G1MKWMNIibsnZ0/VH507DysV/3SrcJzkVDWf dpZ13DBbnqonoSYA9HwBUZ7NJPq4oJSOMdJkky912qpLHbdIesMgi6zvHO63LKXs/37b Fw0WlWBA0TcDZ6TzXltYd0lfHwZn9pbePs8jx10HhwZiGFiUKdST8GKH8BU4eN35tbba WlNxa8y8Twqen/gIbQ4I3Tqbcrs9S/4YnWC7xfGRWfd0s4Wa0AQTgS3wVtNpeKIriXPH FlNAZrRy7S0yTqCF3yCcK11SJUFx7zr2e0NWYx1t6dvojEqO/aDDPlhROyDp4aziTQYB deKQ== X-Forwarded-Encrypted: i=1; AJvYcCWSr6Y86edv0kloNXFPWnhfmfidKgYzBDPShCNnR8SB9ikAOBYSRzMghAsfrOO57XGWvB645jF99Q==@kvack.org X-Gm-Message-State: AOJu0YwOJyyShdK9cyBsITo23K1/WByDAJ7juYGXVUyfopMVrTExLyAW DlWG3FYO8Aqu1/j+0scEtjWCR4aRCbiHXtNE369FgwGkrC4dy1bcSV1Bv2QiTENdCzcFAyY8Bal Efefkx6PhOBp+L8FWNxM2Rom4gqM= X-Gm-Gg: ASbGncvPgdd9e61eXOqnjunKTGubZ2QKRm4uZuEA4HCUC+l+DyKjQtEsBYUn4PtRDmo DJBZgjPqL/FD4Bl3QzW4hEBkNACwuqn6KF5tWYQ== X-Google-Smtp-Source: AGHT+IHe1h2auaaS1BywSg0AAYxGxmeh8jv6DVAXun7itz+nEAD4vbG+OuRZ8HIVMRy7o/yanOO8cfYX28D7Sum8U/c= X-Received: by 2002:a17:906:99c2:b0:aa6:938a:3c40 with SMTP id a640c23a62f3a-ab2c3d4d2c5mr1822957766b.24.1736861106848; Tue, 14 Jan 2025 05:25:06 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Amir Goldstein Date: Tue, 14 Jan 2025 14:24:55 +0100 X-Gm-Features: AbW1kvYfwd1sgIJ-QpvKS-BOMs0QlMN9MyY8-2FxbZu0cU1up3JEh1u7w7Lfm6s Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Predictive readahead of dentries To: Shyam Prasad N Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel , linux-mm@kvack.org, brauner@kernel.org, Matthew Wilcox , David Howells , Jeff Layton , Steve French , trondmy@kernel.org, Shyam Prasad N Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A083640012 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: ahgoe8oiw6gbshf5h9gsghr8kuk9xa6j X-HE-Tag: 1736861108-622937 X-HE-Meta: U2FsdGVkX18ZdvABn8JkO/FppdQbTuSFbGUGxfoUjKUnciGetPufiX5bhXtjDHfWfBOBSRDlPUI0Ii3fBccgSAiLhneUijUgb1/iu/v+AGHYVSL3/TTORBr0TQGlUwqUSF7L0jQ1vLnb//VCdWXglooGewS2wMRzfJTdrGYp5rKMb9iCS+51QFUQDhJIPT0QapNcvsOcPasR3QgaI6Txe79U/6lhO112UOW4RkC84qruKdu6+0u7pLTNShACDVUUE618LrmhqyJHDbnrTTpVJbVlcavD7EhbnKytNL6RA1BlIa2Ya3a4wa8Qpzcth64bR3Uw+7/adwdqQjDey9JJ1lCWHzxuvoa3zV89DuRiceqJ+cpEBpSTQXkdqyxiDiiJ9lx5ey1SSINs4EZjhcSUC9JznqnQ0Y7h7lzRbA62LbksEUm6qYt2PYBkmNE5WaZymo5kXkfqMPsLW8uFAyBs3bcKPC/P87DiTVo9BJyJo7KcNQ8bTKDWCeY7p9P9EBqWnzGKI4ef1ht365obIZRNOTcFYWcwL+l9dZ1qDkGsCvTtFWZ6HtgX4phyY9psMSf+j9DpzuWndh6vSfqw+MVJ5TNYGcUQKBX3FDN5aaXcUJlYQ3/QCBVY5FCWnkyImnbn5UvhanY0sOWBz6h+vdmt7dlV9B5ME2yhO3PGHWMiSSHyJtC+96YwPs6Grvl526VnvRIK6OsSkS1Z6qHs/DgJFj8NkaJu/Sg0ZxqlHtHp6PiBQVJIkhhA4Wj5EAa5W7k4q40ZGZj9RKKprucC4qv2NroynSAXmI/uB1zkmLEcQ2B6ZhSSJepAv8GQ7WCm7bJizEQVtnQDBkWycljxwrFIA5qfKHS64jl9iGcwPittEaLHiAaEudn6wVpNxjt4ElTASVKx5IwsQf1WrbmBc5l14VU18qTsBVZfhJzSrF2Tf5hCXsh6ZOr7olu6itVGSO2T/41KtgaoNa0TkIjgAfP glDlIVCt /X+K8dFO6nb20ShVJC1XpBHVJ3XS2cCL/Y13dv2frNJKUw9xOLAxCxvyArkuFHexQc6o4/aqAcnUCZunawnwG9GSTKWEeIYhWg5hBMyEnCUOUM96l2lxpmagQPlln56jNFJ6jiiUH2UD4/z0L9X6NGU4etz0fFAbWLmWM9AsWAptMM1ty62suXxrtknvQZ6il5Ub1/Rt7i25bivFGbPOynPGVd7B50FU8D8dE5urZzaGYzK5q4SLQ8074uL5+qnL/lEwO2TESuMuLizdGSxvBzeKjg20yxETGbHV/wBXdrwmt08HzVpdMD6HKdfW6WlZTLTE19Rc3GKFralC3dPBi8N7MRQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 14, 2025 at 4:38=E2=80=AFAM Shyam Prasad N wrote: > > The Linux kernel does buffered reads and writes using the page cache > layer, where the filesystem reads and writes are offloaded to the > VM/MM layer. The VM layer does a predictive readahead of data by > optionally asking the filesystem to read more data asynchronously than > what was requested. > > The VFS layer maintains a dentry cache which gets populated during > access of dentries (either during readdir/getdents or during lookup). > This dentries within a directory actually forms the address space for > the directory, which is read sequentially during getdents. For network > filesystems, the dentries are also looked up during revalidate. > > During sequential getdents, it makes sense to perform a readahead > similar to file reads. Even for revalidations and dentry lookups, > there can be some heuristics that can be maintained to know if the > lookups within the directory are sequential in nature. With this, the > dentry cache can be pre-populated for a directory, even before the > dentries are accessed, thereby boosting the performance. This could > give even more benefits for network filesystems by avoiding costly > round trips to the server. > I believe you are referring to READDIRPLUS, which is quite common for network protocols and also supported by FUSE. Unlike network protocols, FUSE decides by server configuration and heuristics whether to "fuse_use_readdirplus" - specifically in readdirplus_= auto mode, FUSE starts with readdirplus, but if nothing calls lookup on the directory inode by the time the next getdents call, it stops with readdirpl= us. I personally ran into the problem that I would like to control from the application, which knows if it is doing "ls" or "ls -l" whether a specific getdents() will use FUSE readdirplus or not, because in some situations where "ls -l" is not needed that can avoid a lot of unneeded IO. I do not know if implementing readdirplus (i.e. populate inode and dentry) makes sense for disk filesystems, but if we do it in VFS level, there has t= o be at an API to control or at least opt-out of readdirplus, like with reada= head. Thanks, Amir.