From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0743AC47422 for ; Fri, 26 Jan 2024 15:16:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9141E6B0074; Fri, 26 Jan 2024 10:16:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89D826B0085; Fri, 26 Jan 2024 10:16:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73E4A6B0087; Fri, 26 Jan 2024 10:16:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5F1056B0074 for ; Fri, 26 Jan 2024 10:16:12 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2A92AA05E0 for ; Fri, 26 Jan 2024 15:16:12 +0000 (UTC) X-FDA: 81721812984.25.E74F445 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf10.hostedemail.com (Postfix) with ESMTP id 9F086C0032 for ; Fri, 26 Jan 2024 15:16:09 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of "SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" designates 145.40.73.55 as permitted sender) smtp.mailfrom="SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706282170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iEWdlYAS+StacIla4jVfSsIoafLpNmLBuhzhPp9dZiE=; b=5+5OBOKVcVtQn6Zrb87Et4UCi+DpBxnWGO2Ql4HgJgqHKGHPk8BeJkeUqvPugyb/V7Msaa xe+nbi8jV6zBb8CnmJ9FIC9Hdre4+QE67UJ6BR254QDiQCfUmftNIw6uWBWlTNxhB22D45 JreAv6IixuSPVfB/87Sfauzoul4OG4c= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of "SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" designates 145.40.73.55 as permitted sender) smtp.mailfrom="SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706282170; a=rsa-sha256; cv=none; b=EAi+oKQOmo1/+6k1pX7f/sw02/LwM8xilzLUTQ0W09gN9qPhdA2at0DY0sHm6eKE5yBuaK 0eOJk8XiVn2b3tCILcA2KaansZXDv71kr8WNcBLr2r1nUT+PtE6x7jvo3+WexsR4d3M9th 1Kcb62SdOBMRcLR76CX8Y1ip6PJQoEk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id DE2EBCE174E; Fri, 26 Jan 2024 15:16:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34C84C43390; Fri, 26 Jan 2024 15:15:51 +0000 (UTC) Date: Fri, 26 Jan 2024 10:15:53 -0500 From: Steven Rostedt To: Greg Kroah-Hartman Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Christian Brauner , Al Viro , Linus Torvalds Subject: Re: [LSF/MM TOPIC] Making pseudo file systems inodes/dentries more like normal file systems Message-ID: <20240126101553.7c22b054@gandalf.local.home> In-Reply-To: <2024012634-rotten-conjoined-0a98@gregkh> References: <20240125104822.04a5ad44@gandalf.local.home> <2024012522-shorten-deviator-9f45@gregkh> <20240125205055.2752ac1c@rorschach.local.home> <2024012528-caviar-gumming-a14b@gregkh> <20240125214007.67d45fcf@rorschach.local.home> <2024012634-rotten-conjoined-0a98@gregkh> X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 9F086C0032 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 3uzis9g9xoizcs68488dx9z83u4gys49 X-HE-Tag: 1706282169-340958 X-HE-Meta: U2FsdGVkX1/AKiQfWJkRR5hFKXKjO/iucS0LqM7rBy5TV/O9nHJFkQ17d8ugODuRg4LP/yvD9EDdqym+RPpyYmcToelpVIsbjWHEYGMAk5pepVx9AL4BxjJ4BUe4pCiQQ2shP63lQ4kMbMLqTzN7Lvg6b0A4637Ms6MuR5U+asonX0j3p+sESY3URKhjDjrbqNXDWPoM8tIQuILQt025tcEaVUHzFtmeTTZIW/KgxllPlfNiZM8xR4IjmwXpcBNHPqQaBdZVHD9i3Lxrl3jmb2SDLwMafvhF327n8gL9ct3l5zSMfj7jyQqi1uyUnVZfE5HfM1R3XTQsHwDXBaq9l5p0qgyvC6O42rz2740UYqBflVzHXIu2Og9SxedIrh5m7s8LMYI02MGK+6rADNqoRnJVgTARTsyXh2ffSyc4uP1SZGGBlTalRkxjHSgx2vOWUCfpLY9c3bo83rcbqclim/NvpyT3USEEmHgHfByzJr4N3cU+8bAViuH0RFP4028ntfveMDdBm2tqqsekcTtSbsNIok2qQM9qFU5cH+QKAXJdpzvSXPxqA5ZZwm1eM2v31pxTQO/dpEMviN9oDq6uDclkisIwQEmyRwiqkDS94VMAtPwiwQzZcqNqzA1FW4QCNakp6sNSnkLQ825XbU9hl2srDRwGDUygD1hYKNw0e4skDIT3d0cBHejWkizc1+uZTdetRrCEyXS+NcBGfybyZpVtqAiGwpOtj4emccLarPGzqVkkeoTs+xAZj52MYc/MaNUaYKKoYe3R2oqS+Hs7uhhHlfxeaRSJ87+QmfoLV3YN9zntTEPtaiC4noI/cCfmhY4VLjcZHw5nXUv3lT+naB6BktsW667lpDzPmLVoHrrSfEHYNcEUoDU7c0pJVD2CVX2/5AtBAww+4+1mIF0h1Fg7OXvKIPQEj/Y4b08xt5jJQXbTJixN0zdII6atIChSA+m792TSQ0fnmkVDFl7 nL4zzqbZ p7XuSdTyc1l8dovjcv2+p/Q7nThaiCT+0nmn47zD+CVbPZ/OmU+DQjSLoO9s8n/RDc+l1aCXmib/7zUtBM8iv/b6BiE40OwVVMlZGPFOkD+MstuQ0v0LtwMJTWUUqNAnLi+uazRmbxVofHIEmrZbmCcGYUkxBeh6Bgx2N8B9JQDf/8H8Hz8JRgXZW7cQSnWaWNkQI6i41wgxfOlIorfq+gE4fCQnxF6wjV6vY/65UsONT9SQFltFPUHzNBLG/ioNnL1IhhYqjDx3yb2KCNrMiRvZboRiWxbM9/lDQ1W1lP/RRaX7degG7O5yMxw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 26 Jan 2024 06:16:38 -0800 Greg Kroah-Hartman wrote: > On Thu, Jan 25, 2024 at 09:40:07PM -0500, Steven Rostedt wrote: > > On Thu, 25 Jan 2024 17:59:40 -0800 > > Greg Kroah-Hartman wrote: > > > > > > I tried to use kernfs when doing a lot of this and I had issues. I > > > > don't remember what those were, but I can revisit it. > > > > > > You might, as kernfs makes it so that the filesystem structures are > > > created on demand, when accessed, and then removed when memory pressure > > > happens. That's what sysfs and configfs and cgroups use quite > > > successfully. > > > > kernfs doesn't look trivial and I can't find any documentation on how > > to use it. > > You have the code :) Really Greg? I can write what I want to do twice as fast than trying to figure out why someone else did what they did in their code, unless there's good documentation on the subject. > > > Should there be work to move debugfs over to kernfs? > > Why? Are you seeing real actual memory use with debugfs that is causing > problems? That is why we made kernfs, because people were seeing this > in sysfs. The reason I brought it up was from Linus's comment about dentries and inodes should not exist if the file system isn't mounted. That's not the case with debugfs. My question is, do we want debugfs to not use dentries as its main handle? > > Don't change stuff unless you need to, right? > > > I could look at it too, but as tracefs, and more specifically eventfs, > > has 10s of thousands of files, I'm very concerned about meta data size. > > Do you have real numbers? If not, then don't worry about it :) I wouldn't be doing any of this without real numbers. They are in the change log of eventfs. See commits: 27152bceea1df27ffebb12ac9cd9adbf2c4c3f35 5790b1fb3d672d9a1fe3881a7181dfdbe741568f > > > Currently eventfs keeps a data structure for every directory, but for > > the files, it only keeps an array of names and callbacks. When a > > directory is registered, it lists the files it needs. eventfs is > > specific that the number of files a directory has is always constant, > > and files will not be removed or added once a directory is created. > > > > This way, the information on how a file is created is done via a > > callback that was registered when the directory was created. > > That's fine, and shouldn't matter. > > > For this use case, I don't think kernfs could be used. But I would > > still like to talk about what I'm trying to accomplish, and perhaps see > > if there's work that can be done to consolidate what is out there. > > Again, look at kernfs if you care about the memory usage of your virtual > filesystem, that's what it is there for, you shouldn't have to reinvent > the wheel. Already did because it was much easier than trying to use kernfs without documentation. I did try at first, and realized it was easier to do it myself. tracefs was based on top of debugfs, and I saw no easy path to go from that to kernfs. > > And the best part is, when people find issues with scaling or other > stuff with kernfs, your filesystem will then benifit (lots of tweaks > have gone into kernfs for this over the past few kernel releases.) Code is already done. It would be a huge effort to try to convert it over to kernfs without even knowing if it will regress the memory issues, which I believe it would (as the second commit saved 2 megs by getting rid of meta data per file, which kernfs would bring back). So, unless there's proof that kernfs would not add that memory footprint back, I have no time to waste on it. -- Steve