From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 013DAC47422 for ; Fri, 26 Jan 2024 16:45:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 61EE86B0088; Fri, 26 Jan 2024 11:45:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CE356B0095; Fri, 26 Jan 2024 11:45:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E46A6B0096; Fri, 26 Jan 2024 11:45:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 405066B0088 for ; Fri, 26 Jan 2024 11:45:01 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 147324068B for ; Fri, 26 Jan 2024 16:45:01 +0000 (UTC) X-FDA: 81722036802.11.550DB0B Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id B655E18001B for ; Fri, 26 Jan 2024 16:44:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf24.hostedemail.com: domain of "SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" designates 145.40.73.55 as permitted sender) smtp.mailfrom="SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706287498; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l3zI+ktjlpLlXWvtgZObNJzX15+deYauMB2zY668Mes=; b=s0ynchITJyGS8yutA45jBTm6LWAWeZQ/5Ut0sMgnK6hHw+CFPdPppnWfFBptBWyTYwp+aS VOL9iS0K3d2kZvscSe8UQamIiAO/dhp/cP+19lCPRFKqw/YRjiCa+g9BoO/e774hw04nnu cr7H2xwerqRYR+pI7I7Or1oHHIfP84U= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf24.hostedemail.com: domain of "SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" designates 145.40.73.55 as permitted sender) smtp.mailfrom="SRS0=9W0d=JE=goodmis.org=rostedt@kernel.org" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706287498; a=rsa-sha256; cv=none; b=WMW6Tbr2ofdI6GU6IGXgjx+9EYLeAIdDK4AziSvrHtr5sjv03Kk5HG7toyH50PVwsVAgfK 3+tgJfenJ3Dyjm9rDVaRv815kokD03oFxXcODtv+xMWoRNV8ZX1klkf0k9esjq8ZO+z0eA WODDTZuuGvA/AFzh9ekP6m2YRMFE6lQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id C2DABCE34DD; Fri, 26 Jan 2024 16:44:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1602C433F1; Fri, 26 Jan 2024 16:44:48 +0000 (UTC) Date: Fri, 26 Jan 2024 11:44:51 -0500 From: Steven Rostedt To: Greg Kroah-Hartman Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Christian Brauner , Al Viro , Linus Torvalds Subject: Re: [LSF/MM TOPIC] Making pseudo file systems inodes/dentries more like normal file systems Message-ID: <20240126114451.17be7e15@gandalf.local.home> In-Reply-To: <2024012600-dose-happiest-f57d@gregkh> References: <20240125104822.04a5ad44@gandalf.local.home> <2024012522-shorten-deviator-9f45@gregkh> <20240125205055.2752ac1c@rorschach.local.home> <2024012528-caviar-gumming-a14b@gregkh> <20240125214007.67d45fcf@rorschach.local.home> <2024012634-rotten-conjoined-0a98@gregkh> <20240126101553.7c22b054@gandalf.local.home> <2024012600-dose-happiest-f57d@gregkh> X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: 39toh4kj96tk19d5gme3quexaukfkx5d X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B655E18001B X-HE-Tag: 1706287497-806221 X-HE-Meta: U2FsdGVkX19brbnGvViD/JYPq5da3BfETyHFTqaLzPPYf33jBD4eziI2+3MKdRH17e5n22QyxFlLlo8ocDlDPafz30UjnYtDMS+VTU8LQVKv/5npgny3PLHSpbo6WP52RdyZU9TGd6f0TvVHdtEN6iuksCjeqw92jJpvC6L0F+U0errRiDdjT9AHUIWm68OAiIJRNp7TZRUzz3vDXN67d0gXi/ipg+VfeYA2T1LCF3oNONZK9ZIGJeL3T6LAad09fvk1eS0S9GOC+0ZHJgLgPB40Ha5l9e8aAJ2CM0ekf0x9YigkQ5/rE7ybQnh5OGn5wtrzLGGWg3iN0Vrmu6hDkN/QoXuOviBQm/fTe7jt/17zOQUaV7UTW+f+R9Gt8dFTdJmWS3zeqVELBEUdpA4Ti5c3p2UiImGWce3+RUxNRidjrCoeJnhorBLLCCJYUmGxCm84ZikIYTt/OnsIBVwvVLYl2fP/vt5QlQP0Xm88z9RJ/nGu4qeaoAsaDDNB+3D3ZWwHZ+96iW6pi2nvVNRRkLLdErbyx5ougDHsFbrUbsK/AbgQ4CJazbSZVDqzf1d0KBwbuXzJrxK8ROHAOZ6wAQ/oTfTo1il7/1trHmJgNG8VGfhSYJPoa7BjAJ1G/f9jfoi+j7yOKcTgbiYuPIjc6FjatJ1vzDTyyPbpj6sVrK/Ov2POpSleV3wd2kc8KhElvRWgL4ZZXeMRhQFqXxkbtLPEpLucg8sdHG1ESzfKWRX60wtN09gLhRrceK7EoncK/frJv0W8Skqz9dKE733ii5YN46knlKIkyqN6vJ0xQ6jZE9MmFVBz8lsjaK56gRWeotEcF3waKGEhC0mFuhFhmVwlo4BR29MShRSrgXVYHzJ1EBOoDOZcjoR/Ef2whspWGhmYmUgxnN8MjIzuZicXpklna2nJr/3u1CMgx1p3nFdNK0O4OoOJMfbXlSkRM4mimJOM20tixZuDeD4EjMl B5Gp1n+B aLgf3rea2+AbDY+3txlvR+lv8KK0qre6gB6SCKLEYTQlkcWuxxwj7cB9Nhk9H3HS5YmToJB6+9uTm9gqJcM/TEmoZ3TGcyA928+omioP1QJLEG4VW+gjqfT/0DcRpkBnIfTrbRxzxwlXP+il1U3t7ivEJ8cIUHhNzRiZtpqhe/UwMkSEKXk5b+o+/KYcm4Ule1kOn9oRwwDGo0eXoy7U1MR8l72mzpAhu8bsWzZn7zCqo4OUUV+fBPIjlcW0hCO2r2ZMWvrbq+Ob6ETdeMr7i0DzQzy0fxrP0dgXxZ7bnl4sDTuZU5A/x9340Xw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 26 Jan 2024 07:41:31 -0800 Greg Kroah-Hartman wrote: > > The reason I brought it up was from Linus's comment about dentries and > > inodes should not exist if the file system isn't mounted. That's not the > > case with debugfs. My question is, do we want debugfs to not use dentries > > as its main handle? > > In the long run, yes, I want the "handle" that all callers to debugfs to > NOT use a dentry, and have been slowly migrating away from allowing > debugfs to actually return a dentry to the caller. When that is > eventually finished, it will be an opaque "handle" that all users of > debugfs has and THEN we can convert debugfs to do whatever it wants to. So it does sound like we are on the same page ;-) > > Again, long-term plans, slowly getting there, if only I had an intern or > 10 to help out with it :) Yeah, this is something we need to think about when people come up to us and say "I'd like to be a kernel developer, is there anything you know of that I can work on?" Add a KTODO? > > But, this is only being driven by my "this feels like the wrong api to > use" ideas, and seeing how debugfs returning a dentry has been abused by > many subsystems in places, not by any real-world measurements of > "debugfs is using up too much memory!" like we have had for sysfs ever > since the beginning. So we have a bit of miscommunication. My motivation for this topic wasn't necessary on memory overhead (but it does help). But more about the correctness of debugfs. I can understand how you could have interpreted my motivation, as eventfs was solely motivated by memory pressure. But this thread was motivated by Linus's comment about dentries not being allocated before mounting. > > If someone comes up with a real workload that shows debugfs is just too > slow or taking up too much memory for their systems for functionality > that they rely on (that's the kicker), then the movement for debugfs to > kernfs would happen much faster as someone would actually have the need > to do so. Another motivation is to prevent another tracefs happening. That is, another pseudo file system that copies debugfs like the way tracefs was created. I've had a few conversations with others that say "we have a special interface in debugfs but we want to move it out". And I've been (incorrectly) telling them what I did with tracefs from debugfs. > > > > Don't change stuff unless you need to, right? > > > > > > > I could look at it too, but as tracefs, and more specifically eventfs, > > > > has 10s of thousands of files, I'm very concerned about meta data size. > > > > > > Do you have real numbers? If not, then don't worry about it :) > > > > I wouldn't be doing any of this without real numbers. They are in the > > change log of eventfs. > > > > See commits: > > > > 27152bceea1df27ffebb12ac9cd9adbf2c4c3f35 > > 5790b1fb3d672d9a1fe3881a7181dfdbe741568f > > Sorry, I mean for debugfs. No problem. This is how I figured we were talking pass each other. eventfs was a big culprit in memory issues, as it has so many files. But now I'm talking about correctness more than memory savings. And this came about from my conversations with Linus pointing out that "I was doing it wrong" ;-) > > > > Again, look at kernfs if you care about the memory usage of your virtual > > > filesystem, that's what it is there for, you shouldn't have to reinvent > > > the wheel. > > > > Already did because it was much easier than trying to use kernfs without > > documentation. I did try at first, and realized it was easier to do it > > myself. tracefs was based on top of debugfs, and I saw no easy path to go > > from that to kernfs. > > Perhaps do some digging into history and see how we moved sysfs to > kernfs, as originally sysfs looked exactly like debugfs. That might > give you some ideas of what to do here. I believe one project that should come out of this (again for those that want to be a kernel developer) is to document how to create a new pseudo file system out of kernfs. > > > > And the best part is, when people find issues with scaling or other > > > stuff with kernfs, your filesystem will then benifit (lots of tweaks > > > have gone into kernfs for this over the past few kernel releases.) > > > > Code is already done. It would be a huge effort to try to convert it over > > to kernfs without even knowing if it will regress the memory issues, which > > I believe it would (as the second commit saved 2 megs by getting rid of > > meta data per file, which kernfs would bring back). > > > > So, unless there's proof that kernfs would not add that memory footprint > > back, I have no time to waste on it. > > That's fine, I was just responding to your "do we need a in-kernel way > to do this type of thing" and I pointed out that kernfs already does > just that. Rolling your own is great, like you did, I'm not saying you > have to move to kernfs at all if you don't want to as I'm not the one > having to maintain eventfs :) Yeah. So now the focus is on keeping others from rolling their own unless they have to. I (or more realistically, someone else) could possibly convert the tracefs portion to kernfs (keeping eventfs separate as it is from tracefs, due to the amount of files). It would probably take the same effort as moving debugfs over to kernfs as the two are pretty much identical. Creating eventfs was a great learning experience for me. But it took much more time than I had allocated for it (putting me way behind in other responsibilities I have). I still like to bring up this discussion with the hopes that someone may be interested in fixing this. Thanks, -- Steve