From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 487B9C46CD2 for ; Sat, 27 Jan 2024 10:15:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2C216B0074; Sat, 27 Jan 2024 05:15:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CDBBF6B0078; Sat, 27 Jan 2024 05:15:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA4B96B007B; Sat, 27 Jan 2024 05:15:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A80E46B0074 for ; Sat, 27 Jan 2024 05:15:14 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3B6404063F for ; Sat, 27 Jan 2024 10:15:14 +0000 (UTC) X-FDA: 81724683348.17.796C6FD Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf09.hostedemail.com (Postfix) with ESMTP id 72A3E140002 for ; Sat, 27 Jan 2024 10:15:12 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OQ3YOrEZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of amir73il@gmail.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=amir73il@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706350512; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u738Q0FabqbaEi1Vqzz4jjzfk5DhIEX6cSXtopolu60=; b=RyvAsT+EXtYieMv2kXS80W8O1naigu1X4FP7dEwui4cK8hJEO79OPa/oifp5PkI1PNp5R2 fRpJTSdoPAHKjxxToJjbq+R1xWThQUejyLbrs+7iynghX7AUGYvqz0E/l4F40fxU/Y1/YF 2XzGDjiu4wKLW5I0uJRU/SacmcLaQ9I= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OQ3YOrEZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of amir73il@gmail.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=amir73il@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706350512; a=rsa-sha256; cv=none; b=kRLiPruj9euTS1WL5qptDNUbe84NpJZHoCOlHZVmEmQHv4RIEewrvlfUjL1ishlP9jALp2 cKPedVzbIHmOG9T+tkCP9U7/eN1b/1hidQ9QjAfLPlRKsaKICAmZ2aBsPlZwe/MR+PEFVM g5VvL6Igw9Z1tVHVQUJZkU1IRbzp+OU= Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6868823be58so8944986d6.0 for ; Sat, 27 Jan 2024 02:15:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706350511; x=1706955311; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=u738Q0FabqbaEi1Vqzz4jjzfk5DhIEX6cSXtopolu60=; b=OQ3YOrEZaOcPkGESvzt9XidvGVtCcgEC9n0VUgrKliDO0XHlwaEJ0PNgfwyYjOT++X EetsJsgQiD1PMEuJkzUv1amepbaNbZoXrb+KP69MvVFqyUrMwTvsNBOsr6Qp3riiWwoq c2Guvk2ZF7w0rnlpCy4k7bnNmaMWar/99DdDJ57/uNSIMlWemjr7dwg1bTARBbximYBG EXbWLePolKTThJ7W96nkeG3R71mlwiT+bIeBpjBulgufjS1Y3NSpe9hTPJjMySLBje/S pbeo+HBIbuomYG/iRWi0hKI0/RN/5x8BaIMZeFv+GIuSETtGto5syQikuwBSeDva15hK 3KAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706350511; x=1706955311; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u738Q0FabqbaEi1Vqzz4jjzfk5DhIEX6cSXtopolu60=; b=JW3qxKh57zKf8EBzZARStgfC0lyLGkXOvM7NylDk7JYFHv2rdUVfxmWuXxuStF/yJ/ 1f1M/cO0Xw1f8ULdQwK0Ie0UDitemu4cdnXBn+v8DQ81Xs2qzna3r/Nf67ZDacZ4SRYx W8XkN7Xyuvp40ZdmtVOANZ1x8CbvwbCwrC3XJNJwx3x0+B5y1ayIppPb0tmC6HRnVNp/ z9YLfQSh9v/FtwiqeXh3BZ5U+5FUGjDNFjkuJ3Hiby+ZZQ/NBe/H27vGqQgNCzxhR2HX FAW8tTI3yNKhP7ePXbHfqXWl02pj6KM7381mHCubjNm+C3Pc6sN6LUFh45WbGh/Gh8a0 hJbw== X-Gm-Message-State: AOJu0Yz3/bKjD8Ek6xuOrvgftsuYDZE6aLnrBi7m8CJqRiw0AuVXJpJW pf2fbAhVDoMSGvugvxv6Nju1O0FZGdhs3pW8u0MocpfU548hKJDevHvkHuXiB5H9LtTzoV/V3zr hV+z8z/wL2Y2f0+gA7cDn6dUFEQ0= X-Google-Smtp-Source: AGHT+IFPxZk0oGttomvcI62HnENt3rw00UC+DOKxhh0dQfPo2rop3Sy3Cjbwza0hXBAh2DrMog+u/3/YQmCSBqhVphw= X-Received: by 2002:ad4:5747:0:b0:681:6d57:65f6 with SMTP id q7-20020ad45747000000b006816d5765f6mr1688582qvx.123.1706350511543; Sat, 27 Jan 2024 02:15:11 -0800 (PST) MIME-Version: 1.0 References: <20240125104822.04a5ad44@gandalf.local.home> <2024012522-shorten-deviator-9f45@gregkh> <20240125205055.2752ac1c@rorschach.local.home> <2024012528-caviar-gumming-a14b@gregkh> <20240125214007.67d45fcf@rorschach.local.home> <2024012634-rotten-conjoined-0a98@gregkh> <20240126101553.7c22b054@gandalf.local.home> <2024012600-dose-happiest-f57d@gregkh> <20240126114451.17be7e15@gandalf.local.home> In-Reply-To: <20240126114451.17be7e15@gandalf.local.home> From: Amir Goldstein Date: Sat, 27 Jan 2024 12:15:00 +0200 Message-ID: Subject: Re: [LSF/MM TOPIC] Making pseudo file systems inodes/dentries more like normal file systems To: Steven Rostedt Cc: Greg Kroah-Hartman , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Christian Brauner , Al Viro , Linus Torvalds Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: qfjffcj5yitjzobbzt349su5cpigmkch X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 72A3E140002 X-HE-Tag: 1706350512-756131 X-HE-Meta: U2FsdGVkX1/Gz3un+d9JJNIhmBan4Syg2895/HtBECV2upiFEtBpn6n0d2tiOphDHEtp3zcAPgE+gLWFxwBaC7WmAR9Xwht8zRfihKRs07eOAYrtUgmFnE1LDGdVdKCpOTQ1EhUGaQArXqiaDlU57yOwTCvpamI9SHC5ImflcRviP51VipFrczdZadXMMaGsj3j17jSpHP243ns2RHcW8WJ0wSONj0AEdpXtBvOuC353HoFZBZ5J3nURrj9CwpHVhwNaXVNGaxp5ppDhsGs8mxSJC4tbCphSkNOIoA1blsfhEzpbOl1MgVs7h34O8qPdTFI5RMIPW6fxvUlbRZg1rfmlOrMQqSWzy4mJG80pTLNjqvezsfVad/GIt/w5REtJ+flbJROWbRfTc0Bf8ghpXyrWAocSnp7A2T8ZJG35Fiaj//dtwVjhOiYT1PkxMvWbmCJVMo+OyAqa0MljXs/2fxAJ2lXxiRPC/T0KAbUi4ul46bNaE+UOsPRKUVbl4mGo8uWK37gX8LazNU1k8YWvHOsuphls/XyU2zlRSozGw1KLSzbw+wiI6/1Px0wde6uMNrhl61qG7mggX1FWksGqQY15KqreS2nKkISliCBg85lRy2vAmk1bZHFT9Qjpl1LcpLOYHjWIkJqK88JKa1MZovxHg7OkGU2wsWBrTzASOsPUSF3zWRz/LAn6yKyGC9ggU3f/irjAPWmhu7l9n25Z6XA96TQhMoDaZT8WeU463W16pKQmSl836SGYDiVrZHhPYAIEswtCSXLj2PPqa+GIqlJtBzd17wFMImBEye4+HWhRtTwt4KcL6iDdjsRBkIW60L2I6ckoPmvomkjNeLnFwB2fphoSCB9oTx/dZj2635j1GXieS8HIKGlm3boz8PyBOqnLu9ivT0Sb4QODyv/88A0FavRl3wxgVXhxFr3XwX4h8QD/li4scbZijIqGtLzop4pt7zhZlpu1fJ25yNd Np2+4z+W BdPI/GgKm3RmO4yhpNv5c6ezNscEcNsUwvruSEpLVDbFwNx1WMRl6oB6a8LuhRV5Tg86nvIeCENKpi1vbV3wy6mbzy3BH3Dy96WF4iaOmu6l9IPHF6R7utjnlaQBshznQ4g1TPPcMVz2sDTO+SRsfJj9w24lX4IhFw5ljsuuczCP7ZxAYOpPCDlwid4+Vko+WnOzSRt92cwt6JJ3LLF2d9Qb5hKh1Glu1c+UnKW7bf6r3MYfmVM0KwDLbl5Uw0whysBbD2zOICEVuAxYKMhCQ5T/xCTHCEBqYo2xfOufOJ9Bmr+vXJoYNm8yOMf0mS67VTHZunBr7/w0sP9Bo5dZIX3z8lC7tYlR54wMid4ID8TtLjNIHHbaJhdVbTRh4WSSEg+e4H3t2Uo4QwZ/nSaxtoBfkSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 26, 2024 at 6:44=E2=80=AFPM Steven Rostedt wrote: > > On Fri, 26 Jan 2024 07:41:31 -0800 > Greg Kroah-Hartman wrote: > > > > The reason I brought it up was from Linus's comment about dentries an= d > > > inodes should not exist if the file system isn't mounted. That's not = the > > > case with debugfs. My question is, do we want debugfs to not use dent= ries > > > as its main handle? > > > > In the long run, yes, I want the "handle" that all callers to debugfs t= o > > NOT use a dentry, and have been slowly migrating away from allowing > > debugfs to actually return a dentry to the caller. When that is > > eventually finished, it will be an opaque "handle" that all users of > > debugfs has and THEN we can convert debugfs to do whatever it wants to. > > So it does sound like we are on the same page ;-) > > > > > Again, long-term plans, slowly getting there, if only I had an intern o= r > > 10 to help out with it :) > > Yeah, this is something we need to think about when people come up to us > and say "I'd like to be a kernel developer, is there anything you know of > that I can work on?" Add a KTODO? > > > > > But, this is only being driven by my "this feels like the wrong api to > > use" ideas, and seeing how debugfs returning a dentry has been abused b= y > > many subsystems in places, not by any real-world measurements of > > "debugfs is using up too much memory!" like we have had for sysfs ever > > since the beginning. > > So we have a bit of miscommunication. My motivation for this topic wasn't > necessary on memory overhead (but it does help). But more about the > correctness of debugfs. I can understand how you could have interpreted m= y > motivation, as eventfs was solely motivated by memory pressure. But this > thread was motivated by Linus's comment about dentries not being allocate= d > before mounting. > > > > > If someone comes up with a real workload that shows debugfs is just too > > slow or taking up too much memory for their systems for functionality > > that they rely on (that's the kicker), then the movement for debugfs to > > kernfs would happen much faster as someone would actually have the need > > to do so. > > Another motivation is to prevent another tracefs happening. That is, > another pseudo file system that copies debugfs like the way tracefs was > created. I've had a few conversations with others that say "we have a > special interface in debugfs but we want to move it out". And I've been > (incorrectly) telling them what I did with tracefs from debugfs. > > > > > > > Don't change stuff unless you need to, right? > > > > > > > > > I could look at it too, but as tracefs, and more specifically eve= ntfs, > > > > > has 10s of thousands of files, I'm very concerned about meta data= size. > > > > > > > > Do you have real numbers? If not, then don't worry about it :) > > > > > > I wouldn't be doing any of this without real numbers. They are in the > > > change log of eventfs. > > > > > > See commits: > > > > > > 27152bceea1df27ffebb12ac9cd9adbf2c4c3f35 > > > 5790b1fb3d672d9a1fe3881a7181dfdbe741568f > > > > Sorry, I mean for debugfs. > > No problem. This is how I figured we were talking pass each other. eventf= s > was a big culprit in memory issues, as it has so many files. But now I'm > talking about correctness more than memory savings. And this came about > from my conversations with Linus pointing out that "I was doing it wrong"= ;-) > > > > > > > Again, look at kernfs if you care about the memory usage of your vi= rtual > > > > filesystem, that's what it is there for, you shouldn't have to rein= vent > > > > the wheel. > > > > > > Already did because it was much easier than trying to use kernfs with= out > > > documentation. I did try at first, and realized it was easier to do i= t > > > myself. tracefs was based on top of debugfs, and I saw no easy path t= o go > > > from that to kernfs. > > > > Perhaps do some digging into history and see how we moved sysfs to > > kernfs, as originally sysfs looked exactly like debugfs. That might > > give you some ideas of what to do here. > > I believe one project that should come out of this (again for those that > want to be a kernel developer) is to document how to create a new pseudo > file system out of kernfs. > > > > > > > And the best part is, when people find issues with scaling or other > > > > stuff with kernfs, your filesystem will then benifit (lots of tweak= s > > > > have gone into kernfs for this over the past few kernel releases.) > > > > > > Code is already done. It would be a huge effort to try to convert it = over > > > to kernfs without even knowing if it will regress the memory issues, = which > > > I believe it would (as the second commit saved 2 megs by getting rid = of > > > meta data per file, which kernfs would bring back). > > > > > > So, unless there's proof that kernfs would not add that memory footpr= int > > > back, I have no time to waste on it. > > > > That's fine, I was just responding to your "do we need a in-kernel way > > to do this type of thing" and I pointed out that kernfs already does > > just that. Rolling your own is great, like you did, I'm not saying you > > have to move to kernfs at all if you don't want to as I'm not the one > > having to maintain eventfs :) > > Yeah. So now the focus is on keeping others from rolling their own unless > they have to. I (or more realistically, someone else) could possibly > convert the tracefs portion to kernfs (keeping eventfs separate as it is > from tracefs, due to the amount of files). It would probably take the sam= e > effort as moving debugfs over to kernfs as the two are pretty much > identical. > > Creating eventfs was a great learning experience for me. But it took much > more time than I had allocated for it (putting me way behind in other > responsibilities I have). > > I still like to bring up this discussion with the hopes that someone may = be > interested in fixing this. > I would like to attend the talk about what happened since we suggested that you use kernfs in LSFMM 2022 and what has happened since. I am being serious, I am not being sarcastic and I am not claiming that you did anything wrong :) Also ,please do not forget to also fill out the Google form: https://forms.gle/TGCgBDH1x5pXiWFo7 So we have your attendance request with suggested topics in our spreadsheet= . Thanks, Amir.