From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70949C433EF for ; Tue, 22 Mar 2022 22:29:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 048486B007B; Tue, 22 Mar 2022 18:29:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F39CD6B007D; Tue, 22 Mar 2022 18:29:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E29EC6B0085; Tue, 22 Mar 2022 18:29:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id D61D66B007B for ; Tue, 22 Mar 2022 18:29:16 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A592A61EE1 for ; Tue, 22 Mar 2022 22:29:16 +0000 (UTC) X-FDA: 79273464312.11.7ED63EF Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by imf31.hostedemail.com (Postfix) with ESMTP id 08EA220010 for ; Tue, 22 Mar 2022 22:29:15 +0000 (UTC) Received: from dread.disaster.area (pa49-186-150-27.pa.vic.optusnet.com.au [49.186.150.27]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 94AD3533BF9; Wed, 23 Mar 2022 09:29:13 +1100 (AEDT) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1nWn04-008gHP-G8; Wed, 23 Mar 2022 09:29:12 +1100 Date: Wed, 23 Mar 2022 09:29:12 +1100 From: Dave Chinner To: Roman Gushchin Cc: Matthew Wilcox , Stephen Brennan , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Gautham Ananthakrishna , khlebnikov@yandex-team.ru Subject: Re: [LSF/MM TOPIC] Better handling of negative dentries Message-ID: <20220322222912.GF1609613@dread.disaster.area> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.4 cv=e9dl9Yl/ c=1 sm=1 tr=0 ts=623a4dba a=sPqof0Mm7fxWrhYUF33ZaQ==:117 a=sPqof0Mm7fxWrhYUF33ZaQ==:17 a=IkcTkHD0fZMA:10 a=o8Y5sQTvuykA:10 a=7-415B0cAAAA:8 a=Mo5KHSmeIGZkFYpUIgAA:9 a=QEXdDO2ut3YA:10 a=biEYGPWJfzWAr4FL6Ov7:22 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 08EA220010 X-Stat-Signature: djgckfrx5pgfub9uq6aiuhem99yunmn6 Authentication-Results: imf31.hostedemail.com; dkim=none; spf=none (imf31.hostedemail.com: domain of david@fromorbit.com has no SPF policy when checking 211.29.132.246) smtp.mailfrom=david@fromorbit.com; dmarc=none X-Rspam-User: X-HE-Tag: 1647988155-831532 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 22, 2022 at 02:19:30PM -0700, Roman Gushchin wrote: > On Tue, Mar 22, 2022 at 08:41:56PM +0000, Matthew Wilcox wrote: > > On Tue, Mar 15, 2022 at 01:56:18PM -0700, Roman Gushchin wrote: > > > I’d be happy to join this discussion. And in my opinion it’s going > > > beyond negative dentries: there are other types of objects which tend > > > to grow beyond any reasonable limits if there is no memory pressure. > > > > > > A perfect example when it happens is when a machine is almost idle > > > for some period of time. Periodically running processes creating > > > various kernel objects (mostly vfs cache) which over time are filling > > > significant portions of the total memory. And when the need for memory > > > arises, we realize that the memory is heavily fragmented and it’s > > > costly to reclaim it back. > > > > When you say "vfs cache", do you mean page cache, inode cache, or > > something else? > > Mostly inodes and dentries, but also in theory some fs-specific objects > (e.g. xfs implements nr_cached_objects/free_cached_objects callbacks). Those aren't independent shrinkers - they are part of the superblock shrinker. XFS just uses these for background management of the VFS inode cache footprint - vfs inodes live a bit longer than ->destroy_inode in XFS - so these callouts from the superblock shrinker are really just part of the existing VFS inode cache management. > Also dentries, for example, can have attached kmalloc'ed areas if the > length of the file's name is larger than x. And probably there are more > examples of indirectly pinned objects. The xfs buffer cache has slab allocated handles that can pin up to 64kB of pages each. The buffer cache can quickly grow to hold tens of gigabytes of memory when you have filesystems with hundreds of gigabytes of metadata in them (which are not that uncommon). It's also not uncommon for the XFS buffer cache to consume more memory than the VFS dentry and inode caches combined. Cheers, Dave. -- Dave Chinner david@fromorbit.com