From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 757A7C433EF for ; Tue, 5 Apr 2022 21:18:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C631C6B0072; Tue, 5 Apr 2022 17:18:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C12936B0073; Tue, 5 Apr 2022 17:18:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADA3E6B0074; Tue, 5 Apr 2022 17:18:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id A12606B0072 for ; Tue, 5 Apr 2022 17:18:32 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 63657AD6E4 for ; Tue, 5 Apr 2022 21:18:22 +0000 (UTC) X-FDA: 79324088844.31.F24271F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id 906D6180016 for ; Tue, 5 Apr 2022 21:18:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7d4jHN3hEVC7czPTgRhPRNegosioytKGOnpN+jNlS3I=; b=A62E7PMTsau07ViNlj8KL+QG3d raIeADbj0mqUWjK39CpcCv3kTB3mlGZxV4RMhqen7LQ9O5+SmC9BLqJ3lzT7iJxXZN0LJhJyis/V+ cSiIAAKBylGreUeWo50FEhGx/xQUXwU0TYercKUbCUI/9uZrH5/Jkf/UgdXQy/0z0Ec/l81NBI7eG eABiiRFdDmkrq3aoYO2c8lfiubAVqzVnnhM/FZ4r8tLE83MrO/6Ih5OEVcsUHmJbXhn31xaKeeK8i ceJUFE3StPeb3Vc+Nsz9RxW6UEOaY2LTzAs5Eu+u1BagGMbd+2dvAzSievhz69Ysb68dRzF+1M2/J cantghkg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbqYr-0073yK-Kv; Tue, 05 Apr 2022 21:18:01 +0000 Date: Tue, 5 Apr 2022 22:18:01 +0100 From: Matthew Wilcox To: Stephen Brennan Cc: Dave Chinner , Hillf Danton , Roman Gushchin , MM , Mel Gorman , Yu Zhao , David Hildenbrand , LKML Subject: Re: [RFC] mm/vmscan: add periodic slab shrinker Message-ID: References: <20220402072103.5140-1-hdanton@sina.com> <20220403005618.5263-1-hdanton@sina.com> <20220404010948.GV1609613@dread.disaster.area> <87ilrn5ttl.fsf@stepbren-lnx.us.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ilrn5ttl.fsf@stepbren-lnx.us.oracle.com> X-Stat-Signature: brfyykddyo5qx816ast8x3qxtuz6bggn Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=A62E7PMT; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 906D6180016 X-HE-Tag: 1649193501-224369 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 05, 2022 at 10:22:14AM -0700, Stephen Brennan wrote: > I can't speak for every slab cache, but I've been coming to the same > conclusion myself regarding the dentry cache. I think that the rate of > stepping through the LRU should be tied to the rate of allocations. > Truly in-use objects shouldn't be harmed by this, as they should get > referenced and rotated to the beginning of the LRU. But the one-offs > which are bloating the cache will be found and removed. I agree with all this. > I've implemented a version of this patch which just takes one step > through the LRU on each d_alloc. It's quite interesting to watch it > work. You can create 5 million negative dentries in directory /A via > stat(), and then create 5 million negative dentries in directory /B. The > total dentry slab size reaches 5 million but never goes past it, since > the old negative dentries from /A aren't really in use, and they get > pruned at the same rate as negative dentries from /B get created. On the > other hand, if you *continue* to stat() on the dentries of /A while you > create negative dentries in /B, then the cache grows to 10 million, > since the /A dentries are actually in use. > > Maybe a solution could involve some generic list_lru machinery that can > let you do these sorts of incremental scans? Maybe batching them up so > instead of doing one every allocation, you do them every 100 or 1000? > It would still be up to the individual user to put this to good use in > the object allocation path. I feel like we need to allow the list to both shrink and grow, depending on how useful the entries in it are. So one counter per LRU, incremented on every add. When that counter gets to 100, reset it to 0 and scan 110 entries. Maybe 0 of them can be reclaimed; maybe 110 of them can be. But the list can shrink over time instead of being a "one in, one out" scenario. Clearly 110 is a magic number, but intuitively, attempting to shrink by 10% feels reasonable. Need to strike a balance between shrinking quickly enough and giving the cache time to figure out which entries are actually useful.