From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7AE2C4332F for ; Wed, 23 Nov 2022 18:09:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B0B86B0071; Wed, 23 Nov 2022 13:09:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 45FB56B0073; Wed, 23 Nov 2022 13:09:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34E526B0074; Wed, 23 Nov 2022 13:09:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 25B0F6B0071 for ; Wed, 23 Nov 2022 13:09:18 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C458F160196 for ; Wed, 23 Nov 2022 18:09:17 +0000 (UTC) X-FDA: 80165493954.12.06A3EF4 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf08.hostedemail.com (Postfix) with ESMTP id 44D0E160014 for ; Wed, 23 Nov 2022 18:09:17 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 3DFECB81F03; Wed, 23 Nov 2022 18:09:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3390C433D6; Wed, 23 Nov 2022 18:09:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669226954; bh=KBd8fI267T1vn3nZ76sXvQiHkudcuy2Zb8Yj692gH+E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=BLKFpFVky1a8HUSn0L45un7ecWdPMISAWZCPzAV4luxrhPF0sls8Dic+lIbVffPjf mHf9m6kZQy8ZPnqMRVsnULiqBGUvB55E224KRxC5GPbYolChrOj5QfikPhOu1/Xlee XRuYY2yOdG9NAMg4wSAA33LfW10c4EcbZtYqYufGlD6FCfPbtC3BDzyDs5yWMue9ES mutzYpLKYvPA4sujAQytdD5S5cpdmDuCO0h7e51kftejDx/5pZa1aZO+g0tMSkwa04 yV/BlyssyFtbyjzm8ltXctLYYOSUkb7aPLGxTyFL6tF13a4dbjMQ9Y2yE+BxnR+/Uz W3mPgPJmFhx1g== Date: Wed, 23 Nov 2022 10:09:13 -0800 From: "Darrick J. Wong" To: Lukas Czerner Cc: Christoph Hellwig , Hugh Dickins , Jan Kara , Eric Sandeen , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH v2 1/3] quota: add quota in-memory format support Message-ID: References: <20221121142854.91109-1-lczerner@redhat.com> <20221121142854.91109-2-lczerner@redhat.com> <20221122142117.epplqsm4ngwx5eyy@fedora> <20221123083615.sj26ptongwhk6wcl@fedora> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221123083615.sj26ptongwhk6wcl@fedora> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669226957; a=rsa-sha256; cv=none; b=7n1OZPaUeIixhTY8/Sq08O2YHwMnllH13W+X2XCa13oh4EZJNGVxBUMG0qrljw43tW1AIU 8usT8lvpBuARapLRMgusj35VdFen3elp5UbAAvrYfISOQgwpDXuTTsEBNSCOVQ7/gqkbUN 5jEAW2E/EolBhAT8S682De4vCP7CxM8= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BLKFpFVk; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of djwong@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=djwong@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669226957; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TSx0Bm3sJdMWxBjCDhiynkOjVfWYYz5uu77WccbvF7g=; b=X1905BFgkPvWEAZ+/mmkHEfdC2uVwJ8FtLJ0dVOTwPsXZS7TbLtu8U55N6yumIxMs2S9+E K4v6WiHwrqZy2xUYUovqbiB6pcd0sYixMQzwb4q/4V9+DGOIFdrwyTowwD85/1uPtqRFk5 4q8MueWkOuIZlj7+tQV7ZfslwRZyh5A= X-Stat-Signature: nip3goh56pbn5wjzhi6td4ido7px9qa1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 44D0E160014 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BLKFpFVk; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of djwong@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=djwong@kernel.org X-Rspam-User: X-HE-Tag: 1669226957-150475 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 23, 2022 at 09:36:15AM +0100, Lukas Czerner wrote: > On Tue, Nov 22, 2022 at 11:58:33PM -0800, Christoph Hellwig wrote: > > On Tue, Nov 22, 2022 at 03:21:17PM +0100, Lukas Czerner wrote: > > > > That seems like a good idea for memory usage, but I think this might > > > > also make the code much simpler, as that just requires fairly trivial > > > > quota_read and quota_write methods in the shmem code instead of new > > > > support for an in-memory quota file. > > > > > > You mean like the implementation in the v1 ? > > > > Having now found it: yes. > > > > Jan, > > do you have any argument for this, since it was your suggestion? > > I also think that the implementation is much simpler with in-memory > dquots because we will avoid all the hassle with creating and > maintaining quota file in a proper format. It's not just reads and > writes it's the entire machinery befind it in quota_v2.c and quota_tree.c. > > But it is true that even with only user modified dquots being > non-reclaimable until unmount it could theoreticaly represent a > substantial memory consumption. Although I do wonder if this problem > is even real. How many user/group ids would you expect extremely heavy > quota user would have the limits set for? 1k, 10k, million, or even > more? Do you know? The last time I checked, some of our container schedulers will heap ~1000 containers onto a single host(!!) at a time. Assuming that a container with a single container might map ~10 uids from the global namespace, that's easily 10,000 at a time. If the container runtime only reuses global uid namespace when it runs out of namespace (i.e. it doesn't immediately recycle them) then you could actually get up in the millions or billions pretty easily. The dquot counters would drop to zero so you might still be able to reclaim the old ones, though it sounds like you'd have to unset any per-dquot limits to get it to do that. That said, fsx in fstests will make all sorts of chown/chgrp calls, which has lead to problems with the XFS quota files reaching their maximum size (~580M per quota type) and filling up the whole fs. --D > -Lukas >