From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC764C47258 for ; Wed, 17 Jan 2024 22:20:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 297886B0087; Wed, 17 Jan 2024 17:20:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 247306B0088; Wed, 17 Jan 2024 17:20:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 110366B0092; Wed, 17 Jan 2024 17:20:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 00FCB6B0087 for ; Wed, 17 Jan 2024 17:20:30 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C41EAA036F for ; Wed, 17 Jan 2024 22:20:30 +0000 (UTC) X-FDA: 81690223020.15.26E8658 Received: from out-177.mta1.migadu.com (out-177.mta1.migadu.com [95.215.58.177]) by imf10.hostedemail.com (Postfix) with ESMTP id DF88AC0011 for ; Wed, 17 Jan 2024 22:20:28 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=oo9nK5OL; spf=pass (imf10.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705530029; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oEcZHshrstC6C+wvUe816o9OqrS9R+Ox74x4Ua6+cNg=; b=PAHooshV2oGtoq6yQrCpVLbSngiW0njuWHSCZdIgqD+FtxKp1Zk/YYzKeqEpeyBSOtdxAv IUPJqM9p0BYx0oI64FwJx2mdxp1HFqJ5krMjF3vougk41m78ywdNemrIm28W/9XRyTMqoA Fm2SoQTsGGg24R40bQQegPwo9EFIfbg= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=oo9nK5OL; spf=pass (imf10.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705530029; a=rsa-sha256; cv=none; b=y/ngjf2AD6mjQIJFVM23jjIUY0GWrCpsu0usgtEwqLAoU/EpeEVrYeujWZTQre4L3sSrHO thGlDQZqOlWSZkGxexxMNCji+3IhYopKKnRap7saJ4O8hUcEW2Fl/8kmdqOqKwJwnyXIvz HrV0KqkQ4GQVGZO0euNDL0nwFtqUcRw= Date: Wed, 17 Jan 2024 14:20:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1705530026; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oEcZHshrstC6C+wvUe816o9OqrS9R+Ox74x4Ua6+cNg=; b=oo9nK5OLo2HbxZhIYJMCnLJpYtsivfboaj7C7gJ0VoDQxxdFw+ieXQh3sl0taplJs8h6VR QHtAdhqNaftf5uCuLcn+Gug1pTa/9k1srBFS3O8bcAo1Y2G6At7h0Cmc+rTWn5qcGvGUQg h1JE6ZW/8v0YWbpn8rfHURYglSWqHUo= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Shakeel Butt Cc: Linus Torvalds , Josh Poimboeuf , Vlastimil Babka , Jeff Layton , Chuck Lever , Johannes Weiner , Michal Hocko , linux-kernel@vger.kernel.org, Jens Axboe , Tejun Heo , Vasily Averin , Michal Koutny , Waiman Long , Muchun Song , Jiri Kosina , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH RFC 1/4] fs/locks: Fix file lock cache accounting, again Message-ID: References: <6667b799702e1815bd4e4f7744eddbc0bd042bb7.camel@kernel.org> <20240117193915.urwueineol7p4hg7@treble> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: DF88AC0011 X-Rspam-User: X-Stat-Signature: 739f35caqzzzh7344wnr4i8mtddzizo8 X-Rspamd-Server: rspam01 X-HE-Tag: 1705530028-832661 X-HE-Meta: U2FsdGVkX18DhfRpC6T5laIeiysZjLPUJMnEmPYfIkGmdW7eRQAlNl8YsQWLHCvVi6Cmnyh5/t6M+K8PaLGs1tlMymu1VFcL5LTRahyTgZZMIt3eJ2GU+MDZZzVpEGl/xWee43piwER45Qg1hINLDIaTl0XSKn7KnGKSjzgQO6Xrp8sO2zZhptthVLNxgsQt/hCl/W6gs+S02BsRTkuwfdPF4HW+dVJEYnIX5FCD2nqCZkv5qAUEveQn/GV//R2FMIn/dt3psqEwrS4IaGZFjGDsPoz9i2bkEHbEbXBhZyfNjMR5jcbdZMWPP2Z41BKC6fGs7qBBAGV++SB5yM4Ldd4cI1depOH4KK+Fns1qhxY1Atcjc7qE24DHJbsGuSPOJlOnq/hj2IgjVJC+AlrBYlzdpIq957XBE4SOf4TCtOaVtVD2Zc+BuYBPNmY8JEUoG1uiYf+04tncnEQvjiFNxZWIqBX+0kFDBJyfY8LQOYrMAgKx09yRbZ5e2JAhIHqJtSv0bATFnw5be/yBbZStILRl+gTIcUCMXoLddxa+mbnj8zn8YTgd+p28b5oedxKeqVp96bnovxNqbvgHnZAFM2xYHH8AnbuzmcjuzSjjYnyWXjZyyhCVcse8rRWyv+iaf8j07C8Astm553DaTgvU0vBqBYQXt0qQMbvflzsXIXZ6bwiikDdffzUHM/jA8nvaQtWxN856rGRdNvwCf2Xx0mXIfzz0amlnV4kBCDCAXlv7/pJQc/ex9r5TQX/F2A4fqlZ+XyBdhdJAcE1ojc+LQUmn9AqEjd1i67X0RWkFOKDguG6YOnd4SPf9g1TpmWmorepYi7c3NwRoWqf8gGTMeIFjysSlQ8gN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 17, 2024 at 01:02:19PM -0800, Shakeel Butt wrote: > On Wed, Jan 17, 2024 at 12:21 PM Linus Torvalds > wrote: > > > > On Wed, 17 Jan 2024 at 11:39, Josh Poimboeuf wrote: > > > > > > That's a good point. If the microbenchmark isn't likely to be even > > > remotely realistic, maybe we should just revert the revert until if/when > > > somebody shows a real world impact. > > > > > > Linus, any objections to that? > > > > We use SLAB_ACCOUNT for much more common allocations like queued > > signals, so I would tend to agree with Jeff that it's probably just > > some not very interesting microbenchmark that shows any file locking > > effects from SLAB_ALLOC, not any real use. > > > > That said, those benchmarks do matter. It's very easy to say "not > > relevant in the big picture" and then the end result is that > > everything is a bit of a pig. > > > > And the regression was absolutely *ENORMOUS*. We're not talking "a few > > percent". We're talking a 33% regression that caused the revert: > > > > https://lore.kernel.org/lkml/20210907150757.GE17617@xsang-OptiPlex-9020/ > > > > I wish our SLAB_ACCOUNT wasn't such a pig. Rather than account every > > single allocation, it would be much nicer to account at a bigger > > granularity, possibly by having per-thread counters first before > > falling back to the obj_cgroup_charge. Whatever. > > > > It's kind of stupid to have a benchmark that just allocates and > > deallocates a file lock in quick succession spend lots of time > > incrementing and decrementing cgroup charges for that repeated > > alloc/free. > > > > However, that problem with SLAB_ACCOUNT is not the fault of file > > locking, but more of a slab issue. > > > > End result: I think we should bring in Vlastimil and whoever else is > > doing SLAB_ACCOUNT things, and have them look at that side. > > > > And then just enable SLAB_ACCOUNT for file locks. But very much look > > at silly costs in SLAB_ACCOUNT first, at least for trivial > > "alloc/free" patterns.. > > > > Vlastimil? Who would be the best person to look at that SLAB_ACCOUNT > > thing? See commit 3754707bcc3e (Revert "memcg: enable accounting for > > file lock caches") for the history here. > > > > Roman last looked into optimizing this code path. I suspect > mod_objcg_state() to be more costly than obj_cgroup_charge(). I will > try to measure this path and see if I can improve it. It's roughly an equal split between mod_objcg_state() and obj_cgroup_charge(). And each is comparable (by order of magnitude) to the slab allocation cost itself. On the free() path a significant cost comes simple from reading the objcg pointer (it's usually a cache miss). So I don't see how we can make it really cheap (say, less than 5% overhead) without caching pre-accounted objects. I thought about merging of charge and stats handling paths, which _maybe_ can shave off another 20-30%, but there still will be a double-digit% accounting overhead. I'm curious to hear other ideas and suggestions. Thanks!