From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7066CC47258 for ; Wed, 17 Jan 2024 21:50:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED8046B008C; Wed, 17 Jan 2024 16:50:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E87FD6B0093; Wed, 17 Jan 2024 16:50:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D76A56B0095; Wed, 17 Jan 2024 16:50:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C87926B008C for ; Wed, 17 Jan 2024 16:50:45 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8BABAA0376 for ; Wed, 17 Jan 2024 21:50:45 +0000 (UTC) X-FDA: 81690148050.05.6D50DDE Received: from out-181.mta1.migadu.com (out-181.mta1.migadu.com [95.215.58.181]) by imf03.hostedemail.com (Postfix) with ESMTP id BD5F020009 for ; Wed, 17 Jan 2024 21:50:42 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WKrTihVi; spf=pass (imf03.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.181 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705528243; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hZH2X8i8fkwb6v6GNbjpSk8FwvFnf/kAGw1sC2lsz24=; b=r1nwvrFzYoU9+XPFTBPr0uE1gGypt8vYHdc3G7NTbRYfs0lDuYdtAd5GRIcE3PxQ011wha WHO0PgvksIWS8dv1APdMJHJaqka3H0WWJN2U60Y2wL1BFZAg93rt5VG5cnXfXE5M2voELd gPgAr9JogCV6pPd+/3j+WHHMOPcp968= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705528243; a=rsa-sha256; cv=none; b=qcfX3YpJTqg4r+EfKVwJ4w3N9zbmg6fVYLKmm88EyVh66iS14tPeOWRbHs35Plg9Ir6YOO Mf+2ND5Sd8PJsvXpVncqoujiH7qxmClLbPFrQS7apLoi6GY5m7Jxm8tRG8aHPh8sa4QKY8 cLPlF0uzUsVTtKa8pmmuEsyJHNArbKo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WKrTihVi; spf=pass (imf03.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.181 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Date: Wed, 17 Jan 2024 13:50:22 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1705528240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hZH2X8i8fkwb6v6GNbjpSk8FwvFnf/kAGw1sC2lsz24=; b=WKrTihViJJAbzutckdH2ar3h3Vh3IGlt4SbVr506sVdORjgjf+0prxWLJBSSjS3+v1YPmF R2WrY51IN8wkCcapp5q8uSlxSIA8KusSSq3CU4pkkspKRlVKzpcSEhFnd/fh3N7qhfuN/a kh7pYOsSrqK2T/wLpvrJMMtdbr8Ciyc= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Linus Torvalds Cc: Josh Poimboeuf , Vlastimil Babka , Jeff Layton , Chuck Lever , Shakeel Butt , Johannes Weiner , Michal Hocko , linux-kernel@vger.kernel.org, Jens Axboe , Tejun Heo , Vasily Averin , Michal Koutny , Waiman Long , Muchun Song , Jiri Kosina , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH RFC 1/4] fs/locks: Fix file lock cache accounting, again Message-ID: References: <6667b799702e1815bd4e4f7744eddbc0bd042bb7.camel@kernel.org> <20240117193915.urwueineol7p4hg7@treble> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: BD5F020009 X-Rspam-User: X-Stat-Signature: bw1py1ybg9esuoj9aftxxubeeqt6hcng X-Rspamd-Server: rspam03 X-HE-Tag: 1705528242-90745 X-HE-Meta: U2FsdGVkX18ohbuojflKSGAkpvDBqj4kNrPSEtRoPae7Bs47GCTLoih/AuvrV0KdkFxteqGnhNfVWAYr8E/wyD+RYAZR4IADOZ6iTWkXOxtLnQyIBzyZuPlFUMTW0eAgOiK7GWindyuDACScY4r/7e1u/5lNDxnjMvVIqxJkA0stf/YYyKVP/QxZ5c7h5WMUO7Yq4dEzXOlOl3VAgFrfBLZNELUbRM1EIUTQeHJTlds0AIHo0GrpygoCBKMISAKJ+9ns+f7+VoLkZXNMyr/fMo0WWmv8pzhMn4TerDn1U4iMex2ecxwJ9rpVnHQXaWrME+YvhLntk5LcfSqaszgqprnLbTcLVbJmHsIAcI0AYG+OUAfpBZeCdCV+ePpbrXl1Bwjb4w+EQRWaJ8Gi3h/ohMTJHL3/PZMDrPSzgCyirIivccuN5oysUUkkK9u3iJv7iQJXjNhEkij0cq63noGCle71z3BR+8ictJUtbWNaLQQDMWUijgr7taE57e4qW/GeTwb+kq4nuejaPhYlb6UZ/rww+oSkRXZMzGGccsnIRP5Ukxhvc6TknzhrlPmkboXlgVLR6wYxsR4JiiOjO0JLuqRgSGhEqJHYUTv++JLrY/sJZFh3Ik2Ra5Ix61J7R3xm4JhxFaGEAIuZO5nNvio17TSFI2fzAZUC7OT5hBB1nW20D96w6s3cZD8c8atMZ7mexi2IJKA17E5+2O0UkYc/nFMvAdbZR0KaV22t055wpVpaxpvelhvfOUvLCq3BntRNqXcT3lcp458tNS21AzpD9dFbxxSruFs6InOgShbVyzkhdsHYUnRHhDT3oU/lpSrsg4aKt/NpTSmuYLYEZwMgTB4PpBJzDLLv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 17, 2024 at 12:20:59PM -0800, Linus Torvalds wrote: > On Wed, 17 Jan 2024 at 11:39, Josh Poimboeuf wrote: > > > > That's a good point. If the microbenchmark isn't likely to be even > > remotely realistic, maybe we should just revert the revert until if/when > > somebody shows a real world impact. > > > > Linus, any objections to that? > > We use SLAB_ACCOUNT for much more common allocations like queued > signals, so I would tend to agree with Jeff that it's probably just > some not very interesting microbenchmark that shows any file locking > effects from SLAB_ALLOC, not any real use. > > That said, those benchmarks do matter. It's very easy to say "not > relevant in the big picture" and then the end result is that > everything is a bit of a pig. > > And the regression was absolutely *ENORMOUS*. We're not talking "a few > percent". We're talking a 33% regression that caused the revert: > > https://lore.kernel.org/lkml/20210907150757.GE17617@xsang-OptiPlex-9020/ > > I wish our SLAB_ACCOUNT wasn't such a pig. Rather than account every > single allocation, it would be much nicer to account at a bigger > granularity, possibly by having per-thread counters first before > falling back to the obj_cgroup_charge. Whatever. > > It's kind of stupid to have a benchmark that just allocates and > deallocates a file lock in quick succession spend lots of time > incrementing and decrementing cgroup charges for that repeated > alloc/free. > > However, that problem with SLAB_ACCOUNT is not the fault of file > locking, but more of a slab issue. > > End result: I think we should bring in Vlastimil and whoever else is > doing SLAB_ACCOUNT things, and have them look at that side. > > And then just enable SLAB_ACCOUNT for file locks. But very much look > at silly costs in SLAB_ACCOUNT first, at least for trivial > "alloc/free" patterns.. > > Vlastimil? Who would be the best person to look at that SLAB_ACCOUNT > thing? Probably me. I recently did some work on improving the kmem accounting performance, which is mentioned in this thread and shaves off about 30%: https://lore.kernel.org/lkml/20231019225346.1822282-1-roman.gushchin@linux.dev/ Overall the SLAB_ACCOUNT overhead looks big on micro-benchmarks simple because SLAB allocation path is really fast, so even touching a per-cpu variable adds a noticeable overhead. There is nothing particularly slow on the kmem allocation and release paths, but saving a memcg/objcg pointer, bumping the charge and stats adds up, even though we have batching in place. I believe the only real way to make it significantly faster is to cache pre-charged slab objects, but it adds to the complexity and increases the memory footprint. So far it was all about micro-benchmarks, I haven't seen any complaints on the performance of real workloads. Thanks!