From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14530C021B2 for ; Sun, 23 Feb 2025 04:27:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B7B76B0083; Sat, 22 Feb 2025 23:27:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 767FF6B0085; Sat, 22 Feb 2025 23:27:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62F1A6B0088; Sat, 22 Feb 2025 23:27:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 461F76B0083 for ; Sat, 22 Feb 2025 23:27:46 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B692A161FC7 for ; Sun, 23 Feb 2025 04:27:45 +0000 (UTC) X-FDA: 83149926090.09.C84543A Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf21.hostedemail.com (Postfix) with ESMTP id DD8D11C0003 for ; Sun, 23 Feb 2025 04:27:43 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="jGlq4v//"; spf=pass (imf21.hostedemail.com: domain of surenb@google.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740284863; a=rsa-sha256; cv=none; b=4RGeq9xK2Ui5TSUs++Pj/64kdsdGzQmBohe23Bad07jEIUGVRIVSobdBitYZGc9/xGSkfr UrfdZVmZV5eV2ZUufA6ONtI8UmuRDPWAf8QPr2CweM2AXLPj5DWJ14VKJxPwXTG9AwChup HILpuEo6NsHaw8IdxmIc3NaAbnTvzxo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="jGlq4v//"; spf=pass (imf21.hostedemail.com: domain of surenb@google.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740284863; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nmm1I+SoVSWD3Lw3CCPR58AX+UVflR4TsL2veWY2GZs=; b=tel9ie7bxh7Y61hUt3OxZRFdw5AXCB3h+JpKxdlN9oYQ8TWjqcDPtrjTZzcnBOIwbMRU5w 0J8LC01+wviSv2EyX/baFeoBdlseCH0+Mdy82JUdd5h1HqlGXyWkLa9YHcCQK2+IOzwIRb 1Gca1gRmaGXnPh7EIBlnyP4ZKzgm3hA= Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-472098e6e75so243481cf.1 for ; Sat, 22 Feb 2025 20:27:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740284863; x=1740889663; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nmm1I+SoVSWD3Lw3CCPR58AX+UVflR4TsL2veWY2GZs=; b=jGlq4v//FNkm277VyIflFWdjHx1u4yDx96DvoXjkAGBaUz5y+KBW7Pd5mTjGSAfZPu lFh+8fPqU3SKeBhZol10qT4Q9w+ijMRvOzndH1p7ghdXO1trLbP1Jv8AvPsdJ1qB1IMH MvQhgEXunssq0R91Sm906INOJ/rjq2NSpQhYaTY+Wm+ADX6A6PufYOjx3K4ShL5bBn+h PJyE0qVqqB81VCOWtz9KtcxeDqLlqdhH0TSxulFdMzczThtrYUnYNbF3xBrX4CQXK4lQ 4SYudbIuRpCy2nAH1GB4/BEa2dFJgDJSzPC/RRLP6tRIeigLa6zScwG9HeHjp5IWfqUY cSsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740284863; x=1740889663; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nmm1I+SoVSWD3Lw3CCPR58AX+UVflR4TsL2veWY2GZs=; b=sgirw1WB9qjxB9r1OprCAwYbE2+Wx1KpH5rS/xHnkZS+GtDiPQ8LWiLVWviogu2yAQ HCp4PtIHuNGIos9B7k1WvZdiVKXPRYW4TK0Ns71HM9oe3G7CGJA1V0mmaqjpCUQSqiD0 O73BIx/pddg3NCWAYldW18F1mdsRLkYaSpbqOfFdkey1DDHOASNm2dS6ey7Zwtyj0kOZ GJNEt1wdXDqgOl1vwpJmswuLhP1SilouU76pu7MalB6uUwJz8BmloYSWljQp9wbCPpXM DQfxcwg8Fef3snsl9ZsCpmT9mBwiYJXvpwzNmfIvd+/SurIVMbH9kgsBEiFTM8m5lsf1 oKeA== X-Forwarded-Encrypted: i=1; AJvYcCVl8HUNmTYzLhWktbWstxukBM6vtQyC9zYrAYrd6X1B/8XAp1XForAr3dSkpLAneNO6QExmCOT1vA==@kvack.org X-Gm-Message-State: AOJu0Yw/MPExXwgNJp2kunio2CY5cMzIZ258dl1dgqZDtRGGMhJjOhhj oF+nxFBr48E0mJEuNk2/v2rzucaCsx49mxcTMeZCwXods1gk8G9VVGkJgVpAbBuocPGlpzP2raf 2v1xEG6P/HytpFTtAIpBnioKnJlY1KlGA7z4g X-Gm-Gg: ASbGnctF+4jOHOcUiX9gorDIxEmWUPNub4dBYY7GkO3nCGQMd7w+eQ2Smv2QAxfExuo 9SK7BoSXBXrQ+ilO1k4uAGSlR4017CUP8focDBNLhnwaAT7qtu7UhxIIszYv2YulR9g8Q+5ba46 Tv7jVCbus= X-Google-Smtp-Source: AGHT+IFKfxPHgKbPeASJWumHoaXB34NvZ31EQJsUHMALoZH4DiPcJzVNOQgIWlGulYe46FriKG+KJhd2DEw1EVddR1I= X-Received: by 2002:a05:622a:206:b0:471:812b:508 with SMTP id d75a77b69052e-472334f363bmr3276441cf.14.1740284862672; Sat, 22 Feb 2025 20:27:42 -0800 (PST) MIME-Version: 1.0 References: <20250214-slub-percpu-caches-v2-0-88592ee0966a@suse.cz> <20250214-slub-percpu-caches-v2-10-88592ee0966a@suse.cz> In-Reply-To: <20250214-slub-percpu-caches-v2-10-88592ee0966a@suse.cz> From: Suren Baghdasaryan Date: Sat, 22 Feb 2025 20:27:30 -0800 X-Gm-Features: AWEUYZl6dIhm_eWSWjzkMiAN7GpBCH5Emzl1y5_M1yS7d4skO0MoR1hBC6n3Cog Message-ID: Subject: Re: [PATCH RFC v2 10/10] maple_tree: use percpu sheaves for maple_node_cache To: Vlastimil Babka Cc: "Liam R. Howlett" , Christoph Lameter , David Rientjes , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Uladzislau Rezki , linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, maple-tree@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 8saeu9sxo4n6cax58rs4cdgd8h9kxjxj X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DD8D11C0003 X-Rspam-User: X-HE-Tag: 1740284863-837837 X-HE-Meta: U2FsdGVkX18y842XQ0uJ/57GBEcShIYKQEbuBKZP5jSZRsW6KyZ7dQR1g5cYFBElCTUDN6WqKHCwnA2yujIIP+pyva1IhfJh17G4DMCUut5Va8bSw3/+XRwlqRt+W5PstB69PMspcrXZF3OP/a3aVzS/M5t4u3AsWFAKj94IDgXUW/fwi0HFuS6A99f8jw040umZQmboMxwVukV1OmNAWKa4LI/7Bfe7iD1lp0PkgkK7HeG7w26s85R95/j37PodprfrK5lpKeo89g9tnMscGLljRcsET7+UyYAhqajEyqiyETCX2fJAn83AfLHOIAwXniyrkEg9qXy0GoNrStx8V8Y0MzxOysrVemf9pzl3pNOv0C4vh+lUQ8HaYjS9qvkrjlMAOvx+ABUBTVnBIkMDZfs5cyWs7EFwEuEp3k11ivm85+HXbazVISqnILJ+Qr3336PtUBQ0xlaTbxdZK2eim/3IrwNUY/V61FD6Jz4168KOedyAsLTCwCp0T2kdfA7NxkCsdajCABA85anvM9E7MSDL75lxeXoM04aAkQgBEYdy3zE4OI0BgQrbioL8kHHcXZPxIjKPXU4joXhMvExFed8IcdukzXhN0n1HQuKdk+d9KUtc4x7utDkllOmJjwqsI/uDtB+v9n7B5u7dRBIVwoQTv1SYGl5FpgSE3aoX8ofcV0N0zoOcPV3hRPV1GffB+x4fhTfestP2nP/t/pzbSytKe/HqMHzRTGCjoY2RFdVeajgVRkgRfr1UwAE9+sR6lqsctGBsKjnz4PY9E3xFYiepQ3en1Di47aPK/qrcQF+v8ECJ/5kH100AfzdcsR772r1X6sqvDZBmQVH0dLyW5biM+2VpU4ZsboL/G5c+9JcHFW8pbWZJBEQELZ3W3iVLPqJomJVImkUhXZYNSPq2oKqc3hP7HIdO/X12Fj6aEP4x5cGMmB6eV3BeMlb+jQ3e13pKo+IJdYvo3M1ji/g 8crcADR1 e/US9k2xUqhEu62hJ8g+P8qdqY+t9354Wz47iH6O7BKlXtTvV5D3X4MvT9PmPxKaCcLS2MqOtrc3xgsj/n42CV3xgxRi/ODGtwMtmUmopZVUQ4FbyhpJGibwKUrMPvNhuWjaKIHOiQD5uoOfm06+dUCrECssTNZMYqM08kQqbe3SyG6soSZDKgXF140hQJ/GYJEIDvmyOxqw/7dE3GFIJkx2ASb3wlpjYrk5DTg8MnorRwwWtO1faNaJH7Gm9BdJ6ryq7FWNTTAS5+X6NBMSlEB6mXrE4N5kwph+Qa/nkY0j/Tfwlv7PADR2pfSISLv0WQnEIkKRPvjuySZezn9FZyStwZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Feb 14, 2025 at 8:28=E2=80=AFAM Vlastimil Babka wr= ote: > > Setup the maple_node_cache with percpu sheaves of size 32 to hopefully > improve its performance. I guess 32 might change in the future based on further testing? > Change the single node rcu freeing in > ma_free_rcu() to use kfree_rcu() instead of the custom callback, which > allows the rcu_free sheaf batching to be used. Note there are other > users of mt_free_rcu() where larger parts of maple tree are submitted to > call_rcu() as a whole, and that cannot use the rcu_free sheaf, but it's > still possible for maple nodes freed this way to be reused via the barn, > even if only some cpus are allowed to process rcu callbacks. > > Signed-off-by: Vlastimil Babka Reviewed-by: Suren Baghdasaryan > --- > lib/maple_tree.c | 11 ++++++++--- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/lib/maple_tree.c b/lib/maple_tree.c > index f7153ade1be5f16423f0ca073846a7f3dfa60523..56e7a00f6f0941bff163091c9= 99a873e4273f071 100644 > --- a/lib/maple_tree.c > +++ b/lib/maple_tree.c > @@ -208,7 +208,7 @@ static void mt_free_rcu(struct rcu_head *head) > static void ma_free_rcu(struct maple_node *node) > { > WARN_ON(node->parent !=3D ma_parent_ptr(node)); > - call_rcu(&node->rcu, mt_free_rcu); > + kfree_rcu(node, rcu); > } > > static void mas_set_height(struct ma_state *mas) > @@ -6258,9 +6258,14 @@ bool mas_nomem(struct ma_state *mas, gfp_t gfp) > > void __init maple_tree_init(void) > { > + struct kmem_cache_args args =3D { > + .align =3D sizeof(struct maple_node), > + .sheaf_capacity =3D 32, > + }; > + > maple_node_cache =3D kmem_cache_create("maple_node", > - sizeof(struct maple_node), sizeof(struct maple_no= de), > - SLAB_PANIC, NULL); > + sizeof(struct maple_node), &args, > + SLAB_PANIC); > } > > /** > > -- > 2.48.1 >