From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A8C2C48286 for ; Thu, 1 Feb 2024 20:56:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA8506B0075; Thu, 1 Feb 2024 15:56:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A584F6B0078; Thu, 1 Feb 2024 15:56:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9488A6B007B; Thu, 1 Feb 2024 15:56:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 822A56B0075 for ; Thu, 1 Feb 2024 15:56:10 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5853F1A0680 for ; Thu, 1 Feb 2024 20:56:10 +0000 (UTC) X-FDA: 81744442500.26.E16CB95 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf30.hostedemail.com (Postfix) with ESMTP id 50C5D80006 for ; Thu, 1 Feb 2024 20:56:08 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706820968; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jsiClJd6YPLVvaVvK8olY8geIlgRYvDTsZoWWqoKjpg=; b=sx1zLwk4b1mAxjRMZbrVSqHxVVXz2CdwH5LD9gHW7EDoA6rHZAmQ1h4kL7yovdGfhchW+0 +XtnxuqmMQ/YzQSuYCTBzXkYziioQnn18xxZtvAdKnlVspEQYW/cjjhmNnNvs38cGbsT86 4+rcabLQxoCGqzYT9ATwg3LgxSmki3I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706820968; a=rsa-sha256; cv=none; b=M5ZZIfmXIVngD8qNYz/x2hj53lOcpo7tdLhVtRIkZVjKY8ATI3MIeqWSFKAS/RuzW+QbSK MndUX6VAFrogeHast9bxBplIcWt8gvWngU/J6OlFqHMeGR3Dxis6ldzmlx4IQZL6boGwKV y1ZLB+BdB/IalclcjX5xvb7It+I2rmQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C4886DA7; Thu, 1 Feb 2024 12:56:49 -0800 (PST) Received: from [10.57.47.101] (unknown [10.57.47.101]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AF6AB3F762; Thu, 1 Feb 2024 12:56:05 -0800 (PST) Message-ID: <02610629-05ef-4956-a122-36b6ac98fbc2@arm.com> Date: Thu, 1 Feb 2024 20:56:04 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] iommu/iova: use named kmem_cache for iova magazines Content-Language: en-GB To: Pasha Tatashin , joro@8bytes.org, will@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com References: <20240201193014.2785570-1-tatashin@google.com> From: Robin Murphy In-Reply-To: <20240201193014.2785570-1-tatashin@google.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 50C5D80006 X-Rspam-User: X-Stat-Signature: 9pozrwx7me3km1x9t7198ezkagqpzbkc X-Rspamd-Server: rspam03 X-HE-Tag: 1706820968-424726 X-HE-Meta: U2FsdGVkX1/yNVt13IDgLnMC/ZU5/E+lSog0XmUMLgtWJYsDn5WOpkjEntA9sa7AX9RdxAKVoFiYqv35kyF808gelxSX5mSRtfxB7qWlOrCTT+vYK3sYBmqzD3W40eiG7t+XsFeOYzb7bd/W84Um7969oF9rZaeo5wKmaCAtbjX/l14mLbgSSWS1E8QlUs9OJ3dLGUN5qq/3S5i393t/r0Dn0v9vVvuo/JqvRzLoW4Nl7pZFY2HRlj1ZUnH9UwjrDmrf+eeKa7r3jJYezpNHR7YhPz+CH+oHju6A9R0Kxp+xKFFfGavNOrjWADHWvdRAJ+N9vlaHZg4mkppWfFV9uNboCnZcJHWtmFWyxpub56L3V3HPIw9xlWsK7GHVAp2UMMFM3Yguzj0uSMPHb9UVUBPfEK1TNlQS56nL0goAKFZC8uCc5CHJFxOSAslKyYCOuobmz99v3m38QvG1YsUtFMnmZ5KPRYuKKaviLgpMVtZgvSAmJJ12IkyUVz1KJa9H93pP2pg3rDUOTLbhWYMSCdwZB6+BYqs8uPmIvTPeEo/uHwaD3m8HGOFptCV49MiOz59Ds31M5OvJeTdxD/GLCcckPn/QZejFm3UY9wPrfOucmw+ZPrDlKC3zBcnvlcXtK/kBLpogm1QjuTIN0dNwLc4Hp7hoZN8QwAazUkr4g85A3ub+njl9kZLhXnZteREUPFMe9Pqw1iwENFJoi2U/7kgZM6lzZFyDEH7gWli3SY1y79/bTUtg50Cl/xzUMA1+S7ya11lXjLN+9UEkFqIGca+71HhVFQJsuazlezkPM5TbVpW1QPE03wfTMav3Bz8x1GGMOokpAZd23EGXE8B2zR71yU9i2/n9JSh5cUIJEKG6TXgcP+dt98w8OlaFj6B3vcTv/+Sl+imz+jFt/I9uLMIE5gZSShGSTZ/rg/NyRKacNF/8Mgu/av9w96JNG1VXyNdfcS7mPHR4TDX7Twu AiFc/bIn QpX1phk4ulYRGNSBd+gCfHDxF2znWrbnAmJob3drb8TkYKjru5UuqpBIwcuAfnAH6mv3mH7FFARZktn9684XOFa+xmMHWBTjl8dv/7QZN/+TfNAiT1rseJjUdQ1MFMstrmSkqkNRksnI9DXatmsfbRC0Y8fyHX4p8ppYxoQEsEQM+5WXo15COZ7B0TfMmJ1HTRD8SIwG5ByEDHR2EdjvnsZq4tU8uwv7Fej7y0wbuT/8iGbFa3cYADmyAC8qWk7oVG03O6vViVQQhCp42cCGlIcPyPy9AaasWnfcG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024-02-01 7:30 pm, Pasha Tatashin wrote: > From: Pasha Tatashin > > The magazine buffers can take gigabytes of kmem memory, dominating all > other allocations. For observability prurpose create named slab cache so > the iova magazine memory overhead can be clearly observed. > > With this change: > >> slabtop -o | head > Active / Total Objects (% used) : 869731 / 952904 (91.3%) > Active / Total Slabs (% used) : 103411 / 103974 (99.5%) > Active / Total Caches (% used) : 135 / 211 (64.0%) > Active / Total Size (% used) : 395389.68K / 411430.20K (96.1%) > Minimum / Average / Maximum Object : 0.02K / 0.43K / 8.00K > > OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > 244412 244239 99% 1.00K 61103 4 244412K iommu_iova_magazine > 91636 88343 96% 0.03K 739 124 2956K kmalloc-32 > 75744 74844 98% 0.12K 2367 32 9468K kernfs_node_cache > > On this machine it is now clear that magazine use 242M of kmem memory. Hmm, something smells there... In the "worst" case there should be a maximum of 6 * 2 * num_online_cpus() empty magazines in the iova_cpu_rcache structures, i.e., 12KB per CPU. Under normal use those will contain at least some PFNs, but mainly every additional magazine stored in a depot is full with 127 PFNs, and each one of those PFNs is backed by a 40-byte struct iova, i.e. ~5KB per 1KB magazine. Unless that machine has many thousands of CPUs, if iova_magazine allocations are the top consumer of memory then something's gone wrong. Thanks, Robin. > Signed-off-by: Pasha Tatashin > --- > drivers/iommu/iova.c | 57 +++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 54 insertions(+), 3 deletions(-) > > diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c > index d30e453d0fb4..617bbc2b79f5 100644 > --- a/drivers/iommu/iova.c > +++ b/drivers/iommu/iova.c > @@ -630,6 +630,10 @@ EXPORT_SYMBOL_GPL(reserve_iova); > > #define IOVA_DEPOT_DELAY msecs_to_jiffies(100) > > +static struct kmem_cache *iova_magazine_cache; > +static unsigned int iova_magazine_cache_users; > +static DEFINE_MUTEX(iova_magazine_cache_mutex); > + > struct iova_magazine { > union { > unsigned long size; > @@ -654,11 +658,51 @@ struct iova_rcache { > struct delayed_work work; > }; > > +static int iova_magazine_cache_init(void) > +{ > + int ret = 0; > + > + mutex_lock(&iova_magazine_cache_mutex); > + > + iova_magazine_cache_users++; > + if (iova_magazine_cache_users > 1) > + goto out_unlock; > + > + iova_magazine_cache = kmem_cache_create("iommu_iova_magazine", > + sizeof(struct iova_magazine), > + 0, SLAB_HWCACHE_ALIGN, NULL); > + > + if (!iova_magazine_cache) { > + pr_err("Couldn't create iova magazine cache\n"); > + ret = -ENOMEM; > + } > + > +out_unlock: > + mutex_unlock(&iova_magazine_cache_mutex); > + > + return ret; > +} > + > +static void iova_magazine_cache_fini(void) > +{ > + mutex_lock(&iova_magazine_cache_mutex); > + > + if (WARN_ON(!iova_magazine_cache_users)) > + goto out_unlock; > + > + iova_magazine_cache_users--; > + if (!iova_magazine_cache_users) > + kmem_cache_destroy(iova_magazine_cache); > + > +out_unlock: > + mutex_unlock(&iova_magazine_cache_mutex); > +} > + > static struct iova_magazine *iova_magazine_alloc(gfp_t flags) > { > struct iova_magazine *mag; > > - mag = kmalloc(sizeof(*mag), flags); > + mag = kmem_cache_alloc(iova_magazine_cache, flags); > if (mag) > mag->size = 0; > > @@ -667,7 +711,7 @@ static struct iova_magazine *iova_magazine_alloc(gfp_t flags) > > static void iova_magazine_free(struct iova_magazine *mag) > { > - kfree(mag); > + kmem_cache_free(iova_magazine_cache, mag); > } > > static void > @@ -766,11 +810,17 @@ int iova_domain_init_rcaches(struct iova_domain *iovad) > unsigned int cpu; > int i, ret; > > + ret = iova_magazine_cache_init(); > + if (ret) > + return -ENOMEM; > + > iovad->rcaches = kcalloc(IOVA_RANGE_CACHE_MAX_SIZE, > sizeof(struct iova_rcache), > GFP_KERNEL); > - if (!iovad->rcaches) > + if (!iovad->rcaches) { > + iova_magazine_cache_fini(); > return -ENOMEM; > + } > > for (i = 0; i < IOVA_RANGE_CACHE_MAX_SIZE; ++i) { > struct iova_cpu_rcache *cpu_rcache; > @@ -948,6 +998,7 @@ static void free_iova_rcaches(struct iova_domain *iovad) > > kfree(iovad->rcaches); > iovad->rcaches = NULL; > + iova_magazine_cache_fini(); > } > > /*