From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FBEEC48286 for ; Thu, 1 Feb 2024 22:11:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 368B46B0075; Thu, 1 Feb 2024 17:11:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F0E86B0078; Thu, 1 Feb 2024 17:11:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 192A36B007B; Thu, 1 Feb 2024 17:11:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 044B36B0075 for ; Thu, 1 Feb 2024 17:11:39 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BD2BBC0DFA for ; Thu, 1 Feb 2024 22:11:38 +0000 (UTC) X-FDA: 81744632676.05.49B52AD Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) by imf13.hostedemail.com (Postfix) with ESMTP id F1CD820019 for ; Thu, 1 Feb 2024 22:11:36 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b="m6/FeA+Q"; spf=none (imf13.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.167.172) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706825497; a=rsa-sha256; cv=none; b=jYK7iOK2YYORFRadQDI1jlknnlZxFfUlnRa/CmL28DGEglFID3fvxEzaoHW4ZrKmacXBcz nz9cNzPSCkiCpKR3qj8ROofArMJ0cQsz3RF9N1qSyzhYalhFunOQ5T3TQHVasQIaxPfdyz we4s1uMsIAlpczUgve62cDpjHbzdVgQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b="m6/FeA+Q"; spf=none (imf13.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.167.172) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706825497; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7CCoiRTZe5/e8MNjxiyM/onfHMZyM22m6CAEHOZonNo=; b=GcUqET4BClbpBeAQEMMb8R6u14R832mCoGEA98uO/b4W6w1TzYBXG5uZasV+HrlrgpiKC5 /33707NPhh7cl/XFP90IfqBdPQXViYSDYO6Z681kXRrYy4djQ995vGKJ/X2AyLGPrtlyKG 6Ezs16MaVQPrtAi6d/CwT2ve4WXmZUk= Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3bbbc6e51d0so956591b6e.3 for ; Thu, 01 Feb 2024 14:11:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1706825496; x=1707430296; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7CCoiRTZe5/e8MNjxiyM/onfHMZyM22m6CAEHOZonNo=; b=m6/FeA+Qfh6Lbp9XfVLpsWGMBTvDkAa5F8+Y6c6YIh8HUZdWsLZnuR3gP7bkkGHgTF EFlHK9NeZdUepDYhFRDkX0sJf9tJnWN9By838B6N40WwiiCtgXJDnu8EEbdDGsAafh8E 6F2hVH8hxQNyFEKqAVT+Ou6fTpikJS3s4FPkq3McHxAZos0IrxmKv1Gf+Xx036o6i7j8 4DUvvozHqjMmI+Cp8k48d2f4Pj6GEBZN9EGuSHo4N0rCAGv+/g7Mf7sZ17boORZErNwy /2ZiafEM4sNSpI5kYWzou5wBorvy1591+EROXDKbd9Fl7hP1F8/pZt9wt1JUeXcZ7erJ k7FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706825496; x=1707430296; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7CCoiRTZe5/e8MNjxiyM/onfHMZyM22m6CAEHOZonNo=; b=AjGhPjR8F/2Oh2lgpnEngySRTJ9yw5Xcs6PuT42DIlb2Sa8jRHhkBO2KDh93LELw7P N3mE3G18UEtZOIxyGkmXJ19sQ2yInuYfqRhTv4wtbnDPQu8k1tu4xH9EJZYqioHtLoPC woaeonl4RS9ItR8aoJO+aYv6srYWGdqgqc31UOWx4wALo227yHESKdno7m3zcedeUA46 IXlLdmYNGxQcHsQgqqtIWOgPkSk2dqhPFdwHCx2eR4y42gCXIwzSm12Oonk6aVpl+etk gs14b7ETW7D8uB061HcHnx9JBE7w2y7lcrghRzdxEkTIjyeEqnH9odgLrgjMrvJWfieQ joIg== X-Gm-Message-State: AOJu0YyJzSdu7SzF2o7vnnvd9zguI8Huns5S8+DjJ8hMHwt5yfNP4xz2 07MSqhAmalATAGPws6oWKVlpaQ5+M3w5vB+OmCzK9QmEQf0tnKo8D3udkJgSaOuINJ8YKv1V1Kn Z8sM/sKgTugPNGtL57X+/TAOWjzfsCxNRmWtPXQ== X-Google-Smtp-Source: AGHT+IHEBN97pDgrEcfxjhjtW1XJtpXigPPu4lYtbTbUci4Hva6kibJ+lbK2oPolooQmP44UMPBQalm+wnWgnFLvZqI= X-Received: by 2002:a05:6808:13ce:b0:3bf:ba1b:e9c7 with SMTP id d14-20020a05680813ce00b003bfba1be9c7mr2080294oiw.55.1706825496199; Thu, 01 Feb 2024 14:11:36 -0800 (PST) MIME-Version: 1.0 References: <20240201193014.2785570-1-tatashin@google.com> <02610629-05ef-4956-a122-36b6ac98fbc2@arm.com> <84c7e816-f749-48d8-a429-8b0ef799cdbb@arm.com> In-Reply-To: <84c7e816-f749-48d8-a429-8b0ef799cdbb@arm.com> From: Pasha Tatashin Date: Thu, 1 Feb 2024 17:10:59 -0500 Message-ID: Subject: Re: [PATCH] iommu/iova: use named kmem_cache for iova magazines To: Robin Murphy Cc: joro@8bytes.org, will@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: F1CD820019 X-Stat-Signature: n63wwoeyqm47rg7bsjso3ebuhbwpr3bx X-Rspam-User: X-HE-Tag: 1706825496-786204 X-HE-Meta: U2FsdGVkX183bgla2eqefz3cG73j3uL6cmltO0Fv1ecIgWLcTPweEtcrAc6S2I/g3QfB4H6p0jqYJ+mhZsAiPqSn+nnhTWtsMUWLx9e2lyF5AXIroIBqckykn9nRxGeElw7QThUfLLGrdp8mjPZSEKcQRZJPtyNbpcEB47n6BELY7wOYwUsWdrd3HJYwjCZqK0FJhjtguUTqrwPdBKtOmSRQ72eZRssjYZu0HGKxtU4OUoKma30l7JYnjbM3Z4smJERiQN9tiog+W9QYo+ZhAp0hTseMgzAkeVl6SN4Y7U4IJd7JILjDn08uednV9FI+cvEyEQJIiaNDw/flFlu+eJlDh/LeVkSJ2MAoc3w0e3Wy5YYYHOpLAV4DX6zfToFrU8EorwKbF20So2raB+jSz87TkN9WO9oson1y5DOMQrWZpHQMFThHO9T9HQgYMwckmEhDqW3wetN4gNadDZ7VLaq9nBWY+FxxDexxutIgGSUg92m06+LrWtBvnyEH74m+bEJ8mY35uM1F6eY/o6+ye0T/ltYQJPFrMH4xCFvVRgfUKdsvQWUMkbxdAXWX2m0V6yo+YjFaLmERBFeJu9a76a1l8IcjxuMfZwFPj1Oq5QjAqsFA7ghzohfmg9U7KFo7BO2x0JLYTd/Xj1LPNZepmP2wO66ZBmSFYgaT2Gg+dMtRNIFnCDQlzhZlAIloVJ9gdVGX3ZKn85F9zRj1k/YKxEMj2qZGLULNpcFV4iKruV9nk5H4PXC85o1PEetstNsFgax/QQWvAo57mAfEMc7M6EpEdkA3XzPC0K2nEY8vg0ZeD8u21PhTfuZbBHgtXIECVoFBJHY2Dq1xTlQzf9LyXcj7Q8DV0ilFDHLkz/b4kV/ew/5+Lq0xB2wPwB/gu45Rg3oMdU0rS0Rrop1HIWusuNA2Karjw2shawWosrfPosqeESILR4dQgF18wfWjz3YnFDOEW4JdTS9DAY7IGA0 +AQpWq+q 04CSV4RsNVhK3SpWQKDTpuudDSWmMspxw0DsnjqvkYikR7nEyEA1aADFE8XqmsQ/nJrFDxTXRgk6yb76RbJPrKd9sDWNtdB5j5uJTVwqQLqzChIuWt2aEuCMvjlSIy/QcC/DoFjRcfi9ma1MR1frmrGwWVGj7ZozXd7RL7zatUglziiF5y6Uaq3WDm5mbsW5stYJH9g8B9R9920GJceW/OSVG9APKWn6EM2nPmq+5IKztoshIu9p8Z2GD/MRYubfoxs8TYueEayrJNkwmYiAmr4zwbmnCVBZPNfZA3ZFa1BwJIMLML7FLSBmGfnRbAaBq4+Zm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 1, 2024 at 4:23=E2=80=AFPM Robin Murphy = wrote: > > On 2024-02-01 9:06 pm, Pasha Tatashin wrote: > > On Thu, Feb 1, 2024 at 3:56=E2=80=AFPM Robin Murphy wrote: > >> > >> On 2024-02-01 7:30 pm, Pasha Tatashin wrote: > >>> From: Pasha Tatashin > >>> > >>> The magazine buffers can take gigabytes of kmem memory, dominating al= l > >>> other allocations. For observability prurpose create named slab cache= so > >>> the iova magazine memory overhead can be clearly observed. > >>> > >>> With this change: > >>> > >>>> slabtop -o | head > >>> Active / Total Objects (% used) : 869731 / 952904 (91.3%) > >>> Active / Total Slabs (% used) : 103411 / 103974 (99.5%) > >>> Active / Total Caches (% used) : 135 / 211 (64.0%) > >>> Active / Total Size (% used) : 395389.68K / 411430.20K (96.1= %) > >>> Minimum / Average / Maximum Object : 0.02K / 0.43K / 8.00K > >>> > >>> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > >>> 244412 244239 99% 1.00K 61103 4 244412K iommu_iova_magaz= ine > >>> 91636 88343 96% 0.03K 739 124 2956K kmalloc-32 > >>> 75744 74844 98% 0.12K 2367 32 9468K kernfs_node_ca= che > >>> > >>> On this machine it is now clear that magazine use 242M of kmem memory= . > >> > >> Hmm, something smells there... > >> > >> In the "worst" case there should be a maximum of 6 * 2 * > >> num_online_cpus() empty magazines in the iova_cpu_rcache structures, > >> i.e., 12KB per CPU. Under normal use those will contain at least some > >> PFNs, but mainly every additional magazine stored in a depot is full > >> with 127 PFNs, and each one of those PFNs is backed by a 40-byte struc= t > >> iova, i.e. ~5KB per 1KB magazine. Unless that machine has many thousan= ds > >> of CPUs, if iova_magazine allocations are the top consumer of memory > >> then something's gone wrong. > > > > This is an upstream kernel + few drivers that is booted on AMD EPYC, > > with 128 CPUs. > > > > It has allocations stacks like these: > > init_iova_domain+0x1ed/0x230 iommu_setup_dma_ops+0xf8/0x4b0 > > amd_iommu_probe_finalize. > > And also init_iova_domain() calls for Google's TPU drivers 242M is > > actually not that much, compared to the size of the system. > > Hmm, I did misspeak slightly (it's late and I really should have left > this for tomorrow...) - that's 12KB per CPU *per domain*, but still that > would seem to imply well over 100 domains if you have 242MB of magazine > allocations while the iommu_iova cache isn't even on the charts... what > the heck is that driver doing? I am not sure what the driver is doing. However, I can check the actual allocation sizes for each init_iova_domain() and report on that later. > > (I don't necessarily disagree with the spirit of the patch BTW, I just > really want to understand the situation that prompted it, and make sure > we don't actually have a subtle leak somewhere.) Yes, the observability is needed here, because there were several optimizations that reduced the size of these magazines, and they still can be large. For example, for a while we had 1032-bytes per-magazine instead of 1024, this caused wasting almost half of magazine memroy with 2K slabs. This was fixed with: b4c9bf178ace iommu/iova: change IOVA_MAG_SIZE to 127 to save memory Also, earlier there was another optimization "32e92d9f6f87 iommu/iova: Separate out rcache init" that reduced cases when magazines need to be allocated. That also reduced overhead on our systems by a factor of 10. Yet, the magazines are still large, and I think it is time to improve observability for the future optimizations, and avoiding future regressions. Pasha