From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77686C433EF for ; Thu, 30 Jun 2022 09:02:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB2B16B0072; Thu, 30 Jun 2022 05:02:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E617D8E0002; Thu, 30 Jun 2022 05:02:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2A738E0001; Thu, 30 Jun 2022 05:02:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C3A166B0072 for ; Thu, 30 Jun 2022 05:02:08 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 9728612123C for ; Thu, 30 Jun 2022 09:02:08 +0000 (UTC) X-FDA: 79634310336.22.F5B3FED Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 1692E10004A for ; Thu, 30 Jun 2022 09:02:07 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 92B291063; Thu, 30 Jun 2022 02:02:07 -0700 (PDT) Received: from [10.57.85.25] (unknown [10.57.85.25]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C38B93F5A1; Thu, 30 Jun 2022 02:02:05 -0700 (PDT) Message-ID: <13db50bb-57c7-0d54-3857-84b8a4591d9e@arm.com> Date: Thu, 30 Jun 2022 10:02:00 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [PATCH] iommu/iova: change IOVA_MAG_SIZE to 127 to save memory Content-Language: en-GB To: Feng Tang , Joerg Roedel , Will Deacon , iommu@lists.linux-foundation.org, iommu@lists.linux.dev, Andrew Morton , Christoph Lameter , Vlastimil Babka Cc: linux-mm@kvack.org, Paul Menzel , linux-kernel@vger.kernel.org References: <20220630073304.26945-1-feng.tang@intel.com> From: Robin Murphy In-Reply-To: <20220630073304.26945-1-feng.tang@intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656579728; a=rsa-sha256; cv=none; b=uxVaJajNVTuVefAq7K3DZAPNFTZJaUZFkh/unNBYglgLNyKw7S8sR5nOj3IUQ84MyougF0 j59Oa6GNlaAAcVyTpy2YiCNq2M/HG7yrwo0Tr5qlYQCwfkwYbhhsMXNB24LFQTrGBlO3kj Np6q4+Rglgpqt75YfqBNUQM+D0K+9Iw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656579728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lWJfGXIJO/6riYZY6WXj48E2VA79XEb1mGLj4eiFtz0=; b=Ixdo9bG+09E3VrT0PVVio7QHqFFoyM/O0vFnmhEWr9LFoz0rKPrSnYHw5Dkz8m66+UsWkr LCHIjoyuSC1b4HibbOoF2VkI7YYb3VKo6KmveI/wqUYfsXwAo8cBkW2oqsSOrJO7kiyJO3 4B3eLmEEjy5XGtbAIREoYa2Jlt0OFkU= X-Rspam-User: Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 1692E10004A X-Stat-Signature: 46q5jxz8nfop9s7puw6a7to8eufg5eoq X-HE-Tag: 1656579727-303270 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022-06-30 08:33, Feng Tang wrote: > kmalloc will round up the request size to power of 2, and current > iova_magazine's size is 1032 (1024+8) bytes, so each instance > allocated will get 2048 bytes from kmalloc, causing around 1KB > waste. > > And in some exstreme case, the memory wasted can trigger OOM as > reported in 2019 on a crash kernel with 256 MB memory [1]. I don't think it really needs pointing out that excessive memory consumption can cause OOM. Especially not in the particularly silly context of a system with only 2MB of RAM per CPU - that's pretty much guaranteed to be doomed one way or another. > [ 4.319253] iommu: Adding device 0000:06:00.2 to group 5 > [ 4.325869] iommu: Adding device 0000:20:01.0 to group 15 > [ 4.332648] iommu: Adding device 0000:20:02.0 to group 16 > [ 4.338946] swapper/0 invoked oom-killer: gfp_mask=0x6040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null), order=0, oom_score_adj=0 > [ 4.350251] swapper/0 cpuset=/ mems_allowed=0 > [ 4.354618] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.57.mx64.282 #1 > [ 4.355612] Hardware name: Dell Inc. PowerEdge R7425/08V001, BIOS 1.9.3 06/25/2019 > [ 4.355612] Call Trace: > [ 4.355612] dump_stack+0x46/0x5b > [ 4.355612] dump_header+0x6b/0x289 > [ 4.355612] out_of_memory+0x470/0x4c0 > [ 4.355612] __alloc_pages_nodemask+0x970/0x1030 > [ 4.355612] cache_grow_begin+0x7d/0x520 > [ 4.355612] fallback_alloc+0x148/0x200 > [ 4.355612] kmem_cache_alloc_trace+0xac/0x1f0 > [ 4.355612] init_iova_domain+0x112/0x170 > [ 4.355612] amd_iommu_domain_alloc+0x138/0x1a0 > [ 4.355612] iommu_group_get_for_dev+0xc4/0x1a0 > [ 4.355612] amd_iommu_add_device+0x13a/0x610 > [ 4.355612] add_iommu_group+0x20/0x30 > [ 4.355612] bus_for_each_dev+0x76/0xc0 > [ 4.355612] bus_set_iommu+0xb6/0xf0 > [ 4.355612] amd_iommu_init_api+0x112/0x132 > [ 4.355612] state_next+0xfb1/0x1165 > [ 4.355612] amd_iommu_init+0x1f/0x67 > [ 4.355612] pci_iommu_init+0x16/0x3f > ... > [ 4.670295] Unreclaimable slab info: > ... > [ 4.857565] kmalloc-2048 59164KB 59164KB > > Change IOVA_MAG_SIZE from 128 to 127 to make size of 'iova_magazine' > 1024 bytes so that no memory will be wasted. > > [1]. https://lkml.org/lkml/2019/8/12/266 > > Signed-off-by: Feng Tang > --- > drivers/iommu/iova.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c > index db77aa675145b..27634ddd9b904 100644 > --- a/drivers/iommu/iova.c > +++ b/drivers/iommu/iova.c > @@ -614,7 +614,12 @@ EXPORT_SYMBOL_GPL(reserve_iova); > * dynamic size tuning described in the paper. > */ > > -#define IOVA_MAG_SIZE 128 > +/* > + * As kmalloc's buffer size is fixed to power of 2, 127 is chosen to > + * assure size of 'iova_magzine' to be 1024 bytes, so that no memory Typo: iova_magazine > + * will be wasted. > + */ > +#define IOVA_MAG_SIZE 127 The change itself seems perfectly reasonable, though. Acked-by: Robin Murphy > #define MAX_GLOBAL_MAGS 32 /* magazines per bin */ > > struct iova_magazine {