From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 760F5ECAAD3 for ; Thu, 1 Sep 2022 14:01:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E10198D0002; Thu, 1 Sep 2022 10:01:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DC0798D0001; Thu, 1 Sep 2022 10:01:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C612F8D0002; Thu, 1 Sep 2022 10:01:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B7CDE8D0001 for ; Thu, 1 Sep 2022 10:01:25 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8F82B120F71 for ; Thu, 1 Sep 2022 14:01:25 +0000 (UTC) X-FDA: 79863678930.01.882CC1F Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf28.hostedemail.com (Postfix) with ESMTP id B338EC0076 for ; Thu, 1 Sep 2022 14:01:22 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id l65so7197230pfl.8 for ; Thu, 01 Sep 2022 07:01:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date; bh=YDOhtt6J+ZY/qg8xzbLdvC6nQ5IzedlQjVCL0rUnymA=; b=ZF3KJ3dJjxVQXTdvX7s5aMb1MKcjiyoCnPrYPQcReHISIExI4Skt+47KA5MoNm4ueM VOveaVDCLb8YjKi09ZcPKXoVrvsoVeCQxEpwP7875v5xmaAxf5egcyh9SWEbbbjl7u5j OPhXCGtSl+ZTh9i5mYR9k3cHqJg6tMVt2TqKfLeVKGCzfMYgOxBdGQbWrxJ2GSFY9klx MgEtFDakufk+aTo65k2+/wGWkv1FLVHqm/qgDkfdb9Xt0KRTTsUcvsdiDm+SbFou0vr8 WNCf5HZAfCQE6eebVZgPFOaQRGRkNZc+Qc5EbzqbK8g/T9Z1e4pT9bNj/0Nh5yUjVqog 17ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date; bh=YDOhtt6J+ZY/qg8xzbLdvC6nQ5IzedlQjVCL0rUnymA=; b=5uhPVvMiwzFtYItOlzKvUz6bZWQS2hTe+lU4Dr+JxKJlAOLbIyE7IhUFNy+ZbLCayy nXd3pucc5gjwutKqTv1LyZ558xKDrONTqvwtqvwrk7lQOxLfVBD4sxoRsmNNIT/2avoV HY2qYg+NZ80PTTSYbKJ2Ajx5d5pa22EOowLpVNzLZKYLgnLUTbFUt+YQCtHKXKHdXlhb iGKsgTufHc2Vh0D7Sof+8wfJ1kM7bGEa67ZytU07od+ier02Uvrqc9MQSU9UhYwzf9pK lPbDjr5UoyzwlYRmgiFEK6a3pedo7EqVqMaGE2BS4ksPIbp0XcyGq02Ww/rvZGuds3Ot hOJA== X-Gm-Message-State: ACgBeo2qG8RXVBYHreItjBUzezVa8ZCsbECWuHsKn4MShT7An0WwPmQO posPA4CpjSn0WvnBzNmx6hc= X-Google-Smtp-Source: AA6agR5QYUk1/ouGe9PG+TcT6iWHvUWeDxUjLXZmVJCjFtZySONTbveX8s3S00IDBKelQ5H+oDD3jg== X-Received: by 2002:a62:a512:0:b0:536:e2bd:e15e with SMTP id v18-20020a62a512000000b00536e2bde15emr32087656pfm.1.1662040881472; Thu, 01 Sep 2022 07:01:21 -0700 (PDT) Received: from hyeyoo ([114.29.91.56]) by smtp.gmail.com with ESMTPSA id t7-20020a17090340c700b0017312bfca95sm13629110pld.253.2022.09.01.07.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Sep 2022 07:01:20 -0700 (PDT) Date: Thu, 1 Sep 2022 23:01:13 +0900 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Feng Tang Cc: Andrew Morton , Vlastimil Babka , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Dmitry Vyukov , Dave Hansen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Robin Murphy , John Garry , Kefeng Wang Subject: Re: [PATCH v4 1/4] mm/slub: enable debugging memory wasting of kmalloc Message-ID: References: <20220829075618.69069-1-feng.tang@intel.com> <20220829075618.69069-2-feng.tang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220829075618.69069-2-feng.tang@intel.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662040882; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YDOhtt6J+ZY/qg8xzbLdvC6nQ5IzedlQjVCL0rUnymA=; b=YL0mdjyb6G29KVVS1kZVGhwryaL7AUE8tlkmK6kdvXbLFa8smyolDNzZtQvdzEUZcWV0XD RYq+l+loXk4Gc2BSfJLCuX2xfWapRIroJ6NWRm6X+HigkWhNe3KyJIciSzY5YWvnvLbE+e O0FFFmuH4YGPGbR/PxHcGjp1kH/7vkI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ZF3KJ3dJ; spf=pass (imf28.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662040882; a=rsa-sha256; cv=none; b=FG2U9mhBYQf5RhxR9Kv2y25vD1aPyKLMyIm2fPPpenCGAtWQ8Sm7IYm1SUUJLVl5783Q2p q1TtPFw1h6lMIXbvOmYrRfcGBL255/a78BaiutH+FA/qaYv1op3SoC+GouFiZs+qM9w2hq lkGQ6jcoTYNk8BDTxFBHwFOqh88TSCc= Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ZF3KJ3dJ; spf=pass (imf28.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: 9h4caotoz6wkbqueotdat6uiq1p8cwc5 X-Rspamd-Queue-Id: B338EC0076 X-Rspamd-Server: rspam05 X-HE-Tag: 1662040882-101549 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Aug 29, 2022 at 03:56:15PM +0800, Feng Tang wrote: > kmalloc's API family is critical for mm, with one nature that it will > round up the request size to a fixed one (mostly power of 2). Say > when user requests memory for '2^n + 1' bytes, actually 2^(n+1) bytes > could be allocated, so in worst case, there is around 50% memory > space waste. > > The wastage is not a big issue for requests that get allocated/freed > quickly, but may cause problems with objects that have longer life > time. > > We've met a kernel boot OOM panic (v5.10), and from the dumped slab > info: > > [ 26.062145] kmalloc-2k 814056KB 814056KB > > From debug we found there are huge number of 'struct iova_magazine', > whose size is 1032 bytes (1024 + 8), so each allocation will waste > 1016 bytes. Though the issue was solved by giving the right (bigger) > size of RAM, it is still nice to optimize the size (either use a > kmalloc friendly size or create a dedicated slab for it). > > And from lkml archive, there was another crash kernel OOM case [1] > back in 2019, which seems to be related with the similar slab waste > situation, as the log is similar: > > [ 4.332648] iommu: Adding device 0000:20:02.0 to group 16 > [ 4.338946] swapper/0 invoked oom-killer: gfp_mask=0x6040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null), order=0, oom_score_adj=0 > ... > [ 4.857565] kmalloc-2048 59164KB 59164KB > > The crash kernel only has 256M memory, and 59M is pretty big here. > (Note: the related code has been changed and optimised in recent > kernel [2], these logs are just picked to demo the problem, also > a patch changing its size to 1024 bytes has been merged) > > So add an way to track each kmalloc's memory waste info, and > leverage the existing SLUB debug framework (specifically > SLUB_STORE_USER) to show its call stack of original allocation, > so that user can evaluate the waste situation, identify some hot > spots and optimize accordingly, for a better utilization of memory. > > The waste info is integrated into existing interface: > '/sys/kernel/debug/slab/kmalloc-xx/alloc_traces', one example of > 'kmalloc-4k' after boot is: > > 126 ixgbe_alloc_q_vector+0xa5/0x4a0 [ixgbe] waste=233856/1856 age=1493302/1493830/1494358 pid=1284 cpus=32 nodes=1 > __slab_alloc.isra.86+0x52/0x80 > __kmalloc_node+0x143/0x350 > ixgbe_alloc_q_vector+0xa5/0x4a0 [ixgbe] > ixgbe_init_interrupt_scheme+0x1a6/0x730 [ixgbe] > ixgbe_probe+0xc8e/0x10d0 [ixgbe] > local_pci_probe+0x42/0x80 > work_for_cpu_fn+0x13/0x20 > process_one_work+0x1c5/0x390 > > which means in 'kmalloc-4k' slab, there are 126 requests of > 2240 bytes which got a 4KB space (wasting 1856 bytes each > and 233856 bytes in total). And when system starts some real > workload like multiple docker instances, there are more > severe waste. > > [1]. https://lkml.org/lkml/2019/8/12/266 > [2]. https://lore.kernel.org/lkml/2920df89-9975-5785-f79b-257d3052dfaf@huawei.com/ > > [Thanks Hyeonggon for pointing out several bugs about sorting/format] > [Thanks Vlastimil for suggesting way to reduce memory usage of > orig_size and keep it only for kmalloc objects] > > Signed-off-by: Feng Tang > Cc: Robin Murphy > Cc: John Garry > Cc: Kefeng Wang > --- > include/linux/slab.h | 2 + > mm/slub.c | 94 +++++++++++++++++++++++++++++++++++++------- > 2 files changed, 81 insertions(+), 15 deletions(-) Would you update Documentation/mm/slub.rst as well? (alloc_traces part) [...] > */ > static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, > - unsigned long addr, struct kmem_cache_cpu *c) > + unsigned long addr, struct kmem_cache_cpu *c, unsigned int orig_size) > { > void *freelist; > struct slab *slab; > @@ -3115,6 +3158,7 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, > > if (s->flags & SLAB_STORE_USER) > set_track(s, freelist, TRACK_ALLOC, addr); > + set_orig_size(s, freelist, orig_size); > > return freelist; > } > @@ -3140,6 +3184,8 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, > */ > if (s->flags & SLAB_STORE_USER) > set_track(s, freelist, TRACK_ALLOC, addr); > + set_orig_size(s, freelist, orig_size); > + > return freelist; > } This patch is okay but with patch 4, init_object() initializes redzone/poison area using s->object_size, and init_kmalloc_object() fixes redzone/poison area using orig_size. Why not do it in init_object() in the first time? Also, updating redzone/poison area after alloc_single_from_new_slab() (outside list_lock, after adding slab to list) will introduce races with validation. So I think doing set_orig_size()/init_kmalloc_object() in alloc_debug_processing() would make more sense. I can miss something, please kindly let me know if I did ;) Anything else looks good to me. Thanks! -- Thanks, Hyeonggon