From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E1EDC4829D for ; Tue, 13 Feb 2024 00:15:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B2926B0075; Mon, 12 Feb 2024 19:15:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6642A6B007B; Mon, 12 Feb 2024 19:15:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52B076B007D; Mon, 12 Feb 2024 19:15:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 413196B0075 for ; Mon, 12 Feb 2024 19:15:01 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0060EC01CC for ; Tue, 13 Feb 2024 00:15:00 +0000 (UTC) X-FDA: 81784860360.24.5A1735A Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by imf05.hostedemail.com (Postfix) with ESMTP id 2E9C0100013 for ; Tue, 13 Feb 2024 00:14:58 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b=iQjkk7AX; dmarc=none; spf=none (imf05.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.210.54) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707783299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zpkbch4s+SL6zR3ce+cP/IOGuaScUrbATNvaMsX6gQU=; b=gGQjU+AZEYprw+W4HXFgEXIKpRjwUezSEkWGwwkCOC2T212cq1RYsZgmBk4HOugpw+qnGc 6xU+0wslOwHaNHDrV6EQgwrqCnaJO2kmTP9YZiEXl7glZz3cdGxF+EKaMAtEvsCF98FyJY PhRY8K8SOuwvgR7mS1JFtgzyp9vJM8Y= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b=iQjkk7AX; dmarc=none; spf=none (imf05.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.210.54) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707783299; a=rsa-sha256; cv=none; b=pb36jhRcH+fzsvFBPt1WNAcMBbmuoiDhUZ3F8lz36UJedpHqQ5fTgj3ughRxv6CC99COI0 uWJD2b8Eq3MrIy/lbeo1SK4i2RGRqnwGaSr8EDB4wopdRJoeUGbpmO+uue3HxCza68Llw1 b2Cg+cPpvKc0elinoDOD91MmzD/C7l8= Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-6e0f43074edso1809074a34.1 for ; Mon, 12 Feb 2024 16:14:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1707783298; x=1708388098; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Zpkbch4s+SL6zR3ce+cP/IOGuaScUrbATNvaMsX6gQU=; b=iQjkk7AXt/yahOJHdgZSqHsx/crmsIeVWwwhcn3dlsLhQFqWsgil2XtjplIlJcbHml heDXGbqcGgKNhqEiZDjdngdCsRTLhZh+btqawagDsLgj73ro0mwmGYhkzKnXU8mYyhwE eau9ZTzztYbBabGS6PKXvYt1SU3yP0Unw+APUbKX8zwt8NR7cU7E618w+DR6XMFSxCdk jEsBnScCUHG9QqVaylAc2VDpJywE8xHfj4FMJT0PfbeQJM5D4tKlW+xUO1Ty3SzFGwfr K6Xd2qk8/gw/BxORZnFGXmcRPv2QLIMqjBDlgK0R3syZRzbVg3j8O2iwzGhYDF28zi/n xfKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707783298; x=1708388098; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zpkbch4s+SL6zR3ce+cP/IOGuaScUrbATNvaMsX6gQU=; b=FuCTAixdiAllxCNNy+gctUgUm0RGyRsjUKonVHdDtCb8T2OZ42FwnfXHi90pAi48py yTAqxY9zFs33Ck3Tkh1t6t5kEEVnAvW9PZe9t/hgs66emqvGrGWGBLXqJ/B3bLeehKSS GGK4YDp+KwlCz+W97mkimw6wlvwO+JiM4wSIjm8kd10wf1LgqwFyGJG0dqzJ6WlFbD6p iP8WfsC4n9bXzfjrL7onwaivXyqlWj+FsEDf+uzf/oI0vMA3KdLR2OaMEGaebJ8AUjI6 h+WoH00Nu/yG6JeYq8B4JJi9GdmNp4VgrCe4VJe3p1rto5KMuGUunbQDvU+UattutL/K 5qmw== X-Forwarded-Encrypted: i=1; AJvYcCV7Bi+K/BIlEIzOfAiSqiTzo35TCGi2R6v8a6pLfWyl/5x5wcuMd1yLFEoRiQ6aALylY0/J5+iGsdmPFev7qnAAZR8= X-Gm-Message-State: AOJu0YwasQGnMK37lC+rvDWOtqWFu5G+850sTQSoBF7mux24qoRWp3Cd Ao/fQ8ChKf8niaklIUFGMpsCOHf3XRclukxhx8MJ7YDVWEt/9mjj0IBlhd7SA3Rro844cRoOWnC xo6mvrTYEUiBYt5MhUYiMctnYZSAY4NnDtb2nHQ== X-Google-Smtp-Source: AGHT+IENA2I2vQZTmviXZDRj9zPa9m0qm240EbRNcQEFbOk5GS9Qd9oHISJT+KIV/8e+A1RHV9ji5tIu3nuHQqetP/o= X-Received: by 2002:a05:6830:12c2:b0:6e2:e953:6fee with SMTP id a2-20020a05683012c200b006e2e9536feemr3152110otq.24.1707783298148; Mon, 12 Feb 2024 16:14:58 -0800 (PST) MIME-Version: 1.0 References: <20240212213922.783301-1-surenb@google.com> In-Reply-To: <20240212213922.783301-1-surenb@google.com> From: Pasha Tatashin Date: Mon, 12 Feb 2024 19:14:20 -0500 Message-ID: Subject: Re: [PATCH v3 00/35] Memory allocation profiling To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 2E9C0100013 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: in55sdd6e1bm1euhggsr71st6sf1p8ox X-HE-Tag: 1707783298-602281 X-HE-Meta: U2FsdGVkX1/ymaYE/lPV1s2szXqmwRN1C0KIwIgPGFOBKRXrchWSFPlKqJEKUkZZhf3+eumkJmArLw/Av/YZjheGWo88i2Jz/FZl7AQpz1SAxG3HieBbzzTKqpIwlZKi6QkWZnqVmYUKXq41jU1/wQIvdiYY5IZDGud82dCmySNIQ2f+E0LuXoX7+fcp9fE99F0KTBP+JXkTsrNVsvhTu6kkaQwnj4iGOux9m6admb8iaPwyNBoyQgeaeGZUNX5GWF+Yu1s3+c0Qg04hiI14pYzfO6UcGjPl9Que5YQdV6qm1XA3ddtSEeKFkqUgcIxS2jKS1bkDpGVm/z4nRuEI101z3ARdCJA1opDVkloskTtZxuPlPiLAXeRUaZKM8DAivLPE+uR2XACujZfLr3i3Ko+npHRUuQmcSQ92kwcQqF856f8hOC6Lpo6AVcf7Eau+kaQYYSZWPmhSCRfRapYaGOlBkZcg6v/vNMseW8WyXwG0adS0CFoc1MaQLYDt0GKZB8Tl/EvDrHaF+YbGkC64ryO2HMZdx0NKx4BjtrN09qil9uZ3X+VcW0QgKA4AA9c/dC8HF2Bv3oKUXuQ8JLtr8zSWyEA0Tivo3JGZ09KxP/FqR8/+T/nv/Qy59mkjQRNPkgFWz+zd2r2RF40bVwDE3QOh1EovQRManfiOWoZ7/oxtGU0pbof3SElndGcET2XpjlTIKzzi5iyiAycXq1KVy/i/gBebeKmIDc/drltVjCufCER6QYL2K9e2NhgF6XSy0REwBIFIZsM+wkNxiEm4X8knLBhpI84Dwf2RAoRN+HMkPa79weKsJGXRq6+RwWdGc5uouGYuhDgsRAwRWB31Fpl8mbqtiimE8LeGl033HFW0ZEgWbP4G5FMNMkRKwizZKVcM/NXLQyk94nyY/okAw9jLu5NZR9JVSrQLwnhw6iep6e5uLGNJ+frxKwJD6eRK0ZdrvRGuFA2w0H57No0 iUh/Cs0q xGo72OQb9MtEEUbhYnnRU77H94ZrXEA+12jmR293TzTZnytXb/yKw/VMK3pXu/KWjqKCUuk1vX7VtZNqE1M9mtW/YBdFjeOjaCvA+jYnm6albrguAbhcLVFKyv64oT8a6QdSVMnOHsVQac0EU511CL9ULwUGW1rRgdoF8xmrh2gg7+YEdODzCCSkYP0FCpkge92cR5IOVuo8s2PdD63eoBMg2pIlqDv6UOtNacvaQD5FCCWeLfXewrGzwXQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 12, 2024 at 4:39=E2=80=AFPM Suren Baghdasaryan wrote: > > Memory allocation, v3 and final: > > Overview: > Low overhead [1] per-callsite memory allocation profiling. Not just for d= ebug > kernels, overhead low enough to be deployed in production. > > We're aiming to get this in the next merge window, for 6.9. The feedback > we've gotten has been that even out of tree this patchset has already > been useful, and there's a significant amount of other work gated on the > code tagging functionality included in this patchset [2]. > > Example output: > root@moria-kvm:~# sort -h /proc/allocinfo|tail > 3.11MiB 2850 fs/ext4/super.c:1408 module:ext4 func:ext4_alloc_inod= e > 3.52MiB 225 kernel/fork.c:356 module:fork func:alloc_thread_stack= _node > 3.75MiB 960 mm/page_ext.c:270 module:page_ext func:alloc_page_ext > 4.00MiB 2 mm/khugepaged.c:893 module:khugepaged func:hpage_coll= apse_alloc_folio > 10.5MiB 168 block/blk-mq.c:3421 module:blk_mq func:blk_mq_alloc_r= qs > 14.0MiB 3594 include/linux/gfp.h:295 module:filemap func:folio_all= oc_noprof > 26.8MiB 6856 include/linux/gfp.h:295 module:memory func:folio_allo= c_noprof > 64.5MiB 98315 fs/xfs/xfs_rmap_item.c:147 module:xfs func:xfs_rui_in= it > 98.7MiB 25264 include/linux/gfp.h:295 module:readahead func:folio_a= lloc_noprof > 125MiB 7357 mm/slub.c:2201 module:slub func:alloc_slab_page This kind of memory profiling would be an incredible asset in cloud environments. Over the past year, we've encountered several kernel memory overhead issues. Two particularly severe cases involved excessively large IOMMU page tables (20GB per machine) and IOVA magazines (up to 8GB). Considering thousands of machines were affected, the cumulative memory waste was huge. While we eventually resolved these issues with custom kernel profiling hacks (some based on this series) and kdump analysis, comprehensive memory profiling would have significantly accelerated the diagnostic process, pinpointing the precise source of the allocations.