From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7D23C48BC3 for ; Wed, 14 Feb 2024 17:15:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 539766B00BB; Wed, 14 Feb 2024 12:15:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E8C56B00BC; Wed, 14 Feb 2024 12:15:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36CD26B00BD; Wed, 14 Feb 2024 12:15:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 228166B00BB for ; Wed, 14 Feb 2024 12:15:01 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EFE7F80E0A for ; Wed, 14 Feb 2024 17:15:00 +0000 (UTC) X-FDA: 81791059560.21.16DBF59 Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) by imf07.hostedemail.com (Postfix) with ESMTP id 1079D40022 for ; Wed, 14 Feb 2024 17:14:58 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="elTg/CEZ"; spf=pass (imf07.hostedemail.com: domain of surenb@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707930899; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LDg0vk5u/oyOZlQ8e31c3DglAzpuw7KxG8umDKGdfj8=; b=ATYXERsRXW8HDcDS45FULf8hpqRDkTvg3UFn6pXTMEpbtlFRIAkCcPidRBg8JdwphgcqEG kY4LpByBsGtVM2hZD9jo5ww/XJ7JuU8xDoLecMGj77RiCTL8TExHd3fsCAj6TQbSUhw+LB wtWynhLP1oWa+oOknTuotX1UdJNajnk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707930899; a=rsa-sha256; cv=none; b=SR04GFkufJnyf4j252CCPU3qvw2hh1m6gm3GHtJtIqj7tyJHtV+JWPWbM1aT2vwfpwR8Nz ZB4d3u80pXoCUCDscwW50FufjCa89dhnZ6WhPh1bgnuK8dBR21VJ0k5D+Msm8o6SBHeBE3 j1zYVS3sEVF/6d5U5bk42fczfUjUcS4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="elTg/CEZ"; spf=pass (imf07.hostedemail.com: domain of surenb@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f174.google.com with SMTP id 3f1490d57ef6-dcbcea9c261so2987473276.3 for ; Wed, 14 Feb 2024 09:14:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707930898; x=1708535698; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=LDg0vk5u/oyOZlQ8e31c3DglAzpuw7KxG8umDKGdfj8=; b=elTg/CEZBIkzYMQjYr12pTmMPdmv3A5wZ2LekQUJYJGAwT7n0+2cvupFhvdJAAnKnp tqqGeOx7Te74DEjqv+oZTiknKIqeTnjfwbOjbQl0PtuDxKBwRm+BhvtrLw0keIsmq0cO cFr/uqzj2EgAxBgVVapWSrnrv/p5n7iMH/5ZMcAfSTPvpYt5Ngfw4l6Flgc4sAW//I+z HNfWcGxh4Jx3q5aBmt2xETuRRMM5SPcpvnNzeYoOiz4vLDh1rPdVZrq6C+/2JfEKM93Y JJfFDW9kkf1owrUSd3jJ6BdnKNvQdSb8Wyo7UOWVYZKnC9+ENc4Cy4+JdxUAMBV/4c9t bSLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707930898; x=1708535698; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LDg0vk5u/oyOZlQ8e31c3DglAzpuw7KxG8umDKGdfj8=; b=KA0wBGDCHVs5PqkF6TvE0NmDIgZNq8HZIUD5G1QdyRyJ/LUFzR7jiXIzdfKi0FoEA7 wlm9dMnG5vi7HGJMAtKhpuQcY20g1r4aaYC4gLFl3KNHQ4cho7GpiGphGf8zs43aahK2 d5wu6fc0PmZdLnhztOEbzAj9qhPF1cyKUGE3XPVlw6pZ72r44laXV2ee9xyZ364aTI55 N6Hg0HztYZv5DxAjEwqKLsW5QyszQZyNVrLrx2jZUNVfa/4ONWfIEfB8PCoGCwTnAJNO yGDS5Evjl3MRYV/mcN0cVoETapFfB4PZ658Cy9HhSv6pFFbMVGF/IyAI3lzNbv2h2+Od quXA== X-Forwarded-Encrypted: i=1; AJvYcCWhEQCdDwoM74QTdKf7d8kULQsd/mb73pJlEgjdtiwtbqW5/bysZA9EGJT2CzCTkeHka2mrCliV5LW2fH/4FUn8EsM= X-Gm-Message-State: AOJu0YyIoEWyDw4rxO0pi3K27eUZ8SP9QdP0QK80ssfP3tEPS+BJACUy 4fURDfL3/NDnf0hiCIN196/syhyym3QKFsPq3HanvGZ2itBLWC29sMcREZFpX3qG3kWzAckhqw2 Wp56ci9P7zg6yHWqU2NmfAXwZmlIZAa2KkXpX X-Google-Smtp-Source: AGHT+IFzocTbMe8vXbVa4j69vx7gDqurZ3EaP/A9DOqxrZVhCKoBgWMQN08N6cMcnP1DyMjMOOc1HMd1b4SoKjvkySI= X-Received: by 2002:a25:7443:0:b0:dcb:e82c:f7d with SMTP id p64-20020a257443000000b00dcbe82c0f7dmr2936932ybc.41.1707930897779; Wed, 14 Feb 2024 09:14:57 -0800 (PST) MIME-Version: 1.0 References: <20240212213922.783301-1-surenb@google.com> <9e14adec-2842-458d-8a58-af6a2d18d823@redhat.com> <2hphuyx2dnqsj3hnzyifp5yqn2hpgfjuhfu635dzgofr5mst27@4a5dixtcuxyi> <6a0f5d8b-9c67-43f6-b25e-2240171265be@redhat.com> <20240214085548.d3608627739269459480d86e@linux-foundation.org> In-Reply-To: <20240214085548.d3608627739269459480d86e@linux-foundation.org> From: Suren Baghdasaryan Date: Wed, 14 Feb 2024 09:14:46 -0800 Message-ID: Subject: Re: [PATCH v3 00/35] Memory allocation profiling To: Andrew Morton Cc: Kent Overstreet , David Hildenbrand , Michal Hocko , vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 1rm7nq6845cans8huzebo4ptjmad183y X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1079D40022 X-Rspam-User: X-HE-Tag: 1707930898-574663 X-HE-Meta: U2FsdGVkX19M9cIgtG5g8DCSEh8mjdw/NAIz8hMeqAM/dV2IP7rbI2mjFoz3bhlr978chV1dVzeR/XJBNtqsci298cO/nJYDT6BGONogq2wgFO/NTJQttAge5pedyewTF1M4I9Nll1z9oI4aZS3hO4cg3/Fz8AfgO3DCRANucqE0kV0AnGpfTsruNHUkSiekqkz2nyGCNHlGWA0DpGpEGG0eipoUJbDYdbCVM77kunZVdrG9VxwiQSJKRhnYdh1gwG8XJdfbNSJCJRiukncUgNakgAGhNcpm3TNEy4nahohV3PI0BLgDZF+NRjPQYGP3x4xdXcmuJOlkDjNS244jeXKgLFdE05U+k8WPRh3cGwGqNoq/K2qHCkR7276YKfwb4FkrAeauO9J9/uKXuj4M8vZuMNlhHucS3SXBk4JzaIsAl3YgmpKfb+Bj/caeRd2Usn9/OhJW2JylRmEIHxYSYeMR0iliVgUobgfbyzDbZAy+HribiwDEYFX2XTP7D08lmftU1fOiphULIup/Qskuy2Ju1x3IRyEvTHjjNbAX73GkdMG5T/UcM+OkT7IO1owdE1o/jaJh39WxMOj6wfsC9t7kNN2xJ1/u7uOFjTHIDJ5PWTyK5geySmt6xpNgaIb1/Ypz3uvpAM1th5ATCpZ4kEw0zGVi4XH8c2w1qZ/LQKJPkRX8M5X5TEYKgRgHqLDEoR6+i0u5ukAEuSpq1NS4tr38rWDeZb6u1LMOxG1MCXHnagYCWPfCqpzDt1hH5g/VZoyjS3vY9CpDj1uHJdxhLGeaCjKo6QOoIiE0P9Bl9qxzh7j9vJZGXUZ17mvw2Zfxyaem4yg+bFCf11b4izUtkkYBx7tJVe0M/peJE1QnMHojahTHB4569oAxXavS/JCn4CeG0oVhpOZ7ONqLA6gH9fFL1fwYsXTltpETLV4Qto1aDdy159rfWJhbWFXqK1aqIbKyFEAPtIYPVR7cWZ8 ZRMY67hm x5YW/EHlTezLrFtSnb/6BfBWoCawecLe60Yi0dMH2gSwLiobkDALaUbvbOlHUUMdQlJlLn8AhvT04BTo6r/GQI1ZV8aepO1tbvbQrhxz6pxWKpkIDImo2AX8gAh3ZOJIJ5rKtyP5P93vqqbhaqRyPyQbxtT0DFxUCezRX41M03/xiu+CGfsN/JJv1rddlo8r+AFmBe841PFhX+/cZWKz0+f4lZgsPXWF913QcNddcLHIJFFgO0Uy1lyK5G3OZ74DUEFAJR873fMeyh76xz0UmyLW05KX94nMcUh+8lElQfPWQ38UJgC9rdqiwTSsZ0Y069fih1vfHwTOvLF+CJsniCI2pnPuB4tY0HOehpTGCXYon+Fe7clkxdamwkGoulSLKsJKsYxLJyfrvaE8IA/ZtAvq4ug== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 14, 2024 at 8:55=E2=80=AFAM Andrew Morton wrote: > > On Tue, 13 Feb 2024 14:59:11 -0800 Suren Baghdasaryan = wrote: > > > > > If you think you can easily achieve what Michal requested without a= ll that, > > > > good. > > > > > > He requested something? > > > > Yes, a cleaner instrumentation. Unfortunately the cleanest one is not > > possible until the compiler feature is developed and deployed. And it > > still would require changes to the headers, so don't think it's worth > > delaying the feature for years. > > Can we please be told much more about this compiler feature? > Description of what it is, what it does, how it will affect this kernel > feature, etc. Sure. The compiler support will be in a form of a new __attribute__, simplified example: // generate data for the wrapper static void _alloc_tag() { static struct alloc_tag _alloc_tag __section ("alloc_tags") =3D { .ct =3D CODE_TAG_INIT, .counter =3D 0 }; } static inline int wrapper (const char *name, int x, int (*callee) (const char *, int), struct alloc_tag *callsite_data) { callsite_data->counter++; printf ("Call #%d from %s:%d (%s)\n", callsite_data->counter, callsite_data->ct.filename, callsite_data->ct.lineno, callsite_data->ct.function); int ret =3D callee (name, x); printf ("Returned: %d\n", ret); return ret; } __attribute__((annotate("callsite_wrapped_by", wrapper, _alloc_tag))) int foo(const char* name, int x); int foo(const char* name, int x) { printf ("Hello %s, %d!\n", name, x); return x; } Which we will be able to attach to a function without changing its name and preserving the namespace (it applies only to functions with that name, not everything else). Note that we will still need _noprof versions of the allocators. > > Who is developing it and when can we expect it to become available? Aleksei Vetrov (google) with the help of Nick Desaulniers (google). Both are CC'ed on this email. After several iterations Aleksei has a POC which we are evaluating (https://github.com/llvm/llvm-project/compare/main...noxwell:llvm-project:c= allsite-wrapper-tree-transform). Once it's in good shape we are going to engage with CLANG and GCC community to get it upstreamed. When it will become available and when the distributions will pick it up is anybody's guess. Upstreaming is usually a lengthy process. > > Will we be able to migrate to it without back-compatibility concerns? > (I think "you need quite recent gcc for memory profiling" is > reasonable). The migration should be quite straight-forward, replacing the macros with functions with that attribute. > > > Because: if the maintainability issues which Michel describes will be > significantly addressed with the gcc support then we're kinda reviewing > the wrong patchset. Yes, it may be a maintenance burden initially, but > at some (yet to be revealed) time in the future, this will be addressed > with the gcc support? That's what I'm aiming for. I just don't want this placed on hold until the compiler support is widely available, which might take years. >