From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08783C48BC3 for ; Wed, 14 Feb 2024 20:00:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E2556B0087; Wed, 14 Feb 2024 15:00:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 391826B0088; Wed, 14 Feb 2024 15:00:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 231D26B008A; Wed, 14 Feb 2024 15:00:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 100A66B0087 for ; Wed, 14 Feb 2024 15:00:32 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8A9221A0F24 for ; Wed, 14 Feb 2024 20:00:31 +0000 (UTC) X-FDA: 81791476662.10.A5AE92C Received: from out-186.mta0.migadu.com (out-186.mta0.migadu.com [91.218.175.186]) by imf06.hostedemail.com (Postfix) with ESMTP id 29F5C180032 for ; Wed, 14 Feb 2024 20:00:28 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V0Dtva7q; spf=pass (imf06.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.186 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707940829; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pf30Mbuh5b/kObn023v7ccfUflKK/+7eCEEfg4Q4/Xo=; b=AglXpzCQGQHxuIMuAky+U/Esbp8YBNSHq0uhujyN2m3wL5zB8pPNW862yjxyH1wG0aKvuH 8UxRbBD5wF0bF/1zW0NrWVGvsPyvPL/paORZ9HZgMUWtqyF/+LcyWRtWCTG9a1n7YHgYj5 SZ2loKIC0gLGK078YQ7MOftJrrwpagU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707940829; a=rsa-sha256; cv=none; b=R5Ai1MJUNrLBB8Z8j/R0h3iUK3krbG/+qU7/gWsPFaKKWypsUY+2iwnPkL/c1lZpLCpqET 9rwrwY9gfpN4E3Y/MuI+HdPpcmwpnBA+z+kEzDCWhk+qI9sV60DHH6HlDH9Laz8EHKanLa 3n4WVaru157hVR1CXeBOJu0Ox6aD2zA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V0Dtva7q; spf=pass (imf06.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.186 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Date: Wed, 14 Feb 2024 15:00:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707940826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pf30Mbuh5b/kObn023v7ccfUflKK/+7eCEEfg4Q4/Xo=; b=V0Dtva7qS2reHkZJi6PJ+HEovBCR0hF1cHRtyU3vqEVbBd8vXfQIrh8jm2X0ifBBI4P+IO H5vrvDEyaz0upotCLVyuYanif0dGbMLFqnWADl59CQ4rEdihhMwhzsvSw9R6sR/elwVruF 7JlRVAQ9ZPkvUkS27LLag/qy7F3NQ4g= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Suren Baghdasaryan Cc: Andrew Morton , David Hildenbrand , Michal Hocko , vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Subject: Re: [PATCH v3 00/35] Memory allocation profiling Message-ID: References: <9e14adec-2842-458d-8a58-af6a2d18d823@redhat.com> <2hphuyx2dnqsj3hnzyifp5yqn2hpgfjuhfu635dzgofr5mst27@4a5dixtcuxyi> <6a0f5d8b-9c67-43f6-b25e-2240171265be@redhat.com> <20240214085548.d3608627739269459480d86e@linux-foundation.org> <7c3walgmzmcygchqaylcz2un5dandlnzdqcohyooryurx6utxr@66adcw7f26c3> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 29F5C180032 X-Rspam-User: X-Stat-Signature: haw4kxf9kwpqdpj6c4bhejaaihdojon9 X-Rspamd-Server: rspam03 X-HE-Tag: 1707940828-639802 X-HE-Meta: U2FsdGVkX19sXEkjKHoRPjlepoqy0bTLx4qZ7GO6kj+1eWdYuHvjpLRBBaZwnQA5SyHNs6nYX6oFFZGS+a9vfAmUJpYQy/sYCOLZeRfLxUBDhHPcmEirow46gelYMnmE2sMEC/6EvFaaGWExfQcqHF7jkiQ8lCxYplQsSXQQbq7iZRnd3miAqMC9+1TVpve1hinOw/nJxJVIh0QZvw8hVy7VOUcVKAtcDSDgla231Kzm/81wD+/l1uzBult1QfDjy7YeOxdMxL/ifSbz/fLr0/dbpUaOXqoIPI/MSFyi7rU7jYQ87fg5o15uCZ/0Ku8m4vNM43A4TmS8w9xRToC6/ZixOu9TJ2Lm09Z6qTKCUrTrsNlb5/OyPfpFcnOrJHlFi17eDtbqMFA2TpgA/u+O/fmnY1w+Mht54Hmvpo+emOGs/hNk+H9qUnm+kWV0WtwLf0MktNrLNMEqr5GFu1406H2W5cnuTTcsxsY9v7BFZvAEkn745Yon0/ND5NcntWeky3xKhTkvHfNuzAJaxAeBvTCxgZ05IKB/VRQx7yQbWuobATIH+y4XAkKq2QGROjW4X7UQ8evRwWF42XNV6mOlB66f8Ggfau1vDSWvMkTkl9cUyqy1NY1iIJpGBQ15iDpSrRfS1EXPyIUwzVPiWJfgyacTUyKM5TcLPcqiIpCmVIBQmoeQoaHlTV0dIVAtUl69CEGEmTFOAZGCCKw8CQ2iYx/cAFNs0k6j3cuxORuaNb+2iId8G2MSg1neYwrYumJI7kLeMSxpyewAod13YLYg0JZECKnlaiUWt9U1s0nJ2nTkIsx/14ggsmN3t0GTIgEZtSEo9NszJ5qyGRQR4c7Zaf9wkIo8voosQh+IpJlBj1uFkZOxG/b7oAoYgkPXI9tTQqdIj3j88h8BiCLRv05kAGn725Po3ZMeOAfqlHg6XHIzuEWykqqcUtVlgIMHCPscLQfWsLhZvMvI0kGZ8AE UpR4ax61 iS8PtiYMgRadWLX/QxraGMTLDay99oCle1k1s1Rqot0KeuCgEqOlEjZCtjjdfRHEO81+72kiLrV7tFEWFDHTD9Cw4upne2tgKuqoM9jR9A2ONgJtF7YOh6/tewiznPRUgTLoOFJYFman7NGobJMt4scQwyvWjoPu01biGFOVLIObWf3p5go+PYzDgeDkY5EOytTJ6329uOYATVlIUsp0PLrqsVg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 14, 2024 at 11:24:23AM -0800, Suren Baghdasaryan wrote: > On Wed, Feb 14, 2024 at 9:52 AM Kent Overstreet > wrote: > > > > On Wed, Feb 14, 2024 at 08:55:48AM -0800, Andrew Morton wrote: > > > On Tue, 13 Feb 2024 14:59:11 -0800 Suren Baghdasaryan wrote: > > > > > > > > > If you think you can easily achieve what Michal requested without all that, > > > > > > good. > > > > > > > > > > He requested something? > > > > > > > > Yes, a cleaner instrumentation. Unfortunately the cleanest one is not > > > > possible until the compiler feature is developed and deployed. And it > > > > still would require changes to the headers, so don't think it's worth > > > > delaying the feature for years. > > > > > > Can we please be told much more about this compiler feature? > > > Description of what it is, what it does, how it will affect this kernel > > > feature, etc. > > > > > > Who is developing it and when can we expect it to become available? > > > > > > Will we be able to migrate to it without back-compatibility concerns? > > > (I think "you need quite recent gcc for memory profiling" is > > > reasonable). > > > > > > > > > > > > Because: if the maintainability issues which Michel describes will be > > > significantly addressed with the gcc support then we're kinda reviewing > > > the wrong patchset. Yes, it may be a maintenance burden initially, but > > > at some (yet to be revealed) time in the future, this will be addressed > > > with the gcc support? > > > > Even if we had compiler magic, after considering it more I don't think > > the patchset would be improved by it - I would still prefer to stick > > with the macro approach. > > > > There's also a lot of unresolved questions about whether the compiler > > approach would even end being what we need; we need macro expansion to > > happen in the caller of the allocation function > > For the record, that's what this attribute will be doing. So it should > cover our usecase. That wasn't clear in the meeting we had the other day; all that was discussed there was the attribute syntax, as I recall. So say that does work out (and I don't think that's a given; if I were a compiler person I don't think I'd be interested in this strange half macro, half inline function beast); all that has accomplished is to get rid of the need for the renaming - the _noprof() versions of functions. So then how do you distinguish where in the callstack the accounting happens? If you say "it happens at the outermost wrapper", then what happens is - Extra overhead for all the inner wrapper invocations, where they have to now check "actually, we already have an alloc tag, don't do anything". That's a cost, and given how much time we spent shaving cycles and branches during development it's not one we want. - Inner allocations that shouldn't be accounted to the outer context are now a major problem, because they silently will be accounted there and never noticed. With our approach, inner allocations are by default (i.e. when we haven't switched them to the _noprof() variant) accounted to their own alloc tag; that way, when we're reading the /proc/allocinfo output, we can examine them and check if they should be collapsed to the outer context. With this approach they won't be seen. So no, we still don't want the compiler approach.