From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D018C77B78 for ; Wed, 3 May 2023 20:08:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C9C0900005; Wed, 3 May 2023 16:08:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65385900002; Wed, 3 May 2023 16:08:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51A94900005; Wed, 3 May 2023 16:08:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) by kanga.kvack.org (Postfix) with ESMTP id 2964A900002 for ; Wed, 3 May 2023 16:08:53 -0400 (EDT) Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-b9a6eec8611so5484798276.0 for ; Wed, 03 May 2023 13:08:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683144532; x=1685736532; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uaEl0+ttAfDVYkQkgOF9inVIlQxPo4MAHa3r5TcHPqs=; b=ZBFmhNUtx4spDPZ2Tc+ao7Umap1oUxGgS6L35zoiaIlLMa3sbvfe1yyzPgLcae3Fty gA+quo34T7AKstKRwlQ7/Ft2WrnkU+rEhs/+hqpbEUBx2dmlPORy4QLjme1jFVo6xrFm 5ILD7V5NtcxuXC6YpDb35KI9Z72miAAGUfujh1KI5hzn6IQVZLP/6kKOSaDUuTufQ8Ku q+1OYE8Cg6APTP1NiSWm7QBLpXw9UsuF2OAhdSzB2PChvOsgj7ZoUWNOcF4i8/Nlqo2k 4O1C3AV8COhnBUUJ7YqqIWrgXPcexM0yzg0bdV03LGdXCG1g+DB8aKT153Xkp0dZ7AtK Zo3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683144532; x=1685736532; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uaEl0+ttAfDVYkQkgOF9inVIlQxPo4MAHa3r5TcHPqs=; b=ErsI7EOJN8ih1IZIEA+d4ikb0K7dgM1GEmN6TYpsn3mYztFPfhkcVsxlcbm0vhB3BK mQZvO+koSAOEsxqM2rrBzHbRf/O7Y9aLAqlzQCRCGi3JxlGtL3T39iqsLE8rnogJOhep CmcPRU++uqjLw3fU7gGZY1GNMHySLF80AUNcXEtxF+RZ9IlUbqZjyrZs7OWiPO6sB7dp Yr5ha5SwoJUcevV89daE0vaeO96s7hBwYMAoSaXf1uripr1sy1cpQhIvumr6flu94yTi smUMH4jNO1CU+OdFztPPry++KUYHiA4qB/26RttvnirBq7WYwYd4GmbFIcPoAyFzZ+bU WBtg== X-Gm-Message-State: AC+VfDyd6JRjZY1mHs013gx4d5vHYAoJaTe6Z1aRdQ7JbtLB/lUXD0cP 9JY4DJm7+8iJTssX29IkGZYXj0SZzVVrBlY1HDSfGQ== X-Google-Smtp-Source: ACHHUZ5Zgn5KmLsK0xUXMtQ1V3cIfppdzeEQFI0bWMD7MbEWCqGn353J4Os9Iuap9y2I3NsIHQCGkXzmHh1rNAt3fns= X-Received: by 2002:a25:3450:0:b0:b9d:9f6e:f1d5 with SMTP id b77-20020a253450000000b00b9d9f6ef1d5mr3049024yba.16.1683144532379; Wed, 03 May 2023 13:08:52 -0700 (PDT) MIME-Version: 1.0 References: <20230503180726.GA196054@cmpxchg.org> In-Reply-To: From: Suren Baghdasaryan Date: Wed, 3 May 2023 13:08:40 -0700 Message-ID: Subject: Re: [PATCH 00/40] Memory allocation profiling To: Tejun Heo Cc: Kent Overstreet , Johannes Weiner , Michal Hocko , akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, ldufour@linux.ibm.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org, Alexei Starovoitov , Andrii Nakryiko Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 3, 2023 at 12:49=E2=80=AFPM Tejun Heo wrote: > > Hello, > > On Wed, May 03, 2023 at 12:41:08PM -0700, Suren Baghdasaryan wrote: > > On Wed, May 3, 2023 at 12:09=E2=80=AFPM Tejun Heo wrote= : > > > > > > On Wed, May 03, 2023 at 08:58:51AM -1000, Tejun Heo wrote: > > > > On Wed, May 03, 2023 at 02:56:44PM -0400, Kent Overstreet wrote: > > > > > On Wed, May 03, 2023 at 08:40:07AM -1000, Tejun Heo wrote: > > > > > > > Yeah, easy / default visibility argument does make sense to m= e. > > > > > > > > > > > > So, a bit of addition here. If this is the thrust, the debugfs = part seems > > > > > > rather redundant, right? That's trivially obtainable with traci= ng / bpf and > > > > > > in a more flexible and performant manner. Also, are we happy wi= th recording > > > > > > just single depth for persistent tracking? > > > > IIUC, by single depth you mean no call stack capturing? > > Yes. > > > If so, that's the idea behind the context capture feature so that we > > can enable it on specific allocations only after we determine there is > > something interesting there. So, with low-cost persistent tracking we > > can determine the suspects and then pay some more to investigate those > > suspects in more detail. > > Yeah, I was wondering whether it'd be useful to have that configurable so > that it'd be possible for a user to say "I'm okay with the cost, please > track more context per allocation". I assume by "more context per allocation" you mean for a specific allocation, not for all allocations. So, in a sense you are asking if the context capture feature can be dropped from this series and implemented using some other means. Is that right? > Given that tracking the immediate caller > is already a huge improvement and narrowing it down from there using > existing tools shouldn't be that difficult, I don't think this is a block= er > in any way. It just bothers me a bit that the code is structured so that > source line is the main abstraction. > > > > > > Not sure what you're envisioning? > > > > > > > > > > I'd consider the debugfs interface pretty integral; it's much mor= e > > > > > discoverable for users, and it's hardly any code out of the whole > > > > > patchset. > > > > > > > > You can do the same thing with a bpftrace one liner tho. That's rat= her > > > > difficult to beat. > > > > debugfs seemed like a natural choice for such information. If another > > interface is more appropriate I'm happy to explore that. > > > > > > > > Ah, shit, I'm an idiot. Sorry. I thought allocations was under /proc = and > > > allocations.ctx under debugfs. I meant allocations.ctx is redundant. > > > > Do you mean that we could display allocation context in > > debugfs/allocations file (for the allocations which we explicitly > > enabled context capturing)? > > Sorry about the fumbled communication. Here's what I mean: > > * Improving memory allocation visibility makes sense to me. To me, a more > natural place for that feels like /proc/allocations next to other memor= y > info files rather than under debugfs. TBH I would love that if this approach is acceptable. > > * The default visibility provided by "allocations" provides something whi= ch > is more difficult or at least cumbersome to obtain using existing traci= ng > tools. However, what's provided by "allocations.ctx" can be trivially > obtained using kprobe and BPF and seems redundant. Hmm. That might be a good way forward. Since context capture has already high performance overhead, maybe choosing not the most performant but more generic solution is the right answer here. I'll need to think about it some more but thanks for the idea! > > Thanks. > > -- > tejun