From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4CCCECAAD3 for ; Thu, 1 Sep 2022 19:39:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0852080048; Thu, 1 Sep 2022 15:39:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0341B8000D; Thu, 1 Sep 2022 15:39:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3DC880048; Thu, 1 Sep 2022 15:39:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D51798000D for ; Thu, 1 Sep 2022 15:39:23 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9DFE01610B6 for ; Thu, 1 Sep 2022 19:39:23 +0000 (UTC) X-FDA: 79864530606.11.BEEDDEF Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) by imf08.hostedemail.com (Postfix) with ESMTP id 6113A160060 for ; Thu, 1 Sep 2022 19:39:23 +0000 (UTC) Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-340f82c77baso231681687b3.1 for ; Thu, 01 Sep 2022 12:39:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=rw5PRlxl7tO0yi2f42RcudRuneYpC4HLzAgnvXZWG5E=; b=lG3CtwEra2JQqN/veO23yk3cgaTJHFCiebuuRZY/xNehRm+m0fM4bIsWwBppGA+dxf mGwrLGImeNf4NmD+qiiJAgJrRcvEAfMmmsjPutkxq2y7CzoFE7qGuyTBDkwrYwo0ghp+ DIdTURnQpZf4mS51OcBm4/JauLAis6X1OUJhsFMZ1Eii1uGdRpU21Q08G5BpjTO5hJLN 9L5Fsbu9o5yfqdUojSVuS9kjs1tQfCEVguOYiQkyDJxdrNwv3a8BIeLV7PQsB1938bTI TTsL41uELqXnesIe6dBtowmfUnvD7Y1/Zc9uEXdIILlsmZXFDiVnDH9hCWWOHKwToPd1 MnoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=rw5PRlxl7tO0yi2f42RcudRuneYpC4HLzAgnvXZWG5E=; b=y9ICM7djTr5kAm+uIkoS4Wr25BxMRL+7YKaodMkzHZfhLnRaHk3GwNmwR9mglwCwi0 EffC1gDECiZJEXH4Tr1bLMNDN8EKwyQHhiolgywMSB8C0KdMiMFMt4PJExtS4fjVX17j PztAibXsrgYbUzg8Mg9+eHQyxfOdil2mabd5VEpINwLLn4E1JJR4T1S40CzNS5waBiP+ 8XrPry4BOZUSuMJRANwBQi9pxPRTyE+mRlK2L/jxhmQQV824NtnOSNar04JX8kZqVLcj jXINrj3YuvK7c5fg8+FxRKd9dHL1VzV70J3ER2aE1sU4CxE4ScponbWzvfzCSM0IQJXz uLHA== X-Gm-Message-State: ACgBeo26WehV59VLS/UkYsCFvj7pyZUh+xoyeGAXU3wfxfqxDb+u7UHa 6lnE7mowqwEQMTVxLzp9cKoDbrTh1tQPyIAZEDyqCg== X-Google-Smtp-Source: AA6agR43mbcplTcd3Q6E6SWUR/a0QzgezQxqYw6L8vriq8koQ7chGwVsb4h58VmaT16qZLx9mcJbzjGRj/xFE0GH57Q= X-Received: by 2002:a0d:d850:0:b0:340:d2c0:b022 with SMTP id a77-20020a0dd850000000b00340d2c0b022mr21758237ywe.469.1662061162404; Thu, 01 Sep 2022 12:39:22 -0700 (PDT) MIME-Version: 1.0 References: <20220830214919.53220-1-surenb@google.com> <20220831084230.3ti3vitrzhzsu3fs@moria.home.lan> <20220831101948.f3etturccmp5ovkl@suse.de> <20220831190154.qdlsxfamans3ya5j@moria.home.lan> In-Reply-To: From: Suren Baghdasaryan Date: Thu, 1 Sep 2022 12:39:11 -0700 Message-ID: Subject: Re: [RFC PATCH 00/30] Code tagging framework and applications To: Michal Hocko Cc: Kent Overstreet , Mel Gorman , Peter Zijlstra , Andrew Morton , Vlastimil Babka , Johannes Weiner , Roman Gushchin , Davidlohr Bueso , Matthew Wilcox , "Liam R. Howlett" , David Vernet , Juri Lelli , Laurent Dufour , Peter Xu , David Hildenbrand , Jens Axboe , mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, changbin.du@intel.com, ytcoode@gmail.com, Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Benjamin Segall , Daniel Bristot de Oliveira , Valentin Schneider , Christopher Lameter , Pekka Enberg , Joonsoo Kim , 42.hyeyoo@gmail.com, Alexander Potapenko , Marco Elver , Dmitry Vyukov , Shakeel Butt , Muchun Song , arnd@arndb.de, jbaron@akamai.com, David Rientjes , Minchan Kim , Kalesh Singh , kernel-team , linux-mm , iommu@lists.linux.dev, kasan-dev@googlegroups.com, io-uring@vger.kernel.org, linux-arch@vger.kernel.org, xen-devel@lists.xenproject.org, linux-bcache@vger.kernel.org, linux-modules@vger.kernel.org, LKML Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662061163; a=rsa-sha256; cv=none; b=aWTNOdHeIUJ5QROU5yDhdM7SV2pNYT/yTrghJLP6xnJBBShbWcP2syHuQ1HxZa7M1fMOsA FnvQr+VeWsdBG5HiBqrOHD5PwtbOKSgs4HbBh45KkkiOMjlqTBfuv5Uyykb3hEZHDKS6zd 0mUWs4+D6qAiNMhlSAA3IyPuIqgg64c= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lG3CtwEr; spf=pass (imf08.hostedemail.com: domain of surenb@google.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662061163; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rw5PRlxl7tO0yi2f42RcudRuneYpC4HLzAgnvXZWG5E=; b=JB8pp70eMYQgrFPG5rXNuiwLmQN35D7ud8xeg4lQxJZ7eRM/nKPEeMrbVaOIpcD8FP68/c nPttUWV/7+5Yu8VcRr5xZw1b1L2iw0fBUPagYn7cXvNE+Aaaukuq4dm/Jcdvd6QXM761K0 VIzJ2f3KJEw11RaAvm5xAuECr5arO5g= X-Stat-Signature: 8z4n5e7twsjbuwuthgip8s9zppnm3r5y X-Rspam-User: X-Rspamd-Queue-Id: 6113A160060 X-Rspamd-Server: rspam07 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lG3CtwEr; spf=pass (imf08.hostedemail.com: domain of surenb@google.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1662061163-101057 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 1, 2022 at 12:15 PM Michal Hocko wrote: > > On Thu 01-09-22 08:33:19, Suren Baghdasaryan wrote: > > On Thu, Sep 1, 2022 at 12:18 AM Michal Hocko wrote: > [...] > > > So I find Peter's question completely appropriate while your response to > > > that not so much! Maybe ftrace is not the right tool for the intented > > > job. Maybe there are other ways and it would be really great to show > > > that those have been evaluated and they are not suitable for a), b) and > > > c) reasons. > > > > That's fair. > > For memory tracking I looked into using kmemleak and page_owner which > > can't match the required functionality at an overhead acceptable for > > production and pre-production testing environments. > > Being more specific would be really helpful. Especially when your cover > letter suggests that you rely on page_owner/memcg metadata as well to > match allocation and their freeing parts. kmemleak is known to be slow and it's even documented [1], so I hope I can skip that part. For page_owner to provide the comparable information we would have to capture the call stacks for all page allocations unlike our proposal which allows to do that selectively for specific call sites. I'll post the overhead numbers of call stack capturing once I'm finished with profiling the latest code, hopefully sometime tomorrow, in the worst case after the long weekend. > > > traces + BPF I > > haven't evaluated myself but heard from other members of my team who > > tried using that in production environment with poor results. I'll try > > to get more specific information on that. > > That would be helpful as well. Ack. > > > > E.g. Oscar has been working on extending page_ext to track number of > > > allocations for specific calltrace[1]. Is this 1:1 replacement? No! But > > > it can help in environments where page_ext can be enabled and it is > > > completely non-intrusive to the MM code. > > > > Thanks for pointing out this work. I'll need to review and maybe > > profile it before making any claims. > > > > > > > > If the page_ext overhead is not desirable/acceptable then I am sure > > > there are other options. E.g. kprobes/LivePatching framework can hook > > > into functions and alter their behavior. So why not use that for data > > > collection? Has this been evaluated at all? > > > > I'm not sure how I can hook into say alloc_pages() to find out where > > it was called from without capturing the call stack (which would > > introduce an overhead at every allocation). Would love to discuss this > > or other alternatives if they can be done with low enough overhead. > > Yes, tracking back the call trace would be really needed. The question > is whether this is really prohibitively expensive. How much overhead are > we talking about? There is no free lunch here, really. You either have > the overhead during runtime when the feature is used or on the source > code level for all the future development (with a maze of macros and > wrappers). Will post the overhead numbers soon. What I hear loud and clear is that we need a kernel command-line kill switch that mitigates the overhead for having this feature. That seems to be the main concern. Thanks, Suren. [1] https://docs.kernel.org/dev-tools/kmemleak.html#limitations-and-drawbacks > > Thanks! > -- > Michal Hocko > SUSE Labs