From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D256C433FE for ; Mon, 28 Mar 2022 09:17:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA8BD8D0002; Mon, 28 Mar 2022 05:17:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C58D18D0001; Mon, 28 Mar 2022 05:17:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B27248D0002; Mon, 28 Mar 2022 05:17:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id A1FAE8D0001 for ; Mon, 28 Mar 2022 05:17:31 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 472B4182890F0 for ; Mon, 28 Mar 2022 09:17:31 +0000 (UTC) X-FDA: 79293241902.18.06CF1EE Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf20.hostedemail.com (Postfix) with ESMTP id C90071C0047 for ; Mon, 28 Mar 2022 09:17:30 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id k14so11812925pga.0 for ; Mon, 28 Mar 2022 02:17:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kpQSKtdwNb3KC+dlFVYdgCuQmYu2WXrTtlnYf3fe1l8=; b=YCsQDYTMhVfFS4jUgjugN+EKYW2QiI11NiiFh/w4yzJf8k0v+rR+WcwnSZ1DS/RnKP 7GkAeIFgVSZcYgH9yYPcRw1VQuH8wf6MSjnx31JR4bmwVt+N7EG5ssSn4iorRC9/5TaF nVsY1CNVZUcfifMRRoRcBAgXAK1dhD0FLbwGOFlGr87RFvJKSgpyNaYT4XugCWPs+KJz S/R7pigyNxrGJ5jGrhVrThjkD5V1lkqRGQBw1/u2heG1ewyU/y1jUYJKmHbLeE1QGjTI MJGui/dwhfeQX32BFZ3xZCjFv+eXoBGlci9kCOx5YWMJxIy/5mPiJFYPZyEUXdgaDDvU weKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kpQSKtdwNb3KC+dlFVYdgCuQmYu2WXrTtlnYf3fe1l8=; b=3UXbdhAiSlD2+bdEAll9IC72Kj38+qDfy4KVSC/DxsKADd7Qvhi/QsBSTw2jo6vLT0 WlKxfImlWeQiWS5yzdCC60MCEm+VHL27JT+lIxqdnsBv/Z9sw6jbT5PuayzNPsOnoltY WiVzrYevAVT5EnJXkn2fJUZPGfqhBjQaqcC8KJ5WeBys7XmVUETQpAnlVZXebqc+C60A y8PTvu5+9choknHwT9A2TXNzC3uRhyHEFdKWDgTKlmlDo0ANxK9T+wMFWzNw2Woo65oF mC0HH3yjRBaDUHHx5QdcCVbU5Cg5/dL9VLeVMsjnEVCZgZMp4rc2grcj7hL3aoAo42Dy u5qg== X-Gm-Message-State: AOAM530I7jUk7XT/qGq37brdJmGwbTTfy+azev0bI9nXiaWFNB41DEwC Sh1lAhSZhBgB6HBs9VWCN2bL4XSTqv6hF7sZquohiw== X-Google-Smtp-Source: ABdhPJzNClzlxXy6do9tw7mpbWwP81WKjZPm+SVTHWOuMA+zQGyhRfvPYFLJ2ILCrxR0H8E1lbcgYU2wYHB9QGa6xK4= X-Received: by 2002:a05:6a00:a0f:b0:4e1:309:83c0 with SMTP id p15-20020a056a000a0f00b004e1030983c0mr22567091pfh.68.1648459049579; Mon, 28 Mar 2022 02:17:29 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yosry Ahmed Date: Mon, 28 Mar 2022 02:16:53 -0700 Message-ID: Subject: Re: [RFC bpf-next] Hierarchical Cgroup Stats Collection Using BPF To: Song Liu Cc: Namhyung Kim , Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Johannes Weiner , Hao Luo , Shakeel Butt , Stanislav Fomichev , David Rientjes , bpf , KP Singh , cgroups@vger.kernel.org, Linux-MM Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: sg8863db3nzmx5ph18e99z6uwxyh8dkc Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=YCsQDYTM; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C90071C0047 X-HE-Tag: 1648459050-857563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 18, 2022 at 12:59 PM Song Liu wrote: > > On Wed, Mar 9, 2022 at 12:27 PM Yosry Ahmed wrote: > > > > Hey everyone, > > > > I would like to discuss an idea to facilitate collection of > > hierarchical cgroup stats using BPF programs. We want to provide a > > simple interface for BPF programs to collect hierarchical cgroup stats > > and integrate with the existing rstat aggregation mechanism in the > > kernel. The most prominent use case is the ability to extend memcg > > stats (and histograms) by BPF programs. > > > > + Namhyung, > > I forgot to mention this in the office hour. The aggregation efficiency > problem is actually similar to Namhyung's work to use BPF programs > to aggregate perf counters. Check: > tools/perf/util/bpf_skel/bperf_cgroup.bpf.c > > Namhyung's solution is to walk up the cgroup tree on cgroup switch > events. This may not be as efficient as rstat flush logic, but I think it > is good enough for many use cases (unless the cgroup tree is very > deep). It also demonstrates how we can implement some cgroup > logic in BPF. > > I hope this helps. > > Thanks, > Song > > [...] Hi Song, Thanks so much for pointing this out. I have an idea of a less intrusive and more generic way to directly use rstat flushing in BPF programs, which basically makes BPF programs maintain their own stats and flushing logic and only using helpers to make calls to rstat (instead of a whole new rstat map). I will work on an RFC patch series for this. If this doesn't work out, I will probably fallback to handling the flushing completely in the BPF programs, similar to the example you provided (which is a lot of help, thanks!).