From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE4C5C433EF for ; Wed, 16 Mar 2022 21:16:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 39F458D0002; Wed, 16 Mar 2022 17:16:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 34DA16B0073; Wed, 16 Mar 2022 17:16:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F10F8D0002; Wed, 16 Mar 2022 17:16:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0076.hostedemail.com [216.40.44.76]) by kanga.kvack.org (Postfix) with ESMTP id 0DD5F6B0072 for ; Wed, 16 Mar 2022 17:16:46 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B4808944CD for ; Wed, 16 Mar 2022 21:16:45 +0000 (UTC) X-FDA: 79251508770.27.4BE1B4A Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf30.hostedemail.com (Postfix) with ESMTP id 4E30780008 for ; Wed, 16 Mar 2022 21:16:45 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id v4so3245713pjh.2 for ; Wed, 16 Mar 2022 14:16:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=b+Cax4UASctLqfboejSMaNs0ZkNZ8/JZnvVprm0tRB8=; b=Ll5AyvuidasmDDHI5sY3pS+tgn3T2IHtEQAUXYk3oj0WUWc0BQAQ8u4q8aId5Uew9l KCAdxbmVmzzoED8VR9uuNtfQib1+Ys/VTJjyMusV7/pvXAJIzVuK4GUqkc4N0CtfvpoZ 1FdaP48riScfIiUy+XHMKYIql+IQI44DQtsj0VoCMzgyZjW3LiLiAAMc9b1a+xIDfBwj hZ6sPzX4ubeHc9GUQJ7e+WQ3f/xMwATFFsfYTnYu9LzquUU+WNvptZZwELVV63LL+F0Y IySvyO+SU/Fu44O3gFd8HaKzMfgFHNSPAMdz1+xUadILOffkzOHVdJMLlClJt8Gqw6wp NYeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=b+Cax4UASctLqfboejSMaNs0ZkNZ8/JZnvVprm0tRB8=; b=yfDfgp25IKllJpkjsWVzoqbeVG3UFZXEwlw4fdBs91OKddUMLikDSK+Z3ztvb5tXSP gPIB7/8fGaspcq5XaScC5t67dTr+nOC5Gbt9MjeiJMlHW7ajKBeD++7yUrTzkAnodCHx w/77J70y9xRkGLm68+KfiFiO4CRfqBjLlzv+WYpZ+jCtiPSm8LPVGsqx/sTIg4h9dwSz FHmM6UlG9MkW+RR9ESQ4Fw0a3YA4nM2FTMs+VPpVsMsc4OV7zEKmnY4GvC4DMPVod1Z9 nYuO+lHrKXKEhwU7mOd8oD0DFUFz0MzUgHApBUsgGE0AMfC9+7288SmoFDY8is7XH5GW 4m7A== X-Gm-Message-State: AOAM531LJAYs6P1rxeUIjxjx8Ch+yK+tSHO8uge9rkoIf+AG369pLdSK Do+7N6k3ifn1/tV4Kh3T9OS3g8TPFoDWcmS4EPBUBg== X-Google-Smtp-Source: ABdhPJw1E/II4kODcCimOQmSevRtgL1x6q1Y8CeiT3dQDz0jYCA/ScZg6gMm/W3Io9oQkh9VPDAs/zWfBvQCSx98rvs= X-Received: by 2002:a17:902:7fc1:b0:151:f80f:494a with SMTP id t1-20020a1709027fc100b00151f80f494amr1330401plb.81.1647465403869; Wed, 16 Mar 2022 14:16:43 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yosry Ahmed Date: Wed, 16 Mar 2022 14:16:07 -0700 Message-ID: Subject: Re: Questions about memcg_slabinfo.py drgn script To: Roman Gushchin Cc: guro@fb.com, vvs@virtuozzo.com, Michal Hocko , Andrew Morton , Shakeel Butt , Johannes Weiner , cl@linux.com, Linux-MM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4E30780008 X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ll5Ayvui; spf=pass (imf30.hostedemail.com: domain of yosryahmed@google.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: ufuk88axwg3jkkn1oudd7xrsh6ei5g4q X-Rspamd-Server: rspam07 X-HE-Tag: 1647465405-521720 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Mar 10, 2022 at 3:41 PM Roman Gushchin wrote: > > On Thu, Mar 10, 2022 at 12:21:44PM -0800, Yosry Ahmed wrote: > > Hi everyone, > > > > I was looking at the memcg_slabinfo.py drgn script that offers a > > replacement to the deprecated memory.kmem.slabinfo. I had some > > questions about how it collects the memcg slab stats: > > Hi Yosry! > > First, I have to admit that I haven't spent too much time optimizing > this script for speed. So, there are almost certainly available opportunities > to enhance it and patches are highly welcome. > > > > > 1. Why does the script loop through all struct pages on the system? > > Wouldn't it be more efficient to loop for every kmem_cache, for every > > online kmem_cache_node, then loop through slabs_free, slabs_full, and > > slabs_partial lists? > > It's somewhat tricky with SLUB because of per-cpu partial pages (I'm less > familiar with SLAB). In theory, we have a single-linked list of such pages, > but idk if we can reliably traverse it (given that it will be changed > concurrently). We also will be way more dependent on SLUB internals. > However it still might be a good optimization. > > > > > This seems more consistent with how /proc/slabinfo works, and more > > efficient. I tested this on SLAB using a crash script as I am unable > > to run drgn on my current setup. I am not sure how correct this would > > be for SLUB though. > > /proc/slabinfo has its own weaknesses, e.g. it shows systematically wrong > numbers for slab utilization because of how it handles per-cpu partial pages > (on SLUB). > Honestly I haven't looked into this for SLUB, but it seems like it is a valid optimization for SLAB. At least it is equivalent to /proc/slabinfo which I assume is somehow accurate for SLAB. > > > > 2. Before looping through pages, why does the script collect all > > objcgs belonging to the desired memcg in a set, and then test every > > objcg in a slab page to see whether it belongs to that memcg. Wouldn't > > it be easier to just check objcg->memcg? AFAICT this gets updated as > > well when the objcg is reparented. > > I can't think of any good reason now, however it's not obviously faster > (I guess dereferencing of a pointer in drgn can be more expensive than doing > few "local" comparison, something to measure). > If it is faster, it will be a good enhancement. > You are right (at least for crash) it is more expensive to dereference the obj_cgroup pointers! > > > > Sorry for my ignorance if any of the assumptions I made are incorrect. > > I just wanted to get more understanding of the implementation > > decisions taken while writing the script. > > You're welcome!