From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18208C433EF for ; Tue, 19 Apr 2022 18:20:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D2406B0072; Tue, 19 Apr 2022 14:20:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 881C06B0073; Tue, 19 Apr 2022 14:20:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 720D86B0074; Tue, 19 Apr 2022 14:20:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 622FF6B0072 for ; Tue, 19 Apr 2022 14:20:34 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3023E24E for ; Tue, 19 Apr 2022 18:20:34 +0000 (UTC) X-FDA: 79374443988.14.D5FE612 Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf02.hostedemail.com (Postfix) with ESMTP id 1CD0B80016 for ; Tue, 19 Apr 2022 18:20:32 +0000 (UTC) Received: by mail-qt1-f182.google.com with SMTP id d14so4964931qtw.5 for ; Tue, 19 Apr 2022 11:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=zeR3V0x/d5dS+XbGlGq1iYQwImzuhf524PVpciJ0wyA=; b=f5MU9YQh2Eunm+v3zEufTrXY3X1K+ZReCWqTSryDhoUg5tqzZbCHuntywVLW4Nrj6T eS3a8FNZ7JhNWz/Mcwt2BdW20BCxmuxR4dA1Tx9M2lJpyDUpNkVwjmYYdjIQbexud61h NSY7ItILuCfFvzk42D8a9+0k9Wdh9pAlc4KDk8QjCQMjFOk7zNYPu35ohvTft8tkAPIm yl9Oxg7/5+HHzCfvXA9V8x6jH4RdwXF/30DxlSar4NPGGNqDzl3zU1NWAcS2oA8FbQ7d r/RdKJFFc3EgwU2+2jeXGdvgDEPMVrIdAZU/E8VpyzSw76zEcrsB8YCy9U4f+rtt1VLJ qq6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=zeR3V0x/d5dS+XbGlGq1iYQwImzuhf524PVpciJ0wyA=; b=0WBXpReF11GvpGoTj2npNZysx2RQm0K4VmCsZYMpHTe4iaq4eqwyMC7Oh2DQ7EJDIw r7ydPTRy9uciaxE/LFpVemigJozROqtqNi/cNQid95CodfWYm/uZsyekilxT6WYiIyJ0 D93Jqct+7DakitkWoz3NgwAdeEVnFX+UMM2biuktQYT4fh0CzM9eRskSEaBufacfOePz pJRfGoWPMRohX+FNlBp0PeWcF5WDkOZ4ao6h4hzqyLlIHOYXmroIujNixyT8ODC6ChO9 W14pQHnF5BuUoENy79SzuAHOtITy+c8qkE/+avTF25o9UJhWw9QKEcYTGnoimjIW5Jch iJWg== X-Gm-Message-State: AOAM531yd/MBjwC8JQP1lsKUxBmm/5BWbK73PU4jb23JOm0aJ3mf/p05 majyp3ZSOuES5aGSIvTSTw== X-Google-Smtp-Source: ABdhPJyWhY5AlGyL4nF/Zmhqbckoo3GR+ES0yQigZFUARWQO7ZzpOXeGJ8v+v296DWXyfRUh4KlRZw== X-Received: by 2002:ac8:578b:0:b0:2e2:324a:7b6c with SMTP id v11-20020ac8578b000000b002e2324a7b6cmr11081009qta.267.1650392432986; Tue, 19 Apr 2022 11:20:32 -0700 (PDT) Received: from moria.home.lan (c-73-219-103-14.hsd1.vt.comcast.net. [73.219.103.14]) by smtp.gmail.com with ESMTPSA id s18-20020a05622a1a9200b002f335693f4bsm400640qtc.38.2022.04.19.11.20.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Apr 2022 11:20:32 -0700 (PDT) Date: Tue, 19 Apr 2022 14:20:30 -0400 From: Kent Overstreet To: Roman Gushchin Cc: linux-mm@kvack.org, Andrew Morton , Dave Chinner , linux-kernel@vger.kernel.org, Johannes Weiner , Michal Hocko , Shakeel Butt , Yang Shi Subject: Re: [PATCH rfc 0/5] mm: introduce shrinker sysfs interface Message-ID: <20220419182030.idqqmtim4slhbked@moria.home.lan> References: <20220416002756.4087977-1-roman.gushchin@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220416002756.4087977-1-roman.gushchin@linux.dev> X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1CD0B80016 X-Stat-Signature: roahtcr1hgdqirdojp5yfhtyihsc1y1d Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=f5MU9YQh; spf=pass (imf02.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1650392432-459177 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 15, 2022 at 05:27:51PM -0700, Roman Gushchin wrote: > There are 50+ different shrinkers in the kernel, many with their own bells and > whistles. Under the memory pressure the kernel applies some pressure on each of > them in the order of which they were created/registered in the system. Some > of them can contain only few objects, some can be quite large. Some can be > effective at reclaiming memory, some not. > > The only existing debugging mechanism is a couple of tracepoints in > do_shrink_slab(): mm_shrink_slab_start and mm_shrink_slab_end. They aren't > covering everything though: shrinkers which report 0 objects will never show up, > there is no support for memcg-aware shrinkers. Shrinkers are identified by their > scan function, which is not always enough (e.g. hard to guess which super > block's shrinker it is having only "super_cache_scan"). They are a passive > mechanism: there is no way to call into counting and scanning of an individual > shrinker and profile it. > > To provide a better visibility and debug options for memory shrinkers > this patchset introduces a /sys/kernel/shrinker interface, to some extent > similar to /sys/kernel/slab. > > For each shrinker registered in the system a folder is created. The folder > contains "count" and "scan" files, which allow to trigger count_objects() > and scan_objects() callbacks. For memcg-aware and numa-aware shrinkers > count_memcg, scan_memcg, count_node, scan_node, count_memcg_node > and scan_memcg_node are additionally provided. They allow to get per-memcg > and/or per-node object count and shrink only a specific memcg/node. Cool! I've been starting to sketch out some shrinker improvements of my own, perhaps we could combine efforts. The issue I've been targeting is that when we hit an OOM, we currently don't get a lot of useful information - shrinkers ought to be included, and we really want information on shrinker's internal state (e.g. object dirtyness) if we're to have a chance at understanding why memory isn't getting reclaimed. https://evilpiepirate.org/git/bcachefs.git/log/?h=shrinker_to_text This adds a .to_text() method - a pretty-printer - that shrinkers can implement, and then on OOM we report on the top 10 shrinkers by memory usage, in sorted order. Another thing I'd like to do is have shrinkers report usage not just in object counts but in bytes; I think it should be obvious why that's desirable. Maybe we could have a memory-reporting-and-shrinker-improvements session at LSF? I'd love to do some collective brainstorming and get some real momementum going in this area.