From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08FE4C433EF for ; Wed, 20 Apr 2022 23:23:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E5EB6B0071; Wed, 20 Apr 2022 19:23:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56C406B0073; Wed, 20 Apr 2022 19:23:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EA1C6B0074; Wed, 20 Apr 2022 19:23:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 2B66A6B0071 for ; Wed, 20 Apr 2022 19:23:20 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 039B3265E0 for ; Wed, 20 Apr 2022 23:23:19 +0000 (UTC) X-FDA: 79378835760.31.2628FDF Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by imf03.hostedemail.com (Postfix) with ESMTP id C1A5B2000C for ; Wed, 20 Apr 2022 23:23:17 +0000 (UTC) Date: Wed, 20 Apr 2022 16:23:10 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1650496997; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jy2W+tJC4EgncPCr4pDQKFAbzSZdFCIXIUs8dOXozQY=; b=slSbIIONuIW5IsYAUO6/Ak9/Uy2xDxxzAglPdGlVUyne4pMkM/92eOlfHPViy6tlnZK58z fq9TeH04R6WXGgidq4F/lr/VQTY65o2Js/Sj+Ac7Qph0h4M8U5Gb7DstJ/CVptYaY068v8 5QJy8+uz8fysOl5azONnmvQ5kYC2EIk= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Yang Shi Cc: Linux MM , Andrew Morton , Dave Chinner , Linux Kernel Mailing List , Johannes Weiner , Michal Hocko , Shakeel Butt Subject: Re: [PATCH rfc 0/5] mm: introduce shrinker sysfs interface Message-ID: References: <20220416002756.4087977-1-roman.gushchin@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-Stat-Signature: cn9nnmgt8aakmxe73b3xrt1tep367nqp X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=slSbIION; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf03.hostedemail.com: domain of roman.gushchin@linux.dev designates 94.23.1.103 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C1A5B2000C X-HE-Tag: 1650496997-639146 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 20, 2022 at 03:24:49PM -0700, Yang Shi wrote: > On Fri, Apr 15, 2022 at 5:28 PM Roman Gushchin wrote: > > > > There are 50+ different shrinkers in the kernel, many with their own bells and > > whistles. Under the memory pressure the kernel applies some pressure on each of > > them in the order of which they were created/registered in the system. Some > > of them can contain only few objects, some can be quite large. Some can be > > effective at reclaiming memory, some not. > > > > The only existing debugging mechanism is a couple of tracepoints in > > do_shrink_slab(): mm_shrink_slab_start and mm_shrink_slab_end. They aren't > > covering everything though: shrinkers which report 0 objects will never show up, > > there is no support for memcg-aware shrinkers. Shrinkers are identified by their > > scan function, which is not always enough (e.g. hard to guess which super > > block's shrinker it is having only "super_cache_scan"). They are a passive > > mechanism: there is no way to call into counting and scanning of an individual > > shrinker and profile it. > > > > To provide a better visibility and debug options for memory shrinkers > > this patchset introduces a /sys/kernel/shrinker interface, to some extent > > similar to /sys/kernel/slab. > > > > For each shrinker registered in the system a folder is created. The folder > > contains "count" and "scan" files, which allow to trigger count_objects() > > and scan_objects() callbacks. For memcg-aware and numa-aware shrinkers > > count_memcg, scan_memcg, count_node, scan_node, count_memcg_node > > and scan_memcg_node are additionally provided. They allow to get per-memcg > > and/or per-node object count and shrink only a specific memcg/node. > > > > To make debugging more pleasant, the patchset also names all shrinkers, > > so that sysfs entries can have more meaningful names. > > > > Usage examples: > > Thanks, Roman. A follow-up question, why do we have to implement this > in kernel if we just count the objects? It seems userspace tools could > achieve it too, for example, drgn :-). Actually I did write a drgn > script for debugging a problem a few months ago, which iterates > specific memcg's lru_list to count the objects by their state. Good question! It's because not all shrinkers are lru_list-based and even some lru_list-based are implementing a custom logic on top of it, e.g. shadow nodes. So there is no simple way to get the count from a generic shrinker. Also I want to be able to reclaim individual shrinkers from userspace (e.g. to profile how effective the shrinking is). Thanks!