From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47CA9C5321E for ; Mon, 26 Aug 2024 03:10:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D38D68D0040; Sun, 25 Aug 2024 23:10:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE9318D0029; Sun, 25 Aug 2024 23:10:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB1008D0040; Sun, 25 Aug 2024 23:10:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9CCC78D0029 for ; Sun, 25 Aug 2024 23:10:22 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 194D51A0A5B for ; Mon, 26 Aug 2024 03:10:22 +0000 (UTC) X-FDA: 82492918284.22.1082DCD Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf27.hostedemail.com (Postfix) with ESMTP id 24E7D40002 for ; Mon, 26 Aug 2024 03:10:19 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WQY2ZF7A; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724641734; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oZX6x/egC07tqWD7jsgxOkn/l2ecZwyFmUDqkxj05sI=; b=thEoWQbYS0hV65MiejFlhe+v8hTAplRFxxnieIHCuyNK7f0Oxf9Z6fWSKshHYRycUQGPPl MynsKPzN5242S6Wl2ULP3/WU9ngTpP2tX4LcMS4tnMxl3oQU/KNHQ8eXyo2pHsA3bjqBMR Hpv5sEcBcnAGfUBTpkAFOKcwNqA1u7g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724641734; a=rsa-sha256; cv=none; b=4+WIPwJmtbosSIe+dKOgE+RDY+O54z+LxP/VFhwk6TrfAton9EAnexU3O6pEc+hGoayaJu dljHBY6hK3kPkTInMnoYZgYz8jzzKN7Chg8Cgl4+yzdi/d171cFdm9JuqjOKnTmv4w6su3 aY5c4Kp841z06xYmju+PEiu5PAkoqgM= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WQY2ZF7A; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2d3da6d3f22so2947010a91.0 for ; Sun, 25 Aug 2024 20:10:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724641818; x=1725246618; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=oZX6x/egC07tqWD7jsgxOkn/l2ecZwyFmUDqkxj05sI=; b=WQY2ZF7AR1fjOfJRrDh9X1xSV2d39NtRutAnenvhAaqUifps2B2wxNZa5q07ZWlyH4 vUCOONZZIFx/DlhAzWVZ8NIkRnpVmoXhjLiqJD5rIRYe4GYDlqA5EcSl1FaVBMwJcI40 pYQT4L9haLKedpix2V65yQA2girwJ0MT+snTMvceNsG+/DTjDBWrFZsD4AlkXm0kNTfP JKNouvErblsbQzYZOi+KNs3qDuPO6uGJGL1jEQwhmiO5+U4OEp4xkyDMDmTVBdjqUO96 XCXWcLvp354XX9RQVFiN5WlLqLfS0doszExQyegWHeTR5SXKae/rL1kdMeWpfA5+h10i DlSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724641818; x=1725246618; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oZX6x/egC07tqWD7jsgxOkn/l2ecZwyFmUDqkxj05sI=; b=pf3pTaBDBRqw12OikoNZqS8mX/V8Oilo1i5iIrPHRcm9HSSWgC2IV/eYHTyEHkkeOy nLizYTtZAaxC1Af5GpzE8Nw67qnOSSZBuIAGVwJ5l8g0XFBGqrblZBtto9AVIKhadd8/ spdss18HK/onH2wnJKHCXy+OLCJqhuz2ZaQcyW6yvHw5fIMOBfVR4+gLcqe/56Op9puk dN8egjSLWVMvGQgq7Rz3TvjoNtox2HGIYE+1gicDgf6BhCJYKcIypx3Xr2/ZmaxqLkQi 5KOfF7s4xjFAQIN4GbaiXElW2tE4dkty/2wO8QUQy172V+Q1gzg7QI6SEmlL+zMqpaG0 7cDg== X-Forwarded-Encrypted: i=1; AJvYcCXrmlz+EU3r5KDQJekVGurukj7cMRKhnOFqfU2zdae9sMDLYNfcVwlveb9yLMLm05WPbr8aazwGFg==@kvack.org X-Gm-Message-State: AOJu0Yx59lQaIvKGUTpIU25Z4jyYAdwskUXVc1BDxGqP0F5ShN4dk7xm 6qZLvFkuYsQZte8U864yVAoAxJ7wWzpIhvXLi2Pq8lrks8STFdgkzg601Pngcm5NI6zBlTZ0hyu V X-Google-Smtp-Source: AGHT+IExSEXEeBc7eCz5/YTr9FjB6mRKMNWSRtKvcbkNd+JY/e0M+CtATZZ02S9q3FxIUNWQsqd7rw== X-Received: by 2002:a17:90b:115:b0:2d3:dcc1:f98f with SMTP id 98e67ed59e1d1-2d646bf282bmr9381893a91.10.1724641818564; Sun, 25 Aug 2024 20:10:18 -0700 (PDT) Received: from [10.4.59.158] ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d613af1496sm8560863a91.40.2024.08.25.20.10.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 25 Aug 2024 20:10:18 -0700 (PDT) Message-ID: <4733d230-0935-4bff-a17e-8c6735ad16b4@bytedance.com> Date: Mon, 26 Aug 2024 11:10:08 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 02/10] mm: shrinker: Add a .to_text() method for shrinkers Content-Language: en-US To: Kent Overstreet Cc: david@fromorbit.com, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Roman Gushchin References: <20240824191020.3170516-1-kent.overstreet@linux.dev> <20240824191020.3170516-3-kent.overstreet@linux.dev> <122be87e-132f-4944-88d9-3d13fd1050ad@bytedance.com> From: Qi Zheng In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: xxqgzcj9z39dc5tkm8s1b3fa4adqqxfm X-Rspamd-Queue-Id: 24E7D40002 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724641819-374037 X-HE-Meta: U2FsdGVkX1+XI7wUVByaDYChA4eBHwTCSK0YybNjv7WEutRZ+uDANAf30Die/7SY568wPPyPc+7JG0DETjioQ2efnSnDIpaeQf9JwGN0PXDECQAtgz+HkaxM6pBPVF+6z+qCl8ENGQK5tFkUEsQPI5OLgp0xXQXXPYfRaBk3m3f8PvSfrw0AbcENGkv6UXn+ZfhbYYES8m4UnEuYkA6cKxQ04X3Uypfyq0FcIlsAAKU8IfvNh/ylvjTmbJ9jFty3HEyPnRthETJPIEx5CbdLtVECtrO02OM2O0QYpr3JD3B70PNouhwZvkamjivQDR96cyIjzCXDomg3pKrqOrVaQCUmYyDI9k9FHn2P+fGMpxY1zqg+WqASbikrWrLTKvNN6mv+HV1GR/J8nD/orDulCqA8tTdoYM7lQM0mLzPhSC0CR54JaGOnpzxmOZ6PpUg9gzBojxtgeJd8HhMja6e/inckLPEJl06Bj4asvBJ7dK0kMDn5L+CoCo6Xp+QV9W/UjKxccTiGMDaCNDjxkkfhhiBOscV1dhlVBMNKmqVVDdjzIdUcJDjEeRbS/Cut/vmsMj2fnzWznIQe9WruqqO4sT0oskmebT0OQI2nprtum8Xk7NKdc1deQKEZuUC0ti/3FS52sHJHTBWIqZ8Fh5cuZF4s2JHcnWA00iEvNbO1pb1f6RYj4f454yMqQCC+hkMhNwjzIFgJNJRLjFoeTcxSI66lo3wxXgsO+8F9UYqAKtL5UclN/EdUPAC+6Wn/9xCL2LInZWaqxbacmIB/tOsbt9wc765Wg+fhrIv7X8ZOd/8zu3LNfvp9qS3lkAQGdmzG15RDt5Q1by3cFXMEzg8CNS0Bru6aku+QfHJUMAz8F2uC1uKRCXv7+ItCLrxCq+hOVA01qHfmZNFBgzzF2o4/B6xgSiax0Xq3Fr5wLwurdZ7fq+8wpeWutR7bO5HqPIdrjQ9sILXUSCZ0CzUNmuT 5PKTsjNi mcvq7CNIqBSx+g3vv6p1MScrt3CVQzoGukK4hfOQ/N+aGpM7FJoZWm+DNN7svixK3IArNIOP1OBPB2lkim4dgmnVWXsW1/NFvNER+T+o9qxbN61G7Zss5ENHSoxhdaoFOfaM5gk4zP9RCSfxJ3etaEjr5Ck3hb9BQuC3Wo+j+ajBAGnQsHbmJAkwa9YVG0Wce8A9W6HRF+ahekQRzckXBdwqJLnvePQzygtHqEAZJd434x/z9/rjVSzgFED7+nTI3IezNT1uLJ7iLl5DV+vdXaRI0L84Mvr9PZMBqPYE0VPKNKt5tI4l5L51HGTeWUA9IkRS0YxLGX+XqyWiI5L48CzoGrGDCVdiN+yV/IJBozCIL4yBkvz8x4NmnJXvtc3/GjJNfxwQIbaiBTR3ILJFAs/J0UmHD+izLPrXFe5YGGCdop5ZnzBrk3nxOq9SxQTNX/sV2NPnXwP/qPzFX4Ece626upJmKQb+nSU7NM1Z32osIod8mk3u7l2nuquAxYFS5bw9w30Sa6FDFRBuSFMlHMC8WaG6SWg4uRa9Q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/8/26 11:04, Kent Overstreet wrote: > On Mon, Aug 26, 2024 at 11:01:50AM GMT, Qi Zheng wrote: >> >> >> On 2024/8/25 03:10, Kent Overstreet wrote: >>> This adds a new callback method to shrinkers which they can use to >>> describe anything relevant to memory reclaim about their internal state, >>> for example object dirtyness. >>> >>> This patch also adds shrinkers_to_text(), which reports on the top 10 >>> shrinkers - by object count - in sorted order, to be used in OOM >>> reporting. >>> >>> Cc: Andrew Morton >>> Cc: Qi Zheng >>> Cc: Roman Gushchin >>> Cc: linux-mm@kvack.org >>> Signed-off-by: Kent Overstreet >>> --- >>> include/linux/shrinker.h | 7 +++- >>> mm/shrinker.c | 73 +++++++++++++++++++++++++++++++++++++++- >>> 2 files changed, 78 insertions(+), 2 deletions(-) >>> >>> diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h >>> index 1a00be90d93a..6193612617a1 100644 >>> --- a/include/linux/shrinker.h >>> +++ b/include/linux/shrinker.h >>> @@ -24,6 +24,8 @@ struct shrinker_info { >>> struct shrinker_info_unit *unit[]; >>> }; >>> +struct seq_buf; >>> + >>> /* >>> * This struct is used to pass information from page reclaim to the shrinkers. >>> * We consolidate the values for easier extension later. >>> @@ -80,10 +82,12 @@ struct shrink_control { >>> * @flags determine the shrinker abilities, like numa awareness >>> */ >>> struct shrinker { >>> + const char *name; >>> unsigned long (*count_objects)(struct shrinker *, >>> struct shrink_control *sc); >>> unsigned long (*scan_objects)(struct shrinker *, >>> struct shrink_control *sc); >>> + void (*to_text)(struct seq_buf *, struct shrinker *); >>> long batch; /* reclaim batch size, 0 = default */ >>> int seeks; /* seeks to recreate an obj */ >>> @@ -110,7 +114,6 @@ struct shrinker { >>> #endif >>> #ifdef CONFIG_SHRINKER_DEBUG >>> int debugfs_id; >>> - const char *name; >>> struct dentry *debugfs_entry; >>> #endif >>> /* objs pending delete, per node */ >>> @@ -135,6 +138,8 @@ __printf(2, 3) >>> struct shrinker *shrinker_alloc(unsigned int flags, const char *fmt, ...); >>> void shrinker_register(struct shrinker *shrinker); >>> void shrinker_free(struct shrinker *shrinker); >>> +void shrinker_to_text(struct seq_buf *, struct shrinker *); >>> +void shrinkers_to_text(struct seq_buf *); >>> static inline bool shrinker_try_get(struct shrinker *shrinker) >>> { >>> diff --git a/mm/shrinker.c b/mm/shrinker.c >>> index dc5d2a6fcfc4..ad52c269bb48 100644 >>> --- a/mm/shrinker.c >>> +++ b/mm/shrinker.c >>> @@ -1,8 +1,9 @@ >>> // SPDX-License-Identifier: GPL-2.0 >>> #include >>> +#include >>> #include >>> +#include >>> #include >>> -#include >>> #include >>> #include "internal.h" >>> @@ -807,3 +808,73 @@ void shrinker_free(struct shrinker *shrinker) >>> call_rcu(&shrinker->rcu, shrinker_free_rcu_cb); >>> } >>> EXPORT_SYMBOL_GPL(shrinker_free); >>> + >>> +void shrinker_to_text(struct seq_buf *out, struct shrinker *shrinker) >>> +{ >>> + struct shrink_control sc = { .gfp_mask = GFP_KERNEL, }; >>> + >>> + seq_buf_puts(out, shrinker->name); >>> + seq_buf_printf(out, " objects: %lu\n", shrinker->count_objects(shrinker, &sc)); >>> + >>> + if (shrinker->to_text) { >>> + shrinker->to_text(out, shrinker); >>> + seq_buf_puts(out, "\n"); >>> + } >>> +} >>> + >>> +/** >>> + * shrinkers_to_text - Report on shrinkers with highest usage >>> + * >>> + * This reports on the top 10 shrinkers, by object counts, in sorted order: >>> + * intended to be used for OOM reporting. >>> + */ >>> +void shrinkers_to_text(struct seq_buf *out) >>> +{ >>> + struct shrinker *shrinker; >>> + struct shrinker_by_mem { >>> + struct shrinker *shrinker; >>> + unsigned long mem; >>> + } shrinkers_by_mem[10]; >>> + int i, nr = 0; >>> + >>> + if (!mutex_trylock(&shrinker_mutex)) { >>> + seq_buf_puts(out, "(couldn't take shrinker lock)"); >>> + return; >>> + } >> >> I remember I pointed out that the RCU + refcount method should be used >> here. Otherwise you will block other shrinkers from >> registering/unregistering, etc. > > The more complex iteration isn't needed here - this is a slowpath > function and we're not doing anything blocking inside it. The shrinker list may be very long, and can you guarantee that shrinkers other than bcachefs won't do anything blocking? ;)