From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6471C5320E for ; Mon, 26 Aug 2024 03:02:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CA926B02D6; Sun, 25 Aug 2024 23:02:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6797E6B02D7; Sun, 25 Aug 2024 23:02:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51A0E6B02D8; Sun, 25 Aug 2024 23:02:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F0DE6B02D6 for ; Sun, 25 Aug 2024 23:02:07 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B594BC0A9F for ; Mon, 26 Aug 2024 03:02:06 +0000 (UTC) X-FDA: 82492897452.07.D6158E9 Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com [209.85.161.52]) by imf07.hostedemail.com (Postfix) with ESMTP id 5FD404000C for ; Mon, 26 Aug 2024 03:02:03 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=CinmAsWW; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.52 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724641229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sqMidpjTyEAXSrDATi1bJfGrokKfF0LLoZX42RYAeCY=; b=vbBCA69JeUpQVDGu4LpAQFGCMLKjD5KIae+G0usYpMST/O1Wjv42kjZBzIqG+scRhpkWGz Q7VIbLADdwIpr2xw+2z51EwQyAzjRH3LquGCCKYgHzwl/2efwCR7FUpfKa3y5qLLptj91d dgj1a6ckVZEXJnTrvuAB2FhqrsEN9Wc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724641229; a=rsa-sha256; cv=none; b=t69p7BIJEWsPXObDD1eDkfFhAhffn8yS914It+QXVWuad99vnIqBfUhA8OAkEmZ4js24Ci xTdwQv9I0VCh+sbI8Of76ynJbIcuF5f4HjvIcxXnqah8/PUWI3BMRIB5fKXJqZMUeQhCp/ bZ+HB4jwL5yVlfPdFc44QVKVbNeU9hs= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=CinmAsWW; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.52 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-oo1-f52.google.com with SMTP id 006d021491bc7-5d5b1e33fa8so2927673eaf.3 for ; Sun, 25 Aug 2024 20:02:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724641322; x=1725246122; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=sqMidpjTyEAXSrDATi1bJfGrokKfF0LLoZX42RYAeCY=; b=CinmAsWWpHbA93vCBW3RTSpfd0+ZsR0iZuhKGP/RBMPqsWfjtIqnMKy9oMggMsO/zy HPmi51KzgQTQT1RrtmNytD9FmuIg5k6LX8B1sFC5aMg7yEWhaMOQIn/Ohn+RbWMULP01 JrB0gpI6EzgzdLCMAYQo+lKabjds2FK34iDi+gFbcKEQwM0mbC6NtYXpvOvT12K9gewn dbSDx5tgnOtE/ROX5OEbJovD03PxriGQJ+VHtSS0kn15HGj6SCzLr0U7eSUr+epqPH4b CIs2e0MiY0cURZAcq3k6ZFUwsSBfYdGKeDn6mowjN7j1Uy/JCUcVG7FJinwFQqW2dIrK eAhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724641322; x=1725246122; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sqMidpjTyEAXSrDATi1bJfGrokKfF0LLoZX42RYAeCY=; b=VXR3HXPkRCAfBmcc8Fv7kMGUhOebN3rvopcLWSJe3HhoagtBO8Shgl+6k+8IkIXhFX mvJz7XETehgnClx/6ZlTvGq1Q+5wRWsL7+ABM6+MuQK7Wupvyshk6XuYMLOkn2hLtTTG BbHj+YIn6/NTObOO22ie0SIoiV2EhwPwguGnVb7d2et6553IGUZdnztbLQGWFTFnipFj i97xA5PX2gyYeBvpQ6wf4aNfFOf4OvIUglQQCsZEisU8+4AZF35UrRCKlXYBYkjZSdvX x8FYAFg1q78NFgD649HwPpr0edhCLGomFXp4LZojj3V0d+20w6OAjljXeNmCqA1k5TMu QDLA== X-Forwarded-Encrypted: i=1; AJvYcCW0tPJM1FiIEWZ8ls1rY0VQ0tUIAcoQ6zcts9RVZtAmxg0JZM/HVTuvuHiFmNKfaBJH5D4WHPiscQ==@kvack.org X-Gm-Message-State: AOJu0YyH2jr8Y6fv1+8IFmQPEo6nR7r413Y+GEmyjY0GdISuMvkKpmFl 5nxzSN2zYNtEVbkqsaw5f/VLoKo9/yJphD4cbtgQpgaRlRpBbyBFaZXcamJmBBk= X-Google-Smtp-Source: AGHT+IFSrd3sCaL6bPpSE2TcGHIxBmpHgqAk+jJoG4ozmh//3gCUtZ8Hmdck95n6sAftDhh2tcOgGA== X-Received: by 2002:a05:6870:9708:b0:26f:ddfa:3564 with SMTP id 586e51a60fabf-273e63d6d03mr11605808fac.6.1724641322133; Sun, 25 Aug 2024 20:02:02 -0700 (PDT) Received: from [10.4.59.158] ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cda0ada9adsm5337433a12.26.2024.08.25.20.01.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 25 Aug 2024 20:02:01 -0700 (PDT) Message-ID: <122be87e-132f-4944-88d9-3d13fd1050ad@bytedance.com> Date: Mon, 26 Aug 2024 11:01:50 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 02/10] mm: shrinker: Add a .to_text() method for shrinkers Content-Language: en-US To: Kent Overstreet Cc: david@fromorbit.com, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Qi Zheng , Roman Gushchin References: <20240824191020.3170516-1-kent.overstreet@linux.dev> <20240824191020.3170516-3-kent.overstreet@linux.dev> From: Qi Zheng In-Reply-To: <20240824191020.3170516-3-kent.overstreet@linux.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5FD404000C X-Stat-Signature: ury8pcjtyik99h6ecbas1ke5a8zj56o5 X-Rspam-User: X-HE-Tag: 1724641323-187411 X-HE-Meta: U2FsdGVkX19lZnzuqLVPqfNb47l6XGK6AIpkhts9K2VkIFRdVc9BVeCKDYVdfx6wMj8YNFfVljnwE/li3OOKwOmfdg7C32Sc+UVsVWJAmK6qN5wE3MmhRkvzrl/khWla1zhRSEkypY2lL5RSaJEvz0mQ1fDM1uhtE2M3CNx1aCP3ztsUtWpFguWcLq2eX4xAVc7NUQSa5u2REfgZ2pczc/Yqyt3XCXu7NhcFuusRhNtyvc6S4bgbACZ6/GQ8ISf7r3PSnyPQSBSAytHxshBcGyPPncXzozEi/NT8Hztq9/JvjhJ+KypSvtQjCC+X0/78ptGg64wNgbv07EYVEK0xXkscOcu/ZIZXbnpuCzMWlUdaWknSzUOcGxDVDRlsI+bYMy2lRqamcp7/v4z62hhHRJ9L39MDtSPXx/l+yiA9MmQglSLjKwlxxjuetyHFE2JoEQwSdedoIFef7y8psHWWT67hBeNixeO0SNz1AI45L5jVSaYUfJz0lAy32wa26o6gTm9TfnrbKDk5iXzcrhxBsm4gA6IOZxwkXEEC/CMXjNRG40GeWBjFqy/Q66AdmOrwoMDRMgz/eY3N9rttJEijRG3/oiKty6GGYfBpP1guz11iagL8MyWmlZXIRFJWS7EqFZ3XpM+X9pjjlJJ8odJa8W8kgMvdUFuNUhb6Gy000dlAF/r40BmbasAqaV51S/1IYLeRkPsgnRDbCemrip1bn2Bc/84er417PUqfzJtF6bEVo4TdXSAUjQRAtTA0+ZsoxZFfvlPJ+Z1mj0c8n6cfQfaE0RzmNL1mMFrxzgMe8mWG83jf9KylMSzgLi4AkWj24eEF7BmUNoOvyxd/v+sx2xkP28WM6OtonREQ7dOje3f7VMZ0+O7gRofV6EWV4dhxSDS08W891M0GSW4fuGJl+8/hBLlPP+8+nOMkVzV3sCgOnVsrPzA+1vV00/MpBwq38TlxloxUwZ1xpltdhZe LU1erb2+ tuxMGztQn9Jb+shlkaM970eXyQu53SuxHxxEY9TTgAd7mYKr/eLu2MmVMCIrd0pe4wAoxPO83KiUnNS+uyy9U5zzHwXoJbwueRLxWtZauRHAZtTN/MT6liaIXDML7FZ1TcMU9iW8PLwnFkH370oiUhYa8eCofRnWDFxU2JzUGiIKSdofwLMvHwL+6LPQKFfIHYgKgyf+YTY/dZvtKAjOlkzksDzObG5xibwMEVFrzW/+q00PMfpId/4QXK85JVW+EmH+YsgUxiD6O93glyx+butY5O1YD4Qbf+DoHqzsRwxeNH64NAZf9RKzqsrNj0+We3v2J5peeR7N50N6b/0Yw9M+nka6DbZnwOdPVRa/GMyx8qiccPwAh+AGJy+3c7+u0AUXaTx0E2jQ/67p3UhHJmw7+AC8NftcH41PTDJW7gXh7+qlTSSkjGnsq50rDsAyinq2du8PJlQyMM38ptKnyBYMVOCIFPkyAavcDw3RUz7YcMS45ebUGW5FiKIeDvdzZFDJnZtoiod1T8ixpoiLWuHGNG3oBjwuWSye6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/8/25 03:10, Kent Overstreet wrote: > This adds a new callback method to shrinkers which they can use to > describe anything relevant to memory reclaim about their internal state, > for example object dirtyness. > > This patch also adds shrinkers_to_text(), which reports on the top 10 > shrinkers - by object count - in sorted order, to be used in OOM > reporting. > > Cc: Andrew Morton > Cc: Qi Zheng > Cc: Roman Gushchin > Cc: linux-mm@kvack.org > Signed-off-by: Kent Overstreet > --- > include/linux/shrinker.h | 7 +++- > mm/shrinker.c | 73 +++++++++++++++++++++++++++++++++++++++- > 2 files changed, 78 insertions(+), 2 deletions(-) > > diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h > index 1a00be90d93a..6193612617a1 100644 > --- a/include/linux/shrinker.h > +++ b/include/linux/shrinker.h > @@ -24,6 +24,8 @@ struct shrinker_info { > struct shrinker_info_unit *unit[]; > }; > > +struct seq_buf; > + > /* > * This struct is used to pass information from page reclaim to the shrinkers. > * We consolidate the values for easier extension later. > @@ -80,10 +82,12 @@ struct shrink_control { > * @flags determine the shrinker abilities, like numa awareness > */ > struct shrinker { > + const char *name; > unsigned long (*count_objects)(struct shrinker *, > struct shrink_control *sc); > unsigned long (*scan_objects)(struct shrinker *, > struct shrink_control *sc); > + void (*to_text)(struct seq_buf *, struct shrinker *); > > long batch; /* reclaim batch size, 0 = default */ > int seeks; /* seeks to recreate an obj */ > @@ -110,7 +114,6 @@ struct shrinker { > #endif > #ifdef CONFIG_SHRINKER_DEBUG > int debugfs_id; > - const char *name; > struct dentry *debugfs_entry; > #endif > /* objs pending delete, per node */ > @@ -135,6 +138,8 @@ __printf(2, 3) > struct shrinker *shrinker_alloc(unsigned int flags, const char *fmt, ...); > void shrinker_register(struct shrinker *shrinker); > void shrinker_free(struct shrinker *shrinker); > +void shrinker_to_text(struct seq_buf *, struct shrinker *); > +void shrinkers_to_text(struct seq_buf *); > > static inline bool shrinker_try_get(struct shrinker *shrinker) > { > diff --git a/mm/shrinker.c b/mm/shrinker.c > index dc5d2a6fcfc4..ad52c269bb48 100644 > --- a/mm/shrinker.c > +++ b/mm/shrinker.c > @@ -1,8 +1,9 @@ > // SPDX-License-Identifier: GPL-2.0 > #include > +#include > #include > +#include > #include > -#include > #include > > #include "internal.h" > @@ -807,3 +808,73 @@ void shrinker_free(struct shrinker *shrinker) > call_rcu(&shrinker->rcu, shrinker_free_rcu_cb); > } > EXPORT_SYMBOL_GPL(shrinker_free); > + > +void shrinker_to_text(struct seq_buf *out, struct shrinker *shrinker) > +{ > + struct shrink_control sc = { .gfp_mask = GFP_KERNEL, }; > + > + seq_buf_puts(out, shrinker->name); > + seq_buf_printf(out, " objects: %lu\n", shrinker->count_objects(shrinker, &sc)); > + > + if (shrinker->to_text) { > + shrinker->to_text(out, shrinker); > + seq_buf_puts(out, "\n"); > + } > +} > + > +/** > + * shrinkers_to_text - Report on shrinkers with highest usage > + * > + * This reports on the top 10 shrinkers, by object counts, in sorted order: > + * intended to be used for OOM reporting. > + */ > +void shrinkers_to_text(struct seq_buf *out) > +{ > + struct shrinker *shrinker; > + struct shrinker_by_mem { > + struct shrinker *shrinker; > + unsigned long mem; > + } shrinkers_by_mem[10]; > + int i, nr = 0; > + > + if (!mutex_trylock(&shrinker_mutex)) { > + seq_buf_puts(out, "(couldn't take shrinker lock)"); > + return; > + } I remember I pointed out that the RCU + refcount method should be used here. Otherwise you will block other shrinkers from registering/unregistering, etc. > + > + list_for_each_entry(shrinker, &shrinker_list, list) { > + struct shrink_control sc = { .gfp_mask = GFP_KERNEL, }; > + unsigned long mem = shrinker->count_objects(shrinker, &sc); > + > + if (!mem || mem == SHRINK_STOP || mem == SHRINK_EMPTY) > + continue; > + > + for (i = 0; i < nr; i++) > + if (mem < shrinkers_by_mem[i].mem) > + break; > + > + if (nr < ARRAY_SIZE(shrinkers_by_mem)) { > + memmove(&shrinkers_by_mem[i + 1], > + &shrinkers_by_mem[i], > + sizeof(shrinkers_by_mem[0]) * (nr - i)); > + nr++; > + } else if (i) { > + i--; > + memmove(&shrinkers_by_mem[0], > + &shrinkers_by_mem[1], > + sizeof(shrinkers_by_mem[0]) * i); > + } else { > + continue; > + } > + > + shrinkers_by_mem[i] = (struct shrinker_by_mem) { > + .shrinker = shrinker, > + .mem = mem, > + }; > + } > + > + for (i = nr - 1; i >= 0; --i) > + shrinker_to_text(out, shrinkers_by_mem[i].shrinker); > + > + mutex_unlock(&shrinker_mutex); > +}