From: Kent Overstreet <kent.overstreet@linux.dev>
To: Qi Zheng <zhengqi.arch@bytedance.com>
Cc: Michal Hocko <mhocko@suse.com>,
Roman Gushchin <roman.gushchin@linux.dev>,
Muchun Song <muchun.song@linux.dev>,
Linux-MM <linux-mm@kvack.org>,
linux-kernel@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Dave Chinner <david@fromorbit.com>
Subject: Re: [PATCH 2/7] mm: shrinker: Add a .to_text() method for shrinkers
Date: Wed, 29 Nov 2023 23:14:44 -0500 [thread overview]
Message-ID: <20231130041444.7cbhbvkvwpcbbhyd@moria.home.lan> (raw)
In-Reply-To: <6f56c8f4-77e3-4ad7-a5f8-a6235b047137@bytedance.com>
On Thu, Nov 30, 2023 at 11:42:45AM +0800, Qi Zheng wrote:
> > Similarly for shrinkers, we're not going to be printing all of them -
> > the patchset picks the top 10 by objects and prints those. Could
> > probably be ~4, there's fewer shrinkers than slabs; also if we can get
> > shrinkers to report on memory owned in bytes, that will help too with
> > deciding what information is pertinent.
>
> I'm not worried about the shrinker's general data. What I'm worried
> about is the shrinker's private data. Except for the corresponding
> developers, others don't know the meaning of the private statistical
> data, and we have no control over the printing quantity and form of
> the private data. This may indeed cause OOM log confusion and failure
> to automatically parse. For this, any thoughts?
If a shrinker is responsible for the OOM, then that private state is
exactly what is needed to debug the issue.
I explained this earlier - shrinkers can skip reclaiming objects for a
variety of reasons; in bcachefs's case that could be trylock() failure,
an IO in flight, the node being dirty, and more. Unlock the system inode
shrinker, it's much too expensive to move objects on and off the
shrinker list whenever they're touched. Again, this all comes from real
world experience.
The show_mem report is already full of numbers with zero explanation of
how they're relevant for debugging OOMS; we really need to improve how
that is presented as well.
next prev parent reply other threads:[~2023-11-30 4:14 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-22 23:25 [PATCH 0/7] shrinker debugging improvements Kent Overstreet
2023-11-22 23:25 ` [PATCH 1/7] seq_buf: seq_buf_human_readable_u64() Kent Overstreet
2023-11-22 23:25 ` [PATCH 2/7] mm: shrinker: Add a .to_text() method for shrinkers Kent Overstreet
[not found] ` <deed9bb1-02b9-4e89-895b-38a84e5a9408@gmail.com>
2023-11-23 21:24 ` Kent Overstreet
2023-11-24 3:08 ` Qi Zheng
2023-11-25 0:30 ` Kent Overstreet
2023-11-28 3:27 ` Muchun Song
2023-11-28 3:53 ` Kent Overstreet
2023-11-28 6:23 ` Qi Zheng
2023-11-29 0:34 ` Roman Gushchin
2023-11-29 9:14 ` Michal Hocko
2023-11-29 23:11 ` Kent Overstreet
2023-11-30 3:09 ` Qi Zheng
2023-11-30 3:21 ` Kent Overstreet
2023-11-30 3:42 ` Qi Zheng
2023-11-30 4:14 ` Kent Overstreet [this message]
2023-11-30 19:01 ` Roman Gushchin
2023-12-01 0:00 ` Kent Overstreet
2023-12-01 1:18 ` Dave Chinner
2023-12-01 20:01 ` Roman Gushchin
2023-12-01 21:51 ` Kent Overstreet
2023-12-06 8:16 ` Dave Chinner
2023-12-06 19:13 ` Kent Overstreet
2023-12-09 1:44 ` Roman Gushchin
2023-12-09 2:04 ` Kent Overstreet
2023-11-30 8:14 ` Michal Hocko
2023-12-01 1:47 ` Kent Overstreet
2023-12-01 10:04 ` Michal Hocko
2023-12-01 21:25 ` Kent Overstreet
2023-12-04 10:33 ` Michal Hocko
2023-12-04 18:15 ` Kent Overstreet
2023-12-05 8:49 ` Michal Hocko
2023-12-05 23:21 ` Kent Overstreet
2023-11-24 11:46 ` kernel test robot
2023-11-28 10:01 ` Michal Hocko
2023-11-28 17:48 ` Kent Overstreet
2023-11-29 16:02 ` Michal Hocko
2023-11-29 22:36 ` Kent Overstreet
2023-11-22 23:25 ` [PATCH 3/7] mm: shrinker: Add new stats for .to_text() Kent Overstreet
2023-11-22 23:25 ` [PATCH 4/7] mm: Centralize & improve oom reporting in show_mem.c Kent Overstreet
2023-11-28 10:07 ` Michal Hocko
2023-11-28 17:54 ` Kent Overstreet
2023-11-29 8:59 ` Michal Hocko
2023-11-22 23:25 ` [PATCH 5/7] mm: shrinker: Add shrinker_to_text() to debugfs interface Kent Overstreet
2023-11-22 23:25 ` [PATCH 6/7] bcachefs: shrinker.to_text() methods Kent Overstreet
2023-11-22 23:25 ` [PATCH 7/7] bcachefs: add counters for failed shrinker reclaim Kent Overstreet
2023-11-28 9:59 ` [PATCH 0/7] shrinker debugging improvements Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231130041444.7cbhbvkvwpcbbhyd@moria.home.lan \
--to=kent.overstreet@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=david@fromorbit.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=muchun.song@linux.dev \
--cc=roman.gushchin@linux.dev \
--cc=zhengqi.arch@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox