linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Liu Bo <bo.liu@linux.alibaba.com>
To: linux-ext4@vger.kernel.org
Cc: fengguang.wu@intel.com, tj@kernel.org, jack@suse.cz,
	cgroups@vger.kernel.org, gthelen@google.com, linux-mm@kvack.org,
	yang.shi@linux.alibaba.com
Subject: ext4 hang and per-memcg dirty throttling
Date: Tue, 11 Sep 2018 17:10:55 -0700	[thread overview]
Message-ID: <20180912001054.bu3x3xwukusnsa26@US-160370MP2.local> (raw)

Hi,

With ext4's data=ordered mode and the underlying blk throttle setting, we
can easily run to hang,

1.
mount /dev/sdc /mnt -odata=ordered
2.
mkdir /sys/fs/cgroup/unified/cg
3.
echo "+io" > /sys/fs/cgroup/unified/cgroup.subtree_control
4.
echo "`cat /sys/block/sdc/dev` wbps=$((1 << 20))" > /sys/fs/cgroup/unified/cg/io.max
5.
echo $$ >  /sys/fs/cgroup/unified/cg/cgroup.procs
6.
// background dirtier
xfs_io -f -c "pwrite 0 1G" $M/dummy &
7.
echo $$ > /sys/fs/cgroup/unified/cgroup.procs
8.
// issue synchronous IO
for i in `seq 1 100`;
do
    xfs_io -f -s -c "pwrite 0 4k" $M/foo > /dev/null
done


And the hang is like

      [jbd2-sdc]
jbd2_journal_commit_transaction                              
  journal_submit_data_buffers
    # file 'dummy' has been written by writeback kthread
  journal_finish_inode_data_buffers
    # wait on page's writeback

Then all the operations of ext4 which need to start journal will have
to wait until journal committing transaction completes.

Since there is no per-memcg throttling, such as dirty ratio or dirty
bytes throttle, balance_dirty_pages() may not be able to slow down the
background dirtier task as expected.

I googled a little bit and found that Greg did the related work[1]
back in 2011, but seems the patch set didn't make it to kernel.

Now that we have writeback aware cgroup, is there any plan to push the
patch set again or are there any alternative solutions/suggestions?

[1]: https://lwn.net/Articles/455341/

thanks,
-liubo

             reply	other threads:[~2018-09-12  0:11 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-12  0:10 Liu Bo [this message]
2018-09-12 12:11 ` Jan Kara
2018-09-12 15:07   ` Theodore Y. Ts'o
2018-09-12 19:22     ` Liu Bo
2018-09-12 19:19   ` Liu Bo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180912001054.bu3x3xwukusnsa26@US-160370MP2.local \
    --to=bo.liu@linux.alibaba.com \
    --cc=cgroups@vger.kernel.org \
    --cc=fengguang.wu@intel.com \
    --cc=gthelen@google.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=tj@kernel.org \
    --cc=yang.shi@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox