linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Keith Busch <kbusch@kernel.org>
To: Ming Lei <ming.lei@redhat.com>
Cc: Theodore Ts'o <tytso@mit.edu>,
	linux-ext4@vger.kernel.org,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	linux-block@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
	Dave Chinner <dchinner@redhat.com>,
	Eric Sandeen <sandeen@redhat.com>, Christoph Hellwig <hch@lst.de>,
	Zhang Yi <yi.zhang@redhat.com>
Subject: Re: [ext4 io hang] buffered write io hang in balance_dirty_pages
Date: Thu, 4 May 2023 09:59:52 -0600	[thread overview]
Message-ID: <ZFPWeOg5xJ7CbCD0@kbusch-mbp.dhcp.thefacebook.com> (raw)
In-Reply-To: <ZEnb7KuOWmu5P+V9@ovpn-8-24.pek2.redhat.com>

On Thu, Apr 27, 2023 at 10:20:28AM +0800, Ming Lei wrote:
> Hello Guys,
> 
> I got one report in which buffered write IO hangs in balance_dirty_pages,
> after one nvme block device is unplugged physically, then umount can't
> succeed.
> 
> Turns out it is one long-term issue, and it can be triggered at least
> since v5.14 until the latest v6.3.
> 
> And the issue can be reproduced reliably in KVM guest:
> 
> 1) run the following script inside guest:
> 
> mkfs.ext4 -F /dev/nvme0n1
> mount /dev/nvme0n1 /mnt
> dd if=/dev/zero of=/mnt/z.img&
> sleep 10
> echo 1 > /sys/block/nvme0n1/device/device/remove
> 
> 2) dd hang is observed and /dev/nvme0n1 is gone actually

Sorry to jump in so late.

For an ungraceful nvme removal, like a surpirse hot unplug, the driver
sets the capacity to 0 and that effectively ends all dirty page writers
that could stall forward progress on the removal. And that 0 capacity
should also cause 'dd' to exit.

But this is not an ungraceful removal, so we're not getting that forced
behavior. Could we use the same capacity trick here after flushing any
outstanding dirty pages?


  parent reply	other threads:[~2023-05-04 15:59 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-27  2:20 Ming Lei
2023-04-27  3:58 ` Matthew Wilcox
2023-04-27  4:50   ` Ming Lei
2023-04-27  6:36     ` Baokun Li
2023-04-27  7:33       ` Baokun Li
2023-04-27 10:01       ` Ming Lei
2023-04-27 11:19         ` Baokun Li
2023-04-27 11:27           ` Ming Lei
2023-04-28  1:41             ` Ming Lei
2023-04-28  3:47               ` Baokun Li
2023-04-28  5:47                 ` Theodore Ts'o
2023-04-29  3:16                   ` Ming Lei
2023-04-29  4:40                     ` Christoph Hellwig
2023-04-29  5:10                       ` Ming Lei
2023-05-01  4:47                         ` Christoph Hellwig
2023-05-02  0:57                           ` Ming Lei
2023-05-02  1:35                             ` Dave Chinner
2023-05-02 15:35                               ` Darrick J. Wong
2023-05-02 22:33                                 ` Dave Chinner
2023-05-02 23:27                                   ` Darrick J. Wong
2023-04-29  4:56                     ` Theodore Ts'o
2023-05-01  2:06                       ` Dave Chinner
2023-05-04  3:09                   ` Baokun Li
2023-04-27 23:33 ` Dave Chinner
2023-04-28  2:56   ` Matthew Wilcox
2023-04-28  5:24     ` Dave Chinner
2023-05-04 15:59 ` Keith Busch [this message]
2023-05-04 16:21   ` Matthew Wilcox
2023-05-05  2:06   ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZFPWeOg5xJ7CbCD0@kbusch-mbp.dhcp.thefacebook.com \
    --to=kbusch@kernel.org \
    --cc=adilger.kernel@dilger.ca \
    --cc=akpm@linux-foundation.org \
    --cc=dchinner@redhat.com \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ming.lei@redhat.com \
    --cc=sandeen@redhat.com \
    --cc=tytso@mit.edu \
    --cc=yi.zhang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox