linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
To: Marc MERLIN <marc@merlins.org>,
	Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michal Hocko <mhocko@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>, linux-mm <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>, Tejun Heo <tj@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free
Date: Tue, 2 May 2017 19:44:47 +0900	[thread overview]
Message-ID: <29c02986-f065-d3be-f176-0c190a72bc58@I-love.SAKURA.ne.jp> (raw)
In-Reply-To: <20170502041235.zqmywvj5tiiom3jk@merlins.org>

On 2017/05/02 13:12, Marc MERLIN wrote:
> Well, sadly, the problem is more or less back is 4.11.0. The system doesn't really 
> crash but it goes into an infinite loop with
> [34776.826800] BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 33s!

Wow, two of workqueues are reaching max active.

[34777.202267] workqueue btrfs-endio-write: flags=0xe
[34777.218313]   pwq 16: cpus=0-7 flags=0x4 nice=0 active=8/8
[34777.236548]     in-flight: 15168:btrfs_endio_write_helper, 13855:btrfs_endio_write_helper, 3360:btrfs_endio_write_helper, 14241:btrfs_endio_write_helper, 27092:btrfs_endio_write_helper, 15194:btrfs_endio_write_helper, 15169:btrfs_endio_write_helper, 27093:btrfs_endio_write_helper
[34777.316225]     delayed: btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper

[34777.450684] workqueue bcache: flags=0x8
[34779.956462]   pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=256/256
[34779.978283]     in-flight: 15320:cached_dev_read_done [bcache], 23385:cached_dev_read_done [bcache], 23371:cached_dev_read_done [bcache], 15321:cached_dev_read_done [bcache], 15395:cached_dev_read_done [bcache], 11101:cached_dev_read_done [bcache], 15300:cached_dev_read_done [bcache], 23349:cached_dev_read_done [bcache], 23425:cached_dev_read_done [bcache], 23399:cached_dev_read_done [bcache], 15293:cached_dev_read_done [bcache], 20529:cached_dev_read_done [bcache], 15402:cached_dev_read_done [bcache], 23422:cached_dev_read_done [bcache], 23417:cached_dev_read_done [bcache], 23409:cached_dev_read_done [bcache], 20539:cached_dev_read_done [bcache], 23431:cached_dev_read_done [bcache], 20544:cached_dev_read_done [bcache], 15355:cached_dev_read_done [bcache], 11085:cached_dev_read_done [bcache], 6511:cached_dev_read_done [bcache]   

Googling with btrfs_endio_write_helper shows a stuck report with 4.8-rc5, but
seems no response ( https://www.spinics.net/lists/linux-btrfs/msg58633.html ).

> Any idea what I should do next?

Maybe you can try collecting list of all in-flight allocations with backtraces
using kmallocwd patches at
http://lkml.kernel.org/r/1489578541-81526-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp
and http://lkml.kernel.org/r/201704272019.JEH26057.SHFOtMLJOOVFQF@I-love.SAKURA.ne.jp
which also tracks mempool allocations.
(Well, the

-	cond_resched();
+	//cond_resched();

change in the latter patch would not be preferable.)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2017-05-02 10:44 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-11-21 15:43 Marc MERLIN
2016-11-21 16:30 ` Marc MERLIN
2016-11-21 21:50 ` Vlastimil Babka
2016-11-21 21:56   ` Marc MERLIN
2016-11-21 23:03     ` [PATCH] block,blkcg: use __GFP_NOWARN for best-effort allocations in blkcg Tejun Heo
2016-11-22 15:47       ` Vlastimil Babka
2016-11-22 16:48         ` Tejun Heo
2016-11-22 22:13           ` Linus Torvalds
2016-11-23  8:50             ` Vlastimil Babka
2016-11-28 17:19               ` Tejun Heo
2016-11-29  7:25                 ` Michal Hocko
2016-11-29 16:38                   ` Tejun Heo
2016-11-29 16:57                     ` Vlastimil Babka
2016-11-29 17:13                       ` Michal Hocko
2016-11-29 17:17                         ` Linus Torvalds
2016-11-29 17:28                           ` Michal Hocko
2016-11-29 17:48                             ` Linus Torvalds
2016-11-22 16:00       ` Jens Axboe
2016-11-22 16:06     ` 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Marc MERLIN
2016-11-22 16:14       ` Vlastimil Babka
2016-11-22 16:25         ` Michal Hocko
2016-11-22 16:47           ` Marc MERLIN
2016-11-22 16:38         ` Greg Kroah-Hartman
2016-11-29 16:25           ` Michal Hocko
2016-11-29 16:43             ` Greg Kroah-Hartman
2016-11-22 19:38         ` Linus Torvalds
2016-11-23  6:34           ` Michal Hocko
2016-11-23  6:53             ` Hillf Danton
2016-11-23  7:00               ` Michal Hocko
2016-11-23  9:18             ` Vlastimil Babka
2016-11-28  7:23             ` Michal Hocko
2016-11-28 20:55               ` Marc MERLIN
2016-11-29 15:55               ` Marc MERLIN
2016-11-29 16:07                 ` Michal Hocko
2016-11-29 16:34                   ` Marc MERLIN
2016-11-29 17:07                     ` Linus Torvalds
2016-11-29 17:40                       ` Marc MERLIN
2016-11-29 18:01                         ` Linus Torvalds
2016-11-30 17:47                           ` Marc MERLIN
2016-11-30 18:14                             ` Linus Torvalds
2016-11-30 18:21                               ` Marc MERLIN
2016-11-30 18:27                               ` Jens Axboe
2016-11-30 20:30                               ` Tejun Heo
2016-12-01 13:50                                 ` Kent Overstreet
2016-12-01 18:16                                   ` Linus Torvalds
2016-12-01 18:30                                     ` Jens Axboe
2016-12-01 18:37                                       ` Linus Torvalds
2016-12-01 18:46                                         ` Jens Axboe
2016-11-29 20:11                         ` Holger Hoffstätte
2016-11-29 23:01                         ` Marc MERLIN
2016-11-30 13:58                           ` Tetsuo Handa
2017-05-02  4:12                           ` Marc MERLIN
2017-05-02  7:44                             ` Michal Hocko
2017-05-02 14:15                               ` Marc MERLIN
2017-05-02 10:44                             ` Tetsuo Handa [this message]
2016-11-29 16:15               ` Marc MERLIN
2016-11-22 21:46         ` Simon Kirby
2016-11-28  8:06           ` Vlastimil Babka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=29c02986-f065-d3be-f176-0c190a72bc58@I-love.SAKURA.ne.jp \
    --to=penguin-kernel@i-love.sakura.ne.jp \
    --cc=gregkh@linuxfoundation.org \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=marc@merlins.org \
    --cc=mhocko@kernel.org \
    --cc=tj@kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=vbabka@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox