linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@elte.hu>,
	Paul McKenney <paulmck@linux.vnet.ibm.com>,
	Tim Chen <tim.c.chen@linux.intel.com>,
	Hugh Dickins <hughd@google.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	David Miller <davem@davemloft.net>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	Russell King <rmk@arm.linux.org.uk>,
	Paul Mundt <lethal@linux-sh.org>, Jeff Dike <jdike@addtoit.com>,
	Richard Weinberger <richard@nod.at>,
	Tony Luck <tony.luck@intel.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Mel Gorman <mel@csn.ul.ie>, Nick Piggin <npiggin@kernel.dk>,
	Namhyung Kim <namhyung@gmail.com>,
	ak@linux.intel.com, shaohua.li@intel.com, alex.shi@intel.com,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	"Rafael J. Wysocki" <rjw@sisk.pl>
Subject: Re: REGRESSION: Performance regressions from switching anon_vma->lock to mutex
Date: Wed, 15 Jun 2011 12:24:35 -0700	[thread overview]
Message-ID: <20110615122435.386731e0.akpm@linux-foundation.org> (raw)
In-Reply-To: <BANLkTikt88KnxTy8TuGGVrBVnXvsnL7nMQ@mail.gmail.com>

On Wed, 15 Jun 2011 12:11:19 -0700
Linus Torvalds <torvalds@linux-foundation.org> wrote:

> So using anonymous kernel threads is actually a real downside. It
> makes it much less obvious what is going on. We saw that exact same
> thing with the generic worker thread conversions: things that used to
> have clear performance issues ("oh, the iwl-phy0 thread is using 3% of
> CPU time because it is polling for IO, and I can see that in 'top'")
> turned into much-harder-to-see issues ("oh, kwork0 us using 3% CPU
> time according to 'top' - I have no idea why").

Yes, this is an issue with the memcg async reclaim patches.  One
implementation uses a per-memcg kswapd and you can then actually see
what it's doing, and see when it goes nuts (as kswapd threads like to
do).  The other implementation uses worker threads and you have no clue
what's going on.

It could be that if more things move away from dedicated threads and
into worker threads, we'll need to build a separate accounting system
so we can see how much time worker threads are spending on a
per-handler basis.  Which means a new top-like tool, etc.

That's all pretty nasty and is a tradeoff which should be considered
when making thread-vs-worker decisions.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-06-15 19:25 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-06-15  0:29 Tim Chen
2011-06-15  0:36 ` Andi Kleen
2011-06-17 19:07   ` Ingo Molnar
2011-06-15  1:21 ` Linus Torvalds
2011-06-15  3:42   ` Linus Torvalds
2011-06-15  1:26 ` Shaohua Li
2011-06-15 11:52   ` Peter Zijlstra
2011-06-15 12:49     ` Peter Zijlstra
2011-06-15 16:18     ` Andi Kleen
2011-06-15 16:45       ` Peter Zijlstra
2011-06-15 16:47         ` Andi Kleen
2011-06-15 18:43         ` Tim Chen
2011-06-15 20:32           ` Peter Zijlstra
2011-06-15 20:57             ` Andi Kleen
2011-06-15 21:12               ` Tim Chen
2011-06-15 21:37                 ` Peter Zijlstra
2011-06-15 21:51                   ` Linus Torvalds
2011-06-15 22:19                     ` Andi Kleen
2011-06-16  0:16                       ` Linus Torvalds
2011-06-16 20:14                         ` Andi Kleen
2011-06-16 20:37                           ` Linus Torvalds
2011-06-17  0:24                             ` Andi Kleen
2011-06-17  9:13                               ` Ingo Molnar
2011-06-15 22:15                   ` Andi Kleen
2011-06-16  1:08                   ` Tim Chen
2011-06-16  1:50                   ` Linus Torvalds
2011-06-16 20:26                     ` Tim Chen
2011-06-16 20:47                       ` Linus Torvalds
2011-06-16 21:05                         ` Linus Torvalds
2011-06-16 21:06                           ` Linus Torvalds
2011-06-16 21:26                             ` Linus Torvalds
2011-06-17  3:58                               ` Linus Torvalds
2011-06-17 11:28                                 ` Peter Zijlstra
2011-06-17 11:54                                   ` Peter Zijlstra
2011-06-17 16:36                                   ` Linus Torvalds
2011-06-17 17:41                                     ` Hugh Dickins
2011-06-17 17:55                                       ` Peter Zijlstra
2011-06-17 18:01                                       ` Linus Torvalds
2011-06-17 18:18                                         ` Peter Zijlstra
2011-06-17 18:32                                           ` Peter Zijlstra
2011-06-17 18:39                                             ` Linus Torvalds
2011-06-17 18:41                                               ` Linus Torvalds
2011-06-17 20:19                                               ` Tim Chen
2011-06-17 22:20                                               ` Hugh Dickins
2011-06-18  4:47                                                 ` Linus Torvalds
2011-06-17 19:53                                             ` [PATCH] mm, memory-failure: Fix spinlock vs mutex order Peter Zijlstra
2011-06-17 20:04                                               ` Andi Kleen
2011-06-17 16:46                                   ` REGRESSION: Performance regressions from switching anon_vma->lock to mutex Linus Torvalds
2011-06-17 17:28                                     ` Linus Torvalds
2011-06-17 19:40                                     ` Andi Kleen
2011-06-18  8:08                                       ` Ingo Molnar
2011-06-17 18:22                                 ` Tim Chen
2011-06-17 19:05                                   ` Ray Lee
2011-06-16 22:00                           ` Andi Kleen
2011-06-15 10:36 ` Peter Zijlstra
2011-06-15 10:58   ` Peter Zijlstra
2011-06-15 11:41     ` Peter Zijlstra
2011-06-15 19:11     ` Linus Torvalds
2011-06-15 19:24       ` Andrew Morton [this message]
2011-06-15 20:16         ` Ingo Molnar
2011-06-15 20:55           ` Linus Torvalds
2011-06-15 20:12       ` [GIT PULL] " Ingo Molnar
2011-06-15 20:29         ` Paul E. McKenney
2011-06-15 20:47           ` Linus Torvalds
2011-06-15 20:54             ` Paul E. McKenney
2011-06-15 21:05         ` Linus Torvalds
2011-06-15 21:15           ` Paul E. McKenney
2011-06-15 21:27             ` Linus Torvalds
2011-06-16  7:03           ` Ingo Molnar
2011-06-16 17:16             ` Paul E. McKenney
2011-06-16 20:25               ` Ingo Molnar
2011-06-16 21:01                 ` Frederic Weisbecker
2011-06-16 23:02                   ` Ingo Molnar
2011-06-17 15:19                     ` Frederic Weisbecker
2011-06-16 21:02                 ` Andi Kleen
2011-06-16 22:21                 ` Benjamin Herrenschmidt
2011-06-16 22:38                   ` Ingo Molnar
2011-06-16 22:47                     ` Andi Kleen
2011-06-16 22:58                       ` Ingo Molnar
2011-06-17  0:45                         ` Paul E. McKenney
2011-06-17  9:43                           ` Ingo Molnar
2011-06-17 16:48                             ` Paul E. McKenney
2011-06-16 23:37                 ` Paul E. McKenney
2011-06-15 20:13       ` Tim Chen
2011-06-15 20:17         ` Ingo Molnar
2011-06-15 20:21           ` Tim Chen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110615122435.386731e0.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=ak@linux.intel.com \
    --cc=alex.shi@intel.com \
    --cc=benh@kernel.crashing.org \
    --cc=davem@davemloft.net \
    --cc=hughd@google.com \
    --cc=jdike@addtoit.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=lethal@linux-sh.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=mingo@elte.hu \
    --cc=namhyung@gmail.com \
    --cc=npiggin@kernel.dk \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --cc=richard@nod.at \
    --cc=rjw@sisk.pl \
    --cc=rmk@arm.linux.org.uk \
    --cc=schwidefsky@de.ibm.com \
    --cc=shaohua.li@intel.com \
    --cc=tim.c.chen@linux.intel.com \
    --cc=tony.luck@intel.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox