linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Frans Pop <elendil@planet.nl>
To: Mel Gorman <mel@csn.ul.ie>
Cc: Chris Mason <chris.mason@oracle.com>,
	David Rientjes <rientjes@google.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	"Rafael J. Wysocki" <rjw@sisk.pl>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Kernel Testers List <kernel-testers@vger.kernel.org>,
	Pekka Enberg <penberg@cs.helsinki.fi>,
	Reinette Chatre <reinette.chatre@intel.com>,
	Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>,
	Karol Lewandowski <karol.k.lewandowski@gmail.com>,
	Mohamed Abbas <mohamed.abbas@intel.com>,
	Jens Axboe <jens.axboe@oracle.com>,
	"John W. Linville" <linville@tuxdriver.com>,
	linux-mm@kvack.org
Subject: Re: [Bug #14141] order 2 page allocation failures in iwlagn
Date: Mon, 26 Oct 2009 22:06:09 +0100	[thread overview]
Message-ID: <200910262206.13146.elendil@planet.nl> (raw)
In-Reply-To: <20091020104839.GC11778@csn.ul.ie>

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

On Tuesday 20 October 2009, Mel Gorman wrote:
> I've attached a patch below that should allow us to cheat. When it's
> applied, it outputs who called congestion_wait(), how long the timeout
> was and how long it waited for. By comparing before and after sleep
> times, we should be able to see which of the callers has significantly
> changed and if it's something easily addressable.

The results from this look fairly interesting (although I may be a bad 
judge as I don't really know what I'm looking at ;-).

I've tested with two kernels:
1) 2.6.31.1: 1 test run
2) 2.6.31.1 + congestion_wait() reverts: 2 test runs

The 1st kernel had the expected "freeze" while reading commits in gitk; 
reading commits with the 2nd kernel was more fluent.
I did 2 runs with the 2nd kernel as the first run had a fairly long music 
skip and more SKB errors than expected. The second run was fairly normal 
with no music skips at all even though it had a few SKB errors.

Data for the tests:
				1st kernel	2nd kernel 1	2nd kernel 2
end reading commits		1:15		1:00		0:55
  "freeze"			yes		no		no
branch data shown		1:55		1:15		1:10
system quiet			2:25		1:50		1:45
# SKB allocation errors		10		53		5

Note that the test is substantially faster with the 2nd kernel and that the 
SKB errors don't really affect the duration of the test.


Attached a tarball with the kernel logs, both the full logs and a stripped 
version with only the lines generated during the actual test.
Something like this will extract the debug data from the logs:
$ grep "delay " <file> | sed "s/^.*\] //"

Also attached a ODF spreadsheet with a summary of the data for all 3 tests.
I've dropped the congestion_wait and sync/rw= columns as they were always 
the same (rw=1 for 1st kernel and sync=0 for 2nd kernel).
I've added a column "weighed delay" and totals for that column and the 
count column.

My layman's observations are:
- without the revert 'background_writeout' is called a lot less frequently,
  but when it's called it gets long delays
- without the revert you have 'wb_kupdate', which is relatively expensive
- with the revert 'shrink_list' is relatively expensive, although not
  really in absolute terms

You people may want to look at exactly what happens directly around the SKB 
allocation errors. I've only looked at totals.

Cheers,
FJP


[-- Attachment #2: logs.tgz --]
[-- Type: application/x-tgz, Size: 151463 bytes --]

[-- Attachment #3: results.ods --]
[-- Type: application/vnd.oasis.opendocument.spreadsheet, Size: 20051 bytes --]

  reply	other threads:[~2009-10-26 21:06 UTC|newest]

Thread overview: 88+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <3onW63eFtRF.A.xXH.oMTxKB@chimera>
     [not found] ` <COE24pZSBH.A.k2B.ZNTxKB@chimera>
     [not found]   ` <200910021111.55749.elendil@planet.nl>
2009-10-05  5:13     ` Frans Pop
2009-10-05  6:50       ` Frans Pop
2009-10-05  8:54         ` Frans Pop
2009-10-05  8:57         ` Mel Gorman
2009-10-05 21:34           ` Frans Pop
2009-10-06  0:04             ` David Rientjes
2009-10-06  1:25               ` KOSAKI Motohiro
2009-10-06  8:53               ` Mel Gorman
2009-10-06  9:14                 ` David Rientjes
2009-10-06  9:22                   ` Mel Gorman
2009-10-06 10:23               ` Frans Pop
2009-10-11 23:10         ` Frans Pop
2009-10-11 23:36           ` Frans Pop
2009-10-12 13:43           ` Mel Gorman
2009-10-12 17:32             ` Frans Pop
2009-10-12 18:43               ` Mel Gorman
2009-10-13 20:38               ` Frans Pop
2009-10-14 10:30                 ` Mel Gorman
2009-10-14 13:10                   ` Frans Pop
2009-10-14 15:40                     ` Mel Gorman
2009-10-14 16:13                       ` Frans Pop
2009-10-14 18:34                       ` Frans Pop
2009-10-14 23:56                         ` Mel Gorman
2009-10-15 20:15                           ` Frans Pop
2009-10-16  9:39                             ` Mel Gorman
2009-10-14 16:30                     ` reinette chatre
2009-10-18 23:33                     ` Frans Pop
2009-10-19  0:36                       ` Pekka Enberg
2009-10-19  2:44                         ` Frans Pop
2009-10-19  9:49                           ` [Bug #14141] order 2 page allocation failures (generic) Tobi Oetiker
2009-10-19  9:54                             ` Pekka Enberg
2009-10-19 14:01                               ` Karol Lewandowski
2009-10-19 14:06                                 ` Mel Gorman
2009-10-19 17:09                                   ` Karol Lewandowski
2009-10-20  1:47                                     ` Karol Lewandowski
2009-10-19 13:31                             ` Mel Gorman
2009-10-19 13:40                               ` Tobias Oetiker
2009-10-19 14:09                                 ` Mel Gorman
2009-10-19 14:16                                   ` Tobias Oetiker
2009-10-19 14:59                                     ` Mel Gorman
2009-10-19 20:12                                       ` Tobias Oetiker
2009-10-19 20:17                                         ` Tobias Oetiker
2009-10-20 10:57                                           ` Mel Gorman
2009-10-20 11:44                                             ` Tobias Oetiker
2009-10-20 12:51                                               ` Mel Gorman
2009-10-20 12:58                                                 ` Tobias Oetiker
2009-10-20 13:39                                                   ` Mel Gorman
2009-10-20 13:50                                                     ` Tobias Oetiker
2009-10-20 14:14                                                       ` Mel Gorman
2009-10-20 14:20                                                         ` Tobias Oetiker
2009-10-22 10:27                                                     ` Tobias Oetiker
2009-10-19  2:52                         ` [Bug #14141] order 2 page allocation failures in iwlagn Jens Axboe
2009-10-19 14:01                       ` Mel Gorman
2009-10-19 16:18                         ` Chris Mason
2009-10-19 17:01                           ` Christoph Hellwig
2009-10-19 21:57                             ` Chris Mason
2009-10-19 17:01                           ` Christoph Hellwig
2009-10-20 10:48                           ` Mel Gorman
2009-10-20 10:48                           ` Mel Gorman
2009-10-26 21:06                             ` Frans Pop [this message]
2009-10-27 14:54                               ` Mel Gorman
2009-10-27 15:16                                 ` KOSAKI Motohiro
2009-10-27 15:21                                   ` Mel Gorman
2009-10-27 15:52                                 ` Mel Gorman
2009-10-27 16:03                                   ` Chris Mason
2009-10-27 17:21                                     ` Frans Pop
2009-10-27 17:21                                     ` Frans Pop
2009-11-05 20:14                               ` Frans Pop
2009-11-06  9:51                                 ` Frans Pop
2009-11-09 19:00                                   ` Mel Gorman
2009-10-25 18:54                           ` Frans Pop
2009-10-14 16:28                   ` reinette chatre
2009-10-14 16:50                     ` Mel Gorman
2009-10-14 20:41                       ` reinette chatre
2009-10-14 21:33                         ` Frans Pop
2009-10-14 21:55                           ` reinette chatre
2009-10-15  2:02                         ` Frans Pop
2009-10-15 15:29                           ` reinette chatre
2009-10-15 19:41                             ` Frans Pop
2009-10-16 17:21                               ` reinette chatre
2009-10-17  5:42                               ` reinette chatre
2009-10-27 11:10                                 ` Frans Pop
2009-10-27 16:15                                   ` reinette chatre
     [not found] ` <COE24pZSBH.A.rP.2MTxKB@chimera>
2009-10-21 20:04   ` [PATCH] SLUB: Don't drop __GFP_NOFAIL completely from allocate_slab() (was: Re: [Bug #14265] ifconfig: page allocation failure. order:5,ode:0x8020 w/ e100) Karol Lewandowski
2009-10-21 21:06     ` David Rientjes
2009-10-21 21:20       ` Karol Lewandowski
2009-10-22 10:20         ` Mel Gorman
2009-10-22 21:33           ` Karol Lewandowski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=200910262206.13146.elendil@planet.nl \
    --to=elendil@planet.nl \
    --cc=bzolnier@gmail.com \
    --cc=chris.mason@oracle.com \
    --cc=jens.axboe@oracle.com \
    --cc=karol.k.lewandowski@gmail.com \
    --cc=kernel-testers@vger.kernel.org \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linville@tuxdriver.com \
    --cc=mel@csn.ul.ie \
    --cc=mohamed.abbas@intel.com \
    --cc=penberg@cs.helsinki.fi \
    --cc=reinette.chatre@intel.com \
    --cc=rientjes@google.com \
    --cc=rjw@sisk.pl \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox