Re: 2.6.22-stable causes oomkiller to be invoked

Re: 2.6.22-stable causes oomkiller to be invoked

[Andrew Morton]

On Fri, 14 Dec 2007 23:58:02 +0530
Dhaval Giani <dhaval@linux.vnet.ibm.com> wrote:

> On Fri, Dec 14, 2007 at 09:50:23AM -0800, Andrew Morton wrote:
> > On Fri, 14 Dec 2007 21:46:37 +0530 Dhaval Giani <dhaval@linux.vnet.ibm.com> wrote:
> > 
> > > On Sat, Dec 15, 2007 at 12:54:09AM +0900, Tejun Heo wrote:
> > > > Dhaval Giani wrote:
> > > > > XXX sysfs_page_cnt=1
> > > > 
> > > > Hmm.. so, sysfs r/w buffer wasn't the culprit.  I'm curious what eats up
> > > > all your low memory.  Please do the following.
> > > > 
> > > > 1. Right after boot, record /proc/meminfo and slabinfo.
> > > > 
> > > > 2. After or near OOM, record /proc/meminfo and slabinfo.  This can be
> > > > tricky but if your machine reliably OOMs after 10mins, run it for 9mins
> > > > and capturing the result should show enough.
> > > > 
> > > 
> > > Attached. The results are after oom, but i think about a min or so after
> > > that. I missed the oom point.
> > 
> > Looking back at your original oom-killer output: something has consumed all
> > your ZONE_NORMAL memory and we cannot tell what it is.
> > 
> > Please run 2.6.24-rc5-mm1 again (with CONFIG_PAGE_OWNER=y) and take a peek
> > at the changelog in
> > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.24-rc5/2.6.24-rc5-mm1/broken-out/page-owner-tracking-leak-detector.patch.
> > 
> > Build up Documentation/page_owner.c then cause the leak to happen then
> > execute page_owner.
> > 
> Hi Andrew
> 
> This is a peek during the leak.
> 
> ...
> 
> [sorted_page_owner.txt  text/plain (100.2KB)]
> 51957 times:
> Page allocated via order 0, mask 0x80d0
> [0xc015b9aa] __alloc_pages+706
> [0xc015b9f0] __get_free_pages+60
> [0xc011b7c9] pgd_alloc+60
> [0xc0122b9e] mm_init+196
> [0xc0122e06] dup_mm+101
> [0xc0122eda] copy_mm+104
> [0xc0123b8c] copy_process+1149
> [0xc0124229] do_fork+141
> 
> 12335 times:
> Page allocated via order 0, mask 0x84d0
> [0xc015b9aa] __alloc_pages+706
> [0xc011b6ca] pte_alloc_one+21
> [0xc01632ac] __pte_alloc+21
> [0xc01634bb] copy_pte_range+67
> [0xc0163827] copy_page_range+284
> [0xc0122a79] dup_mmap+427
> [0xc0122e22] dup_mm+129
> [0xc0122eda] copy_mm+104

OK, so you're leaking pgd's on a fork-intensive load.  It's a 4G i386
highmem system but I'm sure there are enough of those out there (still) for
this bug to have been promptly reported if it was generally occurring.

There's something special about either your setup or the test which you're
running.

Is it really the case that the bug only turns up when you run tests like

	while echo; do cat /sys/kernel/kexec_crash_loaded; done
and
	while echo; do cat /sys/kernel/uevent_seqnum ; done;

or will any fork-intensive workload also do it?  Say,

	while echo ; do true ; done

?

Another interesting factoid here is that after the oomkilling you slabinfo has

mm_struct             38     98    584    7    1 : tunables   32   16    8 : slabdata     14     14      0 : globalstat    2781    196    49   31 				   0    1    0    0    0 : cpustat 368800  11864 368920  11721

so we aren't leaking mm_structs.  In fact we aren't leaking anything from
slab.   But we are leaking pgds.

iirc the most recent change we've made in the pgd_t area is the quicklist
management which went into 2.6.22-rc1.  You say the bug was present in
2.6.22.  Can you test 2.6.21?  

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

> Is it really the case that the bug only turns up when you run tests like
> 
> 	while echo; do cat /sys/kernel/kexec_crash_loaded; done
> and
> 	while echo; do cat /sys/kernel/uevent_seqnum ; done;
> 
> or will any fork-intensive workload also do it?  Say,
> 
> 	while echo ; do true ; done
> 

This does not leak, but having a simple text file and reading it in a
loop causes it.

> ?
> 
> Another interesting factoid here is that after the oomkilling you slabinfo has
> 
> mm_struct             38     98    584    7    1 : tunables   32   16    8 : slabdata     14     14      0 : globalstat    2781    196    49   31 				   0    1    0    0    0 : cpustat 368800  11864 368920  11721
> 
> so we aren't leaking mm_structs.  In fact we aren't leaking anything from
> slab.   But we are leaking pgds.
> 
> iirc the most recent change we've made in the pgd_t area is the quicklist
> management which went into 2.6.22-rc1.  You say the bug was present in
> 2.6.22.  Can you test 2.6.21?  

Nope, leak is not present in 2.6.21.7

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Andrew Morton]

On Sat, 15 Dec 2007 09:22:00 +0530 Dhaval Giani <dhaval@linux.vnet.ibm.com> wrote:

> > Is it really the case that the bug only turns up when you run tests like
> > 
> > 	while echo; do cat /sys/kernel/kexec_crash_loaded; done
> > and
> > 	while echo; do cat /sys/kernel/uevent_seqnum ; done;
> > 
> > or will any fork-intensive workload also do it?  Say,
> > 
> > 	while echo ; do true ; done
> > 
> 
> This does not leak, but having a simple text file and reading it in a
> loop causes it.

hm.

> > ?
> > 
> > Another interesting factoid here is that after the oomkilling you slabinfo has
> > 
> > mm_struct             38     98    584    7    1 : tunables   32   16    8 : slabdata     14     14      0 : globalstat    2781    196    49   31 				   0    1    0    0    0 : cpustat 368800  11864 368920  11721
> > 
> > so we aren't leaking mm_structs.  In fact we aren't leaking anything from
> > slab.   But we are leaking pgds.
> > 
> > iirc the most recent change we've made in the pgd_t area is the quicklist
> > management which went into 2.6.22-rc1.  You say the bug was present in
> > 2.6.22.  Can you test 2.6.21?  
> 
> Nope, leak is not present in 2.6.21.7

Could you try this debug patch please?

It might need some fiddling to get useful output.  Basic idea is to see if
we are failing to empty the quicklists.

--- a/include/linux/quicklist.h~a
+++ a/include/linux/quicklist.h
@@ -69,6 +69,8 @@ static inline void __quicklist_free(int 
 	*(void **)p = q->page;
 	q->page = p;
 	q->nr_pages++;
+	if (q->nr_pages && !(q->nr_pages % 1000))
+		printk("eek: %d\n", q->nr_pages);
 	put_cpu_var(quicklist);
 }
 
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

On Fri, Dec 14, 2007 at 10:00:30PM -0800, Andrew Morton wrote:
> On Sat, 15 Dec 2007 09:22:00 +0530 Dhaval Giani <dhaval@linux.vnet.ibm.com> wrote:
> 
> > > Is it really the case that the bug only turns up when you run tests like
> > > 
> > > 	while echo; do cat /sys/kernel/kexec_crash_loaded; done
> > > and
> > > 	while echo; do cat /sys/kernel/uevent_seqnum ; done;
> > > 
> > > or will any fork-intensive workload also do it?  Say,
> > > 
> > > 	while echo ; do true ; done
> > > 
> > 
> > This does not leak, but having a simple text file and reading it in a
> > loop causes it.
> 
> hm.
> 
> > > ?
> > > 
> > > Another interesting factoid here is that after the oomkilling you slabinfo has
> > > 
> > > mm_struct             38     98    584    7    1 : tunables   32   16    8 : slabdata     14     14      0 : globalstat    2781    196    49   31 				   0    1    0    0    0 : cpustat 368800  11864 368920  11721
> > > 
> > > so we aren't leaking mm_structs.  In fact we aren't leaking anything from
> > > slab.   But we are leaking pgds.
> > > 
> > > iirc the most recent change we've made in the pgd_t area is the quicklist
> > > management which went into 2.6.22-rc1.  You say the bug was present in
> > > 2.6.22.  Can you test 2.6.21?  
> > 
> > Nope, leak is not present in 2.6.21.7
> 
> Could you try this debug patch please?
> 

Here is the dmesg with that patch,

use, ignoring.
PCI: Unable to reserve mem region #2:1000@edffe000 for device 0000:08:0a.1
aic7xxx: <Adaptec AIC-7899 Ultra 160/m SCSI host adapter> at PCI 8/10/1
aic7xxx: I/O ports already in use, ignoring.
megaraid cmm: 2.20.2.7 (Release Date: Sun Jul 16 00:01:03 EST 2006)
megaraid: 2.20.5.1 (Release Date: Thu Nov 16 15:32:35 EST 2006)
megasas: 00.00.03.16-rc1 Thu. Nov. 07 10:09:32 PDT 2007
st: Version 20070203, fixed bufsize 32768, s/g segs 256
osst :I: Tape driver with OnStream support version 0.99.4
osst :I: $Id: osst.c,v 1.73 2005/01/01 21:13:34 wriede Exp $
sd 1:0:0:0: [sda] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Mode Sense: cb 00 00 08
sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 1:0:0:0: [sda] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Mode Sense: cb 00 00 08
sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1
sd 1:0:0:0: [sda] Attached SCSI disk
sd 1:0:1:0: [sdb] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:1:0: [sdb] Write Protect is off
sd 1:0:1:0: [sdb] Mode Sense: cb 00 00 08
sd 1:0:1:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 1:0:1:0: [sdb] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:1:0: [sdb] Write Protect is off
sd 1:0:1:0: [sdb] Mode Sense: cb 00 00 08
sd 1:0:1:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1 sdb2 sdb3 sdb4
sd 1:0:1:0: [sdb] Attached SCSI disk
sd 1:0:2:0: [sdc] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:2:0: [sdc] Write Protect is off
sd 1:0:2:0: [sdc] Mode Sense: cb 00 00 08
sd 1:0:2:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 1:0:2:0: [sdc] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:2:0: [sdc] Write Protect is off
sd 1:0:2:0: [sdc] Mode Sense: cb 00 00 08
sd 1:0:2:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdc: sdc1 sdc2
sd 1:0:2:0: [sdc] Attached SCSI disk
sd 1:0:3:0: [sdd] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:3:0: [sdd] Write Protect is off
sd 1:0:3:0: [sdd] Mode Sense: cb 00 00 08
sd 1:0:3:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 1:0:3:0: [sdd] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:3:0: [sdd] Write Protect is off
sd 1:0:3:0: [sdd] Mode Sense: cb 00 00 08
sd 1:0:3:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdd: sdd1 sdd2 sdd3
sd 1:0:3:0: [sdd] Attached SCSI disk
sd 1:0:4:0: [sde] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:4:0: [sde] Write Protect is off
sd 1:0:4:0: [sde] Mode Sense: cb 00 00 08
sd 1:0:4:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 1:0:4:0: [sde] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:4:0: [sde] Write Protect is off
sd 1:0:4:0: [sde] Mode Sense: cb 00 00 08
sd 1:0:4:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sde: sde1
sd 1:0:4:0: [sde] Attached SCSI disk
sd 1:0:5:0: [sdf] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:5:0: [sdf] Write Protect is off
sd 1:0:5:0: [sdf] Mode Sense: b3 00 10 08
sd 1:0:5:0: [sdf] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 1:0:5:0: [sdf] 71096640 512-byte hardware sectors (36401 MB)
sd 1:0:5:0: [sdf] Write Protect is off
sd 1:0:5:0: [sdf] Mode Sense: b3 00 10 08
sd 1:0:5:0: [sdf] Write cache: disabled, read cache: enabled, supports DPO and FUA
 sdf: sdf1
sd 1:0:5:0: [sdf] Attached SCSI disk
sd 1:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:1:0: Attached scsi generic sg1 type 0
sd 1:0:2:0: Attached scsi generic sg2 type 0
sd 1:0:3:0: Attached scsi generic sg3 type 0
sd 1:0:4:0: Attached scsi generic sg4 type 0
sd 1:0:5:0: Attached scsi generic sg5 type 0
scsi 1:0:8:0: Attached scsi generic sg6 type 3
Fusion MPT base driver 3.04.06
Copyright (c) 1999-2007 LSI Corporation
Fusion MPT SPI Host driver 3.04.06
Fusion MPT FC Host driver 3.04.06
Fusion MPT SAS Host driver 3.04.06
Fusion MPT misc device (ioctl) driver 3.04.06
mptctl: Registered with Fusion MPT base driver
mptctl: /dev/mptctl @ (major,minor=10,220)
ieee1394: raw1394: /dev/raw1394 device initialized
PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x64,0x60 irq 1,12
PNP: PS/2 controller has invalid data port 0x64; using default 0x60
PNP: PS/2 controller has invalid command port 0x60; using default 0x64
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com
oprofile: using NMI interrupt.
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
p4-clockmod: P4/Xeon(TM) CPU On-Demand Clock Modulation available
Testing NMI watchdog ... OK.
Using IPI No-Shortcut mode
Freeing unused kernel memory: 240k freed
XXX sysfs_page_cnt=1
sd_mod: version magic '2.6.15.4 SMP PENTIUM4 REGPARM 4KSTACKS gcc-3.4' should be '2.6.24-rc5-mm1 SMP mod_unload PENTIUM4 '
aic7xxx: version magic '2.6.15.4 SMP PENTIUM4 REGPARM 4KSTACKS gcc-3.4' should be '2.6.24-rc5-mm1 SMP mod_unload PENTIUM4 '
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
XXX sysfs_page_cnt=1
warning: process `kmodule' used the deprecated sysctl system call with 1.23.
EXT3 FS on sdd1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdd3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdf1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 1863532k swap on /dev/sdb3.  Priority:-1 extents:1 across:1863532k
tg3: eth0: Link is up at 100 Mbps, full duplex.
tg3: eth0: Flow control is off for TX and off for RX.
XXX sysfs_page_cnt=1
XXX sysfs_page_cnt=1
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
eek: 1000
XXX sysfs_page_cnt=1
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
eek: 2000
XXX sysfs_page_cnt=1
eek: 3000
eek: 3000
eek: 3000
eek: 3000
eek: 3000
XXX sysfs_page_cnt=1
eek: 3000
eek: 3000
eek: 3000
eek: 3000
eek: 3000
eek: 3000
eek: 3000
eek: 4000
eek: 4000
eek: 4000
eek: 4000
eek: 4000
eek: 4000
XXX sysfs_page_cnt=1
eek: 4000
eek: 4000
eek: 4000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
XXX sysfs_page_cnt=1
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 5000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
XXX sysfs_page_cnt=1
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 6000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
XXX sysfs_page_cnt=1
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 8000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
eek: 7000
XXX sysfs_page_cnt=1
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
eek: 8000
XXX sysfs_page_cnt=1
eek: 10000
eek: 10000
eek: 10000
eek: 10000
eek: 10000
eek: 10000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
eek: 9000
XXX sysfs_page_cnt=1
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
XXX sysfs_page_cnt=1
eek: 10000
eek: 10000
eek: 10000
eek: 10000
eek: 10000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
XXX sysfs_page_cnt=1
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 11000
eek: 13000
eek: 13000
eek: 13000
eek: 13000
eek: 13000
eek: 13000
XXX sysfs_page_cnt=1
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 12000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
XXX sysfs_page_cnt=1
eek: 15000
eek: 13000
eek: 13000
eek: 15000
eek: 15000
eek: 15000
eek: 15000
XXX sysfs_page_cnt=1
eek: 16000
eek: 16000
eek: 16000
eek: 16000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
eek: 14000
eek: 16000
eek: 16000
XXX sysfs_page_cnt=1
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 15000
eek: 15000
eek: 15000
eek: 15000
eek: 15000
eek: 15000
XXX sysfs_page_cnt=1
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
XXX sysfs_page_cnt=1
eek: 16000
eek: 16000
eek: 16000
eek: 16000
eek: 16000
eek: 16000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
XXX sysfs_page_cnt=1
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 17000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
XXX sysfs_page_cnt=1
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 18000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
XXX sysfs_page_cnt=1
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 19000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
XXX sysfs_page_cnt=1
eek: 23000
eek: 23000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 20000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
XXX sysfs_page_cnt=1
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 21000
eek: 24000
eek: 24000
XXX sysfs_page_cnt=1
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
eek: 22000
XXX sysfs_page_cnt=1
eek: 26000
eek: 26000
eek: 26000
eek: 26000
eek: 26000
XXX sysfs_page_cnt=1
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 23000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
XXX sysfs_page_cnt=1
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 24000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
XXX sysfs_page_cnt=1
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 25000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
XXX sysfs_page_cnt=1
eek: 30000
eek: 30000
eek: 26000
eek: 26000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
XXX sysfs_page_cnt=1
eek: 31000
eek: 31000
eek: 31000
eek: 31000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 27000
eek: 31000
eek: 31000
XXX sysfs_page_cnt=1
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
XXX sysfs_page_cnt=1
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 28000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
XXX sysfs_page_cnt=1
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 29000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
XXX sysfs_page_cnt=1
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 30000
eek: 35000
eek: 35000
eek: 35000
eek: 35000
eek: 35000
eek: 35000
XXX sysfs_page_cnt=1
eek: 31000
eek: 31000
eek: 36000
eek: 36000
eek: 36000
XXX sysfs_page_cnt=1
eek: 36000
eek: 36000
eek: 37000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 32000
eek: 37000
eek: 37000
XXX sysfs_page_cnt=1
eek: 37000
eek: 37000
eek: 38000
eek: 38000
eek: 38000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
eek: 33000
XXX sysfs_page_cnt=1
eek: 38000
eek: 38000
eek: 39000
eek: 39000
eek: 39000
eek: 39000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
eek: 34000
XXX sysfs_page_cnt=1
eek: 39000
eek: 39000
eek: 40000
eek: 40000
eek: 40000
eek: 40000
XXX sysfs_page_cnt=1
eek: 40000
eek: 40000
eek: 35000
eek: 35000
eek: 35000
eek: 35000
eek: 35000
eek: 41000
eek: 41000
eek: 41000
eek: 41000
XXX sysfs_page_cnt=1
eek: 41000
eek: 41000
eek: 36000
eek: 36000
eek: 36000
eek: 36000
eek: 36000
eek: 42000
eek: 42000
eek: 42000
eek: 42000
XXX sysfs_page_cnt=1
eek: 42000
eek: 42000
eek: 37000
eek: 37000
eek: 37000
eek: 43000
eek: 43000
eek: 43000
eek: 43000
XXX sysfs_page_cnt=1
eek: 43000
eek: 43000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 38000
eek: 44000
eek: 44000
eek: 44000
eek: 44000
XXX sysfs_page_cnt=1
eek: 44000
eek: 44000
eek: 45000
eek: 45000
eek: 39000
eek: 39000
eek: 45000
eek: 45000
XXX sysfs_page_cnt=1
bash invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 4434, comm: bash Not tainted 2.6.24-rc5-mm1 #9
 [<c010582a>] show_trace_log_lvl+0x12/0x22
 [<c0105847>] show_trace+0xd/0xf
 [<c0105959>] dump_stack+0x57/0x5e
 [<c015a0f3>] oom_kill_process+0x37/0xdb
 [<c015a300>] out_of_memory+0xbd/0xf1
 [<c015b977>] __alloc_pages+0x23f/0x2cc
 [<c011b6ca>] pte_alloc_one+0x15/0x3e
 [<c01632fc>] __pte_alloc+0x15/0xaf
 [<c016350b>] copy_pte_range+0x43/0x293
 [<c0163877>] copy_page_range+0x11c/0x154
 [<c0122ac9>] dup_mmap+0x1ab/0x20c
 [<c0122e72>] dup_mm+0x81/0xd1
 [<c0122f2a>] copy_mm+0x68/0x98
 [<c0123bdc>] copy_process+0x47d/0x9fd
 [<c0124279>] do_fork+0x8d/0x1d2
 [<c0103aba>] sys_clone+0x1f/0x21
 [<c01049fa>] sysenter_past_esp+0x5f/0xa5
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
CPU    4: hi:    0, btch:   1 usd:   0
CPU    5: hi:    0, btch:   1 usd:   0
CPU    6: hi:    0, btch:   1 usd:   0
CPU    7: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd: 183
CPU    2: hi:  186, btch:  31 usd:  21
CPU    3: hi:  186, btch:  31 usd: 168
CPU    4: hi:  186, btch:  31 usd:   6
CPU    5: hi:  186, btch:  31 usd: 184
CPU    6: hi:  186, btch:  31 usd:  30
CPU    7: hi:  186, btch:  31 usd: 180
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:   8
CPU    1: hi:  186, btch:  31 usd: 181
CPU    2: hi:  186, btch:  31 usd:  24
CPU    3: hi:  186, btch:  31 usd: 155
CPU    4: hi:  186, btch:  31 usd:   8
CPU    5: hi:  186, btch:  31 usd: 178
CPU    6: hi:  186, btch:  31 usd:  12
CPU    7: hi:  186, btch:  31 usd: 167
Active:1938 inactive:1234 dirty:0 writeback:0 unstable:0
 free:986779 slab:2584 mapped:851 pagetables:120 bounce:0
DMA free:3496kB min:64kB low:80kB high:96kB active:0kB inactive:0kB present:16016kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 860 4989 4989
Normal free:3604kB min:3720kB low:4648kB high:5580kB active:0kB inactive:0kB present:880880kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 33033 33033
HighMem free:3940016kB min:512kB low:4976kB high:9440kB active:7808kB inactive:4936kB present:4228224kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 3*4kB 4*8kB 4*16kB 4*32kB 4*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3564kB
Normal: 1*4kB 4*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3668kB
HighMem: 127*4kB 535*8kB 270*16kB 93*32kB 32*64kB 8*128kB 1*256kB 1*512kB 2*1024kB 1*2048kB 957*4096kB = 3939892kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 1863532kB
Total swap = 1863532kB
Free swap:       1863532kB
1310720 pages of RAM
1081344 pages of HIGHMEM
140255 reserved pages
3164 pages shared
0 pages swap cached
0 pages dirty
0 pages writeback
841 pages mapped
2584 pages slab
108 pages pagetables
Out of memory: kill process 3837 (slpd) score 897 or a child
Killed process 3837 (slpd)
bash invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 4434, comm: bash Not tainted 2.6.24-rc5-mm1 #9
 [<c010582a>] show_trace_log_lvl+0x12/0x22
 [<c0105847>] show_trace+0xd/0xf
 [<c0105959>] dump_stack+0x57/0x5e
 [<c015a0f3>] oom_kill_process+0x37/0xdb
 [<c015a300>] out_of_memory+0xbd/0xf1
 [<c015b977>] __alloc_pages+0x23f/0x2cc
 [<c011b6ca>] pte_alloc_one+0x15/0x3e
 [<c01632fc>] __pte_alloc+0x15/0xaf
 [<c016350b>] copy_pte_range+0x43/0x293
 [<c0163877>] copy_page_range+0x11c/0x154
 [<c0122ac9>] dup_mmap+0x1ab/0x20c
 [<c0122e72>] dup_mm+0x81/0xd1
 [<c0122f2a>] copy_mm+0x68/0x98
 [<c0123bdc>] copy_process+0x47d/0x9fd
 [<c0124279>] do_fork+0x8d/0x1d2
 [<c0103aba>] sys_clone+0x1f/0x21
 [<c01049fa>] sysenter_past_esp+0x5f/0xa5
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
CPU    4: hi:    0, btch:   1 usd:   0
CPU    5: hi:    0, btch:   1 usd:   0
CPU    6: hi:    0, btch:   1 usd:   0
CPU    7: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  25
CPU    1: hi:  186, btch:  31 usd: 163
CPU    2: hi:  186, btch:  31 usd:  20
CPU    3: hi:  186, btch:  31 usd: 170
CPU    4: hi:  186, btch:  31 usd:  30
CPU    5: hi:  186, btch:  31 usd: 156
CPU    6: hi:  186, btch:  31 usd:  30
CPU    7: hi:  186, btch:  31 usd: 164
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  36
CPU    1: hi:  186, btch:  31 usd: 166
CPU    2: hi:  186, btch:  31 usd:   5
CPU    3: hi:  186, btch:  31 usd: 166
CPU    4: hi:  186, btch:  31 usd:   8
CPU    5: hi:  186, btch:  31 usd: 162
CPU    6: hi:  186, btch:  31 usd:  25
CPU    7: hi:  186, btch:  31 usd: 166
Active:1933 inactive:1266 dirty:0 writeback:0 unstable:0
 free:986883 slab:2560 mapped:826 pagetables:118 bounce:0
DMA free:3552kB min:64kB low:80kB high:96kB active:0kB inactive:0kB present:16016kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 860 4989 4989
Normal free:3964kB min:3720kB low:4648kB high:5580kB active:76kB inactive:128kB present:880880kB pages_scanned:4949 all_unreclaimable? yes
lowmem_reserve[]: 0 0 33033 33033
HighMem free:3940016kB min:512kB low:4976kB high:9440kB active:7656kB inactive:4936kB present:4228224kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 2*4kB 3*8kB 4*16kB 4*32kB 4*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3552kB
Normal: 51*4kB 12*8kB 3*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3964kB
HighMem: 158*4kB 535*8kB 270*16kB 93*32kB 32*64kB 8*128kB 1*256kB 1*512kB 2*1024kB 1*2048kB 957*4096kB = 3940016kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 1863532kB
Total swap = 1863532kB
Free swap:       1863532kB
1310720 pages of RAM
1081344 pages of HIGHMEM
140255 reserved pages
3059 pages shared
0 pages swap cached
0 pages dirty
0 pages writeback
826 pages mapped
2560 pages slab
118 pages pagetables
Out of memory: kill process 3948 (xfs) score 817 or a child
Killed process 3948 (xfs)
bash invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 4434, comm: bash Not tainted 2.6.24-rc5-mm1 #9
 [<c010582a>] show_trace_log_lvl+0x12/0x22
 [<c0105847>] show_trace+0xd/0xf
 [<c0105959>] dump_stack+0x57/0x5e
 [<c015a0f3>] oom_kill_process+0x37/0xdb
 [<c015a300>] out_of_memory+0xbd/0xf1
 [<c015b977>] __alloc_pages+0x23f/0x2cc
 [<c011b6ca>] pte_alloc_one+0x15/0x3e
 [<c01632fc>] __pte_alloc+0x15/0xaf
 [<c016350b>] copy_pte_range+0x43/0x293
 [<c0163877>] copy_page_range+0x11c/0x154
 [<c0122ac9>] dup_mmap+0x1ab/0x20c
 [<c0122e72>] dup_mm+0x81/0xd1
 [<c0122f2a>] copy_mm+0x68/0x98
 [<c0123bdc>] copy_process+0x47d/0x9fd
 [<c0124279>] do_fork+0x8d/0x1d2
 [<c0103aba>] sys_clone+0x1f/0x21
 [<c01049fa>] sysenter_past_esp+0x5f/0xa5
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
CPU    4: hi:    0, btch:   1 usd:   0
CPU    5: hi:    0, btch:   1 usd:   0
CPU    6: hi:    0, btch:   1 usd:   0
CPU    7: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  17
CPU    1: hi:  186, btch:  31 usd: 179
CPU    2: hi:  186, btch:  31 usd:  27
CPU    3: hi:  186, btch:  31 usd: 157
CPU    4: hi:  186, btch:  31 usd:  30
CPU    5: hi:  186, btch:  31 usd: 180
CPU    6: hi:  186, btch:  31 usd:  13
CPU    7: hi:  186, btch:  31 usd: 165
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  21
CPU    1: hi:  186, btch:  31 usd: 178
CPU    2: hi:  186, btch:  31 usd:  18
CPU    3: hi:  186, btch:  31 usd: 182
CPU    4: hi:  186, btch:  31 usd:  15
CPU    5: hi:  186, btch:  31 usd: 179
CPU    6: hi:  186, btch:  31 usd:  22
CPU    7: hi:  186, btch:  31 usd: 164
Active:1764 inactive:1234 dirty:0 writeback:0 unstable:0
 free:987022 slab:2560 mapped:769 pagetables:104 bounce:0
DMA free:3520kB min:64kB low:80kB high:96kB active:0kB inactive:0kB present:16016kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 860 4989 4989
Normal free:3808kB min:3720kB low:4648kB high:5580kB active:204kB inactive:0kB present:880880kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 33033 33033
HighMem free:3940760kB min:512kB low:4976kB high:9440kB active:6852kB inactive:4936kB present:4228224kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 2*4kB 3*8kB 4*16kB 5*32kB 5*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3520kB
Normal: 56*4kB 15*8kB 6*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4056kB
HighMem: 307*4kB 536*8kB 271*16kB 93*32kB 32*64kB 8*128kB 1*256kB 1*512kB 2*1024kB 1*2048kB 957*4096kB = 3940636kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 1863532kB
Total swap = 1863532kB
Free swap:       1863532kB
1310720 pages of RAM
1081344 pages of HIGHMEM
140255 reserved pages
2857 pages shared
0 pages swap cached
0 pages dirty
0 pages writeback
769 pages mapped
2560 pages slab
104 pages pagetables
Out of memory: kill process 3743 (portmap) score 418 or a child
Killed process 3743 (portmap)
bash invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 4434, comm: bash Not tainted 2.6.24-rc5-mm1 #9
 [<c010582a>] show_trace_log_lvl+0x12/0x22
 [<c0105847>] show_trace+0xd/0xf
 [<c0105959>] dump_stack+0x57/0x5e
 [<c015a0f3>] oom_kill_process+0x37/0xdb
 [<c015a300>] out_of_memory+0xbd/0xf1
 [<c015b977>] __alloc_pages+0x23f/0x2cc
 [<c011b6ca>] pte_alloc_one+0x15/0x3e
 [<c01632fc>] __pte_alloc+0x15/0xaf
 [<c016350b>] copy_pte_range+0x43/0x293
 [<c0163877>] copy_page_range+0x11c/0x154
 [<c0122ac9>] dup_mmap+0x1ab/0x20c
 [<c0122e72>] dup_mm+0x81/0xd1
 [<c0122f2a>] copy_mm+0x68/0x98
 [<c0123bdc>] copy_process+0x47d/0x9fd
 [<c0124279>] do_fork+0x8d/0x1d2
 [<c0103aba>] sys_clone+0x1f/0x21
 [<c01049fa>] sysenter_past_esp+0x5f/0xa5
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
CPU    4: hi:    0, btch:   1 usd:   0
CPU    5: hi:    0, btch:   1 usd:   0
CPU    6: hi:    0, btch:   1 usd:   0
CPU    7: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  24
CPU    1: hi:  186, btch:  31 usd: 166
CPU    2: hi:  186, btch:  31 usd:  18
CPU    3: hi:  186, btch:  31 usd: 162
CPU    4: hi:  186, btch:  31 usd:  21
CPU    5: hi:  186, btch:  31 usd: 184
CPU    6: hi:  186, btch:  31 usd:  13
CPU    7: hi:  186, btch:  31 usd: 168
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  21
CPU    1: hi:  186, btch:  31 usd: 171
CPU    2: hi:  186, btch:  31 usd:  24
CPU    3: hi:  186, btch:  31 usd: 161
CPU    4: hi:  186, btch:  31 usd:  26
CPU    5: hi:  186, btch:  31 usd: 164
CPU    6: hi:  186, btch:  31 usd:  26
CPU    7: hi:  186, btch:  31 usd: 156
Active:1810 inactive:1234 dirty:0 writeback:0 unstable:0
 free:987085 slab:2523 mapped:758 pagetables:80 bounce:0
DMA free:3512kB min:64kB low:80kB high:96kB active:0kB inactive:0kB present:16016kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 860 4989 4989
Normal free:4048kB min:3720kB low:4648kB high:5580kB active:204kB inactive:0kB present:880880kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 33033 33033
HighMem free:3940780kB min:512kB low:4976kB high:9440kB active:7036kB inactive:4936kB present:4228224kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 2*4kB 2*8kB 4*16kB 5*32kB 5*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3512kB
Normal: 46*4kB 22*8kB 6*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4072kB
HighMem: 304*4kB 541*8kB 275*16kB 92*32kB 33*64kB 8*128kB 1*256kB 1*512kB 2*1024kB 1*2048kB 957*4096kB = 3940760kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 1863532kB
Total swap = 1863532kB
Free swap:       1863532kB
1310720 pages of RAM
1081344 pages of HIGHMEM
140255 reserved pages
2752 pages shared
0 pages swap cached
0 pages dirty
0 pages writeback
758 pages mapped
2523 pages slab
80 pages pagetables
Out of memory: kill process 4432 (sshd) score 145 or a child
Killed process 4434 (bash)

_

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

> > It was just
> > 
> > 	while echo ; do cat /sys/kernel/<some file> ; done
> > 
> > it's all in the email threads somewhere..
> 
> The patch that was posted in the thread that I mentioned earlier is here. 
> I ran the test for 15 minutes and things are still fine.
> 
> 
> 
> quicklist: Set tlb->need_flush if pages are remaining in quicklist 0
> 
> This ensures that the quicklists are drained. Otherwise draining may only 
> occur when the processor reaches an idle state.
> 

Hi Christoph,

No, it does not stop the oom I am seeing here.

Thanks,

> Signed-off-by: Christoph Lameter <clameter@sgi.com>
> 
> Index: linux-2.6/include/asm-generic/tlb.h
> ===================================================================
> --- linux-2.6.orig/include/asm-generic/tlb.h	2007-12-13 14:45:38.000000000 -0800
> +++ linux-2.6/include/asm-generic/tlb.h	2007-12-13 14:51:07.000000000 -0800
> @@ -14,6 +14,7 @@
>  #define _ASM_GENERIC__TLB_H
> 
>  #include <linux/swap.h>
> +#include <linux/quicklist.h>
>  #include <asm/pgalloc.h>
>  #include <asm/tlbflush.h>
> 
> @@ -85,6 +86,9 @@ tlb_flush_mmu(struct mmu_gather *tlb, un
>  static inline void
>  tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
>  {
> +#ifdef CONFIG_QUICKLIST
> +	tlb->need_flush += &__get_cpu_var(quicklist)[0].nr_pages != 0;
> +#endif
>  	tlb_flush_mmu(tlb, start, end);
> 
>  	/* keep the page table cache within bounds */

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

On Fri, 21 Dec 2007, Dhaval Giani wrote:

> No, it does not stop the oom I am seeing here.

Duh. Disregard that patch. It looks like check_pgt_cache() is not called. 
This could happen if tlb_flush_mmu is never called during the 
fork/terminate sequences in your script. pgd_free is called *after* a 
possible tlb flush so the pgd page is on the quicklist (which is good for 
the next process which needs a pgd). The tlb_flush_mmu's during pte 
eviction should trim the quicklist. For some reason this is not happening 
on your box (it works here).

Could you try this script that insures that check_pgt_cache is called 
after every pgd_free?

Index: linux-2.6/arch/x86/mm/pgtable_32.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:10.000000000 -0800
+++ linux-2.6/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:54.000000000 -0800
@@ -366,6 +366,15 @@ void pgd_free(pgd_t *pgd)
 		}
 	/* in the non-PAE case, free_pgtables() clears user pgd entries */
 	quicklist_free(0, pgd_dtor, pgd);
+
+	/*
+	 * We must call check_pgd_cache() here because the pgd is freed after
+	 * tlb flushing and the call to check_pgd_cache. In some cases the VM
+	 * may not call tlb_flush_mmu during process termination (??).
+	 * If this is repeated then we may never call check_pgd_cache.
+	 * The quicklist will grow and grow. So call check_pgd_cache here.
+	 */
+	check_pgt_cache();
 }
 
 void check_pgt_cache(void)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

On Thu, Dec 27, 2007 at 11:22:34AM -0800, Christoph Lameter wrote:
> On Thu, 27 Dec 2007, Dhaval Giani wrote:
> 
> > anything specific you are looking for? I still hit the oom.
> 
> Weird.... WTH is this? You run an unmodified upstream tree? Can you add a 
> printk in quicklist_trim that shows
> 

Hi,

I am running 2.6.24-rc5-mm1 here.

> A) that it is called
> 
> B) what the control values q->nr_pages and min_pages are?
> 

Trying to print these using printks renders the system unbootable. With
help from RAS folks around me, managed to get a systemtap script, 

probe kernel.statement("quicklist_trim@mm/quicklist.c:56")
{
        printf(" q->nr_pages is %d, min_pages is %d ----> %s\n",
$q->nr_pages, $$
min_pages, execname());
}

we managed to get your required information. Last 10,000 lines are
attached (The uncompressed file comes to 500 kb).

Hope it helps.

Thanks,
-- 
regards,
Dhaval

[-- Attachment #2: systp.out.1.bz2 --]
[-- Type: application/x-bzip2, Size: 7600 bytes --]

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

On Fri, 28 Dec 2007, Dhaval Giani wrote:

> we managed to get your required information. Last 10,000 lines are
> attached (The uncompressed file comes to 500 kb).
> 
> Hope it helps.

Somehow the nr_pages field is truncated to 16 bit and it 
seems that there are sign issues there? We are wrapping around....

 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 46266, min_pages is 25 ----> bash
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 46265, min_pages is 25 ----> bash
 q->nr_pages is 46265, min_pages is 25 ----> cat
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 46265, min_pages is 25 ----> cat
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 0, min_pages is 25 ----> swapper
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 36877, min_pages is 25 ----> swapper
 q->nr_pages is 46265, min_pages is 25 ----> cat


An int is just a 16 bit field on i386? I thought it was 32 bits? Or is 
the result due to the way that systemtap works?

Could you post the neighboring per cpu variables to quicklist (look at the 
System.map). Maybe somehow we corrupt the nr_pages and page contents.

Also could you do another systemtap and also print out the current 
processor? Maybe nr_pages gets only corrupted on a specific processor. I 
see a zero there and sometimes other sane values.



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

Just traced it again on my system: It is okay for the number of pages on 
the quicklist to reach the high count that we see (although the 16 bit 
limits are weird. You have around 4GB of memory in the system?). Up to 
1/16th of free memory of a node can be allocated for quicklists (this 
allows the effective shutting down and restarting of large amounts of 
processes)

The problem may be that this is run on a HIGHMEM system and the 
calculation of allowable pages on the quicklists does not take into 
account that highmem pages are not usable for quicklists (not sure about 
ZONE_MOVABLE on i386. Maybe we need to take that into account as well?)

Here is a patch that removes the HIGHMEM portion from the calculation. 
Does this change anything:

Index: linux-2.6/mm/quicklist.c
===================================================================
--- linux-2.6.orig/mm/quicklist.c	2008-01-02 13:41:10.000000000 -0800
+++ linux-2.6/mm/quicklist.c	2008-01-02 13:44:15.000000000 -0800
@@ -29,6 +29,12 @@ static unsigned long max_pages(unsigned 
 
 	node_free_pages = node_page_state(numa_node_id(),
 			NR_FREE_PAGES);
+#ifdef CONFIG_HIGHMEM
+	/* Take HIGHMEM pages out of consideration */
+	node_free_pages -= zone_page_state(&NODE_DATA(numa_node_id())->node_zones[ZONE_HIGHMEM],
+				NR_FREE_PAGES);
+#endif
+
 	max = node_free_pages / FRACTION_OF_NODE_MEM;
 	return max(max, min_pages);
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

On Wed, Jan 02, 2008 at 01:54:12PM -0800, Christoph Lameter wrote:
> Just traced it again on my system: It is okay for the number of pages on 
> the quicklist to reach the high count that we see (although the 16 bit 
> limits are weird. You have around 4GB of memory in the system?). Up to 
> 1/16th of free memory of a node can be allocated for quicklists (this 
> allows the effective shutting down and restarting of large amounts of 
> processes)
> 
> The problem may be that this is run on a HIGHMEM system and the 
> calculation of allowable pages on the quicklists does not take into 
> account that highmem pages are not usable for quicklists (not sure about 
> ZONE_MOVABLE on i386. Maybe we need to take that into account as well?)
> 
> Here is a patch that removes the HIGHMEM portion from the calculation. 
> Does this change anything:
> 

Yep. This one hits it. I don't see the obvious signs of the oom
happening in the 5 mins I have run the script. I will let it run for
some more time.

Thanks!
-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

On Thu, Jan 03, 2008 at 09:29:42AM +0530, Dhaval Giani wrote:
> On Wed, Jan 02, 2008 at 01:54:12PM -0800, Christoph Lameter wrote:
> > Just traced it again on my system: It is okay for the number of pages on 
> > the quicklist to reach the high count that we see (although the 16 bit 
> > limits are weird. You have around 4GB of memory in the system?). Up to 
> > 1/16th of free memory of a node can be allocated for quicklists (this 
> > allows the effective shutting down and restarting of large amounts of 
> > processes)
> > 
> > The problem may be that this is run on a HIGHMEM system and the 
> > calculation of allowable pages on the quicklists does not take into 
> > account that highmem pages are not usable for quicklists (not sure about 
> > ZONE_MOVABLE on i386. Maybe we need to take that into account as well?)
> > 
> > Here is a patch that removes the HIGHMEM portion from the calculation. 
> > Does this change anything:
> > 
> 
> Yep. This one hits it. I don't see the obvious signs of the oom
> happening in the 5 mins I have run the script. I will let it run for
> some more time.
> 

Yes, no oom even after 20 mins of running (which is double the normal
time for the oom to occur), also no changes in free lowmem.

Thanks for the fix. Feel free to add a 

Tested-by: Dhaval Giani <dhaval@linux.vnet.ibm.com>

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

On Thu, 3 Jan 2008, Dhaval Giani wrote:

> Yes, no oom even after 20 mins of running (which is double the normal
> time for the oom to occur), also no changes in free lowmem.

Ahhh.. Good then lets redo the patchset the right way (the patch so far 
does not address the ZONE_MOVABLE issues) . Does this patch 
also do the trick?



Quicklists: Only consider memory that can be allocated via GFP_KERNEL

Quicklists calculates the size of the quicklists based on the number
of free pages. This must be the number of free pages that can be
allocated with GFP_KERNEL. node_page_state() includes the pages in
ZONE_HIGHMEM and ZONE_MOVABLE. These should not be considered for the 
size calculation.

Signed-off-by: Christoph Lameter <clameter@sgi.com>

Index: linux-2.6/mm/quicklist.c
===================================================================
--- linux-2.6.orig/mm/quicklist.c	2008-01-03 12:22:55.000000000 -0800
+++ linux-2.6/mm/quicklist.c	2008-01-03 13:00:30.000000000 -0800
@@ -26,9 +26,17 @@ DEFINE_PER_CPU(struct quicklist, quickli
 static unsigned long max_pages(unsigned long min_pages)
 {
 	unsigned long node_free_pages, max;
+	struct zone *zones = NODE_DATA(node)->node_zones;
+
+	node_free_pages =
+#ifdef CONFIG_ZONE_DMA
+		zone_page_state(&zones[ZONE_DMA], NR_FREE_PAGES) +
+#endif
+#ifdef CONFIG_ZONE_DMA32
+		zone_page_state(&zones[ZONE_DMA32], NR_FREE_PAGES) +
+#endif
+		zone_page_state(&zones[ZONE_NORMAL], NR_FREE_PAGES);
 
-	node_free_pages = node_page_state(numa_node_id(),
-			NR_FREE_PAGES);
 	max = node_free_pages / FRACTION_OF_NODE_MEM;
 	return max(max, min_pages);
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

Here is the cleaned version of the patch. Dhaval is testing it.


quicklists: Only consider memory that can be used with GFP_KERNEL

Quicklists calculates the size of the quicklists based on the number
of free pages. This must be the number of free pages that can be
allocated with GFP_KERNEL. node_page_state() includes the pages in
ZONE_HIGHMEM and ZONE_MOVABLE which may lead the quicklists to
become too large causing OOM.

Signed-off-by: Christoph Lameter <clameter@sgi.com>

Index: linux-2.6/mm/quicklist.c
===================================================================
--- linux-2.6.orig/mm/quicklist.c	2008-01-07 10:38:13.000000000 -0800
+++ linux-2.6/mm/quicklist.c	2008-01-07 10:38:44.000000000 -0800
@@ -26,9 +26,17 @@ DEFINE_PER_CPU(struct quicklist, quickli
 static unsigned long max_pages(unsigned long min_pages)
 {
 	unsigned long node_free_pages, max;
+	struct zone *zones = NODE_DATA(numa_node_id())->node_zones;
+
+	node_free_pages =
+#ifdef CONFIG_ZONE_DMA
+		zone_page_state(&zones[ZONE_DMA], NR_FREE_PAGES) +
+#endif
+#ifdef CONFIG_ZONE_DMA32
+		zone_page_state(&zones[ZONE_DMA32], NR_FREE_PAGES) +
+#endif
+		zone_page_state(&zones[ZONE_NORMAL], NR_FREE_PAGES);
 
-	node_free_pages = node_page_state(numa_node_id(),
-			NR_FREE_PAGES);
 	max = node_free_pages / FRACTION_OF_NODE_MEM;
 	return max(max, min_pages);
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

On Mon, Jan 07, 2008 at 12:04:06PM -0800, Christoph Lameter wrote:
> Here is the cleaned version of the patch. Dhaval is testing it.
> 
> 
> quicklists: Only consider memory that can be used with GFP_KERNEL
> 
> Quicklists calculates the size of the quicklists based on the number
> of free pages. This must be the number of free pages that can be
> allocated with GFP_KERNEL. node_page_state() includes the pages in
> ZONE_HIGHMEM and ZONE_MOVABLE which may lead the quicklists to
> become too large causing OOM.
> 
> Signed-off-by: Christoph Lameter <clameter@sgi.com>

Does the job here for me.

Tested-by: Dhaval Giani <dhaval@linux.vnet.ibm.com>

> 
> Index: linux-2.6/mm/quicklist.c
> ===================================================================
> --- linux-2.6.orig/mm/quicklist.c	2008-01-07 10:38:13.000000000 -0800
> +++ linux-2.6/mm/quicklist.c	2008-01-07 10:38:44.000000000 -0800
> @@ -26,9 +26,17 @@ DEFINE_PER_CPU(struct quicklist, quickli
>  static unsigned long max_pages(unsigned long min_pages)
>  {
>  	unsigned long node_free_pages, max;
> +	struct zone *zones = NODE_DATA(numa_node_id())->node_zones;
> +
> +	node_free_pages =
> +#ifdef CONFIG_ZONE_DMA
> +		zone_page_state(&zones[ZONE_DMA], NR_FREE_PAGES) +
> +#endif
> +#ifdef CONFIG_ZONE_DMA32
> +		zone_page_state(&zones[ZONE_DMA32], NR_FREE_PAGES) +
> +#endif
> +		zone_page_state(&zones[ZONE_NORMAL], NR_FREE_PAGES);
> 
> -	node_free_pages = node_page_state(numa_node_id(),
> -			NR_FREE_PAGES);
>  	max = node_free_pages / FRACTION_OF_NODE_MEM;
>  	return max(max, min_pages);
>  }

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Ingo Molnar]

* Christoph Lameter <clameter@sgi.com> wrote:

> Index: linux-2.6/arch/x86/mm/pgtable_32.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:10.000000000 -0800
> +++ linux-2.6/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:54.000000000 -0800
> @@ -366,6 +366,15 @@ void pgd_free(pgd_t *pgd)
>  		}
>  	/* in the non-PAE case, free_pgtables() clears user pgd entries */
>  	quicklist_free(0, pgd_dtor, pgd);
> +
> +	/*
> +	 * We must call check_pgd_cache() here because the pgd is freed after
> +	 * tlb flushing and the call to check_pgd_cache. In some cases the VM
> +	 * may not call tlb_flush_mmu during process termination (??).

that's incorrect i think: during process termination exit_mmap() calls 
tlb_finish_mmu() unconditionally which calls tlb_flush_mmu().

> +	 * If this is repeated then we may never call check_pgd_cache.
> +	 * The quicklist will grow and grow. So call check_pgd_cache here.
> +	 */
> +	check_pgt_cache();
>  }

so we still dont seem to understand the failure mode well enough. This 
also looks like a quite dangerous change so late in the v2.6.24 cycle. 
Does it really fix the OOM? If yes, why exactly?

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Dhaval Giani]

On Sun, Dec 30, 2007 at 03:01:16PM +0100, Ingo Molnar wrote:
> 
> * Christoph Lameter <clameter@sgi.com> wrote:
> 
> > Index: linux-2.6/arch/x86/mm/pgtable_32.c
> > ===================================================================
> > --- linux-2.6.orig/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:10.000000000 -0800
> > +++ linux-2.6/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:54.000000000 -0800
> > @@ -366,6 +366,15 @@ void pgd_free(pgd_t *pgd)
> >  		}
> >  	/* in the non-PAE case, free_pgtables() clears user pgd entries */
> >  	quicklist_free(0, pgd_dtor, pgd);
> > +
> > +	/*
> > +	 * We must call check_pgd_cache() here because the pgd is freed after
> > +	 * tlb flushing and the call to check_pgd_cache. In some cases the VM
> > +	 * may not call tlb_flush_mmu during process termination (??).
> 
> that's incorrect i think: during process termination exit_mmap() calls 
> tlb_finish_mmu() unconditionally which calls tlb_flush_mmu().
> 
> > +	 * If this is repeated then we may never call check_pgd_cache.
> > +	 * The quicklist will grow and grow. So call check_pgd_cache here.
> > +	 */
> > +	check_pgt_cache();
> >  }
> 
> so we still dont seem to understand the failure mode well enough. This 
> also looks like a quite dangerous change so late in the v2.6.24 cycle. 
> Does it really fix the OOM? If yes, why exactly?
> 

No it does not. I've sent out some more information if it helps, will
send to you separately.

-- 
regards,
Dhaval

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 2.6.22-stable causes oomkiller to be invoked

[Christoph Lameter]

On Sun, 30 Dec 2007, Ingo Molnar wrote:

> so we still dont seem to understand the failure mode well enough. This 
> also looks like a quite dangerous change so late in the v2.6.24 cycle. 
> Does it really fix the OOM? If yes, why exactly?

Not exactly sure. I suspect that there is some memory corruption. See my 
earlier post from today. I do not see this issue on my system. So it must 
be particular to a certain config.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>