* Question about OOM-Killer
@ 2005-07-18 19:21 James Washer
2005-07-18 19:36 ` James Washer
0 siblings, 1 reply; 14+ messages in thread
From: James Washer @ 2005-07-18 19:21 UTC (permalink / raw)
To: linux-mm
I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory) drops below the minimum, and the OOM-Killer is fired off.
It just strikes me odd that the OOM-Killer would be called at all for DMA memory. What's the chance of regaining DMA memory by killing user land processes?
I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
- jim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-18 19:21 Question about OOM-Killer James Washer
@ 2005-07-18 19:36 ` James Washer
2005-07-23 13:00 ` Marcelo Tosatti
0 siblings, 1 reply; 14+ messages in thread
From: James Washer @ 2005-07-18 19:36 UTC (permalink / raw)
To: James Washer; +Cc: linux-mm
Sorry, I should have added...
2.6.11.10,
x86-64 dual proc (Intel Xeon 3.4GHz)
6GiB ram
Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi0 Channel: 00 Id: 01 Lun: 00
Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
Type: Direct-Access ANSI SCSI revision: 05
On Mon, 18 Jul 2005 12:21:01 -0700
James Washer <washer@trlp.com> wrote:
> I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory) drops below the minimum, and the OOM-Killer is fired off.
>
> It just strikes me odd that the OOM-Killer would be called at all for DMA memory. What's the chance of regaining DMA memory by killing user land processes?
>
> I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
>
> - jim
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-18 19:36 ` James Washer
@ 2005-07-23 13:00 ` Marcelo Tosatti
2005-07-25 19:11 ` James Washer
2005-07-26 13:53 ` Andi Kleen
0 siblings, 2 replies; 14+ messages in thread
From: Marcelo Tosatti @ 2005-07-23 13:00 UTC (permalink / raw)
To: James Washer; +Cc: linux-mm, ak
James,
Can you send the OOM killer output?
I dont know which devices part of an x86-64 system should
be limited to 16Mb of physical addressing. Andi?
I don't think that any devices should have 16MB limitation
On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
> Sorry, I should have added...
> 2.6.11.10,
> x86-64 dual proc (Intel Xeon 3.4GHz)
> 6GiB ram
> Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
> Host: scsi0 Channel: 00 Id: 00 Lun: 00
> Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
> Type: Direct-Access ANSI SCSI revision: 05
> Host: scsi0 Channel: 00 Id: 01 Lun: 00
> Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
> Type: Direct-Access ANSI SCSI revision: 05
>
>
> On Mon, 18 Jul 2005 12:21:01 -0700
> James Washer <washer@trlp.com> wrote:
>
> > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
> drops below the minimum, and the OOM-Killer is fired off.
> >
> > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
> What's the chance of regaining DMA memory by killing user land processes?
> >
> > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-25 19:11 ` James Washer
@ 2005-07-25 12:27 ` Marcelo Tosatti
2005-07-25 22:41 ` Martin J. Bligh
1 sibling, 0 replies; 14+ messages in thread
From: Marcelo Tosatti @ 2005-07-25 12:27 UTC (permalink / raw)
To: James Washer, ak; +Cc: linux-mm
On Mon, Jul 25, 2005 at 12:11:30PM -0700, James Washer wrote:
> Pretty typical message here...
> Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
__GFP_FS|__GFP_IO|__GFP_WAIT|__GFP_DMA
> Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
> Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
> Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
> Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
> Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
> Jul 6 17:31:27 p6 kernel:
> Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
> Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
> Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
> Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
> Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
Andi,
Zone DMA is exhausted, filled with non-LRU pages, and some
allocator is requesting a GFP_DMA page.
Can you enlight us what kind of devices are limited to <16MB
on x86-64, and the reasoning for it ?
> Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
> Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
> Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
> Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
> Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
> Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
> Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
> Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
> Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
> Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-25 22:41 ` Martin J. Bligh
@ 2005-07-25 15:46 ` Marcelo Tosatti
2005-07-26 0:35 ` James Washer
1 sibling, 0 replies; 14+ messages in thread
From: Marcelo Tosatti @ 2005-07-25 15:46 UTC (permalink / raw)
To: Martin J. Bligh; +Cc: James Washer, linux-mm, ak
On Mon, Jul 25, 2005 at 03:41:27PM -0700, Martin J. Bligh wrote:
> Jim, does seem bloody silly to be shooting stuff here, and is
> probably simple to fix ... however, would be useful to see where
> the DMA allocs are coming from as well, any chance you could dump
> a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
>
> M.
The stacktrace should probably be in mainline, along with some sort
of printk ratelimiting...
v2.4 has
if (unlikely(vm_gfp_debug))
dump_stack();
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-26 0:35 ` James Washer
@ 2005-07-25 17:10 ` Marcelo Tosatti
2005-07-26 13:29 ` Martin J. Bligh
1 sibling, 0 replies; 14+ messages in thread
From: Marcelo Tosatti @ 2005-07-25 17:10 UTC (permalink / raw)
To: James Washer
Cc: Martin J. Bligh, linux-mm, ak, Andrea Arcangeli, James Bottomley
On Mon, Jul 25, 2005 at 05:35:13PM -0700, James Washer wrote:
> Already been done... but I'd not had much time to chase it.. This is my desktop/workstation, so difficult for me to refine the debugging, in any event, here's the backtrace. This is consistent across about 20 events.
>
>
> kernel: Call Trace:<ffffffff80156ba8>{out_of_memory+275} <ffffffff80147fe5>{autoremove_wake_function+0}
> kernel: <ffffffff80157e8b>{__alloc_pages+793} <ffffffff8015a99b>{cache_grow+269}
> kernel: <ffffffff8015adf4>{cache_alloc_refill+442} <ffffffff8015a883>{kmem_cache_alloc+92}
> kernel: <ffffffff88000f0b>{:sd_mod:sd_revalidate_disk+155}
> kernel: <ffffffff8015d549>{pagevec_lookup+23} <ffffffff8015d9a0>{invalidate_mapping_pages+208}
> kernel: <ffffffff80176292>{invalidate_bh_lru+0} <ffffffff8011b9ef>{flat_send_IPI_allbutself+20}
> kernel: <ffffffff801198df>{smp_call_function+62} <ffffffff8017b466>{check_disk_change+89}
> kernel: <ffffffff88000625>{:sd_mod:sd_open+257} <ffffffff8017b73e>{do_open+190}
> kernel: <ffffffff8017bb02>{blkdev_open+33} <ffffffff80173768>{dentry_open+224}
> kernel: <ffffffff801738a2>{filp_open+63} <ffffffff8017398a>{get_unused_fd+220}
> kernel: <ffffffff80173a8b>{sys_open+62} <ffffffff8010e29e>{system_call+126}
/**
* sd_revalidate_disk - called the first time a new disk is seen,
* performs disk spin up, read_capacity, etc.
* @disk: struct gendisk we care about
**/
static int sd_revalidate_disk(struct gendisk *disk)
{
...
buffer = kmalloc(512, GFP_KERNEL | __GFP_DMA);
if (!buffer) {
printk(KERN_WARNING "(sd_revalidate_disk:) Memory allocation "
"failure.\n");
goto out_release_request;
}
sd_spinup_disk(sdkp, disk->disk_name, sreq, buffer);
I suppose its playing safe to support 16Mb limited devices. Can't the
gfp_mask be derived from lower level device characteristics?
On the VM side alloc_pages() should do better if conditioned to kill on
lowmem exhaustion and expect allocators to handle failure properly.
Or maybe to always return failure for in-kernel allocations, but one
might argue that kernel allocations _might_ be crucial for system
functionality (its a case by case basis) IIRC Andrea used to favour
failed DMA allocations to trigger the kill, for reasons I don't
recall.
The OOM killer, what a delight.
> On Mon, 25 Jul 2005 15:41:27 -0700
> "Martin J. Bligh" <mbligh@mbligh.org> wrote:
>
> > Jim, does seem bloody silly to be shooting stuff here, and is
> > probably simple to fix ... however, would be useful to see where
> > the DMA allocs are coming from as well, any chance you could dump
> > a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
> >
> > M.
> >
> > --On Monday, July 25, 2005 12:11:30 -0700 James Washer <washer@trlp.com> wrote:
> >
> > > Pretty typical message here...
> > > Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
> > > Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
> > > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
> > > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
> > > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
> > > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
> > > Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
> > > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
> > > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
> > > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
> > > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
> > > Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
> > > Jul 6 17:31:27 p6 kernel:
> > > Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
> > > Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
> > > Jul 6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
> > > Jul 6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
> > > Jul 6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
> > > Jul 6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
> > > Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
> > > Jul 6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
> > > Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
> > > Jul 6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
> > > Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
> > > Jul 6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
> > > Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
> > > Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
> > > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> > > Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
> > > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> > > Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
> > > Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
> > > Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
> > > Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
> > > Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
> > > Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
> > > Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
> > >
> > >
> > > On Sat, 23 Jul 2005 10:00:48 -0300
> > > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> > >
> > >>
> > >> James,
> > >>
> > >> Can you send the OOM killer output?
> > >>
> > >> I dont know which devices part of an x86-64 system should
> > >> be limited to 16Mb of physical addressing. Andi?
> > >>
> > >> I don't think that any devices should have 16MB limitation
> > >>
> > >> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
> > >> > Sorry, I should have added...
> > >> > 2.6.11.10,
> > >> > x86-64 dual proc (Intel Xeon 3.4GHz)
> > >> > 6GiB ram
> > >> > Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
> > >> > Host: scsi0 Channel: 00 Id: 00 Lun: 00
> > >> > Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
> > >> > Type: Direct-Access ANSI SCSI revision: 05
> > >> > Host: scsi0 Channel: 00 Id: 01 Lun: 00
> > >> > Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
> > >> > Type: Direct-Access ANSI SCSI revision: 05
> > >> >
> > >> >
> > >> > On Mon, 18 Jul 2005 12:21:01 -0700
> > >> > James Washer <washer@trlp.com> wrote:
> > >> >
> > >> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
> > >> > drops below the minimum, and the OOM-Killer is fired off.
> > >> > >
> > >> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
> > >> > What's the chance of regaining DMA memory by killing user land processes?
> > >> > >
> > >> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-23 13:00 ` Marcelo Tosatti
@ 2005-07-25 19:11 ` James Washer
2005-07-25 12:27 ` Marcelo Tosatti
2005-07-25 22:41 ` Martin J. Bligh
2005-07-26 13:53 ` Andi Kleen
1 sibling, 2 replies; 14+ messages in thread
From: James Washer @ 2005-07-25 19:11 UTC (permalink / raw)
To: Marcelo Tosatti; +Cc: linux-mm, ak
Pretty typical message here...
Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
Jul 6 17:31:27 p6 kernel:
Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
Jul 6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
Jul 6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
Jul 6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
Jul 6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
Jul 6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
Jul 6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
Jul 6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
On Sat, 23 Jul 2005 10:00:48 -0300
Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
>
> James,
>
> Can you send the OOM killer output?
>
> I dont know which devices part of an x86-64 system should
> be limited to 16Mb of physical addressing. Andi?
>
> I don't think that any devices should have 16MB limitation
>
> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
> > Sorry, I should have added...
> > 2.6.11.10,
> > x86-64 dual proc (Intel Xeon 3.4GHz)
> > 6GiB ram
> > Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
> > Host: scsi0 Channel: 00 Id: 00 Lun: 00
> > Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
> > Type: Direct-Access ANSI SCSI revision: 05
> > Host: scsi0 Channel: 00 Id: 01 Lun: 00
> > Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
> > Type: Direct-Access ANSI SCSI revision: 05
> >
> >
> > On Mon, 18 Jul 2005 12:21:01 -0700
> > James Washer <washer@trlp.com> wrote:
> >
> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
> > drops below the minimum, and the OOM-Killer is fired off.
> > >
> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
> > What's the chance of regaining DMA memory by killing user land processes?
> > >
> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-25 19:11 ` James Washer
2005-07-25 12:27 ` Marcelo Tosatti
@ 2005-07-25 22:41 ` Martin J. Bligh
2005-07-25 15:46 ` Marcelo Tosatti
2005-07-26 0:35 ` James Washer
1 sibling, 2 replies; 14+ messages in thread
From: Martin J. Bligh @ 2005-07-25 22:41 UTC (permalink / raw)
To: James Washer, Marcelo Tosatti; +Cc: linux-mm, ak
Jim, does seem bloody silly to be shooting stuff here, and is
probably simple to fix ... however, would be useful to see where
the DMA allocs are coming from as well, any chance you could dump
a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
M.
--On Monday, July 25, 2005 12:11:30 -0700 James Washer <washer@trlp.com> wrote:
> Pretty typical message here...
> Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
> Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
> Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
> Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
> Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
> Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
> Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
> Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
> Jul 6 17:31:27 p6 kernel:
> Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
> Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
> Jul 6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
> Jul 6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
> Jul 6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
> Jul 6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
> Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
> Jul 6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
> Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
> Jul 6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
> Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
> Jul 6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
> Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
> Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
> Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
> Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
> Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
> Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
> Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
> Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
> Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
> Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
>
>
> On Sat, 23 Jul 2005 10:00:48 -0300
> Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
>
>>
>> James,
>>
>> Can you send the OOM killer output?
>>
>> I dont know which devices part of an x86-64 system should
>> be limited to 16Mb of physical addressing. Andi?
>>
>> I don't think that any devices should have 16MB limitation
>>
>> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
>> > Sorry, I should have added...
>> > 2.6.11.10,
>> > x86-64 dual proc (Intel Xeon 3.4GHz)
>> > 6GiB ram
>> > Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
>> > Host: scsi0 Channel: 00 Id: 00 Lun: 00
>> > Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
>> > Type: Direct-Access ANSI SCSI revision: 05
>> > Host: scsi0 Channel: 00 Id: 01 Lun: 00
>> > Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
>> > Type: Direct-Access ANSI SCSI revision: 05
>> >
>> >
>> > On Mon, 18 Jul 2005 12:21:01 -0700
>> > James Washer <washer@trlp.com> wrote:
>> >
>> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
>> > drops below the minimum, and the OOM-Killer is fired off.
>> > >
>> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
>> > What's the chance of regaining DMA memory by killing user land processes?
>> > >
>> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-25 22:41 ` Martin J. Bligh
2005-07-25 15:46 ` Marcelo Tosatti
@ 2005-07-26 0:35 ` James Washer
2005-07-25 17:10 ` Marcelo Tosatti
2005-07-26 13:29 ` Martin J. Bligh
1 sibling, 2 replies; 14+ messages in thread
From: James Washer @ 2005-07-26 0:35 UTC (permalink / raw)
To: Martin J. Bligh; +Cc: marcelo.tosatti, linux-mm, ak
Already been done... but I'd not had much time to chase it.. This is my desktop/workstation, so difficult for me to refine the debugging, in any event, here's the backtrace. This is consistent across about 20 events.
kernel: Call Trace:<ffffffff80156ba8>{out_of_memory+275} <ffffffff80147fe5>{autoremove_wake_function+0}
kernel: <ffffffff80157e8b>{__alloc_pages+793} <ffffffff8015a99b>{cache_grow+269}
kernel: <ffffffff8015adf4>{cache_alloc_refill+442} <ffffffff8015a883>{kmem_cache_alloc+92}
kernel: <ffffffff88000f0b>{:sd_mod:sd_revalidate_disk+155}
kernel: <ffffffff8015d549>{pagevec_lookup+23} <ffffffff8015d9a0>{invalidate_mapping_pages+208}
kernel: <ffffffff80176292>{invalidate_bh_lru+0} <ffffffff8011b9ef>{flat_send_IPI_allbutself+20}
kernel: <ffffffff801198df>{smp_call_function+62} <ffffffff8017b466>{check_disk_change+89}
kernel: <ffffffff88000625>{:sd_mod:sd_open+257} <ffffffff8017b73e>{do_open+190}
kernel: <ffffffff8017bb02>{blkdev_open+33} <ffffffff80173768>{dentry_open+224}
kernel: <ffffffff801738a2>{filp_open+63} <ffffffff8017398a>{get_unused_fd+220}
kernel: <ffffffff80173a8b>{sys_open+62} <ffffffff8010e29e>{system_call+126}
kernel:
On Mon, 25 Jul 2005 15:41:27 -0700
"Martin J. Bligh" <mbligh@mbligh.org> wrote:
> Jim, does seem bloody silly to be shooting stuff here, and is
> probably simple to fix ... however, would be useful to see where
> the DMA allocs are coming from as well, any chance you could dump
> a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
>
> M.
>
> --On Monday, July 25, 2005 12:11:30 -0700 James Washer <washer@trlp.com> wrote:
>
> > Pretty typical message here...
> > Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
> > Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
> > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
> > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
> > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
> > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
> > Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
> > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
> > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
> > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
> > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
> > Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
> > Jul 6 17:31:27 p6 kernel:
> > Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
> > Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
> > Jul 6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
> > Jul 6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
> > Jul 6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
> > Jul 6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
> > Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
> > Jul 6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
> > Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
> > Jul 6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
> > Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
> > Jul 6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
> > Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
> > Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
> > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> > Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
> > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> > Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
> > Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
> > Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
> > Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
> > Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
> > Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
> > Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
> >
> >
> > On Sat, 23 Jul 2005 10:00:48 -0300
> > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> >
> >>
> >> James,
> >>
> >> Can you send the OOM killer output?
> >>
> >> I dont know which devices part of an x86-64 system should
> >> be limited to 16Mb of physical addressing. Andi?
> >>
> >> I don't think that any devices should have 16MB limitation
> >>
> >> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
> >> > Sorry, I should have added...
> >> > 2.6.11.10,
> >> > x86-64 dual proc (Intel Xeon 3.4GHz)
> >> > 6GiB ram
> >> > Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
> >> > Host: scsi0 Channel: 00 Id: 00 Lun: 00
> >> > Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
> >> > Type: Direct-Access ANSI SCSI revision: 05
> >> > Host: scsi0 Channel: 00 Id: 01 Lun: 00
> >> > Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
> >> > Type: Direct-Access ANSI SCSI revision: 05
> >> >
> >> >
> >> > On Mon, 18 Jul 2005 12:21:01 -0700
> >> > James Washer <washer@trlp.com> wrote:
> >> >
> >> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
> >> > drops below the minimum, and the OOM-Killer is fired off.
> >> > >
> >> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
> >> > What's the chance of regaining DMA memory by killing user land processes?
> >> > >
> >> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
> > --
> > To unsubscribe, send a message with 'unsubscribe linux-mm' in
> > the body to majordomo@kvack.org. For more info on Linux MM,
> > see: http://www.linux-mm.org/ .
> > Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> >
> >
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-26 0:35 ` James Washer
2005-07-25 17:10 ` Marcelo Tosatti
@ 2005-07-26 13:29 ` Martin J. Bligh
2005-07-26 15:17 ` Andi Kleen
1 sibling, 1 reply; 14+ messages in thread
From: Martin J. Bligh @ 2005-07-26 13:29 UTC (permalink / raw)
To: James Washer; +Cc: marcelo.tosatti, linux-mm, ak, James Bottomley
> Already been done... but I'd not had much time to chase it.. This is
> my desktop/workstation, so difficult for me to refine the debugging,
> in any event, here's the backtrace. This is consistent across about
> 20 events.
Humpf. Not sure why generic scsi code would decide it needed __GFP_DMA:
static int sd_revalidate_disk(struct gendisk *disk) {
...
buffer = kmalloc(512, GFP_KERNEL | __GFP_DMA);
if (!buffer) {
printk(KERN_WARNING "(sd_revalidate_disk:) Memory allocation "
"failure.\n");
goto out_release_request;
}
Trouble is, ZONE_DMA seems to mean different things on different platforms.
There's even a comment reflecting this crappiness above __kmalloc:
* Additionally, the %GFP_DMA flag may be set to indicate the memory
* must be suitable for DMA. This can mean different things on different
* platforms. For example, on i386, it means that the memory must come
* from the first 16MB.
But that's really for ISA DMA, which nobody uses any more apart from the
floppy disk, and the stone-tablet adaptor. For now, I'm guessing that if
you remove that __GFP_DMA, your machine will be happier, but it's not
the right fix.
James, am I right in thinking you really want something else there,
maybe dependant on the device (which I doubt you even know in sd.c?)
M.
> kernel: Call Trace:<ffffffff80156ba8>{out_of_memory+275} <ffffffff80147fe5>{autoremove_wake_function+0}
> kernel: <ffffffff80157e8b>{__alloc_pages+793} <ffffffff8015a99b>{cache_grow+269}
> kernel: <ffffffff8015adf4>{cache_alloc_refill+442} <ffffffff8015a883>{kmem_cache_alloc+92}
> kernel: <ffffffff88000f0b>{:sd_mod:sd_revalidate_disk+155}
> kernel: <ffffffff8015d549>{pagevec_lookup+23} <ffffffff8015d9a0>{invalidate_mapping_pages+208}
> kernel: <ffffffff80176292>{invalidate_bh_lru+0} <ffffffff8011b9ef>{flat_send_IPI_allbutself+20}
> kernel: <ffffffff801198df>{smp_call_function+62} <ffffffff8017b466>{check_disk_change+89}
> kernel: <ffffffff88000625>{:sd_mod:sd_open+257} <ffffffff8017b73e>{do_open+190}
> kernel: <ffffffff8017bb02>{blkdev_open+33} <ffffffff80173768>{dentry_open+224}
> kernel: <ffffffff801738a2>{filp_open+63} <ffffffff8017398a>{get_unused_fd+220}
> kernel: <ffffffff80173a8b>{sys_open+62} <ffffffff8010e29e>{system_call+126}
> kernel:
>
>
> On Mon, 25 Jul 2005 15:41:27 -0700
> "Martin J. Bligh" <mbligh@mbligh.org> wrote:
>
>> Jim, does seem bloody silly to be shooting stuff here, and is
>> probably simple to fix ... however, would be useful to see where
>> the DMA allocs are coming from as well, any chance you could dump
>> a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
>>
>> M.
>>
>> --On Monday, July 25, 2005 12:11:30 -0700 James Washer <washer@trlp.com> wrote:
>>
>> > Pretty typical message here...
>> > Jul 6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
>> > Jul 6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
>> > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
>> > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1
>> > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
>> > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1
>> > Jul 6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
>> > Jul 6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
>> > Jul 6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
>> > Jul 6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
>> > Jul 6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
>> > Jul 6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
>> > Jul 6 17:31:27 p6 kernel:
>> > Jul 6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
>> > Jul 6 17:31:37 p6 kernel: Free pages: 16236kB (0kB HighMem)
>> > Jul 6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
>> > Jul 6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
>> > Jul 6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
>> > Jul 6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
>> > Jul 6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
>> > Jul 6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
>> > Jul 6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
>> > Jul 6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
>> > Jul 6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
>> > Jul 6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
>> > Jul 6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
>> > Jul 6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
>> > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
>> > Jul 6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>> > Jul 6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
>> > Jul 6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>> > Jul 6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
>> > Jul 6 17:32:13 p6 kernel: Node 0 HighMem: empty
>> > Jul 6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
>> > Jul 6 17:32:13 p6 kernel: Free swap = 8385728kB
>> > Jul 6 17:32:13 p6 kernel: Total swap = 8385920kB
>> > Jul 6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
>> >
>> >
>> > On Sat, 23 Jul 2005 10:00:48 -0300
>> > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
>> >
>> >>
>> >> James,
>> >>
>> >> Can you send the OOM killer output?
>> >>
>> >> I dont know which devices part of an x86-64 system should
>> >> be limited to 16Mb of physical addressing. Andi?
>> >>
>> >> I don't think that any devices should have 16MB limitation
>> >>
>> >> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
>> >> > Sorry, I should have added...
>> >> > 2.6.11.10,
>> >> > x86-64 dual proc (Intel Xeon 3.4GHz)
>> >> > 6GiB ram
>> >> > Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
>> >> > Host: scsi0 Channel: 00 Id: 00 Lun: 00
>> >> > Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
>> >> > Type: Direct-Access ANSI SCSI revision: 05
>> >> > Host: scsi0 Channel: 00 Id: 01 Lun: 00
>> >> > Vendor: ATA Model: Maxtor 7Y250M0 Rev: YAR5
>> >> > Type: Direct-Access ANSI SCSI revision: 05
>> >> >
>> >> >
>> >> > On Mon, 18 Jul 2005 12:21:01 -0700
>> >> > James Washer <washer@trlp.com> wrote:
>> >> >
>> >> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
>> >> > drops below the minimum, and the OOM-Killer is fired off.
>> >> > >
>> >> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory.
>> >> > What's the chance of regaining DMA memory by killing user land processes?
>> >> > >
>> >> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.
>> > --
>> > To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> > the body to majordomo@kvack.org. For more info on Linux MM,
>> > see: http://www.linux-mm.org/ .
>> > Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>> >
>> >
>>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-23 13:00 ` Marcelo Tosatti
2005-07-25 19:11 ` James Washer
@ 2005-07-26 13:53 ` Andi Kleen
1 sibling, 0 replies; 14+ messages in thread
From: Andi Kleen @ 2005-07-26 13:53 UTC (permalink / raw)
To: Marcelo Tosatti; +Cc: James Washer, linux-mm
On Sat, Jul 23, 2005 at 10:00:48AM -0300, Marcelo Tosatti wrote:
>
> James,
>
> Can you send the OOM killer output?
>
> I dont know which devices part of an x86-64 system should
> be limited to 16Mb of physical addressing. Andi?
Could be old devices like the floppy (it does a single GFP_DMA
allocation). Or a few devices that have >16MB limits (like aacraid
or some old sound chips) but there is no other zone for them
right now.
-Andi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-26 13:29 ` Martin J. Bligh
@ 2005-07-26 15:17 ` Andi Kleen
2005-07-26 16:34 ` Martin J. Bligh
0 siblings, 1 reply; 14+ messages in thread
From: Andi Kleen @ 2005-07-26 15:17 UTC (permalink / raw)
To: Martin J. Bligh; +Cc: James Washer, marcelo.tosatti, linux-mm, James Bottomley
> But that's really for ISA DMA, which nobody uses any more apart from the
> floppy disk, and the stone-tablet adaptor. For now, I'm guessing that if
> you remove that __GFP_DMA, your machine will be happier, but it's not
> the right fix.
iirc the reason for that was that someone could load an old ISA SCSI controller
later as a module and it needs to handle that. Perhaps make it dependent
on CONFIG_ISA ? But even that would not help on distribution kernels.
Another way would be to check in PCI systems if there is a ISA
bridge and for others assume ISA is there.
-Andi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Question about OOM-Killer
2005-07-26 15:17 ` Andi Kleen
@ 2005-07-26 16:34 ` Martin J. Bligh
0 siblings, 0 replies; 14+ messages in thread
From: Martin J. Bligh @ 2005-07-26 16:34 UTC (permalink / raw)
To: Andi Kleen
Cc: James Washer, marcelo.tosatti, linux-mm, James Bottomley, linux-kernel
--Andi Kleen <ak@muc.de> wrote (on Tuesday, July 26, 2005 17:17:54 +0200):
>> But that's really for ISA DMA, which nobody uses any more apart from the
>> floppy disk, and the stone-tablet adaptor. For now, I'm guessing that if
>> you remove that __GFP_DMA, your machine will be happier, but it's not
>> the right fix.
>
> iirc the reason for that was that someone could load an old ISA SCSI controller
> later as a module and it needs to handle that. Perhaps make it dependent
> on CONFIG_ISA ? But even that would not help on distribution kernels.
> Another way would be to check in PCI systems if there is a ISA
> bridge and for others assume ISA is there.
Yeah, the CONFIG_ISA thing makes a lot of sense to me ... however, would
be even better if we can work out (easily) what the disk is attatched to.
Pah, ISA is so shit. Generically, might be useful if we had a
__GFP_DMA_IF_ISA or something defined in the header files, rather than
just shoving ifdef's all over the place.
OTOH, Jim is right ... the OOM killer is being somewhat psycopathic. Seems
we need 2 fixes ;-)
M.
PS. Warning, this is wholly untested, and generally a bit shit.
PPS. jejb ... your mail bounces.
diff -aurpN -X /home/fletch/.diff.exclude virgin/drivers/scsi/sd.c isa_dma/drivers/scsi/sd.c
--- virgin/drivers/scsi/sd.c 2005-07-26 09:25:40.000000000 -0700
+++ isa_dma/drivers/scsi/sd.c 2005-07-26 09:32:05.000000000 -0700
@@ -1468,7 +1468,7 @@ static int sd_revalidate_disk(struct gen
goto out;
}
- buffer = kmalloc(512, GFP_KERNEL | __GFP_DMA);
+ buffer = kmalloc(512, GFP_KERNEL | __GFP_DMA_IF_ISA);
if (!buffer) {
printk(KERN_WARNING "(sd_revalidate_disk:) Memory allocation "
"failure.\n");
diff -aurpN -X /home/fletch/.diff.exclude virgin/include/linux/gfp.h isa_dma/include/linux/gfp.h
--- virgin/include/linux/gfp.h 2005-07-26 09:26:02.000000000 -0700
+++ isa_dma/include/linux/gfp.h 2005-07-26 09:29:38.000000000 -0700
@@ -13,6 +13,11 @@ struct vm_area_struct;
*/
/* Zone modifiers in GFP_ZONEMASK (see linux/mmzone.h - low two bits) */
#define __GFP_DMA 0x01
+#ifdef CONFIG_ISA
+ #define __GFP_DMA_IF_ISA __GFP_DMA
+#else
+ #define __GFP_DMA_IF_ISA 0
+#endif
#define __GFP_HIGHMEM 0x02
/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Question about oom-killer
@ 2023-05-31 8:42 Gou Hao
0 siblings, 0 replies; 14+ messages in thread
From: Gou Hao @ 2023-05-31 8:42 UTC (permalink / raw)
To: linux-mm, linux-kernel
hello everyone,
Recently, my kernel restarted while I was running ltp-oom02(It allocates
memory infinitely in a loop, testing whether the oom-killer works
properly ).
log:
```
[480156.950100] Tasks state (memory values in pages):
[480156.950101] [ pid ] uid tgid total_vm rss pgtables_bytes
swapents oom_score_adj name
[480156.950302] [ 2578] 81 2578 523 0 393216
6 -900 dbus-daemon
[480156.950309] [ 2648] 172 2596 2435 0 393216
5 0 rtkit-daemon
[480156.950322] [ 5256] 0 2826 25411 0 589824
0 0 DetectThread
[480156.950328] [ 5404] 0 5404 412 2 393216
64 -1000 sshd
[480156.950357] [ 10518] 0 10518 2586 0 393216
10 0 at-spi2-registr
[480156.950361] [ 10553] 0 10551 10543 0 458752
9 0 QXcbEventQueue
[480156.950365] [ 10867] 0 10567 17579 0 589824
16 0 QXcbEventQueue
[480156.950370] [ 10928] 0 10921 6999 0 458752
17 0 QXcbEventQueue
[480156.950390] [ 11882] 0 11811 7377 0 458752
10 0 QXcbEventQueue
[480156.950394] [ 12052] 0 12052 5823 0 458752
21 0 fcitx
[480156.950404] [ 12115] 0 12114 11678 0 524288
21 0 QXcbEventQueue
[480156.950408] [ 101558] 0 101558 3549 0 393216
0 0 runltp
[480156.950486] [1068864] 0 1068864 771 6 327680
85 -1000 systemd-udevd
[480156.950552] [1035639] 0 1035639 52 0 393216
14 -1000 oom02
[480156.950556] [1035640] 0 1035640 52 0 393216
23 -1000 oom02
[480156.950561] [1036065] 0 1036065 493 60 393216
0 -250 systemd-journal
[480156.950565] [1036087] 0 1036073 6258739 3543942
37814272 0 0 oom02
[480156.950572] Out of memory and no killable processes...
[480156.950575] Kernel panic - not syncing: System is deadlocked on memory
```
oom02-1036073 has been already killed before crash.
log:
```
[480152.242506] [1035177] 0 1035177 4773 20 393216
115 0 sssd_nss
[480152.242510] [1035376] 0 1035376 25500 391 589824
602 0 tuned
[480152.242514] [1035639] 0 1035639 52 0 393216
14 -1000 oom02
[480152.242517] [1035640] 0 1035640 52 0 393216
19 -1000 oom02
[480152.242522] [1036065] 0 1036065 493 114 393216
62 -250 systemd-journal
[480152.242525] [1036073] 0 1036073 6258739 3540314 37814272
104 0 oom02
[480152.242529] Out of memory: Kill process 1036073 (oom02) score 755 or
sacrifice child
[480152.243869] Killed process 1036073 (oom02) total-vm:400559296kB,
anon-rss:226578368kB, file-rss:1728kB, shmem-rss:0kB
[480152.365804] oom_reaper: reaped process 1036073 (oom02), now
anon-rss:226594048kB, file-rss:0kB, shmem-rss:0kB
```
but its memory can not be reclaimed.I add trace-log to oom_reaper code
in kernel,
I found that there is a large range vma in the memory that cannot be
reclaimed, and the vma has the `VM_LOCKED` flag, so cannot be reclaimed
immediately.
```log
oom_reaper-57 [007] .... 126.063581: __oom_reap_task_mm: gh:
vma is anon:1048691, range=65536
oom_reaper-57 [007] .... 126.063581: __oom_reap_task_mm: gh:
vma is anon:1048691, range=196608
oom_reaper-57 [007] .... 126.063582: __oom_reap_task_mm: gh:
vma continue: 1056883, range:3221225472
oom_reaper-57 [007] .... 126.063583: __oom_reap_task_mm: gh:
vma is anon:112, range=65536
oom_reaper-57 [007] .... 126.063584: __oom_reap_task_mm: gh:
vma is anon:1048691, range=8388608
```
`vma continue: 1056883, range:3221225472` is the memory that can not
reclaims. 1057883(0x102073) is vma->vm_flags, it has VM_LOCKED` flag
oom02 created `nr_cpu` threads and used mmap to allocate memory. mmap
will merge continuous vma into one,
so as long as one thread is still running, the entire vma will not be
released.
In extreme cases, crashes may occur due to the lack of memory reclamation.
I'm not sure if this is a kernel's bug ?
--
thanks,
Gou Hao <gouhao@uniontech.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2023-05-31 8:43 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-07-18 19:21 Question about OOM-Killer James Washer
2005-07-18 19:36 ` James Washer
2005-07-23 13:00 ` Marcelo Tosatti
2005-07-25 19:11 ` James Washer
2005-07-25 12:27 ` Marcelo Tosatti
2005-07-25 22:41 ` Martin J. Bligh
2005-07-25 15:46 ` Marcelo Tosatti
2005-07-26 0:35 ` James Washer
2005-07-25 17:10 ` Marcelo Tosatti
2005-07-26 13:29 ` Martin J. Bligh
2005-07-26 15:17 ` Andi Kleen
2005-07-26 16:34 ` Martin J. Bligh
2005-07-26 13:53 ` Andi Kleen
2023-05-31 8:42 Question about oom-killer Gou Hao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox