Re: Question about OOM-Killer

[Marcelo Tosatti <marcelo.tosatti@cyclades.com>]

On Mon, Jul 25, 2005 at 05:35:13PM -0700, James Washer wrote:
> Already been done... but I'd not had much time to chase it.. This is my desktop/workstation, so difficult for me to refine the debugging, in any event, here's the backtrace. This is consistent across about 20 events.
> 
> 
>  kernel: Call Trace:<ffffffff80156ba8>{out_of_memory+275} <ffffffff80147fe5>{autoremove_wake_function+0}
>  kernel:        <ffffffff80157e8b>{__alloc_pages+793} <ffffffff8015a99b>{cache_grow+269}
>  kernel:        <ffffffff8015adf4>{cache_alloc_refill+442} <ffffffff8015a883>{kmem_cache_alloc+92}
>  kernel:        <ffffffff88000f0b>{:sd_mod:sd_revalidate_disk+155}
>  kernel:        <ffffffff8015d549>{pagevec_lookup+23} <ffffffff8015d9a0>{invalidate_mapping_pages+208}
>  kernel:        <ffffffff80176292>{invalidate_bh_lru+0} <ffffffff8011b9ef>{flat_send_IPI_allbutself+20}
>  kernel:        <ffffffff801198df>{smp_call_function+62} <ffffffff8017b466>{check_disk_change+89}
>  kernel:        <ffffffff88000625>{:sd_mod:sd_open+257} <ffffffff8017b73e>{do_open+190}
>  kernel:        <ffffffff8017bb02>{blkdev_open+33} <ffffffff80173768>{dentry_open+224}
>  kernel:        <ffffffff801738a2>{filp_open+63} <ffffffff8017398a>{get_unused_fd+220}
>  kernel:        <ffffffff80173a8b>{sys_open+62} <ffffffff8010e29e>{system_call+126}

/**
 *      sd_revalidate_disk - called the first time a new disk is seen,
 *      performs disk spin up, read_capacity, etc.
 *      @disk: struct gendisk we care about
 **/
static int sd_revalidate_disk(struct gendisk *disk)
{
...
        buffer = kmalloc(512, GFP_KERNEL | __GFP_DMA);
        if (!buffer) {
                printk(KERN_WARNING "(sd_revalidate_disk:) Memory allocation "
                       "failure.\n");
                goto out_release_request;
        }
	

        sd_spinup_disk(sdkp, disk->disk_name, sreq, buffer);


I suppose its playing safe to support 16Mb limited devices. Can't the 
gfp_mask be derived from lower level device characteristics?

On the VM side alloc_pages() should do better if conditioned to kill on
lowmem exhaustion and expect allocators to handle failure properly.

Or maybe to always return failure for in-kernel allocations, but one
might argue that kernel allocations _might_ be crucial for system
functionality (its a case by case basis) IIRC Andrea used to favour
failed DMA allocations to trigger the kill, for reasons I don't
recall.

The OOM killer, what a delight.

> On Mon, 25 Jul 2005 15:41:27 -0700
> "Martin J. Bligh" <mbligh@mbligh.org> wrote:
> 
> > Jim, does seem bloody silly to be shooting stuff here, and is
> > probably simple to fix ... however, would be useful to see where
> > the DMA allocs are coming from as well, any chance you could dump
> > a stack backtrace in __alloc_pages when we spec a mask for DMA alloc?
> > 
> > M.
> > 
> > --On Monday, July 25, 2005 12:11:30 -0700 James Washer <washer@trlp.com> wrote:
> > 
> > > Pretty typical message here...
> > > Jul  6 17:31:27 p6 kernel: oom-killer: gfp_mask=0xd1
> > > Jul  6 17:31:27 p6 kernel: Node 0 DMA per-cpu:
> > > Jul  6 17:31:27 p6 kernel: cpu 0 hot: low 2, high 6, batch 1
> > > Jul  6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 2, batch 1 
> > > Jul  6 17:31:27 p6 kernel: cpu 1 hot: low 2, high 6, batch 1
> > > Jul  6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 2, batch 1 
> > > Jul  6 17:31:27 p6 kernel: Node 0 Normal per-cpu:
> > > Jul  6 17:31:27 p6 kernel: cpu 0 hot: low 32, high 96, batch 16
> > > Jul  6 17:31:27 p6 kernel: cpu 0 cold: low 0, high 32, batch 16
> > > Jul  6 17:31:27 p6 kernel: cpu 1 hot: low 32, high 96, batch 16
> > > Jul  6 17:31:27 p6 kernel: cpu 1 cold: low 0, high 32, batch 16
> > > Jul  6 17:31:27 p6 kernel: Node 0 HighMem per-cpu: empty
> > > Jul  6 17:31:27 p6 kernel: 
> > > Jul  6 17:31:31 p6 gconfd (washer-7174): SIGHUP received, reloading all databases
> > > Jul  6 17:31:37 p6 kernel: Free pages:       16236kB (0kB HighMem)
> > > Jul  6 17:31:38 p6 su(pam_unix)[9041]: session closed for user root
> > > Jul  6 17:31:38 p6 su(pam_unix)[10645]: session closed for user root
> > > Jul  6 17:31:38 p6 su(pam_unix)[8044]: session closed for user root
> > > Jul  6 17:31:38 p6 su(pam_unix)[7228]: session closed for user root
> > > Jul  6 17:31:38 p6 su(pam_unix)[16136]: session closed for user root
> > > Jul  6 17:31:48 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
> > > Jul  6 17:31:49 p6 kernel: Active:596167 inactive:854867 dirty:624740 writeback:0 unstable:0 free:4059 slab:52688 mapped:595231 pagetables:4862
> > > Jul  6 17:32:00 p6 gconfd (washer-7174): Resolved address "xml:readwrite:/home/washer/.gconf" to a writable configuration source at position 1
> > > Jul  6 17:32:02 p6 kernel: Node 0 DMA free:20kB min:24kB low:28kB high:36kB active:0kB inactive:0kB present:16384kB pages_scanned:1 all_unreclaimable? yes
> > > Jul  6 17:32:04 p6 gconfd (washer-7174): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
> > > Jul  6 17:32:06 p6 kernel: lowmem_reserve[]: 0 7152 7152
> > > Jul  6 17:32:11 p6 kernel: Node 0 Normal free:16216kB min:10808kB low:13508kB high:16212kB active:2384668kB inactive:3419468kB present:7323648kB pages_scanned:0 all_unreclaimable? no
> > > Jul  6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0
> > > Jul  6 17:32:13 p6 kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
> > > Jul  6 17:32:13 p6 kernel: lowmem_reserve[]: 0 0 0 
> > > Jul  6 17:32:13 p6 kernel: Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
> > > Jul  6 17:32:13 p6 kernel: Node 0 Normal: 34*4kB 192*8kB 53*16kB 92*32kB 2*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 2*4096kB = 16216kB
> > > Jul  6 17:32:13 p6 kernel: Node 0 HighMem: empty
> > > Jul  6 17:32:13 p6 kernel: Swap cache: add 48, delete 48, find 0/0, race 0+0
> > > Jul  6 17:32:13 p6 kernel: Free swap  = 8385728kB
> > > Jul  6 17:32:13 p6 kernel: Total swap = 8385920kB
> > > Jul  6 17:32:13 p6 kernel: Out of Memory: Killed process 10475 (firefox-bin).
> > > 
> > > 
> > > On Sat, 23 Jul 2005 10:00:48 -0300
> > > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> > > 
> > >> 
> > >> James,
> > >> 
> > >> Can you send the OOM killer output? 
> > >> 
> > >> I dont know which devices part of an x86-64 system should 
> > >> be limited to 16Mb of physical addressing. Andi? 
> > >> 
> > >> I don't think that any devices should have 16MB limitation
> > >> 
> > >> On Mon, Jul 18, 2005 at 12:36:50PM -0700, James Washer wrote:
> > >> > Sorry, I should have added... 
> > >> > 	2.6.11.10, 
> > >> > 	x86-64 dual proc (Intel Xeon 3.4GHz)
> > >> > 	6GiB ram
> > >> > 	Intel Corporation 82801EB (ICH5) SATA Controller (rev 0)
> > >> > 	Host: scsi0 Channel: 00 Id: 00 Lun: 00
> > >> > 		Vendor: ATA      Model: Maxtor 6Y160M0   Rev: YAR5
> > >> > 		Type:   Direct-Access                    ANSI SCSI revision: 05
> > >> > 	Host: scsi0 Channel: 00 Id: 01 Lun: 00
> > >> > 		Vendor: ATA      Model: Maxtor 7Y250M0   Rev: YAR5
> > >> > 		Type:   Direct-Access                    ANSI SCSI revision: 05
> > >> > 
> > >> > 
> > >> > On Mon, 18 Jul 2005 12:21:01 -0700
> > >> > James Washer <washer@trlp.com> wrote:
> > >> > 
> > >> > > I'm chasing down a system problem where the DMA memory (x86-64, god knows why it is using DMA memory)
> > >> > drops below the minimum, and the OOM-Killer is fired off.
> > >> > > 
> > >> > > It just strikes me odd that the OOM-Killer would be called at all for DMA memory. 
> > >> > What's the chance of regaining DMA memory by killing user land processes?
> > >> > > 
> > >> > > I'll admit, I know very little about linux VM, so perhaps I'm missing how oom killing can be helpful here.  
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>