* kernel: __alloc_pages: 1-order allocation failed
@ 2001-08-27 20:14 Andrew Kay
2001-08-27 22:57 ` Daniel Phillips
0 siblings, 1 reply; 14+ messages in thread
From: Andrew Kay @ 2001-08-27 20:14 UTC (permalink / raw)
To: linux-mm
I am having some rather serious problems with the memory management (i
think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
of these errors in /var/log/messages.
Aug 24 15:08:04 dell63 kernel: __alloc_pages: 1-order allocation failed.
Aug 24 15:08:35 dell63 last message repeated 448 times
Aug 24 15:09:37 dell63 last message repeated 816 times
Aug 24 15:10:38 dell63 last message repeated 1147 times
I am running a Redhat 7.1 distro w/2.4.9 kernel on a Dell poweredge 6300
(4x500Mhz cpu, 4Gb ram). I get this error while running the specmail
2001 benchmarking software against our email server, Intrastore. The
system is very idle from what I can see. The sar output shows user cpu
at around 1% and everything else rather low as well. It seems to pop up
randomly and requires a reboot to fix it.
Is there any workarounds or something I can do to get a more useful
debug message than this? It doesn't seem to throw any other visible
errors. Maybe an older more stable kernel or less memory? I have the
sar output if anyone is interested. This bug is the only current
roadblock for me to publish specmail 2001 results to spec.org. It can
be reproduced fairly easily with a little setup time.
Thanks,
Andy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-27 22:57 ` Daniel Phillips
@ 2001-08-27 22:28 ` Marcelo Tosatti
2001-08-28 0:08 ` Daniel Phillips
2001-08-28 6:27 ` Roger Larsson
0 siblings, 2 replies; 14+ messages in thread
From: Marcelo Tosatti @ 2001-08-27 22:28 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Andrew Kay, linux-mm
On Tue, 28 Aug 2001, Daniel Phillips wrote:
> On August 27, 2001 10:14 pm, Andrew Kay wrote:
> > I am having some rather serious problems with the memory management (i
> > think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
> > of these errors in /var/log/messages.
> >
> > Aug 24 15:08:04 dell63 kernel: __alloc_pages: 1-order allocation failed.
> > Aug 24 15:08:35 dell63 last message repeated 448 times
> > Aug 24 15:09:37 dell63 last message repeated 816 times
> > Aug 24 15:10:38 dell63 last message repeated 1147 times
> >
> > I am running a Redhat 7.1 distro w/2.4.9 kernel on a Dell poweredge 6300
> > (4x500Mhz cpu, 4Gb ram). I get this error while running the specmail
> > 2001 benchmarking software against our email server, Intrastore. The
> > system is very idle from what I can see. The sar output shows user cpu
> > at around 1% and everything else rather low as well. It seems to pop up
> > randomly and requires a reboot to fix it.
> >
> > Is there any workarounds or something I can do to get a more useful
> > debug message than this?
>
> Please apply this patch:
>
> --- 2.4.9.clean/mm/page_alloc.c Thu Aug 16 12:43:02 2001
> +++ 2.4.9/mm/page_alloc.c Mon Aug 20 22:05:40 2001
> @@ -502,7 +502,8 @@
> }
>
> /* No luck.. */
> - printk(KERN_ERR "__alloc_pages: %lu-order allocation failed.\n", order);
> + printk(KERN_ERR "__alloc_pages: %lu-order allocation failed (gfp=0x%x/%i).\n",
> + order, gfp_mask, !!(current->flags & PF_MEMALLOC));
> return NULL;
> }
>
Daniel,
Its probably the bounce buffering thingie.
I'll send a patch to Linus soon.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-27 20:14 kernel: __alloc_pages: 1-order allocation failed Andrew Kay
@ 2001-08-27 22:57 ` Daniel Phillips
2001-08-27 22:28 ` Marcelo Tosatti
0 siblings, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-27 22:57 UTC (permalink / raw)
To: Andrew Kay, linux-mm
On August 27, 2001 10:14 pm, Andrew Kay wrote:
> I am having some rather serious problems with the memory management (i
> think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
> of these errors in /var/log/messages.
>
> Aug 24 15:08:04 dell63 kernel: __alloc_pages: 1-order allocation failed.
> Aug 24 15:08:35 dell63 last message repeated 448 times
> Aug 24 15:09:37 dell63 last message repeated 816 times
> Aug 24 15:10:38 dell63 last message repeated 1147 times
>
> I am running a Redhat 7.1 distro w/2.4.9 kernel on a Dell poweredge 6300
> (4x500Mhz cpu, 4Gb ram). I get this error while running the specmail
> 2001 benchmarking software against our email server, Intrastore. The
> system is very idle from what I can see. The sar output shows user cpu
> at around 1% and everything else rather low as well. It seems to pop up
> randomly and requires a reboot to fix it.
>
> Is there any workarounds or something I can do to get a more useful
> debug message than this?
Please apply this patch:
--- 2.4.9.clean/mm/page_alloc.c Thu Aug 16 12:43:02 2001
+++ 2.4.9/mm/page_alloc.c Mon Aug 20 22:05:40 2001
@@ -502,7 +502,8 @@
}
/* No luck.. */
- printk(KERN_ERR "__alloc_pages: %lu-order allocation failed.\n", order);
+ printk(KERN_ERR "__alloc_pages: %lu-order allocation failed (gfp=0x%x/%i).\n",
+ order, gfp_mask, !!(current->flags & PF_MEMALLOC));
return NULL;
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-27 22:28 ` Marcelo Tosatti
@ 2001-08-28 0:08 ` Daniel Phillips
2001-08-29 13:48 ` Andrew Kay
2001-08-28 6:27 ` Roger Larsson
1 sibling, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-28 0:08 UTC (permalink / raw)
To: Marcelo Tosatti; +Cc: Andrew Kay, linux-mm
On August 28, 2001 12:28 am, Marcelo Tosatti wrote:
> On Tue, 28 Aug 2001, Daniel Phillips wrote:
> > On August 27, 2001 10:14 pm, Andrew Kay wrote:
> > > I am having some rather serious problems with the memory management (i
> > > think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
> > > of these errors in /var/log/messages.
> Its probably the bounce buffering thingie.
>
> I'll send a patch to Linus soon.
That's what I thought too, but I thought, why not give him the patch and be
sure.
--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-27 22:28 ` Marcelo Tosatti
2001-08-28 0:08 ` Daniel Phillips
@ 2001-08-28 6:27 ` Roger Larsson
1 sibling, 0 replies; 14+ messages in thread
From: Roger Larsson @ 2001-08-28 6:27 UTC (permalink / raw)
To: Marcelo Tosatti, Daniel Phillips; +Cc: Andrew Kay, linux-mm
On Tuesdayen den 28 August 2001 00:28, Marcelo Tosatti wrote:
> On Tue, 28 Aug 2001, Daniel Phillips wrote:
> > On August 27, 2001 10:14 pm, Andrew Kay wrote:
> > > I am having some rather serious problems with the memory management (i
> > > think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
> > > of these errors in /var/log/messages.
> > >
> > > Aug 24 15:08:04 dell63 kernel: __alloc_pages: 1-order allocation
> > > failed. Aug 24 15:08:35 dell63 last message repeated 448 times
> > > Aug 24 15:09:37 dell63 last message repeated 816 times
> > > Aug 24 15:10:38 dell63 last message repeated 1147 times
> > >
> > > I am running a Redhat 7.1 distro w/2.4.9 kernel on a Dell poweredge
> > > 6300 (4x500Mhz cpu, 4Gb ram). I get this error while running the
> > > specmail 2001 benchmarking software against our email server,
> > > Intrastore. The system is very idle from what I can see. The sar
> > > output shows user cpu at around 1% and everything else rather low as
> > > well. It seems to pop up randomly and requires a reboot to fix it.
> > >
> > > Is there any workarounds or something I can do to get a more useful
> > > debug message than this?
> >
> > Please apply this patch:
> >
> > --- 2.4.9.clean/mm/page_alloc.c Thu Aug 16 12:43:02 2001
> > +++ 2.4.9/mm/page_alloc.c Mon Aug 20 22:05:40 2001
> > @@ -502,7 +502,8 @@
> > }
> >
> > /* No luck.. */
> > - printk(KERN_ERR "__alloc_pages: %lu-order allocation failed.\n",
> > order); + printk(KERN_ERR "__alloc_pages: %lu-order allocation failed
> > (gfp=0x%x/%i).\n", + order, gfp_mask, !!(current->flags & PF_MEMALLOC));
> > return NULL;
> > }
>
> Daniel,
>
> Its probably the bounce buffering thingie.
>
> I'll send a patch to Linus soon.
I have seen reports of this problem when running without HIGHMEM (from
Stephan von Krawczynski <skraw@ithnet.com>).
But he is running with knfs(d) and raiserfs...
In this configuration I really want to know who is responsible for the allocs.
So, please add a
show_trace(NULL);
too...
/RogerL
--
Roger Larsson
Skelleftea
Sweden
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-28 0:08 ` Daniel Phillips
@ 2001-08-29 13:48 ` Andrew Kay
2001-08-29 15:13 ` Daniel Phillips
0 siblings, 1 reply; 14+ messages in thread
From: Andrew Kay @ 2001-08-29 13:48 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Marcelo Tosatti, linux-mm
Here's some 'cut' output from /var/log/messages. There is a lot more
from where this came from. Some of it looks a bit different, I included
it below the first 3 errors. I can post the 165k gzipped messages file
somewhere if someone wants to look at the whole thing.
__alloc_pages: 1-order allocation failed (gfp=0x20/0).
Call Trace: [<c012db70>] [<c012de1e>] [<c012a69e>] [<c012aa21>]
[<c0211032>]
[<c02392da>] [<c023669f>] [<c02355c1>] [<c02399a1>] [<c01b000f>]
[<c011c9bc>]
[<c01b000f>] [<c02158e6>] [<c021a714>] [<c02158e6>] [<c022173d>]
[<c0221638>]
[<c0221b5d>] [<c0211f53>] [<c0221638>] [<c023099a>] [<c0211f53>]
[<c0211f68>]
[<c02120b9>] [<c023698e>] [<c0236c65>] [<c023711d>] [<c021f07f>]
[<c021f40a>]
[<c01b0571>] [<c0215fae>] [<c0119533>] [<c0108785>] [<c0105230>]
[<c0105230>]
[<c0106e34>] [<c0105230>] [<c0105230>] [<c010525c>] [<c01052c2>]
[<c0105000>]
[<c010505f>]
__alloc_pages: 1-order allocation failed (gfp=0x20/0).
Call Trace: [<c012db70>] [<c012de1e>] [<c012a69e>] [<c012aa21>]
[<c0211032>]
[<c02392da>] [<c023669f>] [<c02399a1>] [<c01b000f>] [<c021a714>]
[<c02158e6>]
[<c022173d>] [<c022245d>] [<c0222a83>] [<c0222890>] [<c01e0c9a>]
[<c01e0e94>]
[<c01e10f4>] [<c023698e>] [<c0236c65>] [<c023711d>] [<c021f07f>]
[<c021f40a>]
[<c01b0571>] [<c0215fae>] [<c0119533>] [<c0108785>] [<c0105230>]
[<c0105230>]
[<c0106e34>] [<c0105230>] [<c0105230>] [<c010525c>] [<c01052c2>]
[<c0105000>]
[<c010505f>]
__alloc_pages: 1-order allocation failed (gfp=0x20/0).
Call Trace: [<c012db70>] [<c012de1e>] [<c012a69e>] [<c012aa21>]
[<c0211032>]
[<c02392da>] [<c023669f>] [<c02399a1>] [<c01b000f>] [<c01b000f>]
[<c02158e6>]
[<c021a714>] [<c02158e6>] [<c022173d>] [<c0221638>] [<c0221b5d>]
[<c0211f53>]
[<c0221638>] [<c023099a>] [<c0230a4a>] [<c0211dca>] [<c0233137>]
[<c023698e>]
[<c0236c65>] [<c023711d>] [<c021f07f>] [<c021f40a>] [<c01b0571>]
[<c0215fae>]
[<c0119533>] [<c0108785>] [<c0105230>] [<c0105230>] [<c0106e34>]
[<c0105230>]
[<c0105230>] [<c010525c>] [<c01052c2>] [<c01ffaf7>] [<c019266e>]
__alloc_pages: 1-order allocation failed (gfp=0x20/0).
__alloc_pages: 1-order allocation failed (gfp=0x20/0).
Call Trace: [<c012db70>] [<c012de1e>] [<c012a69e>] [<c012aa21>]
[<c0211032>]
<3>__alloc_pages: 1-order allocation failed (gfp=0x20/1).
[<c02392da>] Call Trace: [<c023669f>] [<c012db70>] [<c02399a1>]
[<c012de1e>] [<c01b000f>] [<c012a69e>] [<c021a714>] [<c012aa21>]
[<c02158e6>] [<c0211032>]
[<c022173d>] [<c02392da>] [<c0221638>] [<c023669f>] [<c0221b5d>]
[<c0211032>] [<c0221638>] [<c02399a1>] [<c01e0c9a>] [<c02399a1>]
[<c01e0e94>] [<c02399a1>]
[<c01e10f4>] [<c01b000f>] [<c022e9dc>] [<c0112437>] [<c023698e>]
[<c021a714>] [<c0236c65>] [<c02158e6>] [<c023711d>] [<c01e0c9a>]
[<c021f07f>] [<c0112437>]
[<c021f40a>] [<c0112437>] [<c01b0571>] [<c01e10f4>] [<c0215fae>]
[<c023698e>] [<c0119533>] [<c0236c65>] [<c0108785>] [<c023711d>]
[<c0105230>] [<c021f07f>]
[<c0105230>] [<c021f40a>] [<c0106e34>] [<c01b0571>] [<c0105230>]
[<c0215fae>] [<c0105230>] [<c0119533>] [<c010525c>] [<c0108785>]
[<c01052c2>] [<c0106e34>]
[<c01ffaf7>] [<c0120018>] [<c019266e>] [<c012bdfb>]
[<c012beee>] [<c012bf73>] [<c012c04c>] [<c012cf85>]
[<c012d081>] [<c0105000>] [<c0105573>]
Andy
Daniel Phillips wrote:
> On August 28, 2001 12:28 am, Marcelo Tosatti wrote:
>
>>On Tue, 28 Aug 2001, Daniel Phillips wrote:
>>
>>>On August 27, 2001 10:14 pm, Andrew Kay wrote:
>>>
>>>>I am having some rather serious problems with the memory management (i
>>>>think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
>>>>of these errors in /var/log/messages.
>>>>
>>Its probably the bounce buffering thingie.
>>
>>I'll send a patch to Linus soon.
>>
>
> That's what I thought too, but I thought, why not give him the patch and be
> sure.
>
> --
> Daniel
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-29 13:48 ` Andrew Kay
@ 2001-08-29 15:13 ` Daniel Phillips
2001-08-29 16:14 ` Andrew Kay
0 siblings, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-29 15:13 UTC (permalink / raw)
To: Andrew Kay; +Cc: Marcelo Tosatti, linux-mm
On August 29, 2001 03:48 pm, Andrew Kay wrote:
> Here's some 'cut' output from /var/log/messages. There is a lot more
> from where this came from. Some of it looks a bit different, I included
> it below the first 3 errors. I can post the 165k gzipped messages file
> somewhere if someone wants to look at the whole thing.
>
> __alloc_pages: 1-order allocation failed (gfp=0x20/0).
> Call Trace: [<c012db70>] [<c012de1e>] [<c012a69e>] [<c012aa21>]
> [<c0211032>]
> [<c02392da>] [<c023669f>] [<c02355c1>] [<c02399a1>] [<c01b000f>]
> [<c011c9bc>]
> [<c01b000f>] [<c02158e6>] [<c021a714>] [<c02158e6>] [<c022173d>]
> [<c0221638>]
> [<c0221b5d>] [<c0211f53>] [<c0221638>] [<c023099a>] [<c0211f53>]
> [<c0211f68>]
> [<c02120b9>] [<c023698e>] [<c0236c65>] [<c023711d>] [<c021f07f>]
> [<c021f40a>]
> [<c01b0571>] [<c0215fae>] [<c0119533>] [<c0108785>] [<c0105230>]
> [<c0105230>]
> [<c0106e34>] [<c0105230>] [<c0105230>] [<c010525c>] [<c01052c2>]
> [<c0105000>]
> [<c010505f>]
> [similar]
>
> Daniel Phillips wrote:
> > On August 28, 2001 12:28 am, Marcelo Tosatti wrote:
> >
> >>On Tue, 28 Aug 2001, Daniel Phillips wrote:
> >>
> >>>On August 27, 2001 10:14 pm, Andrew Kay wrote:
> >>>
> >>>>I am having some rather serious problems with the memory management (i
> >>>>think) in the 2.4.x kernels. I am currently on the 2.4.9 and get lots
> >>>>of these errors in /var/log/messages.
> >>>>
> >>Its probably the bounce buffering thingie.
> >>
> >>I'll send a patch to Linus soon.
> >>
> >
> > That's what I thought too, but I thought, why not give him the patch and be
> > sure.
OK, it's not a bounce buffer because the allocation isn't __GFP_WAIT (0x10).
It's GFP_ATOMIC and there are several hundred of those throughout the kernel so
I'm not going to try to guess which one. Could you please pass a few of your
backtraces through ksymoops make them meaningful?
--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-29 15:13 ` Daniel Phillips
@ 2001-08-29 16:14 ` Andrew Kay
2001-08-29 18:00 ` Daniel Phillips
0 siblings, 1 reply; 14+ messages in thread
From: Andrew Kay @ 2001-08-29 16:14 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Marcelo Tosatti, linux-mm
> OK, it's not a bounce buffer because the allocation isn't __GFP_WAIT (0x10).
> It's GFP_ATOMIC and there are several hundred of those throughout the kernel so
> I'm not going to try to guess which one. Could you please pass a few of your
> backtraces through ksymoops make them meaningful?
> Daniel
I'm not sure I did this right, but here is my attempt. I ran a
'ksymoops' and gave it a couple of the errors. The parts that look
somewhat recognizable are the sk98lin, which is a Syskonnect gig over
copper card. It is the only module I have running on the system.
Andy
ksymoops 2.4.0 on i686 2.4.9. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.9/ (default)
-m /boot/System.map-2.4.9 (default)
Warning: You did not tell me where to find symbol information. I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc. ksymoops -h explains the options.
Error (regular_file): read_system_map stat /boot/System.map-2.4.9 failed
Warning (compare_maps): mismatch on symbol SkInodeOps , sk98lin says
f88e0500, /lib/modules/2.4.9/kernel/drivers/net/sk98lin/sk98lin.o says
f88df9c0. Ignoring
/lib/modules/2.4.9/kernel/drivers/net/sk98lin/sk98lin.o entry
Aug 29 02:24:49 dell63 kernel: Call Trace: [<c012db70>] [<c012de1e>]
[<c012a69e>] [<c012aa21>] [<c0211032>]
Aug 29 02:24:49 dell63 kernel: [<c02392da>] [<c023669f>] [<c02399a1>]
[<c01b000f>] [<f88c30fa>] [<c01b000f>]
Aug 29 02:24:49 dell63 kernel: [<c021a714>] [<c02158e6>] [<c022173d>]
[<c0221638>] [<c0221b5d>] [<c0112437>]
Aug 29 02:24:49 dell63 kernel: [<c0221638>] [<c023099a>] [<c0112437>]
[<c0211f53>] [<c0211f68>] [<c02120b9>]
Aug 29 02:24:49 dell63 kernel: [<c023698e>] [<c0236c65>] [<c023711d>]
[<c021f07f>] [<c021f40a>] [<c0215fae>]
Aug 29 02:24:49 dell63 kernel: [<c0119533>] [<c0108785>] [<c0105230>]
[<c0105230>] [<c0106e34>] [<c0105230>]
Aug 29 02:24:49 dell63 kernel: [<c0105230>] [<c010525c>] [<c01052c2>]
[<c0105000>] [<c010505f>]
Aug 29 02:24:49 dell63 kernel: Call Trace: [<c012db70>] [<c012de1e>]
[<c012a69e>] [<c012aa21>] [<c0211032>]
Aug 29 02:24:49 dell63 kernel: [<c02392da>] [<c023669f>] [<c02399a1>]
[<c01b000f>] [<c021a714>] [<c02158e6>]
Aug 29 02:24:49 dell63 kernel: [<c022173d>] [<c0221638>] [<c0221b5d>]
[<c0221638>] [<c023099a>] [<c02314f1>]
Aug 29 02:24:49 dell63 kernel: [<c022ea31>] [<c022e9dc>] [<c023698e>]
[<c0236c65>] [<c023711d>] [<c021f07f>]
Aug 29 02:24:49 dell63 kernel: [<c021f40a>] [<c0215fae>] [<c0119533>]
[<c0108785>] [<c0105230>] [<c0105230>]
Aug 29 02:24:49 dell63 kernel: [<c0106e34>] [<c0105230>] [<c0105230>]
[<c010525c>] [<c01052c2>] [<c01ffaf7>]
Aug 29 02:24:49 dell63 kernel: [<c019266e>]
Warning (Oops_read): Code line not seen, dumping what data is available
Trace; c012db70 <_alloc_pages+18/1c>
Trace; c012de1e <__get_free_pages+a/18>
Trace; c012a69e <kmem_cache_destroy+1fe/4d8>
Trace; c012aa21 <kmem_cache_alloc+a9/bc>
Trace; c0211032 <sk_alloc+12/5c>
Trace; c02392da <ip_cmsg_recv+165b6/19944>
Trace; c023669f <ip_cmsg_recv+1397b/19944>
Trace; c02399a1 <ip_cmsg_recv+16c7d/19944>
Trace; c01b000f <loop_unregister_transfer+82a7/c9d4>
Trace; f88c30fa <[sk98lin]SkGeIsrOnePort+12e/144>
Trace; c01b000f <loop_unregister_transfer+82a7/c9d4>
Trace; c021a714 <qdisc_restart+14/250>
Trace; c02158e6 <dev_queue_xmit+136/374>
Trace; c022173d <ip_options_undo+981/1734>
Trace; c0221638 <ip_options_undo+87c/1734>
Trace; c0221b5d <ip_options_undo+da1/1734>
Trace; c0112437 <iounmap+2f7/360>
Trace; c0221638 <ip_options_undo+87c/1734>
Trace; c023099a <ip_cmsg_recv+dc76/19944>
Trace; c0112437 <iounmap+2f7/360>
Trace; c0211f53 <alloc_skb+293/304>
Trace; c0211f68 <alloc_skb+2a8/304>
Trace; c02120b9 <__kfree_skb+f5/fc>
Trace; c023698e <ip_cmsg_recv+13c6a/19944>
Trace; c0236c65 <ip_cmsg_recv+13f41/19944>
Trace; c023711d <ip_cmsg_recv+143f9/19944>
Trace; c021f07f <inet_del_protocol+35b/3d4>
Trace; c021f40a <ip_rcv+312/df4>
Trace; c0215fae <net_call_rx_atomic+1d6/2d4>
Trace; c0119533 <do_softirq+83/e0>
Trace; c0108785 <enable_irq+189/198>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0106e34 <__read_lock_failed+1174/2630>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0105230 <enable_hlt+8/178>
Trace; c010525c <enable_hlt+34/178>
Trace; c01052c2 <enable_hlt+9a/178>
Trace; c0105000 <gdt+4d94/4fb4>
Trace; c010505f <gdt+4df3/4fb4>
Trace; c012db70 <_alloc_pages+18/1c>
Trace; c012de1e <__get_free_pages+a/18>
Trace; c012a69e <kmem_cache_destroy+1fe/4d8>
Trace; c012aa21 <kmem_cache_alloc+a9/bc>
Trace; c0211032 <sk_alloc+12/5c>
Trace; c02392da <ip_cmsg_recv+165b6/19944>
Trace; c023669f <ip_cmsg_recv+1397b/19944>
Trace; c02399a1 <ip_cmsg_recv+16c7d/19944>
Trace; c01b000f <loop_unregister_transfer+82a7/c9d4>
Trace; c021a714 <qdisc_restart+14/250>
Trace; c02158e6 <dev_queue_xmit+136/374>
Trace; c022173d <ip_options_undo+981/1734>
Trace; c0221638 <ip_options_undo+87c/1734>
Trace; c0221b5d <ip_options_undo+da1/1734>
Trace; c0221638 <ip_options_undo+87c/1734>
Trace; c023099a <ip_cmsg_recv+dc76/19944>
Trace; c02314f1 <ip_cmsg_recv+e7cd/19944>
Trace; c022ea31 <ip_cmsg_recv+bd0d/19944>
Trace; c022e9dc <ip_cmsg_recv+bcb8/19944>
Trace; c023698e <ip_cmsg_recv+13c6a/19944>
Trace; c0236c65 <ip_cmsg_recv+13f41/19944>
Trace; c023711d <ip_cmsg_recv+143f9/19944>
Trace; c021f07f <inet_del_protocol+35b/3d4>
Trace; c021f40a <ip_rcv+312/df4>
Trace; c0215fae <net_call_rx_atomic+1d6/2d4>
Trace; c0119533 <do_softirq+83/e0>
Trace; c0108785 <enable_irq+189/198>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0106e34 <__read_lock_failed+1174/2630>
Trace; c0105230 <enable_hlt+8/178>
Trace; c0105230 <enable_hlt+8/178>
Trace; c010525c <enable_hlt+34/178>
Trace; c01052c2 <enable_hlt+9a/178>
Trace; c01ffaf7 <isapnp_resource_change+24c3/353c>
Trace; c019266e <secure_tcp_sequence_number+4746/4b38>
3 warnings and 1 error issued. Results may not be reliable.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-29 16:14 ` Andrew Kay
@ 2001-08-29 18:00 ` Daniel Phillips
2001-08-30 14:24 ` Andrew Kay
0 siblings, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-29 18:00 UTC (permalink / raw)
To: Andrew Kay; +Cc: Marcelo Tosatti, linux-mm
On August 29, 2001 06:14 pm, Andrew Kay wrote:
> > OK, it's not a bounce buffer because the allocation isn't __GFP_WAIT
(0x10).
> > It's GFP_ATOMIC and there are several hundred of those throughout the
kernel so
> > I'm not going to try to guess which one. Could you please pass a few of
your
> > backtraces through ksymoops make them meaningful?
> > Daniel
>
> I'm not sure I did this right, but here is my attempt. I ran a
> 'ksymoops' and gave it a couple of the errors. The parts that look
> somewhat recognizable are the sk98lin, which is a Syskonnect gig over
> copper card. It is the only module I have running on the system.
Close, but see the warning where it tells you your System.map doesn't match
your running kernel. Try symlinking /boot/System.map to the System.map in
the source tree you built from.
> Warning: You did not tell me where to find symbol information. I will
> assume that the log matches the kernel and modules that are running
> right now and I'll use the default options above for symbol resolution.
> If the current kernel and/or modules do not match the log, you can get
> more accurate output by telling me the kernel version and where to find
> map, modules, ksyms etc. ksymoops -h explains the options.
I'm willing to guess at this point that this atomic failure is not a bug, the
only bug is that we print the warning message, potentially slowing things
down. I'd like to see a correct backtrace first.
Do you detect any slowdown in your system when you're getting these messages?
I wouldn't expect so from what you've described so far.
--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-29 18:00 ` Daniel Phillips
@ 2001-08-30 14:24 ` Andrew Kay
2001-08-30 22:19 ` Daniel Phillips
0 siblings, 1 reply; 14+ messages in thread
From: Andrew Kay @ 2001-08-30 14:24 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Marcelo Tosatti, linux-mm
> I'm willing to guess at this point that this atomic failure is not a bug, the
> only bug is that we print the warning message, potentially slowing things
> down. I'd like to see a correct backtrace first.
>
> Do you detect any slowdown in your system when you're getting these messages?
> I wouldn't expect so from what you've described so far.
I don't notice any slowdown in the system, but certain operations hang.
Such as vmstat, ps, and our SMTP server. They lock up completely.
The load average is at 20, but the CPU is completely idle.
Here's another attempt at ksymoops output:
ksymoops 2.4.0 on i686 2.4.9. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.9/ (default)
-m /boot/System.map-2.4.9 (default)
Warning: You did not tell me where to find symbol information. I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc. ksymoops -h explains the options.
Aug 29 19:00:17 dhcp80-252 kernel: Call Trace: [<c0129b60>] [<c0129e0e>]
[<c0126e7a>] [<c0127029>] [<c01f8242>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c021c02a>] [<c0219a6b>]
[<c021c6b1>] [<c01a9024>] [<c0206464>] [<c0200413>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c0206464>] [<c01fc517>]
[<c0206464>] [<c020651d>] [<c0206464>] [<c0206935>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c0106bf4>] [<c0206400>]
[<c01fa441>] [<c0206935>] [<c01fa441>] [<c01f8fc3>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c01f8fd7>] [<c01f910b>]
[<c0219c9e>] [<c0219f3d>] [<c021a35b>] [<c020414f>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c02044a9>] [<c01a8de4>]
[<c01fca3d>] [<c0116f6d>] [<c0107ff4>] [<c01051a0>]
Aug 29 19:00:17 dhcp80-252 kernel: [<c01051a0>] [<c0106bf4>]
[<c01051a0>] [<c01051a0>] [<c01051c3>] [<c0105224>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0105000>] [<c0105027>]
Aug 29 19:00:18 dhcp80-252 kernel: Call Trace: [<c0129b60>] [<c0129e0e>]
[<c0126e7a>] [<c0127029>] [<c01f8242>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c021c02a>] [<c0219a6b>]
[<c021c6b1>] [<c01a9024>] [<c01a954e>] [<c0206464>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0200413>] [<c0206464>]
[<c01fc517>] [<c0206464>] [<c020651d>] [<c0206464>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0206935>] [<c01fc517>]
[<c0206464>] [<c020651d>] [<c0206464>] [<c0214500>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0214627>] [<c02150bf>]
[<c01f8fc3>] [<c01f8fd7>] [<c01f910b>] [<c0219c9e>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0219f3d>] [<c021a35b>]
[<c020414f>] [<c02044a9>] [<c01a8de4>] [<c01fca3d>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c0116f6d>] [<c0107ff4>]
[<c01051a0>] [<c01051a0>] [<c0106bf4>] [<c01051a0>]
Aug 29 19:00:18 dhcp80-252 kernel: [<c01051a0>] [<c01051c3>]
[<c0105224>] [<c0105000>] [<c0105027>]
Warning (Oops_read): Code line not seen, dumping what data is available
Trace; c0129b60 <_alloc_pages+18/1c>
Trace; c0129e0e <__get_free_pages+a/18>
Trace; c0126e7a <kmem_cache_grow+ce/234>
Trace; c0127029 <kmem_cache_alloc+49/58>
Trace; c01f8242 <sk_alloc+12/58>
Trace; c021c02a <tcp_create_openreq_child+16/44c>
Trace; c0219a6b <tcp_v4_syn_recv_sock+57/248>
Trace; c021c6b1 <tcp_check_req+251/380>
Trace; c01a9024 <speedo_refill_rx_buf+40/20c>
Trace; c0206464 <ip_output+0/f0>
Trace; c0200413 <qdisc_restart+13/c8>
Trace; c0206464 <ip_output+0/f0>
Trace; c01fc517 <dev_queue_xmit+117/264>
Trace; c0206464 <ip_output+0/f0>
Trace; c020651d <ip_output+b9/f0>
Trace; c0206464 <ip_output+0/f0>
Trace; c0206935 <ip_queue_xmit+3e1/540>
Trace; c0106bf4 <ret_from_intr+0/7>
Trace; c0206400 <ip_mc_output+108/16c>
Trace; c01fa441 <skb_copy_and_csum_bits+51/35c>
Trace; c0206935 <ip_queue_xmit+3e1/540>
Trace; c01fa441 <skb_copy_and_csum_bits+51/35c>
Trace; c01f8fc3 <skb_release_data+67/70>
Trace; c01f8fd7 <kfree_skbmem+b/58>
Trace; c01f910b <__kfree_skb+e7/f0>
Trace; c0219c9e <tcp_v4_hnd_req+42/150>
Trace; c0219f3d <tcp_v4_do_rcv+91/108>
Trace; c021a35b <tcp_v4_rcv+3a7/618>
Trace; c020414f <ip_local_deliver+eb/164>
Trace; c02044a9 <ip_rcv+2e1/338>
Trace; c01a8de4 <speedo_interrupt+b0/2b0>
Trace; c01fca3d <net_rx_action+135/208>
Trace; c0116f6d <do_softirq+5d/ac>
Trace; c0107ff4 <do_IRQ+98/a8>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051a0 <default_idle+0/28>
Trace; c0106bf4 <ret_from_intr+0/7>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051c3 <default_idle+23/28>
Trace; c0105224 <cpu_idle+3c/50>
Trace; c0105000 <_stext+0/0>
Trace; c0105027 <rest_init+27/28>
Trace; c0129b60 <_alloc_pages+18/1c>
Trace; c0129e0e <__get_free_pages+a/18>
Trace; c0126e7a <kmem_cache_grow+ce/234>
Trace; c0127029 <kmem_cache_alloc+49/58>
Trace; c01f8242 <sk_alloc+12/58>
Trace; c021c02a <tcp_create_openreq_child+16/44c>
Trace; c0219a6b <tcp_v4_syn_recv_sock+57/248>
Trace; c021c6b1 <tcp_check_req+251/380>
Trace; c01a9024 <speedo_refill_rx_buf+40/20c>
Trace; c01a954e <speedo_rx+326/344>
Trace; c0206464 <ip_output+0/f0>
Trace; c0200413 <qdisc_restart+13/c8>
Trace; c0206464 <ip_output+0/f0>
Trace; c01fc517 <dev_queue_xmit+117/264>
Trace; c0206464 <ip_output+0/f0>
Trace; c020651d <ip_output+b9/f0>
Trace; c0206464 <ip_output+0/f0>
Trace; c0206935 <ip_queue_xmit+3e1/540>
Trace; c01fc517 <dev_queue_xmit+117/264>
Trace; c0206464 <ip_output+0/f0>
Trace; c020651d <ip_output+b9/f0>
Trace; c0206464 <ip_output+0/f0>
Trace; c0214500 <tcp_transmit_skb+36c/540>
Trace; c0214627 <tcp_transmit_skb+493/540>
Trace; c02150bf <tcp_write_xmit+18f/2dc>
Trace; c01f8fc3 <skb_release_data+67/70>
Trace; c01f8fd7 <kfree_skbmem+b/58>
Trace; c01f910b <__kfree_skb+e7/f0>
Trace; c0219c9e <tcp_v4_hnd_req+42/150>
Trace; c0219f3d <tcp_v4_do_rcv+91/108>
Trace; c021a35b <tcp_v4_rcv+3a7/618>
Trace; c020414f <ip_local_deliver+eb/164>
Trace; c02044a9 <ip_rcv+2e1/338>
Trace; c01a8de4 <speedo_interrupt+b0/2b0>
Trace; c01fca3d <net_rx_action+135/208>
Trace; c0116f6d <do_softirq+5d/ac>
Trace; c0107ff4 <do_IRQ+98/a8>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051a0 <default_idle+0/28>
Trace; c0106bf4 <ret_from_intr+0/7>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051a0 <default_idle+0/28>
Trace; c01051c3 <default_idle+23/28>
Trace; c0105224 <cpu_idle+3c/50>
Trace; c0105000 <_stext+0/0>
Trace; c0105027 <rest_init+27/28>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-30 14:24 ` Andrew Kay
@ 2001-08-30 22:19 ` Daniel Phillips
2001-08-30 22:39 ` Andrew Kay
0 siblings, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-30 22:19 UTC (permalink / raw)
To: Andrew Kay; +Cc: Marcelo Tosatti, linux-mm
On August 30, 2001 04:24 pm, Andrew Kay wrote:
> > I'm willing to guess at this point that this atomic failure is not a bug, the
> > only bug is that we print the warning message, potentially slowing things
> > down. I'd like to see a correct backtrace first.
> >
> > Do you detect any slowdown in your system when you're getting these messages?
> > I wouldn't expect so from what you've described so far.
>
> I don't notice any slowdown in the system, but certain operations hang.
> Such as vmstat, ps, and our SMTP server. They lock up completely.
> The load average is at 20, but the CPU is completely idle.
Ouch. I don't have any particular difficulty figuring out where the failures
are coming from - I have some ideas on what to do about them, but those lock-ups
don't make sense given the information you've supplied. Would you try this on
2.4.9-ac4 please?
Also, the backtrace now makes sense, but just for the first 6 entries. Now that
you have your System.map properly linked, klogd should be decoding the backtrace.
What version is your klogd? Is your network code in modules or compiled in?
Could you supply a ps -aux that shows your hung processes? Even better,
backtrace with SysReq.
> Here's another attempt at ksymoops output:
> Trace; c0129b60 <_alloc_pages+18/1c>
> Trace; c0129e0e <__get_free_pages+a/18>
> Trace; c0126e7a <kmem_cache_grow+ce/234>
> Trace; c0127029 <kmem_cache_alloc+49/58>
> Trace; c01f8242 <sk_alloc+12/58>
> Trace; c021c02a <tcp_create_openreq_child+16/44c>
> Trace; c0219a6b <tcp_v4_syn_recv_sock+57/248>
> Trace; c021c6b1 <tcp_check_req+251/380>
> Trace; c01a9024 <speedo_refill_rx_buf+40/20c>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c0200413 <qdisc_restart+13/c8>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c01fc517 <dev_queue_xmit+117/264>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c020651d <ip_output+b9/f0>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c0206935 <ip_queue_xmit+3e1/540>
> Trace; c0106bf4 <ret_from_intr+0/7>
> Trace; c0206400 <ip_mc_output+108/16c>
> Trace; c01fa441 <skb_copy_and_csum_bits+51/35c>
> Trace; c0206935 <ip_queue_xmit+3e1/540>
> Trace; c01fa441 <skb_copy_and_csum_bits+51/35c>
> Trace; c01f8fc3 <skb_release_data+67/70>
> Trace; c01f8fd7 <kfree_skbmem+b/58>
> Trace; c01f910b <__kfree_skb+e7/f0>
> Trace; c0219c9e <tcp_v4_hnd_req+42/150>
> Trace; c0219f3d <tcp_v4_do_rcv+91/108>
> Trace; c021a35b <tcp_v4_rcv+3a7/618>
> Trace; c020414f <ip_local_deliver+eb/164>
> Trace; c02044a9 <ip_rcv+2e1/338>
> Trace; c01a8de4 <speedo_interrupt+b0/2b0>
> Trace; c01fca3d <net_rx_action+135/208>
> Trace; c0116f6d <do_softirq+5d/ac>
> Trace; c0107ff4 <do_IRQ+98/a8>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c0106bf4 <ret_from_intr+0/7>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051c3 <default_idle+23/28>
> Trace; c0105224 <cpu_idle+3c/50>
> Trace; c0105000 <_stext+0/0>
> Trace; c0105027 <rest_init+27/28>
> Trace; c0129b60 <_alloc_pages+18/1c>
> Trace; c0129e0e <__get_free_pages+a/18>
> Trace; c0126e7a <kmem_cache_grow+ce/234>
> Trace; c0127029 <kmem_cache_alloc+49/58>
> Trace; c01f8242 <sk_alloc+12/58>
> Trace; c021c02a <tcp_create_openreq_child+16/44c>
> Trace; c0219a6b <tcp_v4_syn_recv_sock+57/248>
> Trace; c021c6b1 <tcp_check_req+251/380>
> Trace; c01a9024 <speedo_refill_rx_buf+40/20c>
> Trace; c01a954e <speedo_rx+326/344>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c0200413 <qdisc_restart+13/c8>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c01fc517 <dev_queue_xmit+117/264>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c020651d <ip_output+b9/f0>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c0206935 <ip_queue_xmit+3e1/540>
> Trace; c01fc517 <dev_queue_xmit+117/264>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c020651d <ip_output+b9/f0>
> Trace; c0206464 <ip_output+0/f0>
> Trace; c0214500 <tcp_transmit_skb+36c/540>
> Trace; c0214627 <tcp_transmit_skb+493/540>
> Trace; c02150bf <tcp_write_xmit+18f/2dc>
> Trace; c01f8fc3 <skb_release_data+67/70>
> Trace; c01f8fd7 <kfree_skbmem+b/58>
> Trace; c01f910b <__kfree_skb+e7/f0>
> Trace; c0219c9e <tcp_v4_hnd_req+42/150>
> Trace; c0219f3d <tcp_v4_do_rcv+91/108>
> Trace; c021a35b <tcp_v4_rcv+3a7/618>
> Trace; c020414f <ip_local_deliver+eb/164>
> Trace; c02044a9 <ip_rcv+2e1/338>
> Trace; c01a8de4 <speedo_interrupt+b0/2b0>
> Trace; c01fca3d <net_rx_action+135/208>
> Trace; c0116f6d <do_softirq+5d/ac>
> Trace; c0107ff4 <do_IRQ+98/a8>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c0106bf4 <ret_from_intr+0/7>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051a0 <default_idle+0/28>
> Trace; c01051c3 <default_idle+23/28>
> Trace; c0105224 <cpu_idle+3c/50>
> Trace; c0105000 <_stext+0/0>
> Trace; c0105027 <rest_init+27/28>
--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-30 22:19 ` Daniel Phillips
@ 2001-08-30 22:39 ` Andrew Kay
2001-08-30 23:04 ` Daniel Phillips
0 siblings, 1 reply; 14+ messages in thread
From: Andrew Kay @ 2001-08-30 22:39 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Marcelo Tosatti, linux-mm
Daniel Phillips wrote:
> Ouch. I don't have any particular difficulty figuring out where the failures
> are coming from - I have some ideas on what to do about them, but those lock-ups
> don't make sense given the information you've supplied. Would you try this on
> 2.4.9-ac4 please?
>
> Also, the backtrace now makes sense, but just for the first 6 entries. Now that
> you have your System.map properly linked, klogd should be decoding the backtrace.
> What version is your klogd? Is your network code in modules or compiled in?
>
> Could you supply a ps -aux that shows your hung processes? Even better,
> backtrace with SysReq.
>
I am running the stock klogd (1.4.0) from the redhat 7.1 install. I'll
give it a try with the 2.4.9-ac4 tomorrow. The output you saw is from a
mostly static kernel (except for reiserfs). Ps -aux shows a bit of
output, but remember that it hangs after encountering the error... Mhsqd
is one of our products. I strongly suspect that the hung process is
SMTPserver, which isn't shown
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1408 80 ? S Aug29 0:03 init [3]
root 2 0.0 0.0 0 0 ? SW Aug29 0:00 [keventd]
root 3 0.0 0.0 0 0 ? SWN Aug29 0:01
[ksoftirqd_CPU0]
root 4 0.1 0.0 0 0 ? SW Aug29 2:01 [kswapd]
root 5 0.0 0.0 0 0 ? SW Aug29 0:00 [kreclaimd]
root 6 0.0 0.0 0 0 ? SW Aug29 0:00 [bdflush]
root 7 0.0 0.0 0 0 ? SW Aug29 0:01 [kupdated]
root 8 0.0 0.0 0 0 ? SW Aug29 0:00 [i2oevtd]
root 9 0.0 0.0 0 0 ? SW Aug29 0:00 [i2oblock]
rpc 443 0.0 0.0 1552 236 ? S Aug29 0:01 portmap
rpcuser 458 0.0 0.0 1640 0 ? SW Aug29 0:00 rpc.statd
root 553 0.0 0.0 18028 288 ? S Aug29 0:00 ypbind
root 554 0.0 0.0 18028 288 ? S Aug29 0:00 ypbind
root 555 0.0 0.0 18028 288 ? S Aug29 0:00 ypbind
root 557 0.0 0.0 18028 288 ? S Aug29 0:00 ypbind
root 604 0.0 0.0 1520 48 ? S Aug29 0:00
/usr/sbin/automount --timeout 60 /misc file /etc/auto.misc
root 630 0.0 0.0 1604 48 ? S Aug29 0:00
/usr/sbin/automount /idisk yp auto.idisk
daemon 648 0.0 0.0 1440 36 ? S Aug29 0:00 /usr/sbin/atd
root 665 0.0 0.0 2624 596 ? S Aug29 0:00
/usr/sbin/sshd
root 686 0.0 0.0 1628 128 ? S Aug29 0:00 crond
xfs 714 0.0 0.0 4368 52 ? S Aug29 0:00 xfs
-droppriv -daemon
root 739 0.0 0.0 1380 4 tty1 S Aug29 0:00
/sbin/mingetty tty1
root 740 0.0 0.0 1380 0 tty2 SW Aug29 0:00
/sbin/mingetty tty2
root 741 0.0 0.0 1380 0 tty3 SW Aug29 0:00
/sbin/mingetty tty3
root 742 0.0 0.0 1380 0 tty4 SW Aug29 0:00
/sbin/mingetty tty4
root 743 0.0 0.0 1380 0 tty5 SW Aug29 0:00
/sbin/mingetty tty5
root 744 0.0 0.0 1380 0 tty6 SW Aug29 0:00
/sbin/mingetty tty6
root 826 0.0 0.0 0 0 ? SW Aug29 0:00 [kreiserfsd]
root 847 0.0 0.0 1468 164 ? S Aug29 0:02 syslogd -m 0
root 853 0.0 0.0 1468 4 ? S Aug29 0:00 klogd -2
root 858 0.0 0.0 0 0 ? SW Aug29 1:24 [rpciod]
root 859 0.0 0.0 0 0 ? SW Aug29 0:00 [lockd]
root 908 0.0 0.0 2892 128 ? S Aug29 0:47
/var/process/exec/mhsqd -s 1000
-Andy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-30 22:39 ` Andrew Kay
@ 2001-08-30 23:04 ` Daniel Phillips
2001-08-31 13:44 ` Andrew Kay
0 siblings, 1 reply; 14+ messages in thread
From: Daniel Phillips @ 2001-08-30 23:04 UTC (permalink / raw)
To: Andrew Kay; +Cc: Marcelo Tosatti, linux-mm
On August 31, 2001 12:39 am, Andrew Kay wrote:
> I am running the stock klogd (1.4.0) from the redhat 7.1 install. I'll
> give it a try with the 2.4.9-ac4 tomorrow. The output you saw is from a
> mostly static kernel (except for reiserfs). Ps -aux shows a bit of
> output, but remember that it hangs after encountering the error... Mhsqd
> is one of our products. I strongly suspect that the hung process is
> SMTPserver, which isn't shown
OK, how about SysReq (t) for a backtrace of the stuck tasks.
--
Daniel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kernel: __alloc_pages: 1-order allocation failed
2001-08-30 23:04 ` Daniel Phillips
@ 2001-08-31 13:44 ` Andrew Kay
0 siblings, 0 replies; 14+ messages in thread
From: Andrew Kay @ 2001-08-31 13:44 UTC (permalink / raw)
To: Daniel Phillips; +Cc: Marcelo Tosatti, linux-mm
Unfortunately, I don't have this compiled in the kernel. I'm
recompiling now with the -ac5 patch and the debugging enabled. I'll
start another test with it and hopefully have better debug info within a
couple hours.
Andy
Daniel Phillips wrote:
> On August 31, 2001 12:39 am, Andrew Kay wrote:
>
>>I am running the stock klogd (1.4.0) from the redhat 7.1 install. I'll
>>give it a try with the 2.4.9-ac4 tomorrow. The output you saw is from a
>> mostly static kernel (except for reiserfs). Ps -aux shows a bit of
>>output, but remember that it hangs after encountering the error... Mhsqd
>>is one of our products. I strongly suspect that the hung process is
>>SMTPserver, which isn't shown
>>
>
> OK, how about SysReq (t) for a backtrace of the stuck tasks.
>
> --
> Daniel
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2001-08-31 13:48 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-08-27 20:14 kernel: __alloc_pages: 1-order allocation failed Andrew Kay
2001-08-27 22:57 ` Daniel Phillips
2001-08-27 22:28 ` Marcelo Tosatti
2001-08-28 0:08 ` Daniel Phillips
2001-08-29 13:48 ` Andrew Kay
2001-08-29 15:13 ` Daniel Phillips
2001-08-29 16:14 ` Andrew Kay
2001-08-29 18:00 ` Daniel Phillips
2001-08-30 14:24 ` Andrew Kay
2001-08-30 22:19 ` Daniel Phillips
2001-08-30 22:39 ` Andrew Kay
2001-08-30 23:04 ` Daniel Phillips
2001-08-31 13:44 ` Andrew Kay
2001-08-28 6:27 ` Roger Larsson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox