OOM-killer for zone DMA?

OOM-killer for zone DMA?

[Takashi Iwai]

Hi,

In the primary version of my DMA allocation patch, I tried to allocate
pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
as fallback.  But this doesn't work.  When the zone DMA is exhausted,
I oberseved endless OOM-killer.

Is this a desired behavior?  I don't think triggering OOM-killer for
zone DMA makes sense, because apps don't allocate pages in this
area...

Note that the driver tried to allocate bunch of single pages with
GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
only (no __GFP_REPEAT or such modifiers).


Takashi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Marcelo Tosatti]

On Fri, Aug 27, 2004 at 07:02:07PM +0200, Takashi Iwai wrote:
> Hi,
> 
> In the primary version of my DMA allocation patch, I tried to allocate
> pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
> as fallback.  But this doesn't work.  When the zone DMA is exhausted,
> I oberseved endless OOM-killer.
> 
> Is this a desired behavior?  I don't think triggering OOM-killer for
> zone DMA makes sense, because apps don't allocate pages in this
> area...
> 
> Note that the driver tried to allocate bunch of single pages with
> GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
> only (no __GFP_REPEAT or such modifiers).

Takashi, 

The OOM killer heuristics seems to be pretty screwed up in current v2.6. 

Must be fixed ASAP.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[William Lee Irwin III]

On Fri, Aug 27, 2004 at 07:02:07PM +0200, Takashi Iwai wrote:
> In the primary version of my DMA allocation patch, I tried to allocate
> pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
> as fallback.  But this doesn't work.  When the zone DMA is exhausted,
> I oberseved endless OOM-killer.
> Is this a desired behavior?  I don't think triggering OOM-killer for
> zone DMA makes sense, because apps don't allocate pages in this
> area...
> Note that the driver tried to allocate bunch of single pages with
> GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
> only (no __GFP_REPEAT or such modifiers).

How are you triggering this?


-- wli
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Takashi Iwai]

At Fri, 27 Aug 2004 23:04:26 -0700,
William Lee Irwin III wrote:
> 
> On Fri, Aug 27, 2004 at 07:02:07PM +0200, Takashi Iwai wrote:
> > In the primary version of my DMA allocation patch, I tried to allocate
> > pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
> > as fallback.  But this doesn't work.  When the zone DMA is exhausted,
> > I oberseved endless OOM-killer.
> > Is this a desired behavior?  I don't think triggering OOM-killer for
> > zone DMA makes sense, because apps don't allocate pages in this
> > area...
> > Note that the driver tried to allocate bunch of single pages with
> > GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
> > only (no __GFP_REPEAT or such modifiers).
> 
> How are you triggering this?

Tried to allocate as many single pages as possible with
dma_alloc_coherent() with 31bit coherent_dma_mask.


Takashi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Nick Piggin]

Takashi Iwai wrote:
> Hi,
> 
> In the primary version of my DMA allocation patch, I tried to allocate
> pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
> as fallback.  But this doesn't work.  When the zone DMA is exhausted,
> I oberseved endless OOM-killer.
> 
> Is this a desired behavior?  I don't think triggering OOM-killer for
> zone DMA makes sense, because apps don't allocate pages in this
> area...
> 

They easily could.

> Note that the driver tried to allocate bunch of single pages with
> GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
> only (no __GFP_REPEAT or such modifiers).
> 

You at least need __GFP_NORETRY to achieve what you want.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Takashi Iwai]

At Sat, 28 Aug 2004 16:10:02 +1000,
Nick Piggin wrote:
> 
> Takashi Iwai wrote:
> > Hi,
> > 
> > In the primary version of my DMA allocation patch, I tried to allocate
> > pages with GFP_DMA as much as possible, then allocate with GFP_KERNEL
> > as fallback.  But this doesn't work.  When the zone DMA is exhausted,
> > I oberseved endless OOM-killer.
> > 
> > Is this a desired behavior?  I don't think triggering OOM-killer for
> > zone DMA makes sense, because apps don't allocate pages in this
> > area...
> > 
> 
> They easily could.
> 
> > Note that the driver tried to allocate bunch of single pages with
> > GFP_DMA, not big pages, by calling dma_alloc_coherent with GFP_DMA
> > only (no __GFP_REPEAT or such modifiers).
> > 
> 
> You at least need __GFP_NORETRY to achieve what you want.

Yes, with that flag it can be avoided.

But it *should* retry.  It's an allocation of single page, and the
caller of dma_alloc_coherent() doesn't know whether it's allocated
from zone DMA or zone normal.  It sets just the coherent_dma_mask to a
value less than 32 bit.

This situation may happen even after applying my patch.
If you have more RAM than mask, allocation in the zone NORMAL may hit
the outside of mask, and tries the zone DMA as fallback, although
there are pretty enough free RAM in the zone NORMAL.

So, triggering oom-killer for zone DMA is non-sense, IMO.


Takashi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Nick Piggin]

Takashi Iwai wrote:
> At Sat, 28 Aug 2004 16:10:02 +1000,
> Nick Piggin wrote:
> 

>>
>>You at least need __GFP_NORETRY to achieve what you want.
> 
> 
> Yes, with that flag it can be avoided.
> 

Great.

> But it *should* retry.

That is precisely the opposite of what you want.

>  It's an allocation of single page, and the
> caller of dma_alloc_coherent() doesn't know whether it's allocated
> from zone DMA or zone normal.  It sets just the coherent_dma_mask to a
> value less than 32 bit.
> 
> This situation may happen even after applying my patch.
> If you have more RAM than mask, allocation in the zone NORMAL may hit
> the outside of mask, and tries the zone DMA as fallback, although
> there are pretty enough free RAM in the zone NORMAL.
> 
> So, triggering oom-killer for zone DMA is non-sense, IMO.
> 

AFAIKS your patch tries ZONE_NORMAL, then falls back to ZONE_DMA, in
which case you possibly do want the oom-killer for ZONE_DMA. Although
if ZONE_DMA gets filled with pinned memory it will take down the system
due to the continual oom-killing :(

If the interface is allowed to fail, it may be an idea to allow it.
I'm not really sure... the other thing might be to do the retries in
the caller (ie. your code).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: OOM-killer for zone DMA?

[Takashi Iwai]

At Mon, 30 Aug 2004 20:36:57 +1000,
Nick Piggin wrote:
> 
> Takashi Iwai wrote:
> > At Sat, 28 Aug 2004 16:10:02 +1000,
> > Nick Piggin wrote:
> > 
> 
> >>
> >>You at least need __GFP_NORETRY to achieve what you want.
> > 
> > 
> > Yes, with that flag it can be avoided.
> > 
> 
> Great.
> 
> > But it *should* retry.
> 
> That is precisely the opposite of what you want.

I mean, retrying for zone NORMAL pages should be ok.


> >  It's an allocation of single page, and the
> > caller of dma_alloc_coherent() doesn't know whether it's allocated
> > from zone DMA or zone normal.  It sets just the coherent_dma_mask to a
> > value less than 32 bit.
> > 
> > This situation may happen even after applying my patch.
> > If you have more RAM than mask, allocation in the zone NORMAL may hit
> > the outside of mask, and tries the zone DMA as fallback, although
> > there are pretty enough free RAM in the zone NORMAL.
> > 
> > So, triggering oom-killer for zone DMA is non-sense, IMO.
> > 
> 
> AFAIKS your patch tries ZONE_NORMAL, then falls back to ZONE_DMA, in
> which case you possibly do want the oom-killer for ZONE_DMA. Although
> if ZONE_DMA gets filled with pinned memory it will take down the system
> due to the continual oom-killing :(

Yes, that's what annoyed me.
Pages allocated in zone DMA are usually buffers for drivers, which
won't be released by oom-killer.

Meanwhile, I fixed the driver codes to always add __GFP_NORETRY to
avoid oom-killer...

> If the interface is allowed to fail, it may be an idea to allow it.
> I'm not really sure... the other thing might be to do the retries in
> the caller (ie. your code).


Takashi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>