refill_inactive()

refill_inactive()

[Ingo Molnar]

i'm wondering about the following piece of code in refill_inactive():

                if (current->need_resched && (gfp_mask & __GFP_IO)) {
                        __set_current_state(TASK_RUNNING);
                        schedule();
                }

shouldnt this be __GFP_WAIT? It's true that __GFP_IO implies __GFP_WAIT
(because IO cannot be done without potentially scheduling), so the code is
not buggy, but the above 'yielding' of the CPU should be done in the
GFP_BUFFER case as well. (which is __GFP_WAIT but not __GFP_IO)

Objections?

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Arjan van de Ven]

In article <Pine.LNX.4.21.0009241148100.2789-100000@elte.hu> you wrote:
> i'm wondering about the following piece of code in refill_inactive():

>                 if (current->need_resched && (gfp_mask & __GFP_IO)) {
>                         __set_current_state(TASK_RUNNING);
>                         schedule();
>                 }

> shouldnt this be __GFP_WAIT? It's true that __GFP_IO implies __GFP_WAIT
> (because IO cannot be done without potentially scheduling), so the code is

Is this also true for starting IO ?

Greetings,
   Arjan van de Ven
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Ingo Molnar]

On Sun, 24 Sep 2000, Arjan van de Ven wrote:

> > shouldnt this be __GFP_WAIT? It's true that __GFP_IO implies __GFP_WAIT
> > (because IO cannot be done without potentially scheduling), so the code is
> 
> Is this also true for starting IO ?

yes. ll_rw_block() might block if there are no more request slots left.
Dirty buffer balancing within buffer.c might block as well.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Rik van Riel]

On Sun, 24 Sep 2000, Ingo Molnar wrote:

> i'm wondering about the following piece of code in refill_inactive():
> 
>                 if (current->need_resched && (gfp_mask & __GFP_IO)) {
>                         __set_current_state(TASK_RUNNING);
>                         schedule();
>                 }
> 
> shouldnt this be __GFP_WAIT? It's true that __GFP_IO implies __GFP_WAIT
> (because IO cannot be done without potentially scheduling), so the code is
> not buggy, but the above 'yielding' of the CPU should be done in the
> GFP_BUFFER case as well. (which is __GFP_WAIT but not __GFP_IO)
> 
> Objections?

1) if __GFP_WAIT isn't set, we cannot run try_to_free_pages at all

2) you are right, we /can/ schedule when __GFP_IO isn't set, this is
   mistake ... now I'm getting confused about what __GFP_IO is all
   about, does anybody know the _exact_ meaning of __GFP_IO ?


regards,


Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Ingo Molnar]

On Mon, 25 Sep 2000, Rik van Riel wrote:

> 2) you are right, we /can/ schedule when __GFP_IO isn't set, this is
>    mistake ... now I'm getting confused about what __GFP_IO is all
>    about, does anybody know the _exact_ meaning of __GFP_IO ?

__GFP_IO set to 1 means that the allocator can afford doing IO implicitly
by the page allocator. Most allocations dont care at all wether swap IO is
started as part of gfp() or not. But a prominent counter-example is
GFP_BUFFER, which is used by the buffer-cache/fs layer, and which cannot
do any IO implicitly. (because it *is* the IO layer already, and it is
already trying to do IO.) The other reason are legacy lowlevel-filesystem
locks like the ext2fs lock, which cannot be taken recursively.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Rik van Riel]

On Mon, 25 Sep 2000, Ingo Molnar wrote:
> 
> On Mon, 25 Sep 2000, Rik van Riel wrote:
> 
> > 2) you are right, we /can/ schedule when __GFP_IO isn't set, this is
> >    mistake ... now I'm getting confused about what __GFP_IO is all
> >    about, does anybody know the _exact_ meaning of __GFP_IO ?
> 
> __GFP_IO set to 1 means that the allocator can afford doing IO implicitly
> by the page allocator. Most allocations dont care at all wether swap IO is
> started as part of gfp() or not. But a prominent counter-example is
> GFP_BUFFER, which is used by the buffer-cache/fs layer, and which cannot
> do any IO implicitly. (because it *is* the IO layer already, and it is
> already trying to do IO.) The other reason are legacy lowlevel-filesystem
> locks like the ext2fs lock, which cannot be taken recursively.

Hmmm, doesn't GFP_BUFFER simply imply that we cannot
allocate new buffer heads to do IO with??

(from reading buffer.c, I can't see much of a reason
why we couldn't start write IO on already allocated
buffers...)

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Linus Torvalds]

On Mon, 25 Sep 2000, Rik van Riel wrote:
> 
> Hmmm, doesn't GFP_BUFFER simply imply that we cannot
> allocate new buffer heads to do IO with??

No.

New buffer heads would be ok - recursion is fine in theory, as long as it
is bounded, and we might bound it some other way (I don't think we
_should_ do recursion here due to the stack limit, but at least it's not
a fundamental problem).

The fundamental problem is that GFP_BUFFER allocations are often done with
some critical filesystem lock held. Which means that we cannot call down
to the filesystem to free up memory.

The name is a misnomer, partly due to historical reasons (the buffer cache
used to be fragile, and if you free'd buffer cache pages while you were
trying to allocate new ones you could cause BadThings(tm) to happen), but
partly just because the only _user_ of it is the buffer cache. 

In theory, filesystems could use it for any other allocations that they
do, but in practice they don't, and the only allocations they do in
critical regions is the buffer allocation. And as this thread has
discussed, even that is really more of a bug than a feature.

		Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Rik van Riel]

On Mon, 25 Sep 2000, Linus Torvalds wrote:
> On Mon, 25 Sep 2000, Rik van Riel wrote:
> > 
> > Hmmm, doesn't GFP_BUFFER simply imply that we cannot
> > allocate new buffer heads to do IO with??
> 
> No.
> 
> New buffer heads would be ok - recursion is fine in theory, as long as it
> is bounded, and we might bound it some other way (I don't think we
> _should_ do recursion here due to the stack limit, but at least it's not
> a fundamental problem).
> 
> The fundamental problem is that GFP_BUFFER allocations are often done with
> some critical filesystem lock held. Which means that we cannot call down
> to the filesystem to free up memory.
> 
> The name is a misnomer, partly due to historical reasons (the buffer cache
> used to be fragile, and if you free'd buffer cache pages while you were
> trying to allocate new ones you could cause BadThings(tm) to happen), but
> partly just because the only _user_ of it is the buffer cache. 
> 
> In theory, filesystems could use it for any other allocations that they
> do, but in practice they don't, and the only allocations they do in
> critical regions is the buffer allocation. And as this thread has
> discussed, even that is really more of a bug than a feature.

Good thing to have this documented ;)

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: refill_inactive()

[Stephen C. Tweedie]

Hi,

On Mon, Sep 25, 2000 at 09:17:54AM -0700, Linus Torvalds wrote:
> 
> On Mon, 25 Sep 2000, Rik van Riel wrote:
> > 
> > Hmmm, doesn't GFP_BUFFER simply imply that we cannot
> > allocate new buffer heads to do IO with??
> 
> No.
> 
> New buffer heads would be ok - recursion is fine in theory, as long as it
> is bounded, and we might bound it some other way (I don't think we
> _should_ do recursion here due to the stack limit, but at least it's not
> a fundamental problem).

Right, but we still need to be careful --- we _were_ getting stack
overflows occassionally before the GFP_BUFFER semantics were set up to
prevent that recursion.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/