A solution for more GFP_xx flags?

A solution for more GFP_xx flags?

[Christoph Lameter]

I wonder if we could pass a pointer to an allocation structure
to the allocators instead of an unsigned long?

Right now the problem is that we need _node allocators to specify nodes 
and the memory policies and cpusets are determined by the allocation 
context of a process. This makes the allocators difficult to handle.

We could define a structure

struct allocation_control {
	unsigned long flags;	/* Traditional flags */
	int node;
	struct cpuset_context *cpuset;
	struct mempol *mpol;
};

We could define struct constants called GFP_KERNEL and GFP_ATOMIC.
const struct allocation_control gfp_kernel {
	GFP_KERNEL, -1, NULL, NULL
}

And then do

alloc_pages(n, gfp_kernel)

?

This would also solve the problem of allocations that do not occur in a 
proper process context. F.e. slab allocations are on behalf of the slab 
allocator and not on behalf of a process. Thus the cpuset and the memory 
policies should not influence that allocation. In that case we could have 
a special allocation_control structure for that context.

It would also get rid off all the xxx_node allocator variations.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Nick Piggin]

Christoph Lameter wrote:

>I wonder if we could pass a pointer to an allocation structure
>to the allocators instead of an unsigned long?
>
>Right now the problem is that we need _node allocators to specify nodes 
>and the memory policies and cpusets are determined by the allocation 
>context of a process. This makes the allocators difficult to handle.
>
>We could define a structure
>
>struct allocation_control {
>	unsigned long flags;	/* Traditional flags */
>	int node;
>	struct cpuset_context *cpuset;
>	struct mempol *mpol;
>};
>
>We could define struct constants called GFP_KERNEL and GFP_ATOMIC.
>const struct allocation_control gfp_kernel {
>	GFP_KERNEL, -1, NULL, NULL
>}
>
>And then do
>
>alloc_pages(n, gfp_kernel)
>
>?
>
>This would also solve the problem of allocations that do not occur in a 
>proper process context. F.e. slab allocations are on behalf of the slab 
>allocator and not on behalf of a process. Thus the cpuset and the memory 
>policies should not influence that allocation. In that case we could have 
>a special allocation_control structure for that context.
>
>It would also get rid off all the xxx_node allocator variations.
>

This seems like a decent approach to make a nice general interface. I guess
existing APIs can be easily implemented by filling in the structure. If you
took this approach I don't think there should be any objections.

A minor point: would we prefer a struct argument to the allocator, or more
function arguments? It is an API that we need to get right...

---

Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Christoph Lameter]

On Tue, 12 Sep 2006, Nick Piggin wrote:

> This seems like a decent approach to make a nice general interface. I guess
> existing APIs can be easily implemented by filling in the structure. If you
> took this approach I don't think there should be any objections.
> 
> A minor point: would we prefer a struct argument to the allocator, or more
> function arguments? It is an API that we need to get right...

If you look at my allocator API (see the latest slab patchset), one could 
add all the allocator methods into the struct in order to objectivize the 
page alloator.

The logical first step towards that would be to have a struct argument
to allow detailed allocation control and to avoid this contextual
memory policy / cpuset mess.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Peter Chubb]

>>>>> "Christoph" == Christoph Lameter <clameter@sgi.com> writes:

Christoph> On Tue, 12 Sep 2006, Nick Piggin wrote:

>> This seems like a decent approach to make a nice general
>> interface. I guess existing APIs can be easily implemented by
>> filling in the structure. If you took this approach I don't think
>> there should be any objections.
>> 
>> A minor point: would we prefer a struct argument to the allocator,
>> or more function arguments? It is an API that we need to get
>> right...

Christoph> If you look at my allocator API (see the latest slab
Christoph> patchset), one could add all the allocator methods into the
Christoph> struct in order to objectivize the page alloator.

Dunno about you, but the thought of passing structs around on the
stack gives me the heebie jeebies, especially if they're going to be
more than a word or so big.

Either pass by reference, or separate out the args, so it's explicit
how much info's being copied around.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
http://www.ertos.nicta.com.au           ERTOS within National ICT Australia

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Christoph Lameter]

On Tue, 12 Sep 2006, Peter Chubb wrote:

> Dunno about you, but the thought of passing structs around on the
> stack gives me the heebie jeebies, especially if they're going to be
> more than a word or so big.

Right.
 
> Either pass by reference, or separate out the args, so it's explicit
> how much info's being copied around.

Maybe I misunderstood Nick but I did not think he was proposing pushing 
structs on the stack.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Peter Chubb]

>>>>> "Christoph" == Christoph Lameter <clameter@sgi.com> writes:

Christoph> On Tue, 12 Sep 2006, Peter Chubb wrote:
>> Dunno about you, but the thought of passing structs around on the
>> stack gives me the heebie jeebies, especially if they're going to
>> be more than a word or so big.

Christoph> Right.
 
>> Either pass by reference, or separate out the args, so it's
>> explicit how much info's being copied around.

Christoph> Maybe I misunderstood Nick but I did not think he was
Christoph> proposing pushing structs on the stack.

Maybe I misunderstood you:

Christoph> We could define a structure

Christoph> struct allocation_control {
Christoph> 	unsigned long flags;	/* Traditional flags */
Christoph> 	int node;
Christoph> 	struct cpuset_context *cpuset;
Christoph>	struct mempol *mpol;
Christoph> };

Christoph>We could define struct constants called GFP_KERNEL and GFP_ATOMIC.
Christoph> const struct allocation_control gfp_kernel {
Christoph>	GFP_KERNEL, -1, NULL, NULL
Christoph> }

Christoph> And then do

Christoph> alloc_pages(n, gfp_kernel)

Did you mean alloc_pages(n, &gfp_kernel) ??


--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
http://www.ertos.nicta.com.au           ERTOS within National ICT Australia

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Christoph Lameter]

On Tue, 12 Sep 2006, Peter Chubb wrote:

> Did you mean alloc_pages(n, &gfp_kernel) ??

Yes.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: A solution for more GFP_xx flags?

[Paul Jackson]

> struct allocation_control {
> 	unsigned long flags;	/* Traditional flags */
> 	int node;
> 	struct cpuset_context *cpuset;

I don't understand what purpose this cpuset pointer has.

The main (heavily traveled) code paths beneath __alloc_pages() don't
look at the tasks cpuset at all, and the less traveled code paths only
look at the current tasks cpuset.

I have no clue how the above cpuset pointer could (usefully) be
anything other than just a copy of current->cpuset.  Also, without
serious reworking of the locking and likely some performance impact, I
have no idea what use this cpuset pointer would be on the main memory
allocation code paths.

The cpuset imposed constraints on an allocation are represented by the
mems_allowed nodemask and the flags PF_SPREAD_PAGE and PF_SPREAD_SLAB
in the task struct.  If the memory constraints imposed by a tasks
cpuset change, then these constraints are transfered to the tasks
mems_allowed and flags by the routine cpuset_update_task_memory_state().

Unlike mm/mempolicy.c NUMA mempolicies, one task -can- change the cpuset
of another task.  This forces us to have fancier (more expensive)
locking on cpusets, and that means we have to keep cpusets off the hot
memory allocation code paths and instead cache their constraints in the
task struct.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>