Allocation of kernel memory >128K

Allocation of kernel memory >128K

[Amit S. Jain]

I have been working on a module in which I copy large amount of data fromn
the user to the kernel area.To do so I allocate using either kmaaloc or
vmalloc or  get_free_pages()large amount of memory(in the range of
MBytes) in the kernel space.However this attempt is not successful.One ofmy 
colleagues informed me that in the kernel space it is safe not to allocate
large amount of memory at one time,should be kept upto 30K...is he
right....could you throw more light on this issue.


Thanking you,
Amit Jain

<

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Eric W. Biederman]

"Amit S. Jain" <amitjain@tifr.res.in> writes:

> I have been working on a module in which I copy large amount of data fromn
> the user to the kernel area.To do so I allocate using either kmaaloc or
> vmalloc or  get_free_pages()large amount of memory(in the range of
> MBytes) in the kernel space.However this attempt is not successful.One ofmy 
> colleagues informed me that in the kernel space it is safe not to allocate
> large amount of memory at one time,should be kept upto 30K...is he
> right....could you throw more light on this issue.

large amounts of memory are o.k. 
large amounts of continuous memory is generally a bad thing.

Allocating everything with multiple calls to get_free_page() should
get the job done.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Amit S. Jain]

Thank u everyone for the response to my question bout allocatin huge
amount of memeory in kernel space....
For a few people who wanted to know why I m allocating such a huge memory
and do i really need contiguous memory...here it is

---- Basically what my module is doing is trying to make the communication
between kernel to kernel in a Linux Cluster transparent tp TCP/IP.
So to transmit the data I copy the data from the user area to the kernel
area and then to the n/w buffers.So what I was trying to do is transfer
the entire data from user to kernel space at one go(allocating huge memory
at kernel)....since this is not possible I can always divide the data into
30K packets and then copy it to the kernel space...

P.S I am new to the Linux Kernel ...hence please excuse ne naive comments
in the above ..


Amit 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Vladimir Dergachev]

Take a look at rvmalloc code in bt848 driver (or a copy of it in km,
http://gatos.sf.net which is somewhat separated out). BT848 and km are 
using it to get large chunks (~300K) of memory which are contiguous in
kernel virtual space but not contiguous physically. Sure makes much
easier to work with buffers than a bunch of separate pages.

                          Vladimir Dergachev

On Wed, 12 Dec 2001, Amit S. Jain wrote:

> 
> Thank u everyone for the response to my question bout allocatin huge
> amount of memeory in kernel space....
> For a few people who wanted to know why I m allocating such a huge memory
> and do i really need contiguous memory...here it is
> 
> ---- Basically what my module is doing is trying to make the communication
> between kernel to kernel in a Linux Cluster transparent tp TCP/IP.
> So to transmit the data I copy the data from the user area to the kernel
> area and then to the n/w buffers.So what I was trying to do is transfer
> the entire data from user to kernel space at one go(allocating huge memory
> at kernel)....since this is not possible I can always divide the data into
> 30K packets and then copy it to the kernel space...
> 
> P.S I am new to the Linux Kernel ...hence please excuse ne naive comments
> in the above ..
> 
> 
> Amit 
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Allocation of kernel memory >128K

[Amit S. Jain]

Hello everyone,
		I hope u can clear this doubt.This question is a
continuation of the question below which i had posted on this site

On Tue, 11 Dec 2001, Amit S. Jain wrote:

> I have been working on a module in which I copy large amount of data fromn
> the user to the kernel area.To do so I allocate using either kmaaloc or
> vmalloc or  get_free_pages()large amount of memory(in the range of
> MBytes) in the kernel space.However this attempt is not successful.One ofmy 
> colleagues informed me that in the kernel space it is safe not to allocate
> large amount of memory at one time,should be kept upto 30K...is he
> right....could you throw more light on this issue.

  I WANT TO KNOW WHAT AMOUNT OF MEMORY ALLOCATION WILL BE SAFE.i.e. even
if i alloc 30K at a time,will I always get a contiguous memory for that
purpose.??
	Is there a set limit in Linux for the amount of memory we obtain
will always be contiguous or always available??
 
 
Thanking you,
 Amit Jain
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Ravi K]

>   I WANT TO KNOW WHAT AMOUNT OF MEMORY ALLOCATION
> WILL BE SAFE.i.e. even
> if i alloc 30K at a time,will I always get a
> contiguous memory for that
> purpose.??
> 	Is there a set limit in Linux for the amount of
> memory we obtain
> will always be contiguous or always available??

 No, there are no guarantees about availability of
contiguous memory. But if you do not specify
GFP_ATOMIC flag when calling kmalloc(), it will sleep
till the requested amount of memory is available.

Ravi.

__________________________________________________
Do You Yahoo!?
Send your FREE holiday greetings online!
http://greetings.yahoo.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Mark_H_Johnson]

Hmm. I don't know if this was necessarily a complete answer. Perhaps a few
extra questions are necessary to clarify what is requested by Amit.
 - do you [Amit] want a continuous range of physical pages? If so, why?
 - do you want a continuous range of pages in the kernel's virtual address
space? If so, why?
I would not necessarily say that "large amounts of continuous memory is a
bad thing" - rather that it is hard to get, and a costly operation (in
time). For example - a number of existing pages must be moved (or swapped)
to get the area you are requesting. Since this is a big performance hit,
you better have a really good reason for doing so.

If you really need contiguous physical pages - get the bigphysarea patch
and use it. We use it for a shared memory interface between a PC and single
board computers in a VME rack. It works well and is pretty easy to use. It
does have the disadvantage of taking memory away from general use.

If you think you need continuous virtual pages - I suggest putting a little
extra effort into your implementation so it also supports a set of smaller
address ranges. Use the code implementing kiobuf as an example. The added
code should not be much and will save the system a lot of effort in getting
your pages.
--Mark H Johnson
  <mailto:Mark_H_Johnson@raytheon.com>


                                                                                                                     
                    ebiederm@xmiss                                                                                   
                    ion.com (Eric         To:     "Amit S. Jain" <amitjain@tifr.res.in>                              
                    W. Biederman)         cc:     linux-mm@kvack.org                                                 
                    Sent by:              Subject:     Re: Allocation of kernel memory >128K                         
                    owner-linux-mm                                                                                   
                    @kvack.org                                                                                       
                                                                                                                     
                                                                                                                     
                    12/11/01 04:41                                                                                   
                    AM                                                                                               
                                                                                                                     
                                                                                                                     




"Amit S. Jain" <amitjain@tifr.res.in> writes:

> I have been working on a module in which I copy large amount of data
fromn
> the user to the kernel area.To do so I allocate using either kmaaloc or
> vmalloc or  get_free_pages()large amount of memory(in the range of
> MBytes) in the kernel space.However this attempt is not successful.One
ofmy
> colleagues informed me that in the kernel space it is safe not to
allocate
> large amount of memory at one time,should be kept upto 30K...is he
> right....could you throw more light on this issue.

large amounts of memory are o.k.
large amounts of continuous memory is generally a bad thing.

Allocating everything with multiple calls to get_free_page() should
get the job done.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Allocation of kernel memory >128K

[Benjamin LaHaise]

On Tue, Dec 11, 2001 at 09:27:36AM -0600, Mark_H_Johnson@Raytheon.com wrote:
> I would not necessarily say that "large amounts of continuous memory is a
> bad thing" - rather that it is hard to get, and a costly operation (in
> time). For example - a number of existing pages must be moved (or swapped)
> to get the area you are requesting. Since this is a big performance hit,
> you better have a really good reason for doing so.

Don't even bother thinking about performance: it is unreliable.

		-ben
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/