Re: Re: Memory allocation problem

Re: Re: Memory allocation problem

[anand kumar]

Hi,

Thanks for immediate response. I used 2.4.20 kernel patched with 
bigphys
area and got it working. I know that this patch is part of Suse
distribution. Is there any plans to incorporate this patch in 
Redhat?
Is redhat 9.0 kernel equipped with this patch?

Rgds
Anand

On Thu, 01 May 2003 Mark_H_Johnson@Raytheon.com wrote :
>
> >The kernel version we are using is 2.4.18 (Redhat 8.0) and the 
>total
> >amount of memory available in the box is 128MB
>
> >Is there any other mechanism to allocate large amount of 
>physically
> >contiguous memory blocks during normal run time of the driver? 
>Is this
> >being addressed in later kernels.
>
>I regularly use a patch (bigphysarea) recommended by Dolphin for 
>use with
>their SCI cards. The copy I use is from a relatively old kernel 
>(2.4.4)
>which applies with a few warnings but is otherwise OK. I did a 
>quick search
>with Google for
>   bigphysarea linux 2.4.18
>and found
>   
>http://frmb.home.cern.ch/frmb/download/bigphysarea-2.4.18.patch
>or a more readable page at
>   http://frmb.home.cern.ch/frmb/linux.html
>which appears to be a version updated for 2.4.18. I believe the 
>original
>patch is maintained at
>   
>http://www.uni-paderborn.de/fachbereich/AG/heiss/linux/bigphysarea.html
>
>There are apparently several drivers that already use this 
>interface, but
>it does require a patched kernel.
>
>I am not aware of any effort to merge this into the main line 
>kernel
>(though I would certainly appreciate that).
>
>--Mark H Johnson
>   <mailto:Mark_H_Johnson@raytheon.com>
>
>





--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Re: Memory allocation problem

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

On Sun, 2003-05-04 at 21:40, anand kumar wrote:
> Hi,
> 
> Thanks for immediate response. I used 2.4.20 kernel patched with 
> bigphys
> area and got it working. I know that this patch is part of Suse
> distribution. Is there any plans to incorporate this patch in 
> Red Hat?
> Is Red Hat 9 kernel equipped with this patch?

no it's not, nor are there plans to add it. BigPhysArea is a hack, not a
solution. The solution is to use the scatter-gather engine on the pci
card instead of needing to chainsaw physical ram always...

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Re: Memory allocation problem

[Mark_H_Johnson]

>On Sun, 2003-05-04 at 21:40, anand kumar wrote:
>> Hi,
>>
>> [query about bigphysarea and Red Hat]
>> Is Red Hat 9 kernel equipped with this patch?

>no it's not, nor are there plans to add it. BigPhysArea is a hack, not a
>solution. The solution is to use the scatter-gather engine on the pci
>card instead of needing to chainsaw physical ram always...

Let me comment on that last point.

It actually depends on the hardware and what your needs are. Using
the SCI (Scaleable Coherent Interface) cards as an example, the hardware
has a limited number of mapping registers (varies by specific card - the
ones we use have 16K of them) to define the memory mapping. Since we map
up to 512 Mbyte of memory, each of the 16K registers map 32 Kbytes. Note
also that the registers that do the mapping are on the machine that is
accessing the remote memory, not the one with the remote memory.

Let me illustrate:

CPU on machine A
  ||
PCI on machine A
  ||
SCI card on machine A [*]
  ||
SCI cables
  ||
SCI card on machine B
  ||
PCI on machine B
  ||
Memory on machine B

When the CPU on machine A does a fetch, the address is mapped to the SCI
card. The SCI card extracts a few bits to look up in the 16K register
the remote machine location / remote address base[*]. Add that to the rest
of the machine A address & send it to machine B. Machine B's SCI card
does a PCI fetch using the address provided (mapped into physcial memory)
and returns the result (eventually making it back to CPU A). Total elapsed
time w/o any caching effects is 2-5 microseconds. A similar process occurs
on writes (though w/ an address / value sent, not recieved).

Note that the driver implementing this card must get memory in chunks
large enough for the largest map on all other machines connected by the
SCI. So, if I mapped 1G or 2G (instead of 512 Mbyte), that means the
driver must be able to allocate 64 Kbyte or 128 Kbyte chunks. You can
get by with smaller chunks only if all other machines agree to use less
mapping space.

Even if the driver was "smart", I believe the linux-mm has a limitation
where you can get to a point where those sizes of fragments are not
available to be allocated. Thus the need for bigphysarea. We have been
doing so for a couple years now. It is also not a "big deal" for us
since we end up adding about a dozen patches total to get the combination
of capabilities we need for our large, real time application.

I can certainly understand not putting something into Red Hat that has
limited applicability. I can also see the point that adding bigphysarea
won't significantly affect the >99% of systems that won't use it (a few
bytes of memory), but would allow those who need it to get it.

  --Mark



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>