RE: Process not given >890MB on a 4MB machine ?????????

RE: Process not given >890MB on a 4MB machine ?????????

[Gabriel.Leen]

Hello again,
And thanks,

	>You will either need to use a true 64-bit machine (POWER, Alpha, 
	>UltraSPARC or MIPS) 

I hope (fingers crossed) that there is some way around this 
I think that Redhat  now supports up to 64GB of ram, 
as the Xeon has 36 address lines, see attached.

I'm only grasping at straws here, but I hope that it is somehow possible 
on this machine?

From:
http://support.intel.com/support/processors/pentiumiii/xeon/esma.htm
 <<...OLE_Obj...>> 
Pentium(R) III Xeon(tm) processors 
Extended Server Memory Architecture
Full 36-bit addressing allows enterprise applications to transcend 
the traditional 4GB (32-bit) memory barrier by adding 4-additional address
bits. 
PSE36 (Page size extensions) adds 4 additional address lines to the current
32 bit address. 
As each bit is added, the cacheability range doubles: 
32 bits= 4 GB 
32 + 1  	= 8 GB 
32 + 1 + 1 = 16 GB 
32 + 1 + 1 + 1 = 32
32 + 1 + 1 + 1 + 1  = 64 GB 	

	The additional headroom allows: 
	Greater than 4-gigabytes of cacheable system memory 


*********************************************************************
Gabriel Leen                        Tel:        00353  61  20 2677
PEI  Technologies               Fax:       00353  61  33 4925
Foundation Building             E-mail:   gabriel.leen@ul.ie
University of Limerick
Limerick
Ireland
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Joseph A Knapka]

"Gabriel.Leen" wrote:
> 
> Hello again,
> And thanks,
> 
>         >You will either need to use a true 64-bit machine (POWER, Alpha,
>         >UltraSPARC or MIPS)
> 
> I hope (fingers crossed) that there is some way around this
> I think that Redhat  now supports up to 64GB of ram,
> as the Xeon has 36 address lines, see attached.
> 
> I'm only grasping at straws here, but I hope that it is somehow possible
> on this machine?

No. You still only get a maximum of 4GB of -virtual- space per
process. The machine can address up to 64GB of -physical- RAM,
but a single process (actually a single page directory) can
see only 4GB at a time. Sorry :-(

-- Joe
# Replace the pink stuff with net to reply.
# "You know how many remote castles there are along the
#  gorges? You can't MOVE for remote castles!" - Lu Tze re. Uberwald
# Linux MM docs:
http://home.earthlink.net/~jknapka/linux-mm/vmoutline.html
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Stephen C. Tweedie]

Hi,

On Mon, Sep 24, 2001 at 09:16:58PM +0000, Joseph A Knapka wrote:

> No. You still only get a maximum of 4GB of -virtual- space per
> process. The machine can address up to 64GB of -physical- RAM,
> but a single process (actually a single page directory) can
> see only 4GB at a time. Sorry :-(

There are hacks to work around this --- for example, you can set up
large amounts of shared memory and map that on demand when you are
looking up your dataset.  However, it's simply not possible for user
space to refer to more than 3GB at once on Linux/Intel.  You *must* go
to a 64-bit architecture if you want more than that.

Cheers,
 Stephen

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[afei]

The current Linux MM design is a 3:1 split of 4G virtual/physical memory.
So a process, under normal condition cannot get beyond 3G memory
allocated.

On Tue, 25 Sep 2001, Stephen C. Tweedie wrote:

> Hi,
>
> On Mon, Sep 24, 2001 at 09:16:58PM +0000, Joseph A Knapka wrote:
>
> > No. You still only get a maximum of 4GB of -virtual- space per
> > process. The machine can address up to 64GB of -physical- RAM,
> > but a single process (actually a single page directory) can
> > see only 4GB at a time. Sorry :-(
>
> There are hacks to work around this --- for example, you can set up
> large amounts of shared memory and map that on demand when you are
> looking up your dataset.  However, it's simply not possible for user
> space to refer to more than 3GB at once on Linux/Intel.  You *must* go
> to a 64-bit architecture if you want more than that.
>
> Cheers,
>  Stephen
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Eric W. Biederman]

afei@jhu.edu writes:

> The current Linux MM design is a 3:1 split of 4G virtual/physical memory.
> So a process, under normal condition cannot get beyond 3G memory
> allocated.

The current Linux i386 MM usage is a 3:1 split of 4G virtual
memory. 3GB for the uesr process.  1GB for the kernel.  With all of
the highmem tricks the kernel can access up to 16TB of physical memory
on a 32 bit system but the i386 architeture only provides for a
maximum of 64GB of physical memory.

A user space process is free to implement it's own paging of a file or
a shared memory region in and out of it's address space but that
usually requires code redesign, so few people go for it.

Eric

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Stephen C. Tweedie]

Hi,

On Tue, Sep 25, 2001 at 01:36:51PM -0400, afei@jhu.edu wrote:

> The current Linux MM design is a 3:1 split of 4G virtual/physical memory.
> So a process, under normal condition cannot get beyond 3G memory
> allocated.

Only on 32-bit machines, and the limit only applies to _mapped_ memory
in process context.  It does not apply to _allocated_ memory --- we
support up to 64GB of physical memory even on Intel.  You just can't
have it all mapped at once, which is why some people use the shared
memory trick to map data in and out of the process's virtual address
space on demand.

Internally, the kernel does not use pointers to memory addresses in
most of the VM.  Instead, it uses 32-bit page numbers to refer to
entire pages, with a separate offset into the page if we need that.
That means that instead of a 12 bit offset and a 20 bit page number
existing in a single 32 bit pointer, we get a full 32 bits of page
number.  That allows the kernel to allocate pages way beyond the
normal 4GB limit of 32 bit pointers.

--Stephen

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: Process not given >890MB on a 4MB machine ?????????

[Rik van Riel]

On Mon, 24 Sep 2001, Gabriel.Leen wrote:

> I hope (fingers crossed) that there is some way around this
> I think that Redhat  now supports up to 64GB of ram,
> as the Xeon has 36 address lines, see attached.
>
> I'm only grasping at straws here, but I hope that it is somehow
> possible on this machine?

The Xeon allows up to 36 bits of physical memory,
but still only 32 bits of virtual address space
per process...

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: Process not given >890MB on a 4MB machine ?????????

[Gabriel.Leen]

	Thank you ALL for your help !!
	And thank you Rik,
	Hoard is a fantastic library, I'm now able to get up to 3GB of
memory !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! :)

	++++++++++++

	Unfortunately my program which is doing "alot" of calculations still
needs more space,

	Is there some way to enable 64 bit support (or something) and get
the swap space active,
	and give it another GB or so ?

	++++++++++++

	I know it sounds a bit crazy the amount of memory required, you see
I'm a PhD student,
	and I've been working on a model of a network for the past 2 years.
What I'm
	doing is formally verifying it and the calculations involved are
enormous !!!
	To finish my work I need to run this computation to completion and
get the results.

	If there are suggestions ideas they are very much appreciated.

	Again thank you ALL for your help,
	Gabriel


+++++++++++++++++++++++++
> Rik van Riel wrote :
>
> On Fri, 21 Sep 2001, Gabriel.Leen wrote:
>
> > Unfortunately the package which I am using is a pre-compiled
> > distribution, so that limits what I can do with it :(
>
> 1) install the hoard malloc library
> 2) LD_PRELOAD=/path/to/libhoard.so
>
> 3) have fun ;)
>
> Rik
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: Process not given >890MB on a 4MB machine ?????????

[Jonathan Morton]

>	Unfortunately my program which is doing "alot" of calculations still
>needs more space,
>
>	Is there some way to enable 64 bit support (or something) and get
>the swap space active,
>	and give it another GB or so ?

Nope, I doubt it.  The x86 architecture is fundamentally 32-bit, and 
thus can't address more than 4Gb for a single process - 1Gb of that 
address space is reserved for the kernel.  You can confirm this by 
looking at the result of sizeof(void*).

You will either need to use a true 64-bit machine (POWER, Alpha, 
UltraSPARC or MIPS) or rewrite your program to use large files 
instead of large amounts of memory.  I suspect the latter would be 
less costly.

-- 
--------------------------------------------------------------
from:     Jonathan "Chromatix" Morton
mail:     chromi@cyberspace.org  (not for attachments)
website:  http://www.chromatix.uklinux.net/vnc/
geekcode: GCS$/E dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$
           V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
tagline:  The key to knowledge is not to rely on people to teach you it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: Process not given >890MB on a 4MB machine ?????????

[Gabriel.Leen]

Hello everybody,
Thanks for your help.

Unfortunately the package which I am using is a pre-compiled distribution,
so that limits what I can do with it :(
But I will hasle the developers and see what version of glibc they used.

QUESTION:
Have you actualy run a single process on the rawhide distribution which uses
~3GB ????????????
Please say YES :)

Red Hat kernel 2.4.9 as in 2.4.9-0.5 from rawhide ?
It should work, you should be able to get close to 3Gb....

++++++++++++++++++++++++++++++++++++++++++++++++++++
Some more info:
I have a small program listing attached which has helped me to identify the
problem.
It just gobbles up memory and writes zeros and f's to it

When I ran this on the kernel distribution out of the Box for Linux 7.1
delux
it would only work up to 1.2GB and then malloc returned NULL, continue
writing the
error messages to the console and eventually terminate normally.

Then I compiled the 2.4.9 kernel with the patch from Alan and it will run up
to 2GB
but not more, now the error occurs straight away when I hit run:
"segmentation fault"

Previously on the older kernel it would run when malloc returned NULL, the
program
continued writing the error and terminated normally.
Now the kernel appears to be psychic or something ?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Thank you again for your help, much appreciated,
Gabriel


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


#include <stdlib.h>

// Change SIZE for experiments
// size in MegaBytes.
#define SIZE 300

// Memory is allocated in SIZE  blocks of allocation units.
// currently all allocation units are stored in one array.
#define DEFAULT_BLOCK 1024
#define ALLOCATION_UNIT 1024

int main() {
        int c, i,j;
        char *ch;
        char *charray[SIZE*DEFAULT_BLOCK];

        for( i = 0; i < SIZE; i++ ) {
                for( c = 0; c < DEFAULT_BLOCK; c++ ) {
                        ch = malloc(ALLOCATION_UNIT);
                        if( ch == NULL )
                                printf("%d FAILED", c);
			charray[(i*DEFAULT_BLOCK)+c]=ch;

                }
                printf("%d\n", i);

                if( i % 10 == 0 )
                        sleep(1);
        }
	printf("writing 0x00-s to memory ...\n");
        for( i = 0; i < SIZE; i++ ) {
                for( c = 0; c < DEFAULT_BLOCK; c++ ) {
                	ch=charray[(i*DEFAULT_BLOCK)+c];
			for(j=0;j<ALLOCATION_UNIT;j++) {
				ch[j]=0x00;
			}
                }
                printf("%d\n", i);

                if( i % 10 == 0 )
                        sleep(1);
        }
	printf("writing 0xFF-s to memory ...\n");
        for( i = 0; i < SIZE; i++ ) {
                for( c = 0; c < DEFAULT_BLOCK; c++ ) {
                	ch=charray[(i*DEFAULT_BLOCK)+c];
			for(j=0;j<ALLOCATION_UNIT;j++) {
				ch[j]=0xFF;
			}
                }
                printf("%d\n", i);

                if( i % 10 == 0 )
                        sleep(1);
        }
	printf("Memory allocation succeeded. Total allocated memory in
kilobytes = %d\n",\

((DEFAULT_BLOCK*SIZE*ALLOCATION_UNIT)+sizeof(charray))/1024);
        printf("sleeping for 30 seconds ...\n");
        sleep(30);

        return 0;
}


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
	END

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: Process not given >890MB on a 4MB machine ?????????

[Rik van Riel]

On Fri, 21 Sep 2001, Gabriel.Leen wrote:

> Unfortunately the package which I am using is a pre-compiled
> distribution, so that limits what I can do with it :(

1) install the hoard malloc library
2) LD_PRELOAD=/path/to/libhoard.so

3) have fun ;)

Rik
-- 
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Process not given >890MB on a 4MB machine ?????????

[Gabriel.Leen]

Hello,
The problem in a nutshell is:

a) I have a 4GB ram 1.7Gh Xeon box
b) I'm running a process which requires around 3GB of ram
c) RedHat 2.4.9 will only give it 890MB, then core dumps with the warning
"segmentation fault"
when it reaches this memory usage and "asks for more"

+++++++++++++++++
Details:

System/OS:
I'm running RedHat 2.4.9, with the 4GB memory support selected 
and the latest patch from
www.kernel.org/pub/linux/kernel/people/alan/linux-2.4/2.4.9 
4GB RAM, slightly less seen by system, (Top, XOSVIEW, etc)
Running as root with the bash shell ulimit command returning: "unlimited"


When it crashes:
according to top): for the process: SIZE 893 MB ,RSS 893 MB, 
"segmentation fault" (core dumped)

+++++++++++++++++
Other Info:

I don't know if the problem has something to do with PAGE_OFFSET in page.h
it is currently set at C0000000
I tried changing this but the machine would not boot then.

I can run 2 or 3 processes using 1 GB at a time but one process using >1GB
is not possible ??

+++++++++++++++++

Any suggestions / advice is very much appreciated, Thanx in advance,

Gabriel

*********************************************************************
Gabriel Leen                        Tel:        00353  61  20 2677
PEI  Technologies               Fax:       00353  61  33 4925
Foundation Building             E-mail:   gabriel.leen@ul.ie
University of Limerick
Limerick
Ireland
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[brian]

On Thu, Sep 20, 2001 at 08:25:37PM +0100, Gabriel.Leen wrote:
> Hello,
> The problem in a nutshell is:
> 
> a) I have a 4GB ram 1.7Gh Xeon box
> b) I'm running a process which requires around 3GB of ram
> c) RedHat 2.4.9 will only give it 890MB, then core dumps with the warning
> "segmentation fault"
> when it reaches this memory usage and "asks for more"

That is exacly what I've seen.

The limit I ran into was in glibc.  My code used malloc, and apparently
some versions of malloc in glibc try "harder" than others to allocate
memory.  Check your version of glibc and try a later one if available.



-- 
Brian Litzinger <brian@worldcontrol.com>

    Copyright (c) 2000 By Brian Litzinger, All Rights Reserved
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Thierry Vignaud]

brian@worldcontrol.com writes:

> > The problem in a nutshell is:
> >
> > a) I have a 4GB ram 1.7Gh Xeon box
> > b) I'm running a process which requires around 3GB of ram
> > c) RedHat 2.4.9 will only give it 890MB, then core dumps with the warning
> > "segmentation fault"
> > when it reaches this memory usage and "asks for more"
>
> That is exacly what I've seen.
>
> The limit I ran into was in glibc.  My code used malloc, and apparently
> some versions of malloc in glibc try "harder" than others to allocate
> memory.  Check your version of glibc and try a later one if available.

the problem is that the glibc has various algo to allocate memory, depending of
the requested size (greater than a page or not), and use the "classic" sbrk() if
lesser (bellow 1Gb) or an anonymous mapping (from 1Gb to a limit that depends of
the virtual memory split between the kernel and the process space).
therefore small malloc will eat space below 1GB and cannot use more than this GB
(minus the process text & data).
anyway for small objects set, there's more efficient techniques (one big malloc
is less costly than several small malloc()).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Benjamin LaHaise]

On Thu, Sep 20, 2001 at 08:25:37PM +0100, Gabriel.Leen wrote:
> Hello,
> The problem in a nutshell is:
> 
> a) I have a 4GB ram 1.7Gh Xeon box
> b) I'm running a process which requires around 3GB of ram
> c) RedHat 2.4.9 will only give it 890MB, then core dumps with the warning
> "segmentation fault"
> when it reaches this memory usage and "asks for more"

That's a limitation of the current memory layout.  If you compile your 
program statically (gcc -static), then you should be able to use closer 
to 3GB.

		-ben
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Process not given >890MB on a 4MB machine ?????????

[Eric W. Biederman]

Benjamin LaHaise <bcrl@redhat.com> writes:

> On Thu, Sep 20, 2001 at 08:25:37PM +0100, Gabriel.Leen wrote:
> > Hello,
> > The problem in a nutshell is:
> >
> > a) I have a 4GB ram 1.7Gh Xeon box
> > b) I'm running a process which requires around 3GB of ram
> > c) RedHat 2.4.9 will only give it 890MB, then core dumps with the warning
> > "segmentation fault"
> > when it reaches this memory usage and "asks for more"
>
> That's a limitation of the current memory layout.  If you compile your
> program statically (gcc -static), then you should be able to use closer
> to 3GB.

But note the maximum possible memory usage for a single process on x86 with 2.4.x
is 3GB...

Eric

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/