Difference between CommitLimit and Comitted_AS?

Difference between CommitLimit and Comitted_AS?

[Westerdale, John]

Hi All,

Am interested in differentiating the meaning of Commit* and Vmalloc*.

I had thought that the Committed_AS was the sum of memory allocations,
and Commit_Limit was the available memory to serve this from.

That said, I winced when I saw that Committed_AS was almost twice the
Commit__Limit.

Vmalloc looks inconsequential, but, the Commit* numbers must be there
for a reason.

Is it safe to continue running with such a perceived over-commit?

Is this evidence of a leak or garbage collection issues?

This system functions as an App/Web front end using  Tomcat servelet
engine, FWIW.

Thanks

John Westerdale


MemTotal:     16634464 kB
MemFree:      11077520 kB
Buffers:        420768 kB
Cached:        4379000 kB
SwapCached:          0 kB
Active:        4577960 kB
Inactive:       685344 kB
HighTotal:    15859440 kB
HighFree:     10987632 kB
LowTotal:       775024 kB
LowFree:         89888 kB
SwapTotal:     4194296 kB
SwapFree:      4194296 kB
Dirty:              12 kB
Writeback:           0 kB
AnonPages:      462748 kB
Mapped:          65420 kB
Slab:           260144 kB
PageTables:      21712 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  12511528 kB
Committed_AS: 22423356 kB
VmallocTotal:   116728 kB
VmallocUsed:      6600 kB
VmallocChunk:   109612 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB	

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Difference between CommitLimit and Comitted_AS?

[KOSAKI Motohiro]

> Hi All,
> 
> Am interested in differentiating the meaning of Commit* and Vmalloc*.
> 
> I had thought that the Committed_AS was the sum of memory allocations,
> and Commit_Limit was the available memory to serve this from.
> 
> That said, I winced when I saw that Committed_AS was almost twice the
> Commit__Limit.

Commit_Limit is only meaningful when overcommit_memory=2.

And, Linux has virtual memory feature. then memory allocation has 2 levels,
virtual address space allocation and physical memory allocation.
Committed_AS mean amount of committed virtual address space. It's not
physical.

Example, if you create thread, libc allocate lots address space for stack.
but typical program don't use so large stack. then physical memory will be 
not allocated.


> 
> Vmalloc looks inconsequential, but, the Commit* numbers must be there
> for a reason.
> 
> Is it safe to continue running with such a perceived over-commit?

Probably safe. Java runtime usually makes a lot of overcommits. but I have
no way to know exactly your system state. 


> 
> Is this evidence of a leak or garbage collection issues?

Maybe no.

> 
> This system functions as an App/Web front end using  Tomcat servelet
> engine, FWIW.
> 
> Thanks
> 
> John Westerdale
> 
> 
> MemTotal:     16634464 kB
> MemFree:      11077520 kB
> Buffers:        420768 kB
> Cached:        4379000 kB
> SwapCached:          0 kB
> Active:        4577960 kB
> Inactive:       685344 kB
> HighTotal:    15859440 kB
> HighFree:     10987632 kB
> LowTotal:       775024 kB
> LowFree:         89888 kB
> SwapTotal:     4194296 kB
> SwapFree:      4194296 kB
> Dirty:              12 kB
> Writeback:           0 kB
> AnonPages:      462748 kB
> Mapped:          65420 kB
> Slab:           260144 kB
> PageTables:      21712 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:  12511528 kB
> Committed_AS: 22423356 kB
> VmallocTotal:   116728 kB
> VmallocUsed:      6600 kB
> VmallocChunk:   109612 kB
> HugePages_Total:     0
> HugePages_Free:      0
> HugePages_Rsvd:      0
> Hugepagesize:     2048 kB	
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
> Don't email: <a hrefmailto:"dont@kvack.org"> email@kvack.org </a>



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Difference between CommitLimit and Comitted_AS?

[Westerdale, John]

[-- Attachment #1: Type: text/plain, Size: 2827 bytes --]

Kosaki,

Thanks very much for your information.

this server is running 25 pct of the projected 

Sent via DROID on Verizon Wireless

-----Original message-----
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: "Westerdale, John" <John.Westerdale@stryker.com>
Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm@kvack.org
Sent: Thu, Dec 2, 2010 01:41:03 GMT+00:00
Subject: Re: Difference between CommitLimit and Comitted_AS?

> Hi All,
> 
> Am interested in differentiating the meaning of Commit* and Vmalloc*.
> 
> I had thought that the Committed_AS was the sum of memory allocations,
> and Commit_Limit was the available memory to serve this from.
> 
> That said, I winced when I saw that Committed_AS was almost twice the
> Commit__Limit.

Commit_Limit is only meaningful when overcommit_memory=2.

And, Linux has virtual memory feature. then memory allocation has 2 levels,
virtual address space allocation and physical memory allocation.
Committed_AS mean amount of committed virtual address space. It's not
physical.

Example, if you create thread, libc allocate lots address space for stack.
but typical program don't use so large stack. then physical memory will be 
not allocated.


> 
> Vmalloc looks inconsequential, but, the Commit* numbers must be there
> for a reason.
> 
> Is it safe to continue running with such a perceived over-commit?

Probably safe. Java runtime usually makes a lot of overcommits. but I have
no way to know exactly your system state. 


> 
> Is this evidence of a leak or garbage collection issues?

Maybe no.

> 
> This system functions as an App/Web front end using  Tomcat servelet
> engine, FWIW.
> 
> Thanks
> 
> John Westerdale
> 
> 
> MemTotal:     16634464 kB
> MemFree:      11077520 kB
> Buffers:        420768 kB
> Cached:        4379000 kB
> SwapCached:          0 kB
> Active:        4577960 kB
> Inactive:       685344 kB
> HighTotal:    15859440 kB
> HighFree:     10987632 kB
> LowTotal:       775024 kB
> LowFree:         89888 kB
> SwapTotal:     4194296 kB
> SwapFree:      4194296 kB
> Dirty:              12 kB
> Writeback:           0 kB
> AnonPages:      462748 kB
> Mapped:          65420 kB
> Slab:           260144 kB
> PageTables:      21712 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:  12511528 kB
> Committed_AS: 22423356 kB
> VmallocTotal:   116728 kB
> VmallocUsed:      6600 kB
> VmallocChunk:   109612 kB
> HugePages_Total:     0
> HugePages_Free:      0
> HugePages_Rsvd:      0
> Hugepagesize:     2048 kB	
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
> Don't email: <a hrefmailto:"dont@kvack.org"> email@kvack.org </a>





[-- Attachment #2: Type: text/html, Size: 3728 bytes --]

Re: Difference between CommitLimit and Comitted_AS?

[Westerdale, John]

[-- Attachment #1: Type: text/plain, Size: 3244 bytes --]

Kosaki, 

Thanks for your contribution.

This system is only running 25 pct of projected load, so I am concerned. 

If we add up physical memory plus swap would that be a comfortable limit for  
committed_AS?

Can tomcat  (or whaever webshere uses for servelets) be tuned to allocate a  
fixed amount per session?  

as there are other applications on the same server, need to set up good  
fences.
 
can I use ulimits to exercise some rough level of control?

thanks

John

ps - sorry about the partial message :)
Sent via DROID on Verizon Wireless

-----Original message-----
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: "Westerdale, John" <John.Westerdale@stryker.com>
Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm@kvack.org
Sent: Thu, Dec 2, 2010 01:41:03 GMT+00:00
Subject: Re: Difference between CommitLimit and Comitted_AS?

> Hi All,
> 
> Am interested in differentiating the meaning of Commit* and Vmalloc*.
> 
> I had thought that the Committed_AS was the sum of memory allocations,
> and Commit_Limit was the available memory to serve this from.
> 
> That said, I winced when I saw that Committed_AS was almost twice the
> Commit__Limit.

Commit_Limit is only meaningful when overcommit_memory=2.

And, Linux has virtual memory feature. then memory allocation has 2 levels,
virtual address space allocation and physical memory allocation.
Committed_AS mean amount of committed virtual address space. It's not
physical.

Example, if you create thread, libc allocate lots address space for stack.
but typical program don't use so large stack. then physical memory will be 
not allocated.


> 
> Vmalloc looks inconsequential, but, the Commit* numbers must be there
> for a reason.
> 
> Is it safe to continue running with such a perceived over-commit?

Probably safe. Java runtime usually makes a lot of overcommits. but I have
no way to know exactly your system state. 


> 
> Is this evidence of a leak or garbage collection issues?

Maybe no.

> 
> This system functions as an App/Web front end using  Tomcat servelet
> engine, FWIW.
> 
> Thanks
> 
> John Westerdale
> 
> 
> MemTotal:     16634464 kB
> MemFree:      11077520 kB
> Buffers:        420768 kB
> Cached:        4379000 kB
> SwapCached:          0 kB
> Active:        4577960 kB
> Inactive:       685344 kB
> HighTotal:    15859440 kB
> HighFree:     10987632 kB
> LowTotal:       775024 kB
> LowFree:         89888 kB
> SwapTotal:     4194296 kB
> SwapFree:      4194296 kB
> Dirty:              12 kB
> Writeback:           0 kB
> AnonPages:      462748 kB
> Mapped:          65420 kB
> Slab:           260144 kB
> PageTables:      21712 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:  12511528 kB
> Committed_AS: 22423356 kB
> VmallocTotal:   116728 kB
> VmallocUsed:      6600 kB
> VmallocChunk:   109612 kB
> HugePages_Total:     0
> HugePages_Free:      0
> HugePages_Rsvd:      0
> Hugepagesize:     2048 kB	
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
> Don't email: <a hrefmailto:"dont@kvack.org"> email@kvack.org </a>





[-- Attachment #2: Type: text/html, Size: 4188 bytes --]

Re: Difference between CommitLimit and Comitted_AS?

[KOSAKI Motohiro]

> Kosaki, 
> 
> Thanks for your contribution.
> 
> This system is only running 25 pct of projected load, so I am concerned. 
> 
> If we add up physical memory plus swap would that be a comfortable limit for  
> committed_AS?
> 
> Can tomcat  (or whaever webshere uses for servelets) be tuned to allocate a  
> fixed amount per session?  
> 
> as there are other applications on the same server, need to set up good  
> fences.
>  
> can I use ulimits to exercise some rough level of control?

When running Java, address space limitation (committed_AS and ulimit) are useless.
Java runtime consume much much AS rather than physical. So, I would recommend you
use memory cgroup.

Thanks.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>