[PATCH] add vmalloc stats to meminfo

[PATCH] add vmalloc stats to meminfo

[Dave Hansen]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

Some workloads like to eat up a lot of vmalloc space.  It is often hard to tell
whether this is because the area is too small, or just too fragmented.  This 
makes it easy to determine.

-- 
Dave Hansen
haveblue@us.ibm.com


[-- Attachment #2: vmalloc-stats-2.5.34-mm4-2.patch --]
[-- Type: text/plain, Size: 2431 bytes --]

diff -ur linux-2.5.34-mm4/fs/proc/proc_misc.c linux-2.5.34-mm4-vmalloc-stats/fs/proc/proc_misc.c
--- linux-2.5.34-mm4/fs/proc/proc_misc.c	Sat Sep 14 21:23:54 2002
+++ linux-2.5.34-mm4-vmalloc-stats/fs/proc/proc_misc.c	Sat Sep 14 22:38:12 2002
@@ -38,6 +38,7 @@
 #include <linux/smp_lock.h>
 #include <linux/seq_file.h>
 #include <linux/times.h>
+#include <linux/vmalloc.h>
 
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
@@ -128,6 +129,40 @@
 	return proc_calc_metrics(page, start, off, count, eof, len);
 }
 
+struct vmalloc_info {
+	unsigned long used;
+	unsigned long largest_chunk;
+};
+
+static struct vmalloc_info get_vmalloc_info(void)
+{
+	unsigned long prev_end = VMALLOC_START;
+	struct vm_struct* vma;
+	struct vmalloc_info vmi;
+	vmi.used = 0;
+
+	read_lock(&vmlist_lock);
+	
+	if(!vmlist) 
+		vmi.largest_chunk = (VMALLOC_END-VMALLOC_START);
+	else 
+		vmi.largest_chunk = 0;
+	
+	for (vma = vmlist; vma; vma = vma->next) {
+		unsigned long free_area_size = 
+			(unsigned long)vma->addr - prev_end;
+		vmi.used += vma->size;
+		if (vmi.largest_chunk < free_area_size ) 
+			vmi.largest_chunk = free_area_size;
+		prev_end = vma->size + (unsigned long)vma->addr;
+	}
+	if(VMALLOC_END-prev_end > vmi.largest_chunk)
+		vmi.largest_chunk = VMALLOC_END-prev_end;
+	
+	read_unlock(&vmlist_lock);
+	return vmi;
+}
+
 extern atomic_t vm_committed_space;
 
 static int meminfo_read_proc(char *page, char **start, off_t off,
@@ -138,6 +173,8 @@
 	struct page_state ps;
 	unsigned long inactive;
 	unsigned long active;
+	unsigned long vmtot;
+	struct vmalloc_info vmi;
 
 	get_page_state(&ps);
 	get_zone_counts(&active, &inactive);
@@ -150,6 +187,11 @@
 	si_swapinfo(&i);
 	committed = atomic_read(&vm_committed_space);
 
+	vmtot = (VMALLOC_END-VMALLOC_START)>>10;
+	vmi = get_vmalloc_info();
+	vmi.used >>= 10;
+	vmi.largest_chunk >>= 10;
+
 	/*
 	 * Tagged format, for easy grepping and expansion.
 	 */
@@ -174,7 +216,10 @@
 		"Slab:         %8lu kB\n"
 		"Committed_AS: %8u kB\n"
 		"PageTables:   %8lu kB\n"
-		"ReverseMaps:  %8lu\n",
+		"ReverseMaps:  %8lu\n"
+		"VmalTotal:    %8lu kB\n"
+		"VmalUsed:     %8lu kB\n"
+		"VmalChunk:    %8lu kB\n",
 		K(i.totalram),
 		K(i.freeram),
 		K(i.sharedram),
@@ -195,7 +240,10 @@
 		K(ps.nr_slab),
 		K(committed),
 		K(ps.nr_page_table_pages),
-		ps.nr_reverse_maps
+		ps.nr_reverse_maps,
+		vmtot,
+		vmi.used,
+		vmi.largest_chunk
 		);
 
 #ifdef CONFIG_HUGETLB_PAGE

Re: [PATCH] add vmalloc stats to meminfo

[Andrew Morton]

Dave Hansen wrote:
> 
> Some workloads like to eat up a lot of vmalloc space.

Which workloads are those?

>  It is often hard to tell
> whether this is because the area is too small, or just too fragmented.  This
> makes it easy to determine.

I do not recall ever having seen any bug/problem reports which this patch
would have helped to solve.  Could you explain in more detai why is it useful?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[William Lee Irwin III]

Dave Hansen wrote:
>>  It is often hard to tell
>> whether this is because the area is too small, or just too fragmented.  This
>> makes it easy to determine.

On Sun, Sep 15, 2002 at 12:17:30AM -0700, Andrew Morton wrote:
> I do not recall ever having seen any bug/problem reports which this patch
> would have helped to solve.  Could you explain in more detai why is it useful?

LDT's were formerly allocated in vmallocspace. This presented difficulties
with many simultaneous threaded applications. Also, given that there is
zero vmallocspace OOM recovery now present in the kernel some method of
monitoring this aspect of system behavior up until the point of failure is
useful for detecting further problem areas (LDT's were addressed by using
non-vmalloc allocations).

Also, dynamic vmalloc allocations may very well be starved by boot-time
allocations on systems where much vmallocspace is required for IO memory.
The failure mode of such is effectively deadlock, since they block
indefinitely waiting for permanent boot-time allocations to be freed up.

Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[William Lee Irwin III]

On Sun, Sep 15, 2002 at 12:11:57AM -0700, William Lee Irwin III wrote:
> Also, dynamic vmalloc allocations may very well be starved by boot-time
> allocations on systems where much vmallocspace is required for IO memory.
> The failure mode of such is effectively deadlock, since they block
> indefinitely waiting for permanent boot-time allocations to be freed up.

This is dead wrong. NFI wtf I was thinking. Ignore that one.


Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[M. Edward (Ed) Borasky]

On Sun, 15 Sep 2002, William Lee Irwin III wrote:

> Dave Hansen wrote:
> >>  It is often hard to tell
> >> whether this is because the area is too small, or just too fragmented.  This
> LDT's were formerly allocated in vmallocspace. This presented difficulties
> with many simultaneous threaded applications. Also, given that there is
> zero vmallocspace OOM recovery now present in the kernel some method of
> monitoring this aspect of system behavior up until the point of failure is
> useful for detecting further problem areas (LDT's were addressed by using
> non-vmalloc allocations).
>
> Also, dynamic vmalloc allocations may very well be starved by boot-time
> allocations on systems where much vmallocspace is required for IO memory.
> The failure mode of such is effectively deadlock, since they block
> indefinitely waiting for permanent boot-time allocations to be freed up.


Thank you!! How difficult would it be to back-port this to 2.4.18?

--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach

znmeb@borasky-research.net
http://www.borasky-research.net/Meta-Trading-Coach.htm
http://groups.yahoo.com/group/meta-trading-coach

ransacked: participated in a sack race.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[William Lee Irwin III]

On Sun, 15 Sep 2002, William Lee Irwin III wrote:
>> Also, dynamic vmalloc allocations may very well be starved by boot-time
>> allocations on systems where much vmallocspace is required for IO memory.
>> The failure mode of such is effectively deadlock, since they block
>> indefinitely waiting for permanent boot-time allocations to be freed up.

On Sun, Sep 15, 2002 at 10:23:24AM -0700, M. Edward (Ed) Borasky wrote:
> Thank you!! How difficult would it be to back-port this to 2.4.18?

Note the follow-up to this saying that the above paragraph was incorrect.
It doesn't sleep except to allocate backing memmory.


Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: [PATCH] add vmalloc stats to meminfo

[M. Edward Borasky]

I'd still like a backport to the (Red Hat 7.3) 2.4.18 kernel if it is
possible. I'm a big fan of statistics and logging them.

M. Edward (Ed) Borasky
mailto: znmeb@borasky-research.net
http://www.pdxneurosemantics.com
http://www.meta-trading-coach.com
http://www.borasky-research.net

Coaching: It's Not Just for Athletes and Executives Any More!

-----Original Message-----
From: William Lee Irwin III [mailto:wli@holomorphy.com]
Sent: Sunday, September 15, 2002 10:26 AM
To: M. Edward (Ed) Borasky
Cc: Andrew Morton; Dave Hansen; Martin J. Bligh;
linux-kernel@vger.kernel.org; linux-mm@kvack.org
Subject: Re: [PATCH] add vmalloc stats to meminfo

On Sun, 15 Sep 2002, William Lee Irwin III wrote:
>> Also, dynamic vmalloc allocations may very well be starved by boot-time
>> allocations on systems where much vmallocspace is required for IO memory.
>> The failure mode of such is effectively deadlock, since they block
>> indefinitely waiting for permanent boot-time allocations to be freed up.

On Sun, Sep 15, 2002 at 10:23:24AM -0700, M. Edward (Ed) Borasky wrote:
> Thank you!! How difficult would it be to back-port this to 2.4.18?

Note the follow-up to this saying that the above paragraph was incorrect.
It doesn't sleep except to allocate backing memmory.


Bill

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[Martin J. Bligh]

>> Some workloads like to eat up a lot of vmalloc space.
> 
> Which workloads are those?
> 
>>  It is often hard to tell
>> whether this is because the area is too small, or just too fragmented.  This
>> makes it easy to determine.
> 
> I do not recall ever having seen any bug/problem reports which this patch
> would have helped to solve.  Could you explain in more detai why is it useful?

Seen on specweb - doubling the size of the vmalloc space made it go
away, but without any counters, it was all really just guesswork.

I am also going to implement per-node slabcache on top of vmalloc,
as slabs have to be in permanently mapped KVA, but all ZONE_NORMAL
is on node 0. That's going to put a lot of pressure on the vmalloc
area.

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [PATCH] add vmalloc stats to meminfo

[Matt Porter]

On Sat, Sep 14, 2002 at 11:03:39PM -0700, Dave Hansen wrote:
> Some workloads like to eat up a lot of vmalloc space.  It is often hard to tell
> whether this is because the area is too small, or just too fragmented.  This 
> makes it easy to determine.

Great, I was going to do something nearly the same to help out
with debugging high-end embedded applications.  It is quite common
for us to see multiple PCI masters with PCI memory windows in sizes
ranging from 256MB-1GB that are being ioremapped and consuming
vmalloc space (along with all the other consumers).  I'd love to
see this in the kernel since it would make it much easier to debug
some folks' custom board ports when they show symptoms of running
out of vmalloc space (i.e. modules not loading).

Regards,
-- 
Matt Porter
porter@cox.net
This is Linux Country. On a quiet night, you can hear Windows reboot.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/