Re: slab fragmentation ?

[Badari Pulavarty <pbadari@us.ibm.com>]

On Tue, 2004-10-05 at 10:58, Manfred Spraul wrote:
> Badari Pulavarty wrote:
> 
> >Here is the /proc/slabinfo output collected every 1 second while
> >running the scsi-debug test. I enabled STATS and DEBUG.
> >
> >  
> >
> Ok, thanks.
> 
> Before test:
> size-40 2088 9760 64 61 1
>     tunables 32 16 8
>     slabdata 160 160 0
>     globalstat 3324 2010 160 0 0 0 173
>     cpustat 5675 213 3945 2
> 
> 2nd value of cpustat ALLOCMISS: 213 calls to cache_alloc_refill.
> first value of slabdata: 160 slabs.
> sane.
> 
> 2nd value of globalstat: maximum 2010 objects allocated.
> first value of the "size" line: 2088 objects active right now.
> sane,too.
> 
> After a few seconds:
> 
> size-40 4582 31110 64 61 1
>     tunables 32 16 8
>     slabdata 510 510 0
>     globalstat 5468 4085 510 0 0 0 173
>     cpustat 7924 347 4247 2
> first value of slabdata: 510 slabs around.
> second value of cpudata: total of 347 cache_alloc_refill calls.
> Huh? Very odd. Each call of cache_alloc_refill causes at most one 
> cache_grow, and a cache_grow creates exactly one slab.

Yep.

> 
> 2nd value of globalstat: max of 4085 objects allocated.
> First value of size line: 4582 objects active.
> Huh? More active objects than kmem_cache_alloc/kmalloc calls?
> 
> Could you add a printk into kmem_cache_alloc_node()? Perhaps with a 
> dump_stack() or something like that. I'd bet that someone calls 
> kmem_cache_alloc_node(). Probably indirectly through alloc_percpu() - 
> hch recently broke the public interface.
> 
> Hmm. init_disk_stats() uses alloc_percpu. What are you testing? Creating 
> lots of disks for testing? If you end up calling kmem_cache_alloc_node() 
> then I know what happens.

Most likely thats whats happening. I am simulating 512 disks using
scsi-debug. As part of this, I am sure init_disk_stats() gets called.

> The fix would be simple: kmem_cache_alloc_node must walk through the 
> list of partial slabs and check if it finds a slab from the correct 
> node. If it does, then just use that slab instead of allocating a new 
> one. And statistics must be added to kmem_cache_alloc_node - I forgot 
> that when I wrote the function.

I will add more debug to find out if this is happening or not.

What stats you want me to update in kmem_cache_alloc_node() ?

Thanks for help me out on this one.

Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>