Re: memcg: slab control

[Pavel Emelyanov <xemul@parallels.com>]

Ying Han wrote:
> On Thu, Nov 26, 2009 at 4:31 AM, Pavel Emelyanov <xemul@parallels.com> wrote:
>>> Aren't there patches to make the kernel track which cgroup caused
>>> which disk I/O? If so, it should be possible to charge the bios to the
>>> right cgroup.
>>>
>>> Maybe one way to decide which kernel allocations should be accounted
>>> would be to look at the calling context: If the allocation is done in
>>> user context (syscall), then it could be counted towards that user,
>>> while if the allocation is done in interrupt or kthread context, it
>>> shouldn't be accounted.
>>>
>>> Of course, this wouldn't be perfect, but it might be a good enough
>>> approximation.
>> I disagree. Bio-s are allocated in user context for all typical reads
>> (unless we requested aio) and are allocated either in pdflush context
>> or (!) in arbitrary task context for writes (e.g. via try_to_free_pages)
>> and thus such bio/buffer_head accounting will be completely random.
>>
>> One of the way to achieve the goal I can propose the following (it's
>> not perfect, but just smth to start discussion from).
>>
>> We implement support for accounting based on a bit on a kmem_cache
>> structure and mark all kmalloc caches as not-accountable. Then we grep
>> the kernel to find all kmalloc-s and think - if a kmalloc is to be
>> accounted we turn this into kmem_cache_alloc() with dedicated
>> kmem_cache and mark it as accountable.
> 
> Well it would be nice to count all kernel memory allocations
> trigger-able by user programs, the kernel
> memory includes kernel slabs as well as the pages directly allocated
> by get_free_pages(). However some
> of the allocations happen asynchronously like in kernel thread or
> interrupt context. We can not charge them
> on the random process happen to run at the time.
> 
> We can either not count those allocations, or do some special
> treatment to remember who owns those allocations.
> In our networking intensive workload, it causes us lots of trouble of
> miscounting the networking slabs for incoming
> packets. So we make changes in the networking stack which records the
> owner of the socket and then charge the
> slab later using that recorded information.

That's the same as what we do, but note, that simple accounting
is not enough for socket buffers (a.k.a. skb-s). In a perfect world
we should implement a memory management similar to what already
exists in the networking.

In particular - sockets should not report errors in case of kmem
shortage, but instead goto waiting state. Besides, TCP sockets should
adjust the TCP window according to the current kmem controller state
and this task is quite complex.

> --Ying
> 
>>> -- Suleiman
>>>
>>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>