RE: CGroup unused allocated slab objects will not get released

["Saeed Karimabadi (skarimab)" <skarimab@cisco.com>]

Hi Roman,

Thanks for your prompt reply and also sharing your patch. 
I did build kernel 5.3.0 with your patch and I can confirm your patch fixes the problem I was describing. 
I used Qemu for this test and the script ran 1000 tasks concurrently in 100 different cgroups.
I'm wondering if your could has gone through any long term regression test?
Do you see any possible simple patch that can fix this excessive memory usage in older kernel code like 4.x versions?

Here are more detail information about the test results:

******************************************************************************
Your proposed patche back-ported to Kernel 5.3.0 :
  https://github.com/rgushchin/linux/tree/new_slab.rfc.v5.3
------------- Before Running the script  -------------
Slab:                      42756 kB
SReclaimable:      25408 kB
SUnreclaim:          17348 kB
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : 
	            tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
task_struct          102    200   3200   10    8 : tunables    0    0    0 : slabdata     20     20      0
------------- After running the script -------------
Slab:                      43736 kB
SReclaimable:      25484 kB
SUnreclaim:         18252 kB
task_struct          149    220   3200   10    8 : tunables    0    0    0 : slabdata     22     22      0

******************************************************************************
Vanilla Kernel 5.3.0 :
------------- Before Running the script  -------------
Slab:                      34704 kB
SReclaimable:      19956 kB
SUnreclaim:          14748 kB
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : 
                           tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
task_struct           99    130   3200   10    8 : tunables    0    0    0 : slabdata     13     13      0
------------- After running the script -------------
Slab:                      59388 kB
SReclaimable:      23580 kB
SUnreclaim:          35808 kB
task_struct         1174   1230   3200   10    8 : tunables    0    0    0 : slabdata    123    123      0

Regards,
Saeed


-----Original Message-----
From: Roman Gushchin <guro@fb.com> 
Sent: Wednesday, September 18, 2019 3:23 PM
To: Saeed Karimabadi (skarimab) <skarimab@cisco.com>
Cc: Christoph Lameter <cl@linux.com>; Pekka Enberg <penberg@kernel.org>; David Rientjes <rientjes@google.com>; Joonsoo Kim <iamjoonsoo.kim@lge.com>; Andrew Morton <akpm@linux-foundation.org>; linux-mm@kvack.org; Tejun Heo <tj@kernel.org>; Li Zefan <lizefan@huawei.com>; Johannes Weiner <hannes@cmpxchg.org>; cgroups@vger.kernel.org; Michal Hocko <mhocko@kernel.org>; Vladimir Davydov <vdavydov.dev@gmail.com>; xe-linux-external(mailer list) <xe-linux-external@cisco.com>
Subject: Re: CGroup unused allocated slab objects will not get released

On Wed, Sep 18, 2019 at 08:31:18PM +0000, Saeed Karimabadi (skarimab) wrote:
> Hi  Kernel Maintainers,
> 
> We are chasing an issue where slab allocator is not releasing task_struct slab objects allocated by cgroups 
> and we are wondering if this is a known issue or an expected behavior ?
> If we stress test the system and spawn multiple tasks with different cgroups, number of active allocated 
> task_struct objects will increase but kernel will never release those memory later on, even though if system 
> goes to the idle state with lower number of the running processes.

Hi Saeed!

I've recently proposed a new slab memory cgroup controller, which aims to solve
the problem you're describing: https://lwn.net/Articles/798605/ . It also generally
reduces the amount of memory used by slabs.

I've been told that not all e-mails in the patchset reached lkml,
so, please, find the original patchset here:
  https://github.com/rgushchin/linux/tree/new_slab.rfc
and it's backport to the 5.3 release here:
  https://github.com/rgushchin/linux/tree/new_slab.rfc.v5.3

If you can try it on your setup, I'd appreciate it a lot, and it also can
help with merging it upstream soon.

Thank you!

Roman