[LSF/MM TOPIC] [ATTEND] memcg: soft limit reclaim (continue) and others

[LSF/MM TOPIC] [ATTEND] memcg: soft limit reclaim (continue) and others

[Ying Han]

some topics that I would like to discuss this year:

1) we talked about soft limit redesign during last LSF, and there are
quite a lot of efforts and changes being pushed after that. I would
like to take this time to sync-up our efforts and also discuss some of
the remaining issues.

Discussion from last year :
http://www.spinics.net/lists/linux-mm/msg17102.html and lots of
changes have been made since then.

2) memory.stat, this is the main stat file for all memcg statistics.
are we planning to keep stuff it for something like per-memcg
vmscan_stat, vmstat or not.

3) root cgroup now becomes quite interesting, especially after we
bring back the exclusive lru to root. To be more specific, root cgroup
now is like a sink which contains pages allocated on its own, and also
pages being re-parented. Those pages won't be reclaimed until there is
a global pressure, and we want to see anything we can do better.

--Ying

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] [ATTEND] memcg: soft limit reclaim (continue) and others

[KAMEZAWA Hiroyuki]

On Tue, 31 Jan 2012 11:59:40 -0800
Ying Han <yinghan@google.com> wrote:

> some topics that I would like to discuss this year:
> 
> 1) we talked about soft limit redesign during last LSF, and there are
> quite a lot of efforts and changes being pushed after that. I would
> like to take this time to sync-up our efforts and also discuss some of
> the remaining issues.
> 
> Discussion from last year :
> http://www.spinics.net/lists/linux-mm/msg17102.html and lots of
> changes have been made since then.
> 

Yes, it seems re-sync is required.

> 2) memory.stat, this is the main stat file for all memcg statistics.
> are we planning to keep stuff it for something like per-memcg
> vmscan_stat, vmstat or not.
> 

Could you calrify ? Do you want to have another stat file like memory.vmstat ?


> 3) root cgroup now becomes quite interesting, especially after we
> bring back the exclusive lru to root. To be more specific, root cgroup
> now is like a sink which contains pages allocated on its own, and also
> pages being re-parented. Those pages won't be reclaimed until there is
> a global pressure, and we want to see anything we can do better.
> 

I'm sorry I can't get your point. 

Do you think it's better to shrink root mem cgroup LRU even if there are
no memory pressure ? The benefit will be reduced memory reclaim latency.
Or Do you think root memcg should have some soft limit and should be
reclaimed in the same schedule line as other memcgs ? The benefit will be fairness.

or other idea ?

Thanks,
-Kame 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] [ATTEND] memcg: soft limit reclaim (continue) and others

[Ying Han]

On Tue, Jan 31, 2012 at 8:54 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Tue, 31 Jan 2012 11:59:40 -0800
> Ying Han <yinghan@google.com> wrote:
>
>> some topics that I would like to discuss this year:
>>
>> 1) we talked about soft limit redesign during last LSF, and there are
>> quite a lot of efforts and changes being pushed after that. I would
>> like to take this time to sync-up our efforts and also discuss some of
>> the remaining issues.
>>
>> Discussion from last year :
>> http://www.spinics.net/lists/linux-mm/msg17102.html and lots of
>> changes have been made since then.
>>
>
> Yes, it seems re-sync is required.
>
>> 2) memory.stat, this is the main stat file for all memcg statistics.
>> are we planning to keep stuff it for something like per-memcg
>> vmscan_stat, vmstat or not.
>>
>
> Could you calrify ? Do you want to have another stat file like memory.vmstat ?

I was planning to add per-memcg vmstat file at one point, but there
were discussions of just extending memory.stat. I don't mind to have
very long memory.stat file since my screen is now vertical anyway.
Just want to sync-up our final decision for later patches.

>
>
>> 3) root cgroup now becomes quite interesting, especially after we
>> bring back the exclusive lru to root. To be more specific, root cgroup
>> now is like a sink which contains pages allocated on its own, and also
>> pages being re-parented. Those pages won't be reclaimed until there is
>> a global pressure, and we want to see anything we can do better.
>>
>
> I'm sorry I can't get your point.
>
> Do you think it's better to shrink root mem cgroup LRU even if there are
> no memory pressure ?

The benefit will be reduced memory reclaim latency.

That is something I am thinking now. Now what we do in removing a
cgroup is re-parent all the pages, and root become a sink with all the
left-over pages. There is no external memory pressure to push those
pages out unless global reclaim, and the machine size will look
smaller and smaller on admin perspective.

I am thinking to use some existing reclaim mechanism to apply pressure
on those pages inside the kernel.

--Ying

> Or Do you think root memcg should have some soft limit and should be
> reclaimed in the same schedule line as other memcgs ? The benefit will be fairness.
>
> or other idea ?
>
> Thanks,
> -Kame
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] [ATTEND] memcg: soft limit reclaim (continue) and others

[KAMEZAWA Hiroyuki]

On Wed, 1 Feb 2012 16:00:44 -0800
Ying Han <yinghan@google.com> wrote:

> On Tue, Jan 31, 2012 at 8:54 PM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@jp.fujitsu.com> wrote:
> > On Tue, 31 Jan 2012 11:59:40 -0800
> > Ying Han <yinghan@google.com> wrote:
> >
> >> some topics that I would like to discuss this year:
> >>
> >> 1) we talked about soft limit redesign during last LSF, and there are
> >> quite a lot of efforts and changes being pushed after that. I would
> >> like to take this time to sync-up our efforts and also discuss some of
> >> the remaining issues.
> >>
> >> Discussion from last year :
> >> http://www.spinics.net/lists/linux-mm/msg17102.html and lots of
> >> changes have been made since then.
> >>
> >
> > Yes, it seems re-sync is required.
> >
> >> 2) memory.stat, this is the main stat file for all memcg statistics.
> >> are we planning to keep stuff it for something like per-memcg
> >> vmscan_stat, vmstat or not.
> >>
> >
> > Could you calrify ? Do you want to have another stat file like memory.vmstat ?
> 
> I was planning to add per-memcg vmstat file at one point, but there
> were discussions of just extending memory.stat. I don't mind to have
> very long memory.stat file since my screen is now vertical anyway.
> Just want to sync-up our final decision for later patches.
> 
> >
> >
> >> 3) root cgroup now becomes quite interesting, especially after we
> >> bring back the exclusive lru to root. To be more specific, root cgroup
> >> now is like a sink which contains pages allocated on its own, and also
> >> pages being re-parented. Those pages won't be reclaimed until there is
> >> a global pressure, and we want to see anything we can do better.
> >>
> >
> > I'm sorry I can't get your point.
> >
> > Do you think it's better to shrink root mem cgroup LRU even if there are
> > no memory pressure ?
> 
> The benefit will be reduced memory reclaim latency.
> 
> That is something I am thinking now. Now what we do in removing a
> cgroup is re-parent all the pages, and root become a sink with all the
> left-over pages. There is no external memory pressure to push those
> pages out unless global reclaim, and the machine size will look
> smaller and smaller on admin perspective.
> 
> I am thinking to use some existing reclaim mechanism to apply pressure
> on those pages inside the kernel.
> 

I considered your re-parent problem a bit...my suggestion is..

How about using cleancache other than re-parent ?

Assume a memcg X contains 100M Bytes of file caches.

# rmdir X 
  => puts all file caches to cleancache, by reclaiming all pages.
     Here, memcg's charge will disappear. and clean cache will have 100MB cache.

If a file cache will be accessed again, it will be re-charged to proper cgroup.
If a file cache won't be accessed soon, it will be dropped from the system.

Furthermore, we may be able to add control knobs
  - re-parent all file caches     (current implemenation)
  - drop all file caches at rmdir
  - drop all file caches to cleancache.

Size of clean cache may be a problem even if it's configurable..

I don't have good idea for ANON pages but I think there are no big problem with
re-parenting them..

Thanks,
-Kame











--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>