Re: Fw: [PATCH] memcg: add reclaim statistics accounting

[KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On Wed, 27 Apr 2011 20:43:58 -0700
Ying Han <yinghan@google.com> wrote:

> On Wed, Apr 27, 2011 at 8:16 PM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@jp.fujitsu.com> wrote:
> > sorry, I had wrong TO:...
> >
> > Begin forwarded message:
> >
> > Date: Thu, 28 Apr 2011 12:02:34 +0900
> > From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> > To: linux-mm@vger.kernel.org
> > Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "nishimura@mxp.nes.nec.co.jp" <nishimura@mxp.nes.nec.co.jp>, "balbir@linux.vnet.ibm.com" <balbir@linux.vnet.ibm.com>, Ying Han <yinghan@google.com>, "akpm@linux-foundation.org" <akpm@linux-foundation.org>
> > Subject: [PATCH] memcg: add reclaim statistics accounting
> >
> >
> >
> > Now, memory cgroup provides poor reclaim statistics per memcg. This
> > patch adds statistics for direct/soft reclaim as the number of
> > pages scans, the number of page freed by reclaim, the nanoseconds of
> > latency at reclaim.
> >
> > It's good to add statistics before we modify memcg/global reclaim, largely.
> > This patch refactors current soft limit status and add an unified update logic.
> >
> > For example, After #cat 195Mfile > /dev/null under 100M limit.
> > A  A  A  A # cat /cgroup/memory/A/memory.stat
> > A  A  A  A ....
> > A  A  A  A limit_freed 24592
> 
> why not "limit_steal" ?
> 
> > A  A  A  A soft_steal 0
> > A  A  A  A limit_scan 43974
> > A  A  A  A soft_scan 0
> > A  A  A  A limit_latency 133837417
> >
> > nearly 96M caches are freed. scanned twice. used 133ms.
> 
> Does it make sense to split up the soft_steal/scan for bg reclaim and
> direct reclaim? The same for the limit_steal/scan. I am now testing
> the patch to add the soft_limit reclaim on global ttfp, and i already
> have the patch to add the following:
> 
> kswapd_soft_steal 0
> kswapd_soft_scan 0
> direct_soft_steal 0
> direct_soft_scan 0
> kswapd_steal 0
> pg_pgsteal 0
> kswapd_pgscan 0
> pg_scan 0
> 

I'll not post updated version until the end of holidays but my latest plan is
adding


limit_direct_free   - # of pages freed by limit in foreground (not stealed, you freed by yourself's limit)
soft_kswapd_steal   - # of pages stealed by kswapd based on soft limit
limit_direct_scan   - # of pages scanned by limit in foreground
soft_kswapd_scan    - # of pages scanned by kswapd based on soft limit

And then, you can add

soft_direct_steal     - # of pages stealed by foreground reclaim based on soft limit
soft_direct_scan        - # of pages scanned by foreground reclaim based on soft limit

And

kern_direct_steal  - # of pages stealed by foreground reclaim at memory shortage.
kern_direct_scan   - # of pages scanned by foreground reclaim at memory shortage.
kern_direct_steal  - # of pages stealed by kswapd at memory shortage
kern_direct_scan   - # of pages scanned by kswapd at memory shortage

(Above kern_xxx number includes soft_xxx in it. ) These will show influence by
other cgroups.

And

wmark_bg_free      - # of pages freed by watermark in background(not kswapd)
wmark_bg_scan      - # of pages scanned by watermark in background(not kswapd)

Hmm ? too many stats ;) 

And making current soft_steal/soft_scan planned to be obsolete...
 
Thanks,
-Kame




















--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>