From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail202.messagelabs.com (mail202.messagelabs.com [216.82.254.227]) by kanga.kvack.org (Postfix) with ESMTP id 5BE478D003B for ; Fri, 22 Apr 2011 03:59:30 -0400 (EDT) Received: from wpaz17.hot.corp.google.com (wpaz17.hot.corp.google.com [172.24.198.81]) by smtp-out.google.com with ESMTP id p3M7xSae032156 for ; Fri, 22 Apr 2011 00:59:28 -0700 Received: from qwf7 (qwf7.prod.google.com [10.241.194.71]) by wpaz17.hot.corp.google.com with ESMTP id p3M7xQnl027080 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Fri, 22 Apr 2011 00:59:27 -0700 Received: by qwf7 with SMTP id 7so193954qwf.10 for ; Fri, 22 Apr 2011 00:59:26 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20110422164622.a8350bc5.kamezawa.hiroyu@jp.fujitsu.com> References: <1303446260-21333-1-git-send-email-yinghan@google.com> <1303446260-21333-5-git-send-email-yinghan@google.com> <20110422133643.6a36d838.kamezawa.hiroyu@jp.fujitsu.com> <20110422140023.949e5737.kamezawa.hiroyu@jp.fujitsu.com> <20110422145943.a8f5a4ef.kamezawa.hiroyu@jp.fujitsu.com> <20110422164622.a8350bc5.kamezawa.hiroyu@jp.fujitsu.com> Date: Fri, 22 Apr 2011 00:59:26 -0700 Message-ID: Subject: Re: [PATCH V7 4/9] Add memcg kswapd thread pool From: Ying Han Content-Type: multipart/alternative; boundary=0016e64aefda993a0904a17d3cd2 Sender: owner-linux-mm@kvack.org List-ID: To: KAMEZAWA Hiroyuki Cc: KOSAKI Motohiro , Minchan Kim , Daisuke Nishimura , Balbir Singh , Tejun Heo , Pavel Emelyanov , Andrew Morton , Li Zefan , Mel Gorman , Christoph Lameter , Johannes Weiner , Rik van Riel , Hugh Dickins , Michal Hocko , Dave Hansen , Zhu Yanhai , linux-mm@kvack.org --0016e64aefda993a0904a17d3cd2 Content-Type: text/plain; charset=ISO-8859-1 On Fri, Apr 22, 2011 at 12:46 AM, KAMEZAWA Hiroyuki < kamezawa.hiroyu@jp.fujitsu.com> wrote: > On Thu, 21 Apr 2011 23:10:58 -0700 > Ying Han wrote: > > > On Thu, Apr 21, 2011 at 10:59 PM, KAMEZAWA Hiroyuki < > > kamezawa.hiroyu@jp.fujitsu.com> wrote: > > > > > On Thu, 21 Apr 2011 22:53:19 -0700 > > > Ying Han wrote: > > > > > > > On Thu, Apr 21, 2011 at 10:00 PM, KAMEZAWA Hiroyuki < > > > > kamezawa.hiroyu@jp.fujitsu.com> wrote: > > > > > > > > > On Thu, 21 Apr 2011 21:49:04 -0700 > > > > > Ying Han wrote: > > > > > > > > > > > On Thu, Apr 21, 2011 at 9:36 PM, KAMEZAWA Hiroyuki < > > > > > > kamezawa.hiroyu@jp.fujitsu.com> wrote: > > > > add a counter for kswapd-scan and kswapd-reclaim, kswapd-pickup will > show > > > you information, if necessary it's good to show some latecy stat. I > think > > > we can add enough information by adding stats (or debug by perf tools.) > > > I'll consider this a a bit more. > > > > > > > Something like "kswapd_pgscan" and "kswapd_steal" per memcg? If we are > going > > to the thread-pool, we definitely need to add more stats to give us > enough > > visibility of per-memcg background reclaim activity. Still, not sure > about > > the cpu-cycles. > > > > BTW, Kosaki requeted me not to have private thread pool implementation and > use workqueue. I think he is right. So, I'd like to write a patch to > enhance > workqueue for using it for memcg (Of couse, I'll make a private workqueue.) > > Hmm. Can you give a bit more details of the logic behind? and what's about the private workqueue? Also, how we plan to solve the better debug-ability issue. > > == > 2. regarding to the alternative workqueue, which is more complicated and we > need to be very careful of work items in the workqueue. We've experienced > in > one workitem stucks and the rest of the work item won't proceed. For > example > in dirty page writeback, one heavily writer cgroup could starve the other > cgroups from flushing dirty pages to the same disk. In the kswapd case, I > can > imagine we might have similar senario. How to prioritize the workitems is > another problem. The order of adding the workitems in the queue reflects > the > order of cgroups being reclaimed. We don't have that restriction currently > but > relying on the cpu scheduler to put kswapd on the right cpu-core to run. We > "might" introduce priority later for reclaim and how are we gonna deal with > that. > == > > From this, I feel I need to use unbound workqueue. BTW, with patches for > current thread pool model, I think starvation problem by dirty pages > cannot be seen. > Anyway, I'll give a try. > Then do you suggest me to wait for your patch for my next post? --Ying > > Thanks, > -Kame > > > > > > --0016e64aefda993a0904a17d3cd2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Fri, Apr 22, 2011 at 12:46 AM, KAMEZA= WA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:
On Thu, 21 Apr 2011 23:10:58 -0700
Ying Han <yingha= n@google.com> wrote:

> On Thu, Apr 21, 2011 at 10:59 PM, KAMEZAWA Hiroyuki <
> kamezawa.hiroyu@jp.f= ujitsu.com> wrote:
>
> > On Thu, 21 Apr 2011 22:53:19 -0700
> > Ying Han <yinghan@google= .com> wrote:
> >
> > > On Thu, Apr 21, 2011 at 10:00 PM, KAMEZAWA Hiroyuki <
> > > kamezawa.h= iroyu@jp.fujitsu.com> wrote:
> > >
> > > > On Thu, 21 Apr 2011 21:49:04 -0700
> > > > Ying Han <ying= han@google.com> wrote:
> > > >
> > > > > On Thu, Apr 21, 2011 at 9:36 PM, KAMEZAWA Hiroyuki= <
> > > > > = kamezawa.hiroyu@jp.fujitsu.com> wrote:

> > add a counter for kswapd-scan and kswapd-= reclaim, kswapd-pickup will show
> > you information, if necessary it's good to show some latecy s= tat. I think
> > we can add enough information by adding stats (or debug by perf t= ools.)
> > I'll consider this a a bit more.
> >
>
> Something like "kswapd_pgscan" and "kswapd_steal" = per memcg? If we are going
> to the thread-pool, we definitely need to add more stats to give us en= ough
> visibility of per-memcg background reclaim activity. Still, not sure a= bout
> the cpu-cycles.
>

BTW, Kosaki requeted me not to have private thread pool implementatio= n and
use workqueue. I think he is right. So, I'd like to write a patch to en= hance
workqueue for using it for memcg (Of couse, I'll make a private workque= ue.)

Hmm. Can you give a bit more details of the logic beh= ind? and what's about the private workqueue? Also, how
we pla= n to solve the better=A0debug-ability issue.
=A0=A0

=3D=3D
2. regarding to the alternative workqueue, which is more complicated and we=
need to be very careful of work items in the workqueue. We've experienc= ed in
one workitem stucks and the rest of the work item won't proceed. For ex= ample
in dirty page writeback, one heavily writer cgroup could starve the other cgroups from flushing dirty pages to the same disk. In the kswapd case, I c= an
imagine we might have similar senario. How to prioritize the workitems is another problem. The order of adding the workitems in the queue reflects th= e
order of cgroups being reclaimed. We don't have that restriction curren= tly but
relying on the cpu scheduler to put kswapd on the right cpu-core to run. We=
"might" introduce priority later for reclaim and how are we gonna= deal with
that.
=3D=3D

>>From this, I feel I need to use unbound workqueue. BTW, with patches for current thread pool model, I think starvation problem by dirty pages
cannot be seen.
Anyway, I'll give a try.

Then do yo= u suggest me to wait for your patch for my next post?=A0

--Ying=A0

Thanks,
-Kame






--0016e64aefda993a0904a17d3cd2-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org