From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail190.messagelabs.com (mail190.messagelabs.com [216.82.249.51]) by kanga.kvack.org (Postfix) with ESMTP id 87E4D8D003B for ; Thu, 21 Apr 2011 12:56:56 -0400 (EDT) Received: from wpaz5.hot.corp.google.com (wpaz5.hot.corp.google.com [172.24.198.69]) by smtp-out.google.com with ESMTP id p3LGupvr012666 for ; Thu, 21 Apr 2011 09:56:51 -0700 Received: from qyk2 (qyk2.prod.google.com [10.241.83.130]) by wpaz5.hot.corp.google.com with ESMTP id p3LGtZQs020947 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Thu, 21 Apr 2011 09:56:50 -0700 Received: by qyk2 with SMTP id 2so1341817qyk.16 for ; Thu, 21 Apr 2011 09:56:50 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <1303185466-2532-1-git-send-email-yinghan@google.com> <20110421124357.c94a03a5.kamezawa.hiroyu@jp.fujitsu.com> <20110421174616.4fd79c5e.kamezawa.hiroyu@jp.fujitsu.com> Date: Thu, 21 Apr 2011 09:56:48 -0700 Message-ID: Subject: Re: [PATCH 1/3] memcg kswapd thread pool (Was Re: [PATCH V6 00/10] memcg: per cgroup background reclaim From: Ying Han Content-Type: multipart/alternative; boundary=0016e64aefda8c629d04a170a0dd Sender: owner-linux-mm@kvack.org List-ID: To: Minchan Kim Cc: KAMEZAWA Hiroyuki , KOSAKI Motohiro , Daisuke Nishimura , Balbir Singh , Tejun Heo , Pavel Emelyanov , Andrew Morton , Li Zefan , Mel Gorman , Christoph Lameter , Johannes Weiner , Rik van Riel , Hugh Dickins , Michal Hocko , Dave Hansen , Zhu Yanhai , linux-mm@kvack.org --0016e64aefda8c629d04a170a0dd Content-Type: text/plain; charset=ISO-8859-1 On Thu, Apr 21, 2011 at 2:05 AM, Minchan Kim wrote: > On Thu, Apr 21, 2011 at 5:46 PM, KAMEZAWA Hiroyuki > wrote: > > On Thu, 21 Apr 2011 17:10:23 +0900 > > Minchan Kim wrote: > > > >> Hi Kame, > >> > >> On Thu, Apr 21, 2011 at 12:43 PM, KAMEZAWA Hiroyuki > >> wrote: > >> > Ying, please take this just a hint, you don't need to implement this > as is. > >> > == > >> > Now, memcg-kswapd is created per a cgroup. Considering there are users > >> > who creates hundreds on cgroup on a system, it consumes too much > >> > resources, memory, cputime. > >> > > >> > This patch creates a thread pool for memcg-kswapd. All memcg which > >> > needs background recalim are linked to a list and memcg-kswapd > >> > picks up a memcg from the list and run reclaim. This reclaimes > >> > SWAP_CLUSTER_MAX of pages and putback the memcg to the lail of > >> > list. memcg-kswapd will visit memcgs in round-robin manner and > >> > reduce usages. > >> > > >> > >> I didn't look at code yet but as I just look over the description, I > >> have a concern. > >> We have discussed LRU separation between global and memcg. > > > > Please discuss global LRU in other thread. memcg-kswapd is not related > > to global LRU _at all_. > > > > And this patch set is independent from the things we discussed at LSF. > > > > > >> The clear goal is that how to keep _fairness_. > >> > >> For example, > >> > >> memcg-1 : # pages of LRU : 64 > >> memcg-2 : # pages of LRU : 128 > >> memcg-3 : # pages of LRU : 256 > >> > >> If we have to reclaim 96 pages, memcg-1 would be lost half of pages. > >> It's much greater than others so memcg 1's page LRU rotation cycle > >> would be very fast, then working set pages in memcg-1 don't have a > >> chance to promote. > >> Is it fair? > >> > >> I think we should consider memcg-LRU size as doing round-robin. > >> > > > > This set doesn't implement a feature to handle your example case, at all. > > Sure. Sorry for the confusing. > I don't mean global LRU but it a fairness although this series is > based on per-memcg targeting. > > > > > This patch set handles > > > > memcg-1: # pages of over watermark : 64 > > memcg-2: # pages of over watermark : 128 > > memcg-3: # pages of over watermark : 256 > > > > And finally reclaim all pages over watermarks which user requested. > > Considering fairness, what we consider is in what order we reclaim > > memory memcg-1, memcg-2, memcg-3 and how to avoid unnecessary cpu > > hogging at reclaiming memory all (64+128+256) > > > > This thread pool reclaim 32 pages per iteration with patch-1 and visit > all > > in round-robin. > > With patch-2, reclaim 32*weight pages per iteration on each memcg. > > > > I should have seen the patch [2/3] before posting the comment. > Maybe you seem consider my concern. > Okay. I will look the idea. > For any ideas on global kswapd and soft_limit reclaim based on round-robin ( discussed in LSF), please move the discussion to : [RFC no patch yet] memcg: revisit soft_limit reclaim on contention: http://permalink.gmane.org/gmane.linux.kernel.mm/60966" I already started with the patch and hopefully to post some result soon. --Ying > > > > Thanks, > > -Kame > > > > > > > > -- > Kind regards, > Minchan Kim > --0016e64aefda8c629d04a170a0dd Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Thu, Apr 21, 2011 at 2:05 AM, Minchan= Kim <minchan= .kim@gmail.com> wrote:
On Thu, Apr 21, 2011 at 5:46 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Thu, 21 Apr 2011 17:10:23 +0900
> Minchan Kim <minchan.kim@g= mail.com> wrote:
>
>> Hi Kame,
>>
>> On Thu, Apr 21, 2011 at 12:43 PM, KAMEZAWA Hiroyuki
>> <kamezawa.hir= oyu@jp.fujitsu.com> wrote:
>> > Ying, please take this just a hint, you don't need to imp= lement this as is.
>> > =3D=3D
>> > Now, memcg-kswapd is created per a cgroup. Considering there = are users
>> > who creates hundreds on cgroup on a system, it consumes too m= uch
>> > resources, memory, cputime.
>> >
>> > This patch creates a thread pool for memcg-kswapd. All memcg = which
>> > needs background recalim are linked to a list and memcg-kswap= d
>> > picks up a memcg from the list and run reclaim. This reclaime= s
>> > SWAP_CLUSTER_MAX of pages and putback the memcg to the lail o= f
>> > list. memcg-kswapd will visit memcgs in round-robin manner an= d
>> > reduce usages.
>> >
>>
>> I didn't look at code yet but as I just look over the descript= ion, I
>> have a concern.
>> We have discussed LRU separation between global and memcg.
>
> Please discuss global LRU in other thread. memcg-kswapd is not related=
> to global LRU _at all_.
>
> And this patch set is independent from the things we discussed at LSF.=
>
>
>> The clear goal is that how to keep _fairness_.
>>
>> For example,
>>
>> memcg-1 : # pages of LRU : 64
>> memcg-2 : # pages of LRU : 128
>> memcg-3 : # pages of LRU : 256
>>
>> If we have to reclaim 96 pages, memcg-1 would be lost half of page= s.
>> It's much greater than others so memcg 1's page LRU rotati= on cycle
>> would be very fast, then working set pages in memcg-1 don't ha= ve a
>> chance to promote.
>> Is it fair?
>>
>> I think we should consider memcg-LRU size as doing round-robin. >>
>
> This set doesn't implement a feature to handle your example case, = at all.

Sure. Sorry for the confusing.
I don't mean global LRU but it a fairness although this series is
based on per-memcg targeting.

>
> This patch set handles
>
> memcg-1: # pages of over watermark : 64
> memcg-2: # pages of over watermark : 128
> memcg-3: # pages of over watermark : 256
>
> And finally reclaim all pages over watermarks which user requested. > Considering fairness, what we consider is in what order we reclaim
> memory memcg-1, memcg-2, memcg-3 and how to avoid unnecessary cpu
> hogging at reclaiming memory all (64+128+256)
>
> This thread pool reclaim 32 pages per iteration with patch-1 and visit= all
> in round-robin.
> With patch-2, reclaim 32*weight pages per iteration on each memcg.
>

I should have seen the patch [2/3] before posting the comment.
Maybe you seem consider my concern.
Okay. I will look the idea.

For any ide= as on global kswapd and soft_limit reclaim based on round-robin ( discussed= in LSF), please move the discussion to :

[RFC = no patch yet] memcg: revisit soft_limit reclaim on contention:

<= h2 class=3D"title" style=3D"margin-top: 0px; margin-right: 0px; margin-bott= om: 20px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-b= ottom: 0px; padding-left: 0px; font-family: palatino, georgia, 'times n= ew roman', serif; font-size: large; color: rgb(0, 51, 102); "> http://perm= alink.gmane.org/gmane.linux.kernel.mm/60966"
I already started with the patch and hopefully to post so= me result soon.

--Ying
=A0
>
> Thanks,
> -Kame
>
>



--
Kind regards,
Minchan Kim

--0016e64aefda8c629d04a170a0dd-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org