From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx113.postini.com [74.125.245.113]) by kanga.kvack.org (Postfix) with SMTP id 122556B0034 for ; Thu, 22 Aug 2013 05:08:36 -0400 (EDT) Received: by mail-wi0-f177.google.com with SMTP id hq12so365585wib.16 for ; Thu, 22 Aug 2013 02:08:34 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20130820140406.694b248b41611883878f8245@linux-foundation.org> References: <1376767883-4411-1-git-send-email-hannes@cmpxchg.org> <20130820140406.694b248b41611883878f8245@linux-foundation.org> Date: Thu, 22 Aug 2013 12:08:34 +0300 Message-ID: Subject: Re: [patch 9/9] mm: thrash detection-based file cache sizing v4 From: Metin Doslu Content-Type: multipart/alternative; boundary=f46d0444e855792f7104e485a273 Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Johannes Weiner , Andi Kleen , Andrea Arcangeli , Greg Thelen , Christoph Hellwig , Hugh Dickins , Jan Kara , KOSAKI Motohiro , Mel Gorman , Minchan Kim , Peter Zijlstra , Rik van Riel , Michel Lespinasse , Seth Jennings , Roman Gushchin , Ozgun Erdogan , Vlastimil Babka , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org --f46d0444e855792f7104e485a273 Content-Type: text/plain; charset=ISO-8859-1 Hey everbody, I run following tests, and it shows in what cases this patch is beneficial for us. Test Environment: * Ubuntu Server 12.04.2 LTS Linux 3.2.0-40-virtual #64-Ubuntu on EC2. * 15 GB memory (DMA32 4GB + Normal 11GB). Test Settings: We have two PostgreSQL tables with same size of 9.75GB (65% of total memory), where these tables contain clickstream events for March and April. We call these two tables "events_march" and "events_april" respectively. Problem (Before Patch is Applied): I pass over events_march data twice with an example query, such as "select count(*) from events_march". This activates all of events_march's pages. I then pass over events_april dozens of times with a similar query. No matter how many times I query events_april, I can't get completely get this table's pages into memory. This happens even when events_march isn't touched at all, events_april easily fits into memory, and events_april has been referenced dozens of times. After Patch is Applied: This time, after three passes over events_april, all the pages are cached in memory. (4th access is completely served from memory.) I also repeated this test with a bigger dataset of size 12GB (80% of total memory) for both events_march and events_april, and observed the same results. (after 3rd pass, all of pages in events_april are cached.) Thank you, Metin On Wed, Aug 21, 2013 at 12:04 AM, Andrew Morton wrote: > On Sat, 17 Aug 2013 15:31:14 -0400 Johannes Weiner > wrote: > > > This series solves the problem by maintaining a history of pages > > evicted from the inactive list, enabling the VM to tell streaming IO > > from thrashing and rebalance the page cache lists when appropriate. > > I can't say I'm loving the patchset. It adds significant bloat to the > inode (of all things!), seems to add some runtime overhead and > certainly adds boatloads of complexity. > > In return for which we get... well, I don't know what we get - no data > was included. It had better be good! > > To aid in this decision, please go through the patchset and calculate > and itemize the overhead: increased inode size, increased radix-tree > consumption, lengthier code paths, anything else I missed Others can > make their own judgements regarding complexity increase. > > Then please carefully describe the benefits, then see if you can > convince us that one is worth the other! > > --f46d0444e855792f7104e485a273 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hey everbody,

I run following tests, an= d it shows in what cases this patch is beneficial for us.

Test Environment:

* Ubuntu Server 12.= 04.2 LTS Linux 3.2.0-40-virtual #64-Ubuntu on EC2.
* 15 GB memory (DMA32 4GB + Normal 11GB).

Tes= t Settings:

We have two PostgreSQL tables with sam= e size of 9.75GB (65% of total memory), where these tables contain clickstr= eam events for March and April. We call these two tables "events_march= " and "events_april" respectively.

Problem (Before Patch is Applied):

=
I pass over events_march data twice with an example query, such as &qu= ot;select count(*) from events_march". This activates all of events_ma= rch's pages.

I then pass over events_april dozens of times with a si= milar query. No matter how many times I query events_april, I can't get= completely get this table's pages into memory. This happens even when = events_march isn't touched at all, events_april easily fits into memory= , and events_april has been referenced dozens of times.

After Patch is Applied:

This t= ime, after three passes over events_april, all the pages are cached in memo= ry. (4th access is completely served from memory.)

I also repeated this test with a bigger dataset of size 12GB (80% of t= otal memory) for both events_march and events_april, and observed the same = results. (after 3rd pass, all of pages in events_april are cached.)

Thank you,
Metin


On Wed, Aug 21, 2013 at= 12:04 AM, Andrew Morton <akpm@linux-foundation.org>= wrote:
On Sat, 17 Aug 2013 15:31:= 14 -0400 Johannes Weiner <hannes@c= mpxchg.org> wrote:

> This series solves the problem by maintaining a history of pages
> evicted from the inactive list, enabling the VM to tell streaming IO > from thrashing and rebalance the page cache lists when appropriate.
I can't say I'm loving the patchset. =A0It adds significant b= loat to the
inode (of all things!), seems to add some runtime overhead and
certainly adds boatloads of complexity.

In return for which we get... =A0well, I don't know what we get - no da= ta
was included. =A0It had better be good!

To aid in this decision, please go through the patchset and calculate
and itemize the overhead: increased inode size, increased radix-tree
consumption, lengthier code paths, anything else I missed =A0Others can
make their own judgements regarding complexity increase.

Then please carefully describe the benefits, then see if you can
convince us that one is worth the other!


--f46d0444e855792f7104e485a273-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org