Re: [patch 9/9] mm: thrash detection-based file cache sizing v4

[Metin Doslu <metin@citusdata.com>]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

Hey everbody,

I run following tests, and it shows in what cases this patch is beneficial
for us.

Test Environment:

* Ubuntu Server 12.04.2 LTS Linux 3.2.0-40-virtual #64-Ubuntu on EC2.
* 15 GB memory (DMA32 4GB + Normal 11GB).

Test Settings:

We have two PostgreSQL tables with same size of 9.75GB (65% of total
memory), where these tables contain clickstream events for March and April.
We call these two tables "events_march" and "events_april" respectively.

Problem (Before Patch is Applied):

I pass over events_march data twice with an example query, such as "select
count(*) from events_march". This activates all of events_march's pages.

I then pass over events_april dozens of times with a similar query. No
matter how many times I query events_april, I can't get completely get this
table's pages into memory. This happens even when events_march isn't
touched at all, events_april easily fits into memory, and events_april has
been referenced dozens of times.

After Patch is Applied:

This time, after three passes over events_april, all the pages are cached
in memory. (4th access is completely served from memory.)

I also repeated this test with a bigger dataset of size 12GB (80% of total
memory) for both events_march and events_april, and observed the same
results. (after 3rd pass, all of pages in events_april are cached.)

Thank you,
Metin


On Wed, Aug 21, 2013 at 12:04 AM, Andrew Morton
<akpm@linux-foundation.org>wrote:

> On Sat, 17 Aug 2013 15:31:14 -0400 Johannes Weiner <hannes@cmpxchg.org>
> wrote:
>
> > This series solves the problem by maintaining a history of pages
> > evicted from the inactive list, enabling the VM to tell streaming IO
> > from thrashing and rebalance the page cache lists when appropriate.
>
> I can't say I'm loving the patchset.  It adds significant bloat to the
> inode (of all things!), seems to add some runtime overhead and
> certainly adds boatloads of complexity.
>
> In return for which we get...  well, I don't know what we get - no data
> was included.  It had better be good!
>
> To aid in this decision, please go through the patchset and calculate
> and itemize the overhead: increased inode size, increased radix-tree
> consumption, lengthier code paths, anything else I missed  Others can
> make their own judgements regarding complexity increase.
>
> Then please carefully describe the benefits, then see if you can
> convince us that one is worth the other!
>
>

[-- Attachment #2: Type: text/html, Size: 3278 bytes --]