From: Glauber Costa <glommer@parallels.com>
To: Glauber Costa <glommer@openvz.org>,
Dave Chinner <david@fromorbit.com>, Mel Gorman <mgorman@suse.de>
Cc: linux-mm@kvack.org, cgroups@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Greg Thelen <gthelen@google.com>,
kamezawa.hiroyu@jp.fujitsu.com, Michal Hocko <mhocko@suse.cz>,
Johannes Weiner <hannes@cmpxchg.org>,
linux-fsdevel@vger.kernel.org, hughd@google.com
Subject: Re: [PATCH v7 00/34] kmemcg shrinkers
Date: Tue, 21 May 2013 11:03:33 +0400 [thread overview]
Message-ID: <519B1C45.5090201@parallels.com> (raw)
In-Reply-To: <1368994047-5997-1-git-send-email-glommer@openvz.org>
[-- Attachment #1: Type: text/plain, Size: 1754 bytes --]
On 05/20/2013 12:06 AM, Glauber Costa wrote:
> Initial notes:
> ==============
>
> Please pay attention to new patches that are debuting in this series. Patch1
> changes our unused countries for int to long, since Dave noticed that it wasn't
> being enough in some cases. Aside from that, the major change is that we now
> compute and keep deferred work per-node (Patch13). The biggest effect of this,
> is that to avoid storing a new nodemask in the stack, I am passing only the
> node id down to the API. This means that the lru API *does not* take a nodemask
> any longer, which in turn, makes it simpler.
>
> I deeply considered this matter, and decided this would be the best way to go.
> It is not different from what I have already done for memcgs: Only a single one
> is passed down, and the complexity of scanning them is moved upwards to the
> caller, where all the scanning logic should belong anyway.
>
> If you want, you can also grab from branch "kmemcg-lru-shrinker" at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/glommer/memcg.git
>
> I hope the performance problems are all gone. My testing now shows a smoother
> and steady state for the objects during the lifetime of the workload, and
> postmark numbers are closer to base, although we do deviate a bit.
>
Mel, Dave, et. al.
I have applied some more fixes for things I have found here and there as
a result of a new round of testing. I won't post the result here until
Thursday or Friday, to avoid patchbombing you guys. In the meantime I
will be merging comments I receive from this version.
My git tree is up to date, so if you want to test it further, please
pick that up.
I am attaching the result of my postmark run. I think the results look
really good now.
[-- Attachment #2: postmark --]
[-- Type: text/plain, Size: 2973 bytes --]
postmark
bas memcg
base final
Ops/sec Transactions 13.00 ( 0.00%) 12.00 ( -7.69%)
Ops/sec FilesCreate 25.00 ( 0.00%) 26.00 ( 4.00%)
Ops/sec CreateTransact 6.00 ( 0.00%) 6.00 ( 0.00%)
Ops/sec FilesDeleted 3935.00 ( 0.00%) 3373.00 (-14.28%)
Ops/sec DeleteTransact 6.00 ( 0.00%) 6.00 ( 0.00%)
Ops/sec DataRead/MB 6.58 ( 0.00%) 6.63 ( 0.76%)
Ops/sec DataWrite/MB 48.63 ( 0.00%) 48.99 ( 0.74%)
bas memcg
base final
User 44.33 43.58
System 447.86 488.46
Elapsed 3000.82 2977.86
bas memcg
base final
Page Ins 10810400 10843604
Page Outs 151486632 150640884
Swap Ins 0 0
Swap Outs 0 0
Direct pages scanned 0 0
Kswapd pages scanned 23410722 23458232
Kswapd pages reclaimed 23399868 23447276
Direct pages reclaimed 0 0
Kswapd efficiency 99% 99%
Kswapd velocity 7801.442 7877.547
Direct efficiency 100% 100%
Direct velocity 0.000 0.000
Percentage direct scans 0% 0%
Page writes by reclaim 0 0
Page writes file 0 0
Page writes anon 0 0
Page reclaim immediate 0 0
Page rescued immediate 0 0
Slabs scanned 194304 234368
Direct inode steals 0 0
Kswapd inode steals 27412 2600
Kswapd skipped wait 0 0
THP fault alloc 6 6
THP collapse alloc 43 39
THP splits 1 1
THP fault fallback 0 0
THP collapse fail 0 0
Compaction stalls 0 0
Compaction success 0 0
Compaction failures 0 0
Page migrate success 0 0
Page migrate failure 0 0
Compaction pages isolated 0 0
Compaction migrate scanned 0 0
Compaction free scanned 0 0
Compaction cost 0 0
NUMA PTE updates 0 0
NUMA hint faults 0 0
NUMA hint local faults 0 0
NUMA pages migrated 0 0
AutoNUMA cost 0 0
next prev parent reply other threads:[~2013-05-21 7:02 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-19 20:06 Glauber Costa
2013-05-19 20:06 ` [PATCH v7 01/34] fs: bump inode and dentry counters to long Glauber Costa
2013-05-19 20:06 ` [PATCH v7 02/34] super: fix calculation of shrinkable objects for small numbers Glauber Costa
2013-05-19 20:06 ` [PATCH v7 03/34] dcache: convert dentry_stat.nr_unused to per-cpu counters Glauber Costa
2013-05-19 20:06 ` [PATCH v7 04/34] dentry: move to per-sb LRU locks Glauber Costa
2013-05-19 20:06 ` [PATCH v7 05/34] dcache: remove dentries from LRU before putting on dispose list Glauber Costa
2013-05-19 20:06 ` [PATCH v7 06/34] mm: new shrinker API Glauber Costa
2013-05-19 20:07 ` [PATCH v7 07/34] shrinker: convert superblock shrinkers to new API Glauber Costa
2013-05-20 16:39 ` Glauber Costa
2013-05-20 23:40 ` Dave Chinner
2013-05-19 20:07 ` [PATCH v7 08/34] list: add a new LRU list type Glauber Costa
2013-05-19 20:07 ` [PATCH v7 09/34] inode: convert inode lru list to generic lru list code Glauber Costa
2013-05-19 20:07 ` [PATCH v7 10/34] dcache: convert to use new lru list infrastructure Glauber Costa
2013-05-19 20:07 ` [PATCH v7 11/34] list_lru: per-node " Glauber Costa
2013-05-19 20:07 ` [PATCH v7 12/34] shrinker: add node awareness Glauber Costa
2013-05-19 20:07 ` [PATCH v7 13/34] vmscan: per-node deferred work Glauber Costa
2013-05-19 20:07 ` [PATCH v7 14/34] list_lru: per-node API Glauber Costa
2013-05-19 20:07 ` [PATCH v7 15/34] fs: convert inode and dentry shrinking to be node aware Glauber Costa
2013-05-19 20:07 ` [PATCH v7 16/34] xfs: convert buftarg LRU to generic code Glauber Costa
2013-05-19 20:07 ` [PATCH v7 17/34] xfs: convert dquot cache lru to list_lru Glauber Costa
2013-05-19 20:07 ` [PATCH v7 18/34] fs: convert fs shrinkers to new scan/count API Glauber Costa
2013-05-20 8:25 ` Steven Whitehouse
2013-05-20 13:46 ` Glauber Costa
2013-05-20 15:25 ` Glauber Costa
2013-05-20 23:38 ` Dave Chinner
2013-05-20 23:42 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 19/34] drivers: convert shrinkers to new count/scan API Glauber Costa
2013-06-03 20:03 ` Kent Overstreet
2013-06-04 9:06 ` Glauber Costa
2013-06-04 9:10 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 20/34] i915: bail out earlier when shrinker cannot acquire mutex Glauber Costa
2013-05-19 20:07 ` [PATCH v7 21/34] shrinker: convert remaining shrinkers to count/scan API Glauber Costa
2013-05-19 20:07 ` [PATCH v7 22/34] hugepage: convert huge zero page shrinker to new shrinker API Glauber Costa
2013-05-19 20:07 ` [PATCH v7 23/34] shrinker: Kill old ->shrink API Glauber Costa
2013-05-19 20:07 ` [PATCH v7 24/34] vmscan: also shrink slab in memcg pressure Glauber Costa
2013-05-19 20:07 ` [PATCH v7 25/34] memcg,list_lru: duplicate LRUs upon kmemcg creation Glauber Costa
2013-05-19 20:07 ` [PATCH v7 26/34] lru: add an element to a memcg list Glauber Costa
2013-05-19 20:07 ` [PATCH v7 27/34] list_lru: per-memcg walks Glauber Costa
2013-05-19 20:07 ` [PATCH v7 28/34] memcg: per-memcg kmem shrinking Glauber Costa
2013-05-19 20:07 ` [PATCH v7 29/34] memcg: scan cache objects hierarchically Glauber Costa
2013-05-19 20:07 ` [PATCH v7 30/34] vmscan: take at least one pass with shrinkers Glauber Costa
2013-05-19 20:07 ` [PATCH v7 31/34] super: targeted memcg reclaim Glauber Costa
2013-05-19 20:07 ` [PATCH v7 32/34] memcg: move initialization to memcg creation Glauber Costa
2013-05-19 20:07 ` [PATCH v7 33/34] vmpressure: in-kernel notifications Glauber Costa
2013-05-19 20:07 ` [PATCH v7 34/34] memcg: reap dead memcgs upon global memory pressure Glauber Costa
2013-05-21 7:03 ` Glauber Costa [this message]
2013-05-21 7:18 ` [PATCH v7 00/34] kmemcg shrinkers Dave Chinner
2013-05-21 7:27 ` Glauber Costa
2013-05-22 6:26 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=519B1C45.5090201@parallels.com \
--to=glommer@parallels.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=david@fromorbit.com \
--cc=glommer@openvz.org \
--cc=gthelen@google.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox