From: riel@redhat.com
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, peterz@infradead.org, mgorman@suse.de,
mingo@redhat.com, chegu_vinod@hp.com
Subject: [PATCH v3 0/6] pseudo-interleaving for automatic NUMA balancing
Date: Mon, 20 Jan 2014 14:21:01 -0500 [thread overview]
Message-ID: <1390245667-24193-1-git-send-email-riel@redhat.com> (raw)
The current automatic NUMA balancing code base has issues with
workloads that do not fit on one NUMA load. Page migration is
slowed down, but memory distribution between the nodes where
the workload runs is essentially random, often resulting in a
suboptimal amount of memory bandwidth being available to the
workload.
In order to maximize performance of workloads that do not fit in one NUMA
node, we want to satisfy the following criteria:
1) keep private memory local to each thread
2) avoid excessive NUMA migration of pages
3) distribute shared memory across the active nodes, to
maximize memory bandwidth available to the workload
This patch series identifies the NUMA nodes on which the workload
is actively running, and balances (somewhat lazily) the memory
between those nodes, satisfying the criteria above.
As usual, the series has had some performance testing, but it
could always benefit from more testing, on other systems.
Changes since v2:
- dropped tracepoint (for now?)
- implement obvious improvements suggested by Peter
- use the scheduler maintained CPU use statistics, drop
the NUMA specific ones for now. We can add those later
if they turn out to be beneficial
Changes since v1:
- fix divide by zero found by Chegu Vinod
- improve comment, as suggested by Peter Zijlstra
- do stats calculations in task_numa_placement in local variables
Some performance numbers, with two 40-warehouse specjbb instances
on an 8 node system with 10 CPU cores per node, using a pre-cleanup
version of these patches, courtesy of Chegu Vinod:
numactl manual pinning
spec1.txt: throughput = 755900.20 SPECjbb2005 bops
spec2.txt: throughput = 754914.40 SPECjbb2005 bops
NO-pinning results (Automatic NUMA balancing, with patches)
spec1.txt: throughput = 706439.84 SPECjbb2005 bops
spec2.txt: throughput = 729347.75 SPECjbb2005 bops
NO-pinning results (Automatic NUMA balancing, without patches)
spec1.txt: throughput = 667988.47 SPECjbb2005 bops
spec2.txt: throughput = 638220.45 SPECjbb2005 bops
No Automatic NUMA and NO-pinning results
spec1.txt: throughput = 544120.97 SPECjbb2005 bops
spec2.txt: throughput = 453553.41 SPECjbb2005 bops
My own performance numbers are not as relevant, since I have been
running with a more hostile workload on purpose, and I have run
into a scheduler issue that caused the workload to run on only
two of the four NUMA nodes on my test system...
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2014-01-20 19:21 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-20 19:21 riel [this message]
2014-01-20 19:21 ` [PATCH 1/6] numa,sched,mm: remove p->numa_migrate_deferred riel
2014-01-21 11:52 ` Mel Gorman
2014-01-20 19:21 ` [PATCH 2/6] numa,sched: track from which nodes NUMA faults are triggered riel
2014-01-21 12:21 ` Mel Gorman
2014-01-21 22:26 ` Rik van Riel
2014-01-24 14:14 ` Mel Gorman
2014-01-20 19:21 ` [PATCH 3/6] numa,sched: build per numa_group active node mask from faults_from statistics riel
2014-01-21 14:19 ` Mel Gorman
2014-01-21 15:09 ` Rik van Riel
2014-01-21 15:41 ` Mel Gorman
2014-01-20 19:21 ` [PATCH 4/6] numa,sched,mm: use active_nodes nodemask to limit numa migrations riel
2014-01-21 15:08 ` Mel Gorman
2014-01-20 19:21 ` [PATCH 5/6] numa,sched: normalize faults_from stats and weigh by CPU use riel
2014-01-21 15:56 ` Mel Gorman
2014-01-21 21:05 ` Rik van Riel
2014-01-20 19:21 ` [PATCH 6/6] numa,sched: do statistics calculation using local variables only riel
2014-01-21 16:15 ` Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1390245667-24193-1-git-send-email-riel@redhat.com \
--to=riel@redhat.com \
--cc=chegu_vinod@hp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox