* Memory management in 2.6
@ 2004-01-20 4:51 Nick Piggin
2004-01-20 4:58 ` Andrew Morton
2004-01-20 7:00 ` Nigel Cunningham
0 siblings, 2 replies; 8+ messages in thread
From: Nick Piggin @ 2004-01-20 4:51 UTC (permalink / raw)
To: linux-mm; +Cc: Andrew Morton
[-- Attachment #1: Type: text/plain, Size: 538 bytes --]
Hi,
In the interest of helping improve 2.6 VM performance when
under heavy swapping load, I'm putting together a few regression
tests.
If anyone has any suggestions of workloads I could use, I will
try to include them, or code them up if you want a simple concept
tested. Also, any suggestions of what information I should gather?
loads should be runnable on about 64MB, preferably give decently
repeatable results in under an hour.
I'll be happy to test patches. Here is one (results are a bit
wild because it was only 1 run).
Nick
[-- Attachment #2: vm-akpm-balance-pgdat.patch --]
[-- Type: text/plain, Size: 2507 bytes --]
linux-2.6-npiggin/mm/vmscan.c | 30 +++++++++++++++++++++++-------
1 files changed, 23 insertions(+), 7 deletions(-)
diff -puN mm/vmscan.c~vm-akpm-balance-pgdat mm/vmscan.c
--- linux-2.6/mm/vmscan.c~vm-akpm-balance-pgdat 2004-01-17 20:35:39.000000000 +1100
+++ linux-2.6-npiggin/mm/vmscan.c 2004-01-17 20:35:42.000000000 +1100
@@ -941,11 +941,12 @@ static int balance_pgdat(pg_data_t *pgda
int nr_mapped = 0;
int max_scan;
int to_reclaim;
+ int reclaimed;
if (zone->all_unreclaimable && priority != DEF_PRIORITY)
continue;
- if (nr_pages && to_free > 0) { /* Software suspend */
+ if (nr_pages && nr_pages > 0) { /* Software suspend */
to_reclaim = min(to_free, SWAP_CLUSTER_MAX*8);
} else { /* Zone balancing */
to_reclaim = zone->pages_high-zone->free_pages;
@@ -953,28 +954,43 @@ static int balance_pgdat(pg_data_t *pgda
continue;
}
zone->temp_priority = priority;
- all_zones_ok = 0;
max_scan = zone->nr_inactive >> priority;
if (max_scan < to_reclaim * 2)
max_scan = to_reclaim * 2;
if (max_scan < SWAP_CLUSTER_MAX)
max_scan = SWAP_CLUSTER_MAX;
- to_free -= shrink_zone(zone, max_scan, GFP_KERNEL,
+ reclaimed = shrink_zone(zone, max_scan, GFP_KERNEL,
to_reclaim, &nr_mapped, ps, priority);
if (i < ZONE_HIGHMEM) {
reclaim_state->reclaimed_slab = 0;
shrink_slab(max_scan + nr_mapped, GFP_KERNEL);
- to_free -= reclaim_state->reclaimed_slab;
+ reclaimed += reclaim_state->reclaimed_slab;
}
+ to_free -= reclaimed;
if (zone->all_unreclaimable)
continue;
if (zone->pages_scanned > zone->present_pages * 2)
zone->all_unreclaimable = 1;
+ /*
+ * If this scan failed to reclaim `to_reclaim' or more
+ * pages, we're getting into trouble. Need to scan
+ * some more, and throttle kswapd. Note that this zone
+ * may now have sufficient free pages due to freeing
+ * activity by some other process. That's OK - we'll
+ * pick that info up on the next pass through the loop.
+ */
+ if (reclaimed < to_reclaim)
+ all_zones_ok = 0;
}
- if (all_zones_ok)
- break;
if (to_free > 0)
- blk_congestion_wait(WRITE, HZ/10);
+ continue; /* swsusp: need to do more work */
+ if (all_zones_ok)
+ break; /* kswapd: all done */
+ /*
+ * OK, kswapd is getting into trouble. Take a nap, then take
+ * another pass across the zones.
+ */
+ blk_congestion_wait(WRITE, HZ/10);
}
for (i = 0; i < pgdat->nr_zones; i++) {
_
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 4:51 Memory management in 2.6 Nick Piggin
@ 2004-01-20 4:58 ` Andrew Morton
2004-01-20 5:05 ` Nick Piggin
2004-01-20 7:00 ` Nigel Cunningham
1 sibling, 1 reply; 8+ messages in thread
From: Andrew Morton @ 2004-01-20 4:58 UTC (permalink / raw)
To: Nick Piggin; +Cc: linux-mm
Nick Piggin <piggin@cyberone.com.au> wrote:
>
> loads should be runnable on about 64MB, preferably give decently
> repeatable results in under an hour.
In under three minutes, IMO.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 4:58 ` Andrew Morton
@ 2004-01-20 5:05 ` Nick Piggin
2004-01-20 13:11 ` Roger Luethi
0 siblings, 1 reply; 8+ messages in thread
From: Nick Piggin @ 2004-01-20 5:05 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm
Andrew Morton wrote:
>Nick Piggin <piggin@cyberone.com.au> wrote:
>
>>loads should be runnable on about 64MB, preferably give decently
>> repeatable results in under an hour.
>>
>
>In under three minutes, IMO.
>
That would be nice, but sometimes hard, with multiple processes
and fairly heavy swapping load.
As you can see, kbuild time for example increases very quickly. It
would be preferable to "do something for 2 minutes and measure how
far we got", but kbuild doesn't lend itself particularly well to
that.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 4:51 Memory management in 2.6 Nick Piggin
2004-01-20 4:58 ` Andrew Morton
@ 2004-01-20 7:00 ` Nigel Cunningham
2004-01-20 7:19 ` Michael Frank
1 sibling, 1 reply; 8+ messages in thread
From: Nigel Cunningham @ 2004-01-20 7:00 UTC (permalink / raw)
To: Nick Piggin; +Cc: Linux Memory Management, Andrew Morton, Michael Frank
[-- Attachment #1: Type: text/plain, Size: 3696 bytes --]
Hi.
Michael Frank has written some scripts we've used to stress test
software suspend. Perhaps you might be able to make some mileage from
them?
Regards,
Nigel
On Tue, 2004-01-20 at 17:51, Nick Piggin wrote:
> Hi,
> In the interest of helping improve 2.6 VM performance when
> under heavy swapping load, I'm putting together a few regression
> tests.
>
> If anyone has any suggestions of workloads I could use, I will
> try to include them, or code them up if you want a simple concept
> tested. Also, any suggestions of what information I should gather?
>
> loads should be runnable on about 64MB, preferably give decently
> repeatable results in under an hour.
>
> I'll be happy to test patches. Here is one (results are a bit
> wild because it was only 1 run).
>
> Nick
>
>
> ______________________________________________________________________
> linux-2.6-npiggin/mm/vmscan.c | 30 +++++++++++++++++++++++-------
> 1 files changed, 23 insertions(+), 7 deletions(-)
>
> diff -puN mm/vmscan.c~vm-akpm-balance-pgdat mm/vmscan.c
> --- linux-2.6/mm/vmscan.c~vm-akpm-balance-pgdat 2004-01-17 20:35:39.000000000 +1100
> +++ linux-2.6-npiggin/mm/vmscan.c 2004-01-17 20:35:42.000000000 +1100
> @@ -941,11 +941,12 @@ static int balance_pgdat(pg_data_t *pgda
> int nr_mapped = 0;
> int max_scan;
> int to_reclaim;
> + int reclaimed;
>
> if (zone->all_unreclaimable && priority != DEF_PRIORITY)
> continue;
>
> - if (nr_pages && to_free > 0) { /* Software suspend */
> + if (nr_pages && nr_pages > 0) { /* Software suspend */
> to_reclaim = min(to_free, SWAP_CLUSTER_MAX*8);
> } else { /* Zone balancing */
> to_reclaim = zone->pages_high-zone->free_pages;
> @@ -953,28 +954,43 @@ static int balance_pgdat(pg_data_t *pgda
> continue;
> }
> zone->temp_priority = priority;
> - all_zones_ok = 0;
> max_scan = zone->nr_inactive >> priority;
> if (max_scan < to_reclaim * 2)
> max_scan = to_reclaim * 2;
> if (max_scan < SWAP_CLUSTER_MAX)
> max_scan = SWAP_CLUSTER_MAX;
> - to_free -= shrink_zone(zone, max_scan, GFP_KERNEL,
> + reclaimed = shrink_zone(zone, max_scan, GFP_KERNEL,
> to_reclaim, &nr_mapped, ps, priority);
> if (i < ZONE_HIGHMEM) {
> reclaim_state->reclaimed_slab = 0;
> shrink_slab(max_scan + nr_mapped, GFP_KERNEL);
> - to_free -= reclaim_state->reclaimed_slab;
> + reclaimed += reclaim_state->reclaimed_slab;
> }
> + to_free -= reclaimed;
> if (zone->all_unreclaimable)
> continue;
> if (zone->pages_scanned > zone->present_pages * 2)
> zone->all_unreclaimable = 1;
> + /*
> + * If this scan failed to reclaim `to_reclaim' or more
> + * pages, we're getting into trouble. Need to scan
> + * some more, and throttle kswapd. Note that this zone
> + * may now have sufficient free pages due to freeing
> + * activity by some other process. That's OK - we'll
> + * pick that info up on the next pass through the loop.
> + */
> + if (reclaimed < to_reclaim)
> + all_zones_ok = 0;
> }
> - if (all_zones_ok)
> - break;
> if (to_free > 0)
> - blk_congestion_wait(WRITE, HZ/10);
> + continue; /* swsusp: need to do more work */
> + if (all_zones_ok)
> + break; /* kswapd: all done */
> + /*
> + * OK, kswapd is getting into trouble. Take a nap, then take
> + * another pass across the zones.
> + */
> + blk_congestion_wait(WRITE, HZ/10);
> }
>
> for (i = 0; i < pgdat->nr_zones; i++) {
>
> _
--
My work on Software Suspend is graciously brought to you by
LinuxFund.org.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 7:00 ` Nigel Cunningham
@ 2004-01-20 7:19 ` Michael Frank
2004-01-20 7:37 ` Nick Piggin
0 siblings, 1 reply; 8+ messages in thread
From: Michael Frank @ 2004-01-20 7:19 UTC (permalink / raw)
To: ncunningham, Nick Piggin; +Cc: Linux Memory Management, Andrew Morton
[-- Attachment #1: Type: text/plain, Size: 4723 bytes --]
I had already sent a version (ti-tests) dedicated to stress testing to LKML in October.
Just in case I enclose those again. README inside.
I ran 2.6 again for the first time since -test9 yesterday to test
swsusp and immeadiatly went to elevator=deadline as aio is unusable
at high io loads which I need to break swsusp in.
If you do on a 2GHz+ machine:
$ti stat ub17 ddw 4 5000
This gives a load avg of ~40 on a 2.4G P4 with 533MHz FSB
2.4.23 behaves "proportionate" to load - at these loads mouse is jerky but has best throughput.
2.6 with deadline is similar but a bit slower and the mouse is very smooth.
2.6 with aio the mouse is smooth but no io throughput and io is highly intermittent. AFAICS similar to -test9.
It must be recognized that these tests act in a non-anticipatory manner - this is what they are designed for ;)
Regards
Michael
On Tuesday 20 January 2004 15:00, Nigel Cunningham wrote:
> Hi.
>
> Michael Frank has written some scripts we've used to stress test
> software suspend. Perhaps you might be able to make some mileage from
> them?
>
> Regards,
>
> Nigel
>
> On Tue, 2004-01-20 at 17:51, Nick Piggin wrote:
> > Hi,
> > In the interest of helping improve 2.6 VM performance when
> > under heavy swapping load, I'm putting together a few regression
> > tests.
> >
> > If anyone has any suggestions of workloads I could use, I will
> > try to include them, or code them up if you want a simple concept
> > tested. Also, any suggestions of what information I should gather?
> >
> > loads should be runnable on about 64MB, preferably give decently
> > repeatable results in under an hour.
> >
> > I'll be happy to test patches. Here is one (results are a bit
> > wild because it was only 1 run).
> >
> > Nick
> >
> >
> > ______________________________________________________________________
> > linux-2.6-npiggin/mm/vmscan.c | 30 +++++++++++++++++++++++-------
> > 1 files changed, 23 insertions(+), 7 deletions(-)
> >
> > diff -puN mm/vmscan.c~vm-akpm-balance-pgdat mm/vmscan.c
> > --- linux-2.6/mm/vmscan.c~vm-akpm-balance-pgdat 2004-01-17 20:35:39.000000000 +1100
> > +++ linux-2.6-npiggin/mm/vmscan.c 2004-01-17 20:35:42.000000000 +1100
> > @@ -941,11 +941,12 @@ static int balance_pgdat(pg_data_t *pgda
> > int nr_mapped = 0;
> > int max_scan;
> > int to_reclaim;
> > + int reclaimed;
> >
> > if (zone->all_unreclaimable && priority != DEF_PRIORITY)
> > continue;
> >
> > - if (nr_pages && to_free > 0) { /* Software suspend */
> > + if (nr_pages && nr_pages > 0) { /* Software suspend */
> > to_reclaim = min(to_free, SWAP_CLUSTER_MAX*8);
> > } else { /* Zone balancing */
> > to_reclaim = zone->pages_high-zone->free_pages;
> > @@ -953,28 +954,43 @@ static int balance_pgdat(pg_data_t *pgda
> > continue;
> > }
> > zone->temp_priority = priority;
> > - all_zones_ok = 0;
> > max_scan = zone->nr_inactive >> priority;
> > if (max_scan < to_reclaim * 2)
> > max_scan = to_reclaim * 2;
> > if (max_scan < SWAP_CLUSTER_MAX)
> > max_scan = SWAP_CLUSTER_MAX;
> > - to_free -= shrink_zone(zone, max_scan, GFP_KERNEL,
> > + reclaimed = shrink_zone(zone, max_scan, GFP_KERNEL,
> > to_reclaim, &nr_mapped, ps, priority);
> > if (i < ZONE_HIGHMEM) {
> > reclaim_state->reclaimed_slab = 0;
> > shrink_slab(max_scan + nr_mapped, GFP_KERNEL);
> > - to_free -= reclaim_state->reclaimed_slab;
> > + reclaimed += reclaim_state->reclaimed_slab;
> > }
> > + to_free -= reclaimed;
> > if (zone->all_unreclaimable)
> > continue;
> > if (zone->pages_scanned > zone->present_pages * 2)
> > zone->all_unreclaimable = 1;
> > + /*
> > + * If this scan failed to reclaim `to_reclaim' or more
> > + * pages, we're getting into trouble. Need to scan
> > + * some more, and throttle kswapd. Note that this zone
> > + * may now have sufficient free pages due to freeing
> > + * activity by some other process. That's OK - we'll
> > + * pick that info up on the next pass through the loop.
> > + */
> > + if (reclaimed < to_reclaim)
> > + all_zones_ok = 0;
> > }
> > - if (all_zones_ok)
> > - break;
> > if (to_free > 0)
> > - blk_congestion_wait(WRITE, HZ/10);
> > + continue; /* swsusp: need to do more work */
> > + if (all_zones_ok)
> > + break; /* kswapd: all done */
> > + /*
> > + * OK, kswapd is getting into trouble. Take a nap, then take
> > + * another pass across the zones.
> > + */
> > + blk_congestion_wait(WRITE, HZ/10);
> > }
> >
> > for (i = 0; i < pgdat->nr_zones; i++) {
> >
> > _
> --
> My work on Software Suspend is graciously brought to you by
> LinuxFund.org.
>
[-- Attachment #2: ti-tests-1.0-beta-2.tar.bz2 --]
[-- Type: application/x-tbz, Size: 39712 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 7:19 ` Michael Frank
@ 2004-01-20 7:37 ` Nick Piggin
0 siblings, 0 replies; 8+ messages in thread
From: Nick Piggin @ 2004-01-20 7:37 UTC (permalink / raw)
To: Michael Frank; +Cc: ncunningham, Linux Memory Management, Andrew Morton
Michael Frank wrote:
>I had already sent a version (ti-tests) dedicated to stress testing to LKML in October.
>
>Just in case I enclose those again. README inside.
>
>I ran 2.6 again for the first time since -test9 yesterday to test
>swsusp and immeadiatly went to elevator=deadline as aio is unusable
>at high io loads which I need to break swsusp in.
>
>If you do on a 2GHz+ machine:
>
>$ti stat ub17 ddw 4 5000
>
>This gives a load avg of ~40 on a 2.4G P4 with 533MHz FSB
>
>2.4.23 behaves "proportionate" to load - at these loads mouse is jerky but has best throughput.
>
>2.6 with deadline is similar but a bit slower and the mouse is very smooth.
>
>2.6 with aio the mouse is smooth but no io throughput and io is highly intermittent. AFAICS similar to -test9.
>
>It must be recognized that these tests act in a non-anticipatory manner - this is what they are designed for ;)
>
Hmm... thats a bit alarming. I'll have to take a look at why that is so,
thanks.
(2.6 -bk and -mm kernels have some as-iosched changes by the way)
I'll also see if they might be useful as an mm regression test.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory management in 2.6
2004-01-20 5:05 ` Nick Piggin
@ 2004-01-20 13:11 ` Roger Luethi
0 siblings, 0 replies; 8+ messages in thread
From: Roger Luethi @ 2004-01-20 13:11 UTC (permalink / raw)
To: Nick Piggin; +Cc: Andrew Morton, linux-mm
On Tue, 20 Jan 2004 16:05:52 +1100, Nick Piggin wrote:
>
> Andrew Morton wrote:
>
> >Nick Piggin <piggin@cyberone.com.au> wrote:
> >
> >>loads should be runnable on about 64MB, preferably give decently
> >>repeatable results in under an hour.
> >
> >In under three minutes, IMO.
>
> That would be nice, but sometimes hard, with multiple processes
> and fairly heavy swapping load.
efax exhibits a much higher run time variance in 2.6 than in 2.4, and
that's only one process. The reason we can't say anything conclusive
after three minutes is not a lack of short benchmarks, but the fact
that most benchmarks need to be repeated a dozen times to get reliable
numbers.
> would be preferable to "do something for 2 minutes and measure how
> far we got", but kbuild doesn't lend itself particularly well to
> that.
What you can do for kbuild is to build only part of it. I used something
like:
rm arch/*/*/*.o arch/i386/boot/bzImage
time make -j24
Roger
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Memory Management in 2.6
[not found] ` <40185564.8020709@cyberone.com.au>
@ 2004-01-29 0:54 ` Nick Piggin
0 siblings, 0 replies; 8+ messages in thread
From: Nick Piggin @ 2004-01-29 0:54 UTC (permalink / raw)
To: linux-mm, Andrew Morton; +Cc: Nikita Danilov, Roger Luethi
Hi,
I have done a bit more benchmarking with Nikita's patch
dont-rotate-active-list (I call it -lru, sorry), and my
mapped pages fairness patch.
Together they're nearly twice as fast as the standard VM
under heavier make loads, which is pleasing.
http://www.kerneltrap.org/~npiggin/vm/2/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2004-01-29 0:54 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-01-20 4:51 Memory management in 2.6 Nick Piggin
2004-01-20 4:58 ` Andrew Morton
2004-01-20 5:05 ` Nick Piggin
2004-01-20 13:11 ` Roger Luethi
2004-01-20 7:00 ` Nigel Cunningham
2004-01-20 7:19 ` Michael Frank
2004-01-20 7:37 ` Nick Piggin
[not found] <20040127162346.37b75f6c.cliffw@osdl.org>
[not found] ` <40185564.8020709@cyberone.com.au>
2004-01-29 0:54 ` Memory Management " Nick Piggin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox