[PATCH] 0/2 Buddy allocator with placement policy + prezeroing

[PATCH] 0/2 Buddy allocator with placement policy + prezeroing

[Mel Gorman]

Hi,

In the two following emails are the latest version of the placement policy
for the binary buddy allocator to reduce fragmentation and the prezeroing
patch. The changelogs are with the patches although the most significant change
to the placement policy is a fix for a bug in the usemap size calculation
(pointed out by Mike Kravetz). 

The placement policy is Even Better than previous versions and can allocate
over 100 2**10 blocks of pages under loads in excess of 30 so I still
consider it ready for inclusion to the mainline. The prezeroing patches
main contribution is a handy accounting scheme for the scrubbing daemon. The
patch records how many times blocks were zeroed and what size they were. I
found that order-0 is the most common size to zero because of the per-cpu
cache. For example, after the usual stress test completed, /proc/buddyinfo
reported the following;

Zeroblock count 1775307   7696   2048   1046   2577    871    164     17     18
8     39

That means that the majority of zeroing calls was for order-0 pages. What is
of greater concern is that the prezeroing patch seriously regresses how well
fragmentation is handled making it perform almost as badly as the standard
allocator. 

The patches were developed and tested heavily on 2.6.11-rc4 but are known
to patch cleanly and pass a stress test on 2.6.11-rc5.

-- 
Mel Gorman
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] 0/2 Buddy allocator with placement policy + prezeroing

[Dave Hansen]

On Sun, 2005-02-27 at 13:42 +0000, Mel Gorman wrote:
> In the two following emails are the latest version of the placement policy
> for the binary buddy allocator to reduce fragmentation and the prezeroing
> patch. The changelogs are with the patches although the most significant change
> to the placement policy is a fix for a bug in the usemap size calculation
> (pointed out by Mike Kravetz). 
> 
> The placement policy is Even Better than previous versions and can allocate
> over 100 2**10 blocks of pages under loads in excess of 30 so I still
> consider it ready for inclusion to the mainline.
...

This patch does some important things for memory hotplug: it explicitly
marks the different types of kernel allocations, and it separates those
different types in the allocator.  When it comes to memory hot-remove
this is certainly something we were going to have to do anyway.  Plus, I
believe there are already at least two prototype patches that do this.  

Anything that makes future memory hotplug work easier is good in my
book. :)

-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] 0/2 Buddy allocator with placement policy + prezeroing

[Mel Gorman]

On Mon, 28 Feb 2005, Dave Hansen wrote:

> On Sun, 2005-02-27 at 13:42 +0000, Mel Gorman wrote:
> > In the two following emails are the latest version of the placement policy
> > for the binary buddy allocator to reduce fragmentation and the prezeroing
> > patch. The changelogs are with the patches although the most significant change
> > to the placement policy is a fix for a bug in the usemap size calculation
> > (pointed out by Mike Kravetz).
> >
> > The placement policy is Even Better than previous versions and can allocate
> > over 100 2**10 blocks of pages under loads in excess of 30 so I still
> > consider it ready for inclusion to the mainline.
> ...
>
> This patch does some important things for memory hotplug: it explicitly
> marks the different types of kernel allocations, and it separates those
> different types in the allocator.  When it comes to memory hot-remove
> this is certainly something we were going to have to do anyway.  Plus, I
> believe there are already at least two prototype patches that do this.
>

I have read through Matt Tolentino's version at least and this version of
the placement policy should be an easier merge for hotplug. Particularly,
stuff from his patch like the removal of magic numbers (which I should
have done anyway) are present in this version of the placement policy
patch. I will also move how a MAX_ORDER-1 block of pages is removed from
the global list and put it in it's own inline function.

> Anything that makes future memory hotplug work easier is good in my
> book. :)
>

If there are any other changes that might make hotplug's life easier, I'm
sure someone will shout :)

-- 
Mel Gorman
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>