[PATCH] no buddy bitmap patch : intro and includes [0/2]

[PATCH] no buddy bitmap patch : intro and includes [0/2]

[Hiroyuki KAMEZAWA]

Hi,

Followings are patches for removing bitmaps from buddy allocator, against 2.6.9-rc3.
I think this version is much clearer than ones I posted a month ago.

The problem I was worried about was how to deal with memmap's holes in a zone.
and I decided to use pfn_valid() simply. Now, there is no messy codes :)

pfn_valid() is used when macro "HOLES_IN_ZONE" is defined.
It is defined only in ia64 now.

Here is kernbench result on my tiger4 (Itanium2 1.3GHz x2, 8 Gbytes memory)

Average Optimal -j 8 Load Run (Perfroming 5 run of make -j 8 on linux-2.6.8 source tree):

                Elapsed Time  User Time  System Time Percent CPU  Context Switches   Sleeps
2.6.9-rc3         699.906      1322.01     39.336       194            64390         74416.8
no-bitmap         698.334      1321.79     38.58        194.2          64435.4       74622.2

If there is unclear point, please tell me.

Thanks.
Kame <kamezawa.hiroyu@jp.fujitsu.com>

=== patch for include files ===

This patch removes bitmap from zone->free_area[] in include/linux/mmzone.h,
and adds some comments on page->private field in include/linux/mm.h.

non-atomic ops for changing PG_private bit is added in include/page-flags.h.
zone->lock is always acquired when PG_private of "a free page" is changed.

Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>


---

 test-kernel-kamezawa/include/linux/mm.h         |    2 ++
 test-kernel-kamezawa/include/linux/mmzone.h     |    1 -
 test-kernel-kamezawa/include/linux/page-flags.h |    2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

diff -puN include/linux/mm.h~eliminate-bitmap-includes include/linux/mm.h
--- test-kernel/include/linux/mm.h~eliminate-bitmap-includes	2004-10-07 17:18:34.062982800 +0900
+++ test-kernel-kamezawa/include/linux/mm.h	2004-10-07 17:18:34.070981584 +0900
@@ -209,6 +209,8 @@ struct page {
 					 * usually used for buffer_heads
 					 * if PagePrivate set; used for
 					 * swp_entry_t if PageSwapCache
+					 * When page is free, this indicates
+					 * order in the buddy system.
 					 */
 	struct address_space *mapping;	/* If low bit clear, points to
 					 * inode address_space, or NULL.
diff -puN include/linux/mmzone.h~eliminate-bitmap-includes include/linux/mmzone.h
--- test-kernel/include/linux/mmzone.h~eliminate-bitmap-includes	2004-10-07 17:18:34.065982344 +0900
+++ test-kernel-kamezawa/include/linux/mmzone.h	2004-10-07 17:18:34.071981432 +0900
@@ -22,7 +22,6 @@

 struct free_area {
 	struct list_head	free_list;
-	unsigned long		*map;
 };

 struct pglist_data;
diff -puN include/linux/page-flags.h~eliminate-bitmap-includes include/linux/page-flags.h
--- test-kernel/include/linux/page-flags.h~eliminate-bitmap-includes	2004-10-07 17:18:34.067982040 +0900
+++ test-kernel-kamezawa/include/linux/page-flags.h	2004-10-07 17:18:34.071981432 +0900
@@ -238,6 +238,8 @@ extern unsigned long __read_page_state(u
 #define SetPagePrivate(page)	set_bit(PG_private, &(page)->flags)
 #define ClearPagePrivate(page)	clear_bit(PG_private, &(page)->flags)
 #define PagePrivate(page)	test_bit(PG_private, &(page)->flags)
+#define __SetPagePrivate(page)  __set_bit(PG_private, &(page)->flags)
+#define __ClearPagePrivate(page) __clear_bit(PG_private, &(page)->flags)

 #define PageWriteback(page)	test_bit(PG_writeback, &(page)->flags)
 #define SetPageWriteback(page)						\

_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Martin J. Bligh]

> Followings are patches for removing bitmaps from buddy allocator, against 2.6.9-rc3.
> I think this version is much clearer than ones I posted a month ago.
...
> If there is unclear point, please tell me.

What was the purpose behind this, again? Sorry, has been too long since
I last looked.

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [Lhms-devel] Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Dave McCracken]

--On Thursday, October 07, 2004 07:45:21 -0700 "Martin J. Bligh"
<mbligh@aracnet.com> wrote:

>> Followings are patches for removing bitmaps from buddy allocator,
>> against 2.6.9-rc3. I think this version is much clearer than ones I
>> posted a month ago.
> ...
>> If there is unclear point, please tell me.
> 
> What was the purpose behind this, again? Sorry, has been too long since
> I last looked.

The memory allocator bitmaps are the main remaining reason we need the
concept of linear memory.  If we can get rid of them, it's one step closer
to managing memory as a set of sections.

Dave McCracken

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Hiroyuki KAMEZAWA]

Martin J. Bligh wrote:
 >>>>What was the purpose behind this, again? Sorry, has been too long since
 >>>>I last looked.

>>On Thu, 2004-10-07 at 08:03, Tolentino, Matthew E wrote:
>>
>>For one, it avoids the otherwise requisite resizing of the bitmaps=20
>>during memory hotplug operations...
>>

 >> Dave McCracken wrote:
>> The memory allocator bitmaps are the main remaining reason we need the
>> concept of linear memory.  If we can get rid of them, it's one step closer
>> to managing memory as a set of sections.

 >>--Dave Hansen <haveblue@us.ibm.com> wrote (on Thursday, October 07, 2004 08:39:38 -0700)
>>It also simplifies the nonlinear implementation.  The whole reason we
>>had the lpfn (Linear) stuff was so that the bitmaps could represent a
>>sparse physical address space in a much more linear fashion.  With no
>>bitmaps, this isn't an issue, and gets rid of a lot of code, and a
>>*huge* source of bugs where lpfns and pfns are confused for each other. 
> 
> 
> Makese sense on both counts. Would be nice to add the justification to 
> the changelog ;-)
> 

It seems all I should answer is already answered.
Thank you all.

I'll add the purpose to the changelog.

Kame <kamezawa.hiroyu@jp.fujitsu.com>

> M.
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Martin J. Bligh]

>> Makese sense on both counts. Would be nice to add the justification to 
>> the changelog ;-)
> 
> Would you mind running these through your normal set of tests on the
> NUMAQ?  The last time I ran them, I didn't see a performance impact
> either way, and I'd be good to check again.

Makes no difference in performance that I can see.

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Martin J. Bligh]

--Dave Hansen <haveblue@us.ibm.com> wrote (on Thursday, October 07, 2004 09:10:19 -0700):

> On Thu, 2004-10-07 at 08:57, Martin J. Bligh wrote:
>> Makese sense on both counts. Would be nice to add the justification to 
>> the changelog ;-)
> 
> Would you mind running these through your normal set of tests on the
> NUMAQ?  The last time I ran them, I didn't see a performance impact
> either way, and I'd be good to check again.

Will do. What they're doing looks like it might be expensive. will check.

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Dave Hansen]

On Thu, 2004-10-07 at 08:57, Martin J. Bligh wrote:
> Makese sense on both counts. Would be nice to add the justification to 
> the changelog ;-)

Would you mind running these through your normal set of tests on the
NUMAQ?  The last time I ran them, I didn't see a performance impact
either way, and I'd be good to check again.

-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Martin J. Bligh]

--Dave Hansen <haveblue@us.ibm.com> wrote (on Thursday, October 07, 2004 08:39:38 -0700):

> On Thu, 2004-10-07 at 08:03, Tolentino, Matthew E wrote:
>> >> Followings are patches for removing bitmaps from buddy=20
>> > allocator, against 2.6.9-rc3.
>> >> I think this version is much clearer than ones I posted a month ago.
>> > ...
>> >> If there is unclear point, please tell me.
>> > 
>> > What was the purpose behind this, again? Sorry, has been too long since
>> > I last looked.
>> 
>> For one, it avoids the otherwise requisite resizing of the bitmaps=20
>> during memory hotplug operations...
> 
> It also simplifies the nonlinear implementation.  The whole reason we
> had the lpfn (Linear) stuff was so that the bitmaps could represent a
> sparse physical address space in a much more linear fashion.  With no
> bitmaps, this isn't an issue, and gets rid of a lot of code, and a
> *huge* source of bugs where lpfns and pfns are confused for each other. 

Makese sense on both counts. Would be nice to add the justification to 
the changelog ;-)

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: [PATCH] no buddy bitmap patch : intro and includes [0/2]

[Dave Hansen]

On Thu, 2004-10-07 at 08:03, Tolentino, Matthew E wrote:
> >> Followings are patches for removing bitmaps from buddy=20
> >allocator, against 2.6.9-rc3.
> >> I think this version is much clearer than ones I posted a month ago.
> >...
> >> If there is unclear point, please tell me.
> >
> >What was the purpose behind this, again? Sorry, has been too long since
> >I last looked.
> 
> For one, it avoids the otherwise requisite resizing of the bitmaps=20
> during memory hotplug operations...

It also simplifies the nonlinear implementation.  The whole reason we
had the lpfn (Linear) stuff was so that the bitmaps could represent a
sparse physical address space in a much more linear fashion.  With no
bitmaps, this isn't an issue, and gets rid of a lot of code, and a
*huge* source of bugs where lpfns and pfns are confused for each other. 
-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>