[PATCH 1/1] mm: Export split_page().

[PATCH 1/1] mm: Export split_page().

[K. Y. Srinivasan]

The split_page() function will be very useful for balloon drivers. On Hyper-V,
it will be very efficient to use 2M allocations in the guest as this (a) makes
the ballooning protocol with the host that much more efficient and (b) moving
memory in 2M chunks minimizes fragmentation in the host. Export the split_page()
function to let the guest allocations be in 2M chunks while the host is free to
return this memory at arbitrary granularity.


Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
---
 mm/page_alloc.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6cacfee..7e0ead6 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 }
+EXPORT_SYMBOL_GPL(split_page);
 
 static int __isolate_free_page(struct page *page, unsigned int order)
 {
-- 
1.7.4.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] mm: Export split_page().

[Greg KH]

On Sun, Mar 03, 2013 at 06:27:55PM -0800, K. Y. Srinivasan wrote:
> The split_page() function will be very useful for balloon drivers. On Hyper-V,
> it will be very efficient to use 2M allocations in the guest as this (a) makes
> the ballooning protocol with the host that much more efficient and (b) moving
> memory in 2M chunks minimizes fragmentation in the host. Export the split_page()
> function to let the guest allocations be in 2M chunks while the host is free to
> return this memory at arbitrary granularity.
> 
> 
> Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> ---
>  mm/page_alloc.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 6cacfee..7e0ead6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
>  	for (i = 1; i < (1 << order); i++)
>  		set_page_refcounted(page + i);
>  }
> +EXPORT_SYMBOL_GPL(split_page);

When you export a symbol, you also need to post the code that is going
to use that symbol, otherwise people don't really know how to judge this
request.

Can you just make this a part of your balloon driver update patch series
instead?

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH 1/1] mm: Export split_page().

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Sunday, March 03, 2013 9:08 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org; olaf@aepfle.de;
> apw@canonical.com; andi@firstfloor.org; akpm@linux-foundation.org; linux-
> mm@kvack.org
> Subject: Re: [PATCH 1/1] mm: Export split_page().
> 
> On Sun, Mar 03, 2013 at 06:27:55PM -0800, K. Y. Srinivasan wrote:
> > The split_page() function will be very useful for balloon drivers. On Hyper-V,
> > it will be very efficient to use 2M allocations in the guest as this (a) makes
> > the ballooning protocol with the host that much more efficient and (b) moving
> > memory in 2M chunks minimizes fragmentation in the host. Export the
> split_page()
> > function to let the guest allocations be in 2M chunks while the host is free to
> > return this memory at arbitrary granularity.
> >
> >
> > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > ---
> >  mm/page_alloc.c |    1 +
> >  1 files changed, 1 insertions(+), 0 deletions(-)
> >
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index 6cacfee..7e0ead6 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
> >  	for (i = 1; i < (1 << order); i++)
> >  		set_page_refcounted(page + i);
> >  }
> > +EXPORT_SYMBOL_GPL(split_page);
> 
> When you export a symbol, you also need to post the code that is going
> to use that symbol, otherwise people don't really know how to judge this
> request.
> 
> Can you just make this a part of your balloon driver update patch series
> instead?

Fair enough; I was hoping to see how inclined the mm folks were with regards to
exporting this symbol before I went ahead and modified the balloon driver code to
leverage this. Looking at the Windows guests on Hyper-V, I am convinced 2M balloon
allocations in the Linux (Hyper-V) balloon driver will make significant difference. As you
suggest, I will post this patch as part of the balloon driver changes that use this exported
symbol. I am still hoping to get some feedback from the mm guys on this.

Regards,

K. Y


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] mm: Export split_page().

[Greg KH]

On Mon, Mar 04, 2013 at 02:14:02AM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > Sent: Sunday, March 03, 2013 9:08 PM
> > To: KY Srinivasan
> > Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org; olaf@aepfle.de;
> > apw@canonical.com; andi@firstfloor.org; akpm@linux-foundation.org; linux-
> > mm@kvack.org
> > Subject: Re: [PATCH 1/1] mm: Export split_page().
> > 
> > On Sun, Mar 03, 2013 at 06:27:55PM -0800, K. Y. Srinivasan wrote:
> > > The split_page() function will be very useful for balloon drivers. On Hyper-V,
> > > it will be very efficient to use 2M allocations in the guest as this (a) makes
> > > the ballooning protocol with the host that much more efficient and (b) moving
> > > memory in 2M chunks minimizes fragmentation in the host. Export the
> > split_page()
> > > function to let the guest allocations be in 2M chunks while the host is free to
> > > return this memory at arbitrary granularity.
> > >
> > >
> > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > ---
> > >  mm/page_alloc.c |    1 +
> > >  1 files changed, 1 insertions(+), 0 deletions(-)
> > >
> > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > > index 6cacfee..7e0ead6 100644
> > > --- a/mm/page_alloc.c
> > > +++ b/mm/page_alloc.c
> > > @@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
> > >  	for (i = 1; i < (1 << order); i++)
> > >  		set_page_refcounted(page + i);
> > >  }
> > > +EXPORT_SYMBOL_GPL(split_page);
> > 
> > When you export a symbol, you also need to post the code that is going
> > to use that symbol, otherwise people don't really know how to judge this
> > request.
> > 
> > Can you just make this a part of your balloon driver update patch series
> > instead?
> 
> Fair enough; I was hoping to see how inclined the mm folks were with regards to
> exporting this symbol before I went ahead and modified the balloon driver code to
> leverage this. Looking at the Windows guests on Hyper-V, I am convinced 2M balloon
> allocations in the Linux (Hyper-V) balloon driver will make significant difference. As you
> suggest, I will post this patch as part of the balloon driver changes that use this exported
> symbol. I am still hoping to get some feedback from the mm guys on this.

I guess the most obvious question about exporting this symbol is, "Why
doesn't any of the other hypervisor balloon drivers need this?  What is
so special about hyper-v?"

Or can those other drivers also need/use it as well, and they were just
too chicken to be asking for the export?  :)

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH 1/1] mm: Export split_page().

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Sunday, March 03, 2013 9:25 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org; olaf@aepfle.de;
> apw@canonical.com; andi@firstfloor.org; akpm@linux-foundation.org; linux-
> mm@kvack.org
> Subject: Re: [PATCH 1/1] mm: Export split_page().
> 
> On Mon, Mar 04, 2013 at 02:14:02AM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > Sent: Sunday, March 03, 2013 9:08 PM
> > > To: KY Srinivasan
> > > Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org;
> olaf@aepfle.de;
> > > apw@canonical.com; andi@firstfloor.org; akpm@linux-foundation.org; linux-
> > > mm@kvack.org
> > > Subject: Re: [PATCH 1/1] mm: Export split_page().
> > >
> > > On Sun, Mar 03, 2013 at 06:27:55PM -0800, K. Y. Srinivasan wrote:
> > > > The split_page() function will be very useful for balloon drivers. On Hyper-V,
> > > > it will be very efficient to use 2M allocations in the guest as this (a) makes
> > > > the ballooning protocol with the host that much more efficient and (b)
> moving
> > > > memory in 2M chunks minimizes fragmentation in the host. Export the
> > > split_page()
> > > > function to let the guest allocations be in 2M chunks while the host is free to
> > > > return this memory at arbitrary granularity.
> > > >
> > > >
> > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > > ---
> > > >  mm/page_alloc.c |    1 +
> > > >  1 files changed, 1 insertions(+), 0 deletions(-)
> > > >
> > > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > > > index 6cacfee..7e0ead6 100644
> > > > --- a/mm/page_alloc.c
> > > > +++ b/mm/page_alloc.c
> > > > @@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int
> order)
> > > >  	for (i = 1; i < (1 << order); i++)
> > > >  		set_page_refcounted(page + i);
> > > >  }
> > > > +EXPORT_SYMBOL_GPL(split_page);
> > >
> > > When you export a symbol, you also need to post the code that is going
> > > to use that symbol, otherwise people don't really know how to judge this
> > > request.
> > >
> > > Can you just make this a part of your balloon driver update patch series
> > > instead?
> >
> > Fair enough; I was hoping to see how inclined the mm folks were with regards
> to
> > exporting this symbol before I went ahead and modified the balloon driver
> code to
> > leverage this. Looking at the Windows guests on Hyper-V, I am convinced 2M
> balloon
> > allocations in the Linux (Hyper-V) balloon driver will make significant difference.
> As you
> > suggest, I will post this patch as part of the balloon driver changes that use this
> exported
> > symbol. I am still hoping to get some feedback from the mm guys on this.
> 
> I guess the most obvious question about exporting this symbol is, "Why
> doesn't any of the other hypervisor balloon drivers need this?  What is
> so special about hyper-v?"

The balloon protocol that Hyper-V has specified is designed around the ability to
move 2M pages. While the protocol can handle 4k allocations, it is going to be very chatty
with 4K allocations. Furthermore, the Memory Balancer on the host is also designed to work
best with memory moving around in 2M chunks. While I have not seen the code on the Windows
host that does this memory balancing, looking at how Windows guests behave in this environment,
(relative to Linux) I have to assume that the 2M allocations that Windows guests do are a big part of
the difference we see.
 
> 
> Or can those other drivers also need/use it as well, and they were just
> too chicken to be asking for the export?  :)

The 2M balloon allocations would make sense if the host is designed accordingly.

Regards,

K. Y


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] mm: Export split_page().

[Dave Hansen]

On 03/03/2013 06:36 PM, KY Srinivasan wrote:
>> I guess the most obvious question about exporting this symbol is, "Why
>> doesn't any of the other hypervisor balloon drivers need this?  What is
>> so special about hyper-v?"
> 
> The balloon protocol that Hyper-V has specified is designed around the ability to
> move 2M pages. While the protocol can handle 4k allocations, it is going to be very chatty
> with 4K allocations.

What does "very chatty" mean?  Do you think that there will be a
noticeable performance difference ballooning 2M pages vs 4k?

> Furthermore, the Memory Balancer on the host is also designed to work
> best with memory moving around in 2M chunks. While I have not seen the code on the Windows
> host that does this memory balancing, looking at how Windows guests behave in this environment,
> (relative to Linux) I have to assume that the 2M allocations that Windows guests do are a big part of
> the difference we see.

You've been talking about differences.  Could you elaborate on what the
differences in behavior are that you are trying to rectify here?

>> Or can those other drivers also need/use it as well, and they were just
>> too chicken to be asking for the export?  :)
> 
> The 2M balloon allocations would make sense if the host is designed accordingly.

How does the guest decide which size pages to allocate?  It seems like a
relatively bad idea to be inflating the balloon with 2M pages from the
guest in the case where the guest is under memory pressure _and_
fragmented.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH 1/1] mm: Export split_page().

[KY Srinivasan]

> -----Original Message-----
> From: Dave Hansen [mailto:dave@linux.vnet.ibm.com]
> Sent: Monday, March 04, 2013 12:06 PM
> To: KY Srinivasan
> Cc: Greg KH; linux-kernel@vger.kernel.org; devel@linuxdriverproject.org;
> olaf@aepfle.de; apw@canonical.com; andi@firstfloor.org; akpm@linux-
> foundation.org; linux-mm@kvack.org
> Subject: Re: [PATCH 1/1] mm: Export split_page().
> 
> On 03/03/2013 06:36 PM, KY Srinivasan wrote:
> >> I guess the most obvious question about exporting this symbol is, "Why
> >> doesn't any of the other hypervisor balloon drivers need this?  What is
> >> so special about hyper-v?"
> >
> > The balloon protocol that Hyper-V has specified is designed around the ability
> to
> > move 2M pages. While the protocol can handle 4k allocations, it is going to be
> very chatty
> > with 4K allocations.
> 
> What does "very chatty" mean?  Do you think that there will be a
> noticeable performance difference ballooning 2M pages vs 4k?

The balloon protocol that Hyper-V host specified allows you to specify page
ranges - start_pfn: num_pfn. With 2M pages the number of messages that need
to be exchanges is significantly fewer than with 4K page allocations.
> 
> > Furthermore, the Memory Balancer on the host is also designed to work
> > best with memory moving around in 2M chunks. While I have not seen the
> code on the Windows
> > host that does this memory balancing, looking at how Windows guests behave
> in this environment,
> > (relative to Linux) I have to assume that the 2M allocations that Windows
> guests do are a big part of
> > the difference we see.
> 
> You've been talking about differences.  Could you elaborate on what the
> differences in behavior are that you are trying to rectify here?

As I look at how smoothly memory is balanced on Windows guests with changing load conditions
in the guest relative to what I see with Linux, I see Linux taking more time to reach the steady state
during a balancing operation.  I will experiment with 2M allocations and report if this issue is addressed.

> 
> >> Or can those other drivers also need/use it as well, and they were just
> >> too chicken to be asking for the export?  :)
> >
> > The 2M balloon allocations would make sense if the host is designed
> accordingly.
> 
> How does the guest decide which size pages to allocate?  It seems like a
> relatively bad idea to be inflating the balloon with 2M pages from the
> guest in the case where the guest is under memory pressure _and_
> fragmented.

I want to start with 2M allocations and if they fail, fall back onto lower order allocations.
As I said, the host can support 4K allocations and that will be the final fallback position
(that is what I have currently implemented). If the guest memory is fragmented, then
obviously we will go in for lower order allocations.

Regards,

K. Y 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>