[PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Zhaoyang Huang]

For the type of 'ALLOC_HARDER' page allocation, there is an express
highway for the whole process which lead the allocation reach __rmqueue_xxx
easier than other type.
However, when CMA is enabled, the free_page within zone_watermark_ok() will
be deducted for number the pages in CMA type, which may cause the watermark
check fail, but there are possible enough HighAtomic or Unmovable and
Reclaimable pages in the zone. So add 'alloc_harder' here to
count CMA pages in to clean the obstacles on the way to the final.

Signed-off-by: Zhaoyang Huang <zhaoyang.huang@spreadtrum.com>
---
 mm/page_alloc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 635d7dd..cc18620 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3045,8 +3045,11 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 
 
 #ifdef CONFIG_CMA
-	/* If allocation can't use CMA areas don't use free CMA pages */
-	if (!(alloc_flags & ALLOC_CMA))
+	/*
+	 * If allocation can't use CMA areas and no alloc_harder set for none
+	 * order0 allocation, don't use free CMA pages.
+	 */
+	if (!(alloc_flags & ALLOC_CMA) && (!alloc_harder || !order))
 		free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
 #endif
 
-- 
1.9.1

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Michal Hocko]

On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> For the type of 'ALLOC_HARDER' page allocation, there is an express
> highway for the whole process which lead the allocation reach __rmqueue_xxx
> easier than other type.
> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> be deducted for number the pages in CMA type, which may cause the watermark
> check fail, but there are possible enough HighAtomic or Unmovable and
> Reclaimable pages in the zone. So add 'alloc_harder' here to
> count CMA pages in to clean the obstacles on the way to the final.

This is no longer the case in the current mmotm tree. Have a look at
Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com

> Signed-off-by: Zhaoyang Huang <zhaoyang.huang@spreadtrum.com>
> ---
>  mm/page_alloc.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 635d7dd..cc18620 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3045,8 +3045,11 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
>  
>  
>  #ifdef CONFIG_CMA
> -	/* If allocation can't use CMA areas don't use free CMA pages */
> -	if (!(alloc_flags & ALLOC_CMA))
> +	/*
> +	 * If allocation can't use CMA areas and no alloc_harder set for none
> +	 * order0 allocation, don't use free CMA pages.
> +	 */
> +	if (!(alloc_flags & ALLOC_CMA) && (!alloc_harder || !order))
>  		free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
>  #endif
>  
> -- 
> 1.9.1
> 

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Zhaoyang Huang]

On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
>> For the type of 'ALLOC_HARDER' page allocation, there is an express
>> highway for the whole process which lead the allocation reach __rmqueue_xxx
>> easier than other type.
>> However, when CMA is enabled, the free_page within zone_watermark_ok() will
>> be deducted for number the pages in CMA type, which may cause the watermark
>> check fail, but there are possible enough HighAtomic or Unmovable and
>> Reclaimable pages in the zone. So add 'alloc_harder' here to
>> count CMA pages in to clean the obstacles on the way to the final.
>
> This is no longer the case in the current mmotm tree. Have a look at
> Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
>
Thanks for the information. However, I can't find the commit in the
latest mainline, is it merged?
>> Signed-off-by: Zhaoyang Huang <zhaoyang.huang@spreadtrum.com>
>> ---
>>  mm/page_alloc.c | 7 +++++--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 635d7dd..cc18620 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -3045,8 +3045,11 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
>>
>>
>>  #ifdef CONFIG_CMA
>> -     /* If allocation can't use CMA areas don't use free CMA pages */
>> -     if (!(alloc_flags & ALLOC_CMA))
>> +     /*
>> +      * If allocation can't use CMA areas and no alloc_harder set for none
>> +      * order0 allocation, don't use free CMA pages.
>> +      */
>> +     if (!(alloc_flags & ALLOC_CMA) && (!alloc_harder || !order))
>>               free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
>>  #endif
>>
>> --
>> 1.9.1
>>
>
> --
> Michal Hocko
> SUSE Labs

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Michal Hocko]

On Fri 23-03-18 17:19:26, Zhaoyang Huang wrote:
> On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> >> For the type of 'ALLOC_HARDER' page allocation, there is an express
> >> highway for the whole process which lead the allocation reach __rmqueue_xxx
> >> easier than other type.
> >> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> >> be deducted for number the pages in CMA type, which may cause the watermark
> >> check fail, but there are possible enough HighAtomic or Unmovable and
> >> Reclaimable pages in the zone. So add 'alloc_harder' here to
> >> count CMA pages in to clean the obstacles on the way to the final.
> >
> > This is no longer the case in the current mmotm tree. Have a look at
> > Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
> >
> Thanks for the information. However, I can't find the commit in the
> latest mainline, is it merged?

Not yet. It is still sitting in the mmomt tree. I am not sure what is
the merge plan but I guess it is still waiting for some review feedback.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Andrew Morton]

On Fri, 23 Mar 2018 10:33:27 +0100 Michal Hocko <mhocko@kernel.org> wrote:

> On Fri 23-03-18 17:19:26, Zhaoyang Huang wrote:
> > On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > > On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> > >> For the type of 'ALLOC_HARDER' page allocation, there is an express
> > >> highway for the whole process which lead the allocation reach __rmqueue_xxx
> > >> easier than other type.
> > >> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> > >> be deducted for number the pages in CMA type, which may cause the watermark
> > >> check fail, but there are possible enough HighAtomic or Unmovable and
> > >> Reclaimable pages in the zone. So add 'alloc_harder' here to
> > >> count CMA pages in to clean the obstacles on the way to the final.
> > >
> > > This is no longer the case in the current mmotm tree. Have a look at
> > > Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
> > >
> > Thanks for the information. However, I can't find the commit in the
> > latest mainline, is it merged?
> 
> Not yet. It is still sitting in the mmomt tree. I am not sure what is
> the merge plan but I guess it is still waiting for some review feedback.

http://lkml.kernel.org/r/20171222001113.GA1729@js1304-P5Q-DELUXE

That patchset has been floating about since December and still has
unresolved issues.

Joonsoo, can you please let us know the status?

Thanks.

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Joonsoo Kim]

On Fri, Mar 23, 2018 at 01:04:08PM -0700, Andrew Morton wrote:
> On Fri, 23 Mar 2018 10:33:27 +0100 Michal Hocko <mhocko@kernel.org> wrote:
> 
> > On Fri 23-03-18 17:19:26, Zhaoyang Huang wrote:
> > > On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > > > On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> > > >> For the type of 'ALLOC_HARDER' page allocation, there is an express
> > > >> highway for the whole process which lead the allocation reach __rmqueue_xxx
> > > >> easier than other type.
> > > >> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> > > >> be deducted for number the pages in CMA type, which may cause the watermark
> > > >> check fail, but there are possible enough HighAtomic or Unmovable and
> > > >> Reclaimable pages in the zone. So add 'alloc_harder' here to
> > > >> count CMA pages in to clean the obstacles on the way to the final.
> > > >
> > > > This is no longer the case in the current mmotm tree. Have a look at
> > > > Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
> > > >
> > > Thanks for the information. However, I can't find the commit in the
> > > latest mainline, is it merged?
> > 
> > Not yet. It is still sitting in the mmomt tree. I am not sure what is
> > the merge plan but I guess it is still waiting for some review feedback.
> 
> http://lkml.kernel.org/r/20171222001113.GA1729@js1304-P5Q-DELUXE
> 
> That patchset has been floating about since December and still has
> unresolved issues.
> 
> Joonsoo, can you please let us know the status?

Hello, Andrew.
Sorry for a late response.

Today, I finally have answered the question from Michal and it seems
that there is no problem at all.

http://lkml.kernel.org/r/CAAmzW4NGv7RyCYyokPoj4aR3ySKub4jaBZ3k=pt_YReFbByvsw@mail.gmail.com

You can merge the patch as is.

Thanks.

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Andrew Morton]

On Wed, 4 Apr 2018 09:31:10 +0900 Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:

> On Fri, Mar 23, 2018 at 01:04:08PM -0700, Andrew Morton wrote:
> > On Fri, 23 Mar 2018 10:33:27 +0100 Michal Hocko <mhocko@kernel.org> wrote:
> > 
> > > On Fri 23-03-18 17:19:26, Zhaoyang Huang wrote:
> > > > On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > > > > On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> > > > >> For the type of 'ALLOC_HARDER' page allocation, there is an express
> > > > >> highway for the whole process which lead the allocation reach __rmqueue_xxx
> > > > >> easier than other type.
> > > > >> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> > > > >> be deducted for number the pages in CMA type, which may cause the watermark
> > > > >> check fail, but there are possible enough HighAtomic or Unmovable and
> > > > >> Reclaimable pages in the zone. So add 'alloc_harder' here to
> > > > >> count CMA pages in to clean the obstacles on the way to the final.
> > > > >
> > > > > This is no longer the case in the current mmotm tree. Have a look at
> > > > > Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
> > > > >
> > > > Thanks for the information. However, I can't find the commit in the
> > > > latest mainline, is it merged?
> > > 
> > > Not yet. It is still sitting in the mmomt tree. I am not sure what is
> > > the merge plan but I guess it is still waiting for some review feedback.
> > 
> > http://lkml.kernel.org/r/20171222001113.GA1729@js1304-P5Q-DELUXE
> > 
> > That patchset has been floating about since December and still has
> > unresolved issues.
> > 
> > Joonsoo, can you please let us know the status?
> 
> Hello, Andrew.
> Sorry for a late response.
> 
> Today, I finally have answered the question from Michal and it seems
> that there is no problem at all.
> 
> http://lkml.kernel.org/r/CAAmzW4NGv7RyCYyokPoj4aR3ySKub4jaBZ3k=pt_YReFbByvsw@mail.gmail.com
> 
> You can merge the patch as is.
> 

hm.

There was also a performance regression reported:
http://lkml.kernel.org/r/20180102063528.GG30397@yexl-desktop

Re: [PATCH v1] mm: help the ALLOC_HARDER allocation pass the watermarki when CMA on

[Joonsoo Kim]

On Wed, Apr 04, 2018 at 03:37:03PM -0700, Andrew Morton wrote:
> On Wed, 4 Apr 2018 09:31:10 +0900 Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> 
> > On Fri, Mar 23, 2018 at 01:04:08PM -0700, Andrew Morton wrote:
> > > On Fri, 23 Mar 2018 10:33:27 +0100 Michal Hocko <mhocko@kernel.org> wrote:
> > > 
> > > > On Fri 23-03-18 17:19:26, Zhaoyang Huang wrote:
> > > > > On Fri, Mar 23, 2018 at 4:38 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > > > > > On Fri 23-03-18 15:57:32, Zhaoyang Huang wrote:
> > > > > >> For the type of 'ALLOC_HARDER' page allocation, there is an express
> > > > > >> highway for the whole process which lead the allocation reach __rmqueue_xxx
> > > > > >> easier than other type.
> > > > > >> However, when CMA is enabled, the free_page within zone_watermark_ok() will
> > > > > >> be deducted for number the pages in CMA type, which may cause the watermark
> > > > > >> check fail, but there are possible enough HighAtomic or Unmovable and
> > > > > >> Reclaimable pages in the zone. So add 'alloc_harder' here to
> > > > > >> count CMA pages in to clean the obstacles on the way to the final.
> > > > > >
> > > > > > This is no longer the case in the current mmotm tree. Have a look at
> > > > > > Joonsoo's zone movable based CMA patchset http://lkml.kernel.org/r/1512114786-5085-1-git-send-email-iamjoonsoo.kim@lge.com
> > > > > >
> > > > > Thanks for the information. However, I can't find the commit in the
> > > > > latest mainline, is it merged?
> > > > 
> > > > Not yet. It is still sitting in the mmomt tree. I am not sure what is
> > > > the merge plan but I guess it is still waiting for some review feedback.
> > > 
> > > http://lkml.kernel.org/r/20171222001113.GA1729@js1304-P5Q-DELUXE
> > > 
> > > That patchset has been floating about since December and still has
> > > unresolved issues.
> > > 
> > > Joonsoo, can you please let us know the status?
> > 
> > Hello, Andrew.
> > Sorry for a late response.
> > 
> > Today, I finally have answered the question from Michal and it seems
> > that there is no problem at all.
> > 
> > http://lkml.kernel.org/r/CAAmzW4NGv7RyCYyokPoj4aR3ySKub4jaBZ3k=pt_YReFbByvsw@mail.gmail.com
> > 
> > You can merge the patch as is.
> > 
> 
> hm.
> 
> There was also a performance regression reported:
> http://lkml.kernel.org/r/20180102063528.GG30397@yexl-desktop

I analyze the report and may find the reason.

When we uses more zones, min_free_kbytes is increased for avoiding
fragmentation if THP is enabled. This patch uses one more zone to
manage CMA memory so min_free_kbytes and thus min_watermark is increased.
It would reduce our usable memory and cause regression.

However, this reservation for fragmentation isn't needed for
ZONE_MOVABLE since it has only movable pages so I send a patch to fix it.

http://lkml.kernel.org/r/<1522913236-15776-1-git-send-email-iamjoonsoo.kim@lge.com>

I'm not sure that it is a root cause of above performance regression
but I highly anticipate that they are related. I will ask the reporter
to test this patch on top of that.

Thanks.