NR_FREE_CMA_PAGES larger than total CMA size

NR_FREE_CMA_PAGES larger than total CMA size

[Lisa Du]

Dear Sir
Recently I met one issue that after system run for a long time, free cma pages recorded
in vm_stat[NR_FREE_CMA_PAGES] are larger than total CMA size declared.
For example, I declared 64MB CMA size, but found free cma was 70MB.

I added some trace to track how it happen, and found the reason maybe like below:
1) alloc_contig_range() want to allocate a range [start, end], for example [0x1e040, 0x1e050];

2) start_isolate_page_range() will isolate the range [pfn_max_align_down(start),
  pfn_max_align_up(end)]; for this example it's [0x1e000, 0x1e400] (MAX_ORDER is 11);

3) drain_all_pages() would be called as follows, if there's some pages belong to the range
  [0x1e000, 0x1e400] was freed from the pcp_list, also if the page was MIGRATE_CMA,
  then vm_stat[NR_FREE_CMA_PAGES] would increase and also NR_FREE_PAGES;

4) if the freed pages in #3 was not the range of [start, end], then at last undo_isolate_page_range()
  will be called, and the pages would be calculated again as free pages in unset_migratetype_isolate(),
  and __mod_zone_freepage_state() will increased again for these pages for both NR_FREE_CMA_PAGES
  and NR_FREE_PAGES. 
  The function calling flow as below, the free pages in move_freepages() was calculated again.
  undo_isolate_page_range()
	--> unset_migratetype_isolate()
		--> move_freepages_block()
			--> move_freepages()
	--> __mod_zone_freepage_state()

Shall we add some check in move_freepages() if the page was already in CMA free list, 
then exclude it from the pages_moved?

I found this issue in kernel v3.4, but seems there's no fix in latest kernel code base.
Not sure if anyone else has met such issue? Anyone would help to comment? Thanks a lot!

Thanks!

Best Regards
Lisa Du

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: NR_FREE_CMA_PAGES larger than total CMA size

[Joonsoo Kim]

On Sat, Jul 05, 2014 at 01:13:17AM -0700, Lisa Du wrote:
> Dear Sir
> Recently I met one issue that after system run for a long time, free cma pages recorded
> in vm_stat[NR_FREE_CMA_PAGES] are larger than total CMA size declared.
> For example, I declared 64MB CMA size, but found free cma was 70MB.
> 
> I added some trace to track how it happen, and found the reason maybe like below:
> 1) alloc_contig_range() want to allocate a range [start, end], for example [0x1e040, 0x1e050];
> 
> 2) start_isolate_page_range() will isolate the range [pfn_max_align_down(start),
>   pfn_max_align_up(end)]; for this example it's [0x1e000, 0x1e400] (MAX_ORDER is 11);
> 
> 3) drain_all_pages() would be called as follows, if there's some pages belong to the range
>   [0x1e000, 0x1e400] was freed from the pcp_list, also if the page was MIGRATE_CMA,
>   then vm_stat[NR_FREE_CMA_PAGES] would increase and also NR_FREE_PAGES;
> 
> 4) if the freed pages in #3 was not the range of [start, end], then at last undo_isolate_page_range()
>   will be called, and the pages would be calculated again as free pages in unset_migratetype_isolate(),
>   and __mod_zone_freepage_state() will increased again for these pages for both NR_FREE_CMA_PAGES
>   and NR_FREE_PAGES. 
>   The function calling flow as below, the free pages in move_freepages() was calculated again.
>   undo_isolate_page_range()
> 	--> unset_migratetype_isolate()
> 		--> move_freepages_block()
> 			--> move_freepages()
> 	--> __mod_zone_freepage_state()
> 
> Shall we add some check in move_freepages() if the page was already in CMA free list, 
> then exclude it from the pages_moved?
> 
> I found this issue in kernel v3.4, but seems there's no fix in latest kernel code base.
> Not sure if anyone else has met such issue? Anyone would help to comment? Thanks a lot!

Hello,

Maybe this bug is relevant for my recent patchset.
I don't have much time to investigate your problem, so
if you have interest on my patchset, please look at it on below link

https://lkml.org/lkml/2014/7/4/79

If they work for you, please let me know. :)

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: NR_FREE_CMA_PAGES larger than total CMA size

[Lisa Du]

>-----Original Message-----
>From: Joonsoo Kim [mailto:iamjoonsoo.kim@lge.com]
>Sent: 2014年7月7日 12:54
>To: Lisa Du
>Cc: linux-mm@kvack.org
>Subject: Re: NR_FREE_CMA_PAGES larger than total CMA size
>
>On Sat, Jul 05, 2014 at 01:13:17AM -0700, Lisa Du wrote:
>> Dear Sir
>> Recently I met one issue that after system run for a long time, free
>> cma pages recorded in vm_stat[NR_FREE_CMA_PAGES] are larger than total CMA size declared.
>> For example, I declared 64MB CMA size, but found free cma was 70MB.
>>
>> I added some trace to track how it happen, and found the reason maybe like below:
>> 1) alloc_contig_range() want to allocate a range [start, end], for
>> example [0x1e040, 0x1e050];
>>
>> 2) start_isolate_page_range() will isolate the range [pfn_max_align_down(start),
>>   pfn_max_align_up(end)]; for this example it's [0x1e000, 0x1e400]
>> (MAX_ORDER is 11);
>>
>> 3) drain_all_pages() would be called as follows, if there's some pages belong to the range
>>   [0x1e000, 0x1e400] was freed from the pcp_list, also if the page was MIGRATE_CMA,
>>   then vm_stat[NR_FREE_CMA_PAGES] would increase and also
>> NR_FREE_PAGES;
>>
>> 4) if the freed pages in #3 was not the range of [start, end], then at last undo_isolate_page_range()
>>   will be called, and the pages would be calculated again as free pages in unset_migratetype_isolate(),
>>   and __mod_zone_freepage_state() will increased again for these pages for both NR_FREE_CMA_PAGES
>>   and NR_FREE_PAGES.
>>   The function calling flow as below, the free pages in move_freepages() was calculated again.
>>   undo_isolate_page_range()
>> 	--> unset_migratetype_isolate()
>> 		--> move_freepages_block()
>> 			--> move_freepages()
>> 	--> __mod_zone_freepage_state()
>>
>> Shall we add some check in move_freepages() if the page was already in
>> CMA free list, then exclude it from the pages_moved?
>>
>> I found this issue in kernel v3.4, but seems there's no fix in latest kernel code base.
>> Not sure if anyone else has met such issue? Anyone would help to comment? Thanks a lot!
>
>Hello,
>
>Maybe this bug is relevant for my recent patchset.
>I don't have much time to investigate your problem, so if you have interest on my patchset, please look at it on below link
>
>https://lkml.org/lkml/2014/7/4/79
>
>If they work for you, please let me know. :)
Below patches works for me, thanks a lot!
  mm/page_alloc: handle page on pcp correctly if it's pageblock is isolated
  mm/page_alloc: carefully free the page on isolate pageblock
>
>Thanks.