[patch 13/15] mm/page_owner: align with pageblock_nr pages

[patch 13/15] mm/page_owner: align with pageblock_nr pages

[akpm]

From: zhong jiang <zhongjiang@huawei.com>
Subject: mm/page_owner: align with pageblock_nr pages

When pfn_valid(pfn) returns false, pfn should be aligned with
pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
because the skipped 2M may be valid pfn, as a result, early allocated
count will not be accurate.

Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
Signed-off-by: zhong jiang <zhongjiang@huawei.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/page_owner.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff -puN mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages mm/page_owner.c
--- a/mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages
+++ a/mm/page_owner.c
@@ -544,7 +544,7 @@ static void init_pages_in_zone(pg_data_t
 	 */
 	for (; pfn < end_pfn; ) {
 		if (!pfn_valid(pfn)) {
-			pfn = ALIGN(pfn + 1, MAX_ORDER_NR_PAGES);
+			pfn = ALIGN(pfn + 1, pageblock_nr_pages);
 			continue;
 		}
 
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Vlastimil Babka]

On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
> From: zhong jiang <zhongjiang@huawei.com>
> Subject: mm/page_owner: align with pageblock_nr pages
> 
> When pfn_valid(pfn) returns false, pfn should be aligned with
> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
> because the skipped 2M may be valid pfn, as a result, early allocated
> count will not be accurate.
> 
> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> Cc: Michal Hocko <mhocko@kernel.org>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

The author never responded and Michal Hocko basically NAKed it in
https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
I think we should drop it.

> ---
> 
>  mm/page_owner.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff -puN mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages mm/page_owner.c
> --- a/mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages
> +++ a/mm/page_owner.c
> @@ -544,7 +544,7 @@ static void init_pages_in_zone(pg_data_t
>  	 */
>  	for (; pfn < end_pfn; ) {
>  		if (!pfn_valid(pfn)) {
> -			pfn = ALIGN(pfn + 1, MAX_ORDER_NR_PAGES);
> +			pfn = ALIGN(pfn + 1, pageblock_nr_pages);
>  			continue;
>  		}
>  
> _
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Michal Hocko]

On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
> > From: zhong jiang <zhongjiang@huawei.com>
> > Subject: mm/page_owner: align with pageblock_nr pages
> > 
> > When pfn_valid(pfn) returns false, pfn should be aligned with
> > pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
> > because the skipped 2M may be valid pfn, as a result, early allocated
> > count will not be accurate.
> > 
> > Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
> > Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> > Cc: Michal Hocko <mhocko@kernel.org>
> > Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 
> The author never responded and Michal Hocko basically NAKed it in
> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
> I think we should drop it.

Or extend the changelog to actually describe what kind of problem it
fixes and do an additional step to unigy
MAX_ORDER_NR_PAGES/pageblock_nr_pages
 
> > ---
> > 
> >  mm/page_owner.c |    2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff -puN mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages mm/page_owner.c
> > --- a/mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages
> > +++ a/mm/page_owner.c
> > @@ -544,7 +544,7 @@ static void init_pages_in_zone(pg_data_t
> >  	 */
> >  	for (; pfn < end_pfn; ) {
> >  		if (!pfn_valid(pfn)) {
> > -			pfn = ALIGN(pfn + 1, MAX_ORDER_NR_PAGES);
> > +			pfn = ALIGN(pfn + 1, pageblock_nr_pages);
> >  			continue;
> >  		}
> >  
> > _
> > 
> > --
> > To unsubscribe, send a message with 'unsubscribe linux-mm' in
> > the body to majordomo@kvack.org.  For more info on Linux MM,
> > see: http://www.linux-mm.org/ .
> > Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> > 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[zhong jiang]

On 2017/12/2 1:15, Michal Hocko wrote:
> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>> From: zhong jiang <zhongjiang@huawei.com>
>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>
>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>> count will not be accurate.
>>>
>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>> Cc: Michal Hocko <mhocko@kernel.org>
>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>> The author never responded and Michal Hocko basically NAKed it in
>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>> I think we should drop it.
> Or extend the changelog to actually describe what kind of problem it
> fixes and do an additional step to unigy
> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>  
  Hi, Michal
   
        IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
    
        when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
       it maybe result in normal pages leak.

        meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
   
        I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
       and repost it.

    Michal,  Do you think ?

 Thanks
zhongjiang
>>> ---
>>>
>>>  mm/page_owner.c |    2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff -puN mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages mm/page_owner.c
>>> --- a/mm/page_owner.c~mm-page_owner-align-with-pageblock_nr-pages
>>> +++ a/mm/page_owner.c
>>> @@ -544,7 +544,7 @@ static void init_pages_in_zone(pg_data_t
>>>  	 */
>>>  	for (; pfn < end_pfn; ) {
>>>  		if (!pfn_valid(pfn)) {
>>> -			pfn = ALIGN(pfn + 1, MAX_ORDER_NR_PAGES);
>>> +			pfn = ALIGN(pfn + 1, pageblock_nr_pages);
>>>  			continue;
>>>  		}
>>>  
>>> _
>>>
>>> --
>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>>> the body to majordomo@kvack.org.  For more info on Linux MM,
>>> see: http://www.linux-mm.org/ .
>>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Michal Hocko]

On Mon 04-12-17 19:51:12, zhong jiang wrote:
> On 2017/12/2 1:15, Michal Hocko wrote:
> > On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
> >> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
> >>> From: zhong jiang <zhongjiang@huawei.com>
> >>> Subject: mm/page_owner: align with pageblock_nr pages
> >>>
> >>> When pfn_valid(pfn) returns false, pfn should be aligned with
> >>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
> >>> because the skipped 2M may be valid pfn, as a result, early allocated
> >>> count will not be accurate.
> >>>
> >>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
> >>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> >>> Cc: Michal Hocko <mhocko@kernel.org>
> >>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> >> The author never responded and Michal Hocko basically NAKed it in
> >> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
> >> I think we should drop it.
> > Or extend the changelog to actually describe what kind of problem it
> > fixes and do an additional step to unigy
> > MAX_ORDER_NR_PAGES/pageblock_nr_pages
> >  
>   Hi, Michal
>    
>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>     
>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>        it maybe result in normal pages leak.
> 
>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>    
>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>        and repost it.
> 
>     Michal,  Do you think ?

Yes, rewrite the patch changelog and make it _clear_ what it fixes and
under _what_ conditions. There are also other places using
MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
updated as well?
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[zhong jiang]

On 2017/12/4 20:01, Michal Hocko wrote:
> On Mon 04-12-17 19:51:12, zhong jiang wrote:
>> On 2017/12/2 1:15, Michal Hocko wrote:
>>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>>>
>>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>>>> count will not be accurate.
>>>>>
>>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>>>> Cc: Michal Hocko <mhocko@kernel.org>
>>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>>>> The author never responded and Michal Hocko basically NAKed it in
>>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>>>> I think we should drop it.
>>> Or extend the changelog to actually describe what kind of problem it
>>> fixes and do an additional step to unigy
>>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>>>  
>>   Hi, Michal
>>    
>>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>>     
>>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>>        it maybe result in normal pages leak.
>>
>>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>>    
>>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>>        and repost it.
>>
>>     Michal,  Do you think ?
> Yes, rewrite the patch changelog and make it _clear_ what it fixes and
> under _what_ conditions. There are also other places using
> MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
> updated as well?
 in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)

  Thanks
  zhongjiang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Michal Hocko]

On Mon 04-12-17 20:23:49, zhong jiang wrote:
> On 2017/12/4 20:01, Michal Hocko wrote:
> > On Mon 04-12-17 19:51:12, zhong jiang wrote:
> >> On 2017/12/2 1:15, Michal Hocko wrote:
> >>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
> >>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
> >>>>> From: zhong jiang <zhongjiang@huawei.com>
> >>>>> Subject: mm/page_owner: align with pageblock_nr pages
> >>>>>
> >>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
> >>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
> >>>>> because the skipped 2M may be valid pfn, as a result, early allocated
> >>>>> count will not be accurate.
> >>>>>
> >>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
> >>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> >>>>> Cc: Michal Hocko <mhocko@kernel.org>
> >>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> >>>> The author never responded and Michal Hocko basically NAKed it in
> >>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
> >>>> I think we should drop it.
> >>> Or extend the changelog to actually describe what kind of problem it
> >>> fixes and do an additional step to unigy
> >>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
> >>>  
> >>   Hi, Michal
> >>    
> >>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
> >>     
> >>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
> >>        it maybe result in normal pages leak.
> >>
> >>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
> >>    
> >>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
> >>        and repost it.
> >>
> >>     Michal,  Do you think ?
> > Yes, rewrite the patch changelog and make it _clear_ what it fixes and
> > under _what_ conditions. There are also other places using
> > MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
> > updated as well?
>  in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)

git grep says some in page_ext.c, memory_hotplug.c and few in the arch
code. I belive we really want to describe and document the distinction
between the two constants and explain when to use which one.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[zhong jiang]

On 2017/12/4 20:35, Michal Hocko wrote:
> On Mon 04-12-17 20:23:49, zhong jiang wrote:
>> On 2017/12/4 20:01, Michal Hocko wrote:
>>> On Mon 04-12-17 19:51:12, zhong jiang wrote:
>>>> On 2017/12/2 1:15, Michal Hocko wrote:
>>>>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>>>>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>>>>>
>>>>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>>>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>>>>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>>>>>> count will not be accurate.
>>>>>>>
>>>>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>>>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>>>>>> Cc: Michal Hocko <mhocko@kernel.org>
>>>>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>>>>>> The author never responded and Michal Hocko basically NAKed it in
>>>>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>>>>>> I think we should drop it.
>>>>> Or extend the changelog to actually describe what kind of problem it
>>>>> fixes and do an additional step to unigy
>>>>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>>>>>  
>>>>   Hi, Michal
>>>>    
>>>>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>>>>     
>>>>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>>>>        it maybe result in normal pages leak.
>>>>
>>>>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>>>>    
>>>>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>>>>        and repost it.
>>>>
>>>>     Michal,  Do you think ?
>>> Yes, rewrite the patch changelog and make it _clear_ what it fixes and
>>> under _what_ conditions. There are also other places using
>>> MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
>>> updated as well?
>>  in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)
> git grep says some in page_ext.c, memory_hotplug.c and few in the arch
> code. I belive we really want to describe and document the distinction
> between the two constants and explain when to use which one.
>
 yes,   limited by my knowledge and english.  Maybe Vlastimil  can  address it  in detail.  

Thanks
zhongjiang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Vlastimil Babka]

On 12/04/2017 01:56 PM, zhong jiang wrote:
> On 2017/12/4 20:35, Michal Hocko wrote:
>> On Mon 04-12-17 20:23:49, zhong jiang wrote:
>>> On 2017/12/4 20:01, Michal Hocko wrote:
>>>> On Mon 04-12-17 19:51:12, zhong jiang wrote:
>>>>> On 2017/12/2 1:15, Michal Hocko wrote:
>>>>>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>>>>>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>>>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>>>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>>>>>>
>>>>>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>>>>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>>>>>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>>>>>>> count will not be accurate.
>>>>>>>>
>>>>>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>>>>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>>>>>>> Cc: Michal Hocko <mhocko@kernel.org>
>>>>>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>>>>>>> The author never responded and Michal Hocko basically NAKed it in
>>>>>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>>>>>>> I think we should drop it.
>>>>>> Or extend the changelog to actually describe what kind of problem it
>>>>>> fixes and do an additional step to unigy
>>>>>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>>>>>>  
>>>>>   Hi, Michal
>>>>>    
>>>>>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>>>>>     
>>>>>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>>>>>        it maybe result in normal pages leak.
>>>>>
>>>>>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>>>>>    
>>>>>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>>>>>        and repost it.
>>>>>
>>>>>     Michal,  Do you think ?
>>>> Yes, rewrite the patch changelog and make it _clear_ what it fixes and
>>>> under _what_ conditions. There are also other places using
>>>> MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
>>>> updated as well?
>>>  in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)
>> git grep says some in page_ext.c, memory_hotplug.c and few in the arch
>> code. I belive we really want to describe and document the distinction
>> between the two constants and explain when to use which one.
>>
>  yes,   limited by my knowledge and english.  Maybe Vlastimil  can  address it  in detail.  

Hi, on a fresh look, I believe this patch doesn't improve anything in
practice. It potentially makes init_pages_in_zone() catch more early
allocations, if a hole happens to be placed in the beginning of
MAX_ORDER block, and the following pageblock within the block was early
allocated.

However, read_page_owner() skips whole MAX_ORDER block as well in this
situation, so we won't be able to read the info anyway...

Also the problem is not as simple as documenting MAX_ORDER_NR_PAGES vs
pabeblock_nr_pages. We discussed it year ago when this patch was first
posted, how skipping over holes would have to be made more robust, and
how architectures should define hole granularity to avoid checking each
individual pfn in what appears to be a hole, to see if the hole has ended.

> Thanks
> zhongjiang
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Vlastimil Babka]

On 12/05/2017 12:22 PM, Vlastimil Babka wrote:
> On 12/04/2017 01:56 PM, zhong jiang wrote:
>> On 2017/12/4 20:35, Michal Hocko wrote:
>>> On Mon 04-12-17 20:23:49, zhong jiang wrote:
>>>> On 2017/12/4 20:01, Michal Hocko wrote:
>>>>> On Mon 04-12-17 19:51:12, zhong jiang wrote:
>>>>>> On 2017/12/2 1:15, Michal Hocko wrote:
>>>>>>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>>>>>>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>>>>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>>>>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>>>>>>>
>>>>>>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>>>>>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>>>>>>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>>>>>>>> count will not be accurate.
>>>>>>>>>
>>>>>>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>>>>>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>>>>>>>> Cc: Michal Hocko <mhocko@kernel.org>
>>>>>>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>>>>>>>> The author never responded and Michal Hocko basically NAKed it in
>>>>>>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>>>>>>>> I think we should drop it.
>>>>>>> Or extend the changelog to actually describe what kind of problem it
>>>>>>> fixes and do an additional step to unigy
>>>>>>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>>>>>>>  
>>>>>>   Hi, Michal
>>>>>>    
>>>>>>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>>>>>>     
>>>>>>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>>>>>>        it maybe result in normal pages leak.
>>>>>>
>>>>>>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>>>>>>    
>>>>>>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>>>>>>        and repost it.
>>>>>>
>>>>>>     Michal,  Do you think ?
>>>>> Yes, rewrite the patch changelog and make it _clear_ what it fixes and
>>>>> under _what_ conditions. There are also other places using
>>>>> MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
>>>>> updated as well?
>>>>  in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)
>>> git grep says some in page_ext.c, memory_hotplug.c and few in the arch
>>> code. I belive we really want to describe and document the distinction
>>> between the two constants and explain when to use which one.
>>>
>>  yes,   limited by my knowledge and english.  Maybe Vlastimil  can  address it  in detail.  
> 
> Hi, on a fresh look, I believe this patch doesn't improve anything in
> practice. It potentially makes init_pages_in_zone() catch more early
> allocations, if a hole happens to be placed in the beginning of
> MAX_ORDER block, and the following pageblock within the block was early
> allocated.
> 
> However, read_page_owner() skips whole MAX_ORDER block as well in this
> situation, so we won't be able to read the info anyway...
> 
> Also the problem is not as simple as documenting MAX_ORDER_NR_PAGES vs
> pabeblock_nr_pages. We discussed it year ago when this patch was first
> posted, how skipping over holes would have to be made more robust, and
> how architectures should define hole granularity to avoid checking each
> individual pfn in what appears to be a hole, to see if the hole has ended.

OK, let's see. There are three archs that define CONFIG_HOLES_IN_ZONE.
This means for those arches, pfn_valid_within() is not defined as true,
but calls actual pfn_valid():

arm64 - pfn_valid(pfn) is memblock_is_map_memory(pfn << PAGE_SHIFT)
What is the granularity of memblock allocator? If it's less than
MAX_ORDER, could we just sacrifice the memory in MAX_ORDER block that
has a hole inside?

mips - pfn_valid() is a simple pfn-within-boundaries comparison. That
shouldn't make any difference, if we observe zone end pfn right! They
added the config in 465aaed0030b2 for Cavium Octeon and apparently it
fixed a crash, but I wonder what was really going on then...

ia64 - uses ia64_pfn_valid() which I have no idea what it does...

> 
>> Thanks
>> zhongjiang
>>
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[zhong jiang]

On 2017/12/5 19:22, Vlastimil Babka wrote:
> On 12/04/2017 01:56 PM, zhong jiang wrote:
>> On 2017/12/4 20:35, Michal Hocko wrote:
>>> On Mon 04-12-17 20:23:49, zhong jiang wrote:
>>>> On 2017/12/4 20:01, Michal Hocko wrote:
>>>>> On Mon 04-12-17 19:51:12, zhong jiang wrote:
>>>>>> On 2017/12/2 1:15, Michal Hocko wrote:
>>>>>>> On Fri 01-12-17 17:58:28, Vlastimil Babka wrote:
>>>>>>>> On 11/30/2017 11:15 PM, akpm@linux-foundation.org wrote:
>>>>>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>>>>>> Subject: mm/page_owner: align with pageblock_nr pages
>>>>>>>>>
>>>>>>>>> When pfn_valid(pfn) returns false, pfn should be aligned with
>>>>>>>>> pageblock_nr_pages other than MAX_ORDER_NR_PAGES in init_pages_in_zone,
>>>>>>>>> because the skipped 2M may be valid pfn, as a result, early allocated
>>>>>>>>> count will not be accurate.
>>>>>>>>>
>>>>>>>>> Link: http://lkml.kernel.org/r/1468938136-24228-1-git-send-email-zhongjiang@huawei.com
>>>>>>>>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>>>>>>>>> Cc: Michal Hocko <mhocko@kernel.org>
>>>>>>>>> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>>>>>>>> The author never responded and Michal Hocko basically NAKed it in
>>>>>>>> https://lkml.kernel.org/r/<20160812130727.GI3639@dhcp22.suse.cz>
>>>>>>>> I think we should drop it.
>>>>>>> Or extend the changelog to actually describe what kind of problem it
>>>>>>> fixes and do an additional step to unigy
>>>>>>> MAX_ORDER_NR_PAGES/pageblock_nr_pages
>>>>>>>  
>>>>>>   Hi, Michal
>>>>>>    
>>>>>>         IIRC,  I had explained the reason for patch.  if it not. I am so sorry for that.
>>>>>>     
>>>>>>         when we select MAX_ORDER_NR_PAGES,   the second 2M will be skiped.
>>>>>>        it maybe result in normal pages leak.
>>>>>>
>>>>>>         meanwhile.  as you had said.  it make the code consistent.  why do not we do it.
>>>>>>    
>>>>>>         I think it is reasonable to upstream the patch.  maybe I should rewrite the changelog
>>>>>>        and repost it.
>>>>>>
>>>>>>     Michal,  Do you think ?
>>>>> Yes, rewrite the patch changelog and make it _clear_ what it fixes and
>>>>> under _what_ conditions. There are also other places using
>>>>> MAX_ORDER_NR_PAGES rathern than pageblock_nr_pages. Do they need to be
>>>>> updated as well?
>>>>  in the lastest kernel.  according to correspond context,   I  can not find the candidate. :-)
>>> git grep says some in page_ext.c, memory_hotplug.c and few in the arch
>>> code. I belive we really want to describe and document the distinction
>>> between the two constants and explain when to use which one.
>>>
>>  yes,   limited by my knowledge and english.  Maybe Vlastimil  can  address it  in detail.  
> Hi, on a fresh look, I believe this patch doesn't improve anything in
> practice. It potentially makes init_pages_in_zone() catch more early
> allocations, if a hole happens to be placed in the beginning of
> MAX_ORDER block, and the following pageblock within the block was early
> allocated.
 Hi, Vlastimil

  I have a stupid question about holes

  because a hole is possible to have within a MAX_ORDER_NR_PAGES, it indeed
  exist in first pfn. it that is true, why we must skip the whole MAX_ORDER block?
  Any limit ?  I can not find the answer.

  Thanks
  zhongjiang
> However, read_page_owner() skips whole MAX_ORDER block as well in this
> situation, so we won't be able to read the info anyway...
>
> Also the problem is not as simple as documenting MAX_ORDER_NR_PAGES vs
> pabeblock_nr_pages. We discussed it year ago when this patch was first
> posted, how skipping over holes would have to be made more robust, and
> how architectures should define hole granularity to avoid checking each
> individual pfn in what appears to be a hole, to see if the hole has ended.
>
>> Thanks
>> zhongjiang
>>
>
> .
>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch 13/15] mm/page_owner: align with pageblock_nr pages

[Vlastimil Babka]

On 12/05/2017 01:50 PM, zhong jiang wrote:
>>>  yes,   limited by my knowledge and english.  Maybe Vlastimil  can  address it  in detail.  
>> Hi, on a fresh look, I believe this patch doesn't improve anything in
>> practice. It potentially makes init_pages_in_zone() catch more early
>> allocations, if a hole happens to be placed in the beginning of
>> MAX_ORDER block, and the following pageblock within the block was early
>> allocated.
>  Hi, Vlastimil
> 
>   I have a stupid question about holes
> 
>   because a hole is possible to have within a MAX_ORDER_NR_PAGES, it indeed
>   exist in first pfn. it that is true, why we must skip the whole MAX_ORDER block?
>   Any limit ?  I can not find the answer.

It's not that we "must skip". If I understand it correctly, on kernels
without CONFIG_HOLES_IN_ZONE, we can skip a MAX_ORDER block if *any* pfn
(including the first pfn) is invalid, because we know that the whole
block is invalid. On CONFIG_HOLES_IN_ZONE, there is no such guarantee.

So if we see that the first pfn is valid, we continue with the block,
but use pfn_valid_within() (which is defined as pfn_valid() on
CONFIG_HOLES_IN_ZONE and hardcoded "true" elsewhere) to validate each
pfn. This is slow, but the arches pay the price for CONFIG_HOLES_IN_ZONE.

If we see that first pfn is invalid, we are safe to skip the MAX_ORDER
block when CONFIG_HOLES_IN_ZONE=n and we know we won't miss anything. On
CONFIG_HOLES_IN_ZONE we might miss something, so to be sure we don't
miss something, we should validate each pfn. The potential price there
is probably worse, because we might be validating arbitrary large holes
not limited by physical amount of RAM. So e.g. compaction doesn't pay
this price, and MAX_ORDER blocks that would have hole at the beginning
and end (with valid pages in the middle) are skipped.

page_owner on the other hand is a debugging feature not normally
enabled, with significant overhead, so paying the price there might not
be an issue. But it means rewriting both init_pages_in_zone() and
read_page_owner() to not skip MAX_ORDER block (nor pageblock_order) when
CONFIG_HOLES_IN_ZONE=y. I don't think there's a simple wrapper similar
to pfn_valid_within() for that, but it could be created (input: current
pfn, output: start pfn of next MAX_ORDER block if
CONFIG_HOLES_IN_ZONE=n, pfn+1 when CONFIG_HOLES_IN_ZONE=y).

>   Thanks
>   zhongjiang
>> However, read_page_owner() skips whole MAX_ORDER block as well in this
>> situation, so we won't be able to read the info anyway...
>>
>> Also the problem is not as simple as documenting MAX_ORDER_NR_PAGES vs
>> pabeblock_nr_pages. We discussed it year ago when this patch was first
>> posted, how skipping over holes would have to be made more robust, and
>> how architectures should define hole granularity to avoid checking each
>> individual pfn in what appears to be a hole, to see if the hole has ended.
>>
>>> Thanks
>>> zhongjiang
>>>
>>
>> .
>>
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>