[PATCH] mm: respect pre-allocated storage mapping for memmap

[PATCH] mm: respect pre-allocated storage mapping for memmap

[zhongjiang]

From: zhong jiang <zhongjiang@huawei.com>

At present, we skip the reservation storage by the driver for
the zone_dvice. but the free pages set aside for the memmap is
ignored. And since the free pages is only used as the memmap,
so we can also skip the corresponding pages.

Signed-off-by: zhong jiang <zhongjiang@huawei.com>
---
 mm/page_alloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index d604d25..51d8d03 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5047,7 +5047,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
 	 * memory
 	 */
 	if (altmap && start_pfn == altmap->base_pfn)
-		start_pfn += altmap->reserve;
+		start_pfn += vmem_altmap_offset(altmap);
 
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 		/*
-- 
1.8.3.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[Anshuman Khandual]

On 01/16/2017 07:08 PM, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
> 
> At present, we skip the reservation storage by the driver for
> the zone_dvice. but the free pages set aside for the memmap is
> ignored. And since the free pages is only used as the memmap,
> so we can also skip the corresponding pages.

But these free pages used for memmap mapping should also be accounted
inside the zone, no ?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[zhong jiang]

On 2017/1/16 22:43, Anshuman Khandual wrote:
> On 01/16/2017 07:08 PM, zhongjiang wrote:
>> From: zhong jiang <zhongjiang@huawei.com>
>>
>> At present, we skip the reservation storage by the driver for
>> the zone_dvice. but the free pages set aside for the memmap is
>> ignored. And since the free pages is only used as the memmap,
>> so we can also skip the corresponding pages.
> But these free pages used for memmap mapping should also be accounted
> inside the zone, no ?
  That's confusing.  because first_pfn for zone_device after reserve and free page for
 memmap mapping.  That is used as a actuallly pfn for zone_device.

 Thanks
 zhongjiang
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[Michal Hocko]

On Mon 16-01-17 21:38:05, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
> 
> At present, we skip the reservation storage by the driver for
> the zone_dvice. but the free pages set aside for the memmap is
> ignored. And since the free pages is only used as the memmap,
> so we can also skip the corresponding pages.

I have really hard time to understand what this patch does and why it
matters.  Could you please rephrase the changelog to state, the problem,
how it affects users and what is the fix please?
 
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> ---
>  mm/page_alloc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index d604d25..51d8d03 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5047,7 +5047,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>  	 * memory
>  	 */
>  	if (altmap && start_pfn == altmap->base_pfn)
> -		start_pfn += altmap->reserve;
> +		start_pfn += vmem_altmap_offset(altmap);
>  
>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>  		/*
> -- 
> 1.8.3.1
> 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[zhong jiang]

On 2017/1/17 18:25, Michal Hocko wrote:
> On Mon 16-01-17 21:38:05, zhongjiang wrote:
>> From: zhong jiang <zhongjiang@huawei.com>
>>
>> At present, we skip the reservation storage by the driver for
>> the zone_dvice. but the free pages set aside for the memmap is
>> ignored. And since the free pages is only used as the memmap,
>> so we can also skip the corresponding pages.
> I have really hard time to understand what this patch does and why it
> matters.  Could you please rephrase the changelog to state, the problem,
> how it affects users and what is the fix please?
>  
  Hi, Michal
 
  The patch maybe incorrect if free pages for memmap mapping is accouted for zone_device.
  I am just a little confusing about the implement.  it maybe simple and  stupid.

  first pfn for dev_mappage come from vmem_altmap_offset, and free pages reserved for
  memmap mapping need to be accounted. I do not know the meaning.

  Another issue is in sparse_remove_one_section.  A section belongs to  zone_device is not
  always need to consider the  map_offset. is it right ?  From pfn_first to end , that section
  should no need to consider the map_offet.

  Thanks
  zhongjiang
 
>> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
>> ---
>>  mm/page_alloc.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index d604d25..51d8d03 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -5047,7 +5047,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>>  	 * memory
>>  	 */
>>  	if (altmap && start_pfn == altmap->base_pfn)
>> -		start_pfn += altmap->reserve;
>> +		start_pfn += vmem_altmap_offset(altmap);
>>  
>>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>>  		/*
>> -- 
>> 1.8.3.1
>>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[Dan Williams]

On Tue, Jan 17, 2017 at 5:58 AM, zhong jiang <zhongjiang@huawei.com> wrote:
> On 2017/1/17 18:25, Michal Hocko wrote:
>> On Mon 16-01-17 21:38:05, zhongjiang wrote:
>>> From: zhong jiang <zhongjiang@huawei.com>
>>>
>>> At present, we skip the reservation storage by the driver for
>>> the zone_dvice. but the free pages set aside for the memmap is
>>> ignored. And since the free pages is only used as the memmap,
>>> so we can also skip the corresponding pages.
>> I have really hard time to understand what this patch does and why it
>> matters.  Could you please rephrase the changelog to state, the problem,
>> how it affects users and what is the fix please?
>>
>   Hi, Michal
>
>   The patch maybe incorrect if free pages for memmap mapping is accouted for zone_device.
>   I am just a little confusing about the implement.  it maybe simple and  stupid.

The patch is incorrect, the struct page initialization starts
immediately after altmap->reserve.

>   first pfn for dev_mappage come from vmem_altmap_offset, and free pages reserved for
>   memmap mapping need to be accounted. I do not know the meaning.
>
>   Another issue is in sparse_remove_one_section.  A section belongs to  zone_device is not
>   always need to consider the  map_offset. is it right ?  From pfn_first to end , that section
>   should no need to consider the map_offet.

No that's not right. devm_memremap_pages() will specify the full
physical address range that was initially hotplugged. At removal time
the first page of the memmap starts at pfn_to_page(phys_start_pfn +
map_offset).

However, I always need to remind myself of these rules every time I
read the code, so the documentation needs improvement.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[zhong jiang]

On 2017/1/18 1:15, Dan Williams wrote:
> On Tue, Jan 17, 2017 at 5:58 AM, zhong jiang <zhongjiang@huawei.com> wrote:
>> On 2017/1/17 18:25, Michal Hocko wrote:
>>> On Mon 16-01-17 21:38:05, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> At present, we skip the reservation storage by the driver for
>>>> the zone_dvice. but the free pages set aside for the memmap is
>>>> ignored. And since the free pages is only used as the memmap,
>>>> so we can also skip the corresponding pages.
>>> I have really hard time to understand what this patch does and why it
>>> matters.  Could you please rephrase the changelog to state, the problem,
>>> how it affects users and what is the fix please?
>>>
>>   Hi, Michal
>>
>>   The patch maybe incorrect if free pages for memmap mapping is accouted for zone_device.
>>   I am just a little confusing about the implement.  it maybe simple and  stupid.
> The patch is incorrect, the struct page initialization starts
> immediately after altmap->reserve.
>
>>   first pfn for dev_mappage come from vmem_altmap_offset, and free pages reserved for
>>   memmap mapping need to be accounted. I do not know the meaning.
>>
>>   Another issue is in sparse_remove_one_section.  A section belongs to  zone_device is not
>>   always need to consider the  map_offset. is it right ?  From pfn_first to end , that section
>>   should no need to consider the map_offet.
> No that's not right. devm_memremap_pages() will specify the full
> physical address range that was initially hotplugged. At removal time
> the first page of the memmap starts at pfn_to_page(phys_start_pfn +
> map_offset).
>
> However, I always need to remind myself of these rules every time I
> read the code, so the documentation needs improvement.
 The rules ensure that (reserve + free) need to less than one section size.
 if it is so, or we add WARNON to explicitly the limits.


 Thanks
 zhongjiang
> .
>
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[zhong jiang]

On 2017/1/18 1:15, Dan Williams wrote:
> On Tue, Jan 17, 2017 at 5:58 AM, zhong jiang <zhongjiang@huawei.com> wrote:
>> On 2017/1/17 18:25, Michal Hocko wrote:
>>> On Mon 16-01-17 21:38:05, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> At present, we skip the reservation storage by the driver for
>>>> the zone_dvice. but the free pages set aside for the memmap is
>>>> ignored. And since the free pages is only used as the memmap,
>>>> so we can also skip the corresponding pages.
>>> I have really hard time to understand what this patch does and why it
>>> matters.  Could you please rephrase the changelog to state, the problem,
>>> how it affects users and what is the fix please?
>>>
>>   Hi, Michal
>>
>>   The patch maybe incorrect if free pages for memmap mapping is accouted for zone_device.
>>   I am just a little confusing about the implement.  it maybe simple and  stupid.
> The patch is incorrect, the struct page initialization starts
> immediately after altmap->reserve.
>
>>   first pfn for dev_mappage come from vmem_altmap_offset, and free pages reserved for
>>   memmap mapping need to be accounted. I do not know the meaning.
>>
>>   Another issue is in sparse_remove_one_section.  A section belongs to  zone_device is not
>>   always need to consider the  map_offset. is it right ?  From pfn_first to end , that section
>>   should no need to consider the map_offet.
> No that's not right. devm_memremap_pages() will specify the full
> physical address range that was initially hotplugged. At removal time
> the first page of the memmap starts at pfn_to_page(phys_start_pfn +
> map_offset).
>
> However, I always need to remind myself of these rules every time I
> read the code, so the documentation needs improvement.
>
> .
>
Addtional, if we just offline some of sectiones of dev_mappage. That will have question.
Therefore,  we must offline the whole dev_mappages page that belongs to zone_device.

is it right?  please

Thanks
zhongjiang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: respect pre-allocated storage mapping for memmap

[Dan Williams]

On Wed, Jan 18, 2017 at 5:27 AM, zhong jiang <zhongjiang@huawei.com> wrote:
> On 2017/1/18 1:15, Dan Williams wrote:
>> On Tue, Jan 17, 2017 at 5:58 AM, zhong jiang <zhongjiang@huawei.com> wrote:
>>> On 2017/1/17 18:25, Michal Hocko wrote:
>>>> On Mon 16-01-17 21:38:05, zhongjiang wrote:
>>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>>
>>>>> At present, we skip the reservation storage by the driver for
>>>>> the zone_dvice. but the free pages set aside for the memmap is
>>>>> ignored. And since the free pages is only used as the memmap,
>>>>> so we can also skip the corresponding pages.
>>>> I have really hard time to understand what this patch does and why it
>>>> matters.  Could you please rephrase the changelog to state, the problem,
>>>> how it affects users and what is the fix please?
>>>>
>>>   Hi, Michal
>>>
>>>   The patch maybe incorrect if free pages for memmap mapping is accouted for zone_device.
>>>   I am just a little confusing about the implement.  it maybe simple and  stupid.
>> The patch is incorrect, the struct page initialization starts
>> immediately after altmap->reserve.
>>
>>>   first pfn for dev_mappage come from vmem_altmap_offset, and free pages reserved for
>>>   memmap mapping need to be accounted. I do not know the meaning.
>>>
>>>   Another issue is in sparse_remove_one_section.  A section belongs to  zone_device is not
>>>   always need to consider the  map_offset. is it right ?  From pfn_first to end , that section
>>>   should no need to consider the map_offet.
>> No that's not right. devm_memremap_pages() will specify the full
>> physical address range that was initially hotplugged. At removal time
>> the first page of the memmap starts at pfn_to_page(phys_start_pfn +
>> map_offset).
>>
>> However, I always need to remind myself of these rules every time I
>> read the code, so the documentation needs improvement.
>>
>> .
>>
> Addtional, if we just offline some of sectiones of dev_mappage. That will have question.
> Therefore,  we must offline the whole dev_mappages page that belongs to zone_device.
>
> is it right?  please
>

We never online ZONE_DEVICE memory so consequently it can never be
offlined. The only way to access the pages is via get_user_pages() of
a DAX mapping.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>