Re: [PATCH] mem-hotplug: fix node spanned pages when we have a node with only zone_movable

[Oscar Salvador <osalvador@suse.de>]

On Tue, Apr 02, 2019 at 12:11:16PM +0800, Linxu Fang wrote:
> commit <342332e6a925> ("mm/page_alloc.c: introduce kernelcore=mirror
> option") and series patches rewrote the calculation of node spanned
> pages.
> commit <e506b99696a2> (mem-hotplug: fix node spanned pages when we have a
> movable node), but the current code still has problems,
> when we have a node with only zone_movable and the node id is not zero,
> the size of node spanned pages is double added.
> That's because we have an empty normal zone, and zone_start_pfn or
> zone_end_pfn is not between arch_zone_lowest_possible_pfn and
> arch_zone_highest_possible_pfn, so we need to use clamp to constrain the
> range just like the commit <96e907d13602> (bootmem: Reimplement
> __absent_pages_in_range() using for_each_mem_pfn_range()).

So, let me see if I understood this correctly:

When calling zone_spanned_pages_in_node() for any node which is not node 0,

> *zone_start_pfn = arch_zone_lowest_possible_pfn[zone_type];
> *zone_end_pfn = arch_zone_highest_possible_pfn[zone_type];

will actually set zone_start_pfn/zone_end_pfn to the values from node0's
ZONE_NORMAL?

So we use clamp to actually check if such values fall within what node1's
memory spans, and ignore them otherwise?

Btw, mem-hotplug does not hit this path anymore.


> 
> e.g.
> Zone ranges:
>   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
>   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
>   Normal   [mem 0x0000000100000000-0x000000023fffffff]
> Movable zone start for each node
>   Node 0: 0x0000000100000000
>   Node 1: 0x0000000140000000
> Early memory node ranges
>   node   0: [mem 0x0000000000001000-0x000000000009efff]
>   node   0: [mem 0x0000000000100000-0x00000000bffdffff]
>   node   0: [mem 0x0000000100000000-0x000000013fffffff]
>   node   1: [mem 0x0000000140000000-0x000000023fffffff]
> 
> node 0 DMA	spanned:0xfff   present:0xf9e   absent:0x61
> node 0 DMA32	spanned:0xff000 present:0xbefe0	absent:0x40020
> node 0 Normal	spanned:0	present:0	absent:0
> node 0 Movable	spanned:0x40000 present:0x40000 absent:0
> On node 0 totalpages(node_present_pages): 1048446
> node_spanned_pages:1310719
> node 1 DMA	spanned:0	    present:0		absent:0
> node 1 DMA32	spanned:0	    present:0		absent:0
> node 1 Normal	spanned:0x100000    present:0x100000	absent:0
> node 1 Movable	spanned:0x100000    present:0x100000	absent:0
> On node 1 totalpages(node_present_pages): 2097152
> node_spanned_pages:2097152
> Memory: 6967796K/12582392K available (16388K kernel code, 3686K rwdata,
> 4468K rodata, 2160K init, 10444K bss, 5614596K reserved, 0K
> cma-reserved)
> 
> It shows that the current memory of node 1 is double added.
> After this patch, the problem is fixed.
> 
> node 0 DMA	spanned:0xfff   present:0xf9e   absent:0x61
> node 0 DMA32	spanned:0xff000 present:0xbefe0	absent:0x40020
> node 0 Normal	spanned:0	present:0	absent:0
> node 0 Movable	spanned:0x40000 present:0x40000 absent:0
> On node 0 totalpages(node_present_pages): 1048446
> node_spanned_pages:1310719
> node 1 DMA	spanned:0	    present:0		absent:0
> node 1 DMA32	spanned:0	    present:0		absent:0
> node 1 Normal	spanned:0	    present:0		absent:0
> node 1 Movable	spanned:0x100000    present:0x100000	absent:0
> On node 1 totalpages(node_present_pages): 1048576
> node_spanned_pages:1048576
> memory: 6967796K/8388088K available (16388K kernel code, 3686K rwdata,
> 4468K rodata, 2160K init, 10444K bss, 1420292K reserved, 0K
> cma-reserved)
> 
> Signed-off-by: Linxu Fang <fanglinxu@huawei.com>
> ---
>  mm/page_alloc.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 3eb01de..5cd0cb2 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -6233,13 +6233,15 @@ static unsigned long __init zone_spanned_pages_in_node(int nid,
>  					unsigned long *zone_end_pfn,
>  					unsigned long *ignored)
>  {
> +	unsigned long zone_low = arch_zone_lowest_possible_pfn[zone_type];
> +	unsigned long zone_high = arch_zone_highest_possible_pfn[zone_type];
>  	/* When hotadd a new node from cpu_up(), the node should be empty */
>  	if (!node_start_pfn && !node_end_pfn)
>  		return 0;
>  
>  	/* Get the start and end of the zone */
> -	*zone_start_pfn = arch_zone_lowest_possible_pfn[zone_type];
> -	*zone_end_pfn = arch_zone_highest_possible_pfn[zone_type];
> +	*zone_start_pfn = clamp(node_start_pfn, zone_low, zone_high);
> +	*zone_end_pfn = clamp(node_end_pfn, zone_low, zone_high);
>  	adjust_zone_range_for_zone_movable(nid, zone_type,
>  				node_start_pfn, node_end_pfn,
>  				zone_start_pfn, zone_end_pfn);
> -- 
> 1.8.5.6
> 
> 

-- 
Oscar Salvador
SUSE L3