Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug

[Yang Shi <shy828301@gmail.com>]

>
> >
> > 2) if memmap_on_memory is on, and hotplug capacity (node1) is
> >     zone_movable - then each memory block (256MB) should appear
> >     as 252MB (-4MB of 64-byte page structs).  For 256GB (my system)
> >     I should see a total of 252GB of onlined memory (-4GB of page struct)
>
> In memory_block_online(), we have:
>
>         /*
>          * Account once onlining succeeded. If the zone was unpopulated, it is
>          * now already properly populated.
>          */
>         if (nr_vmemmap_pages)
>                 adjust_present_page_count(pfn_to_page(start_pfn), mem->group,
>                                           nr_vmemmap_pages);
>
> So we'd add the vmemmap pages to
> * zone->present_pages
> * zone->zone_pgdat->node_present_pages
>
> (mhp_init_memmap_on_memory() moved the vmemmap pages to ZONE_MOVABLE)
>
> However, we don't add them to
> * zone->managed_pages
> * totalram pages
>
> /proc/zoneinfo would show them as present but not managed.
> /proc/meminfo would not include them in MemTotal
>
> We could adjust the latter two, if there is a problem.
> (just needs some adjust_managed_page_count() calls)
>
> So yes, staring at MemTotal, you should see an increase by 252 MiB right now.
>
> >
> >     2a) when dropping these memory blocks, I should see node0 memory use
> >         stay the same - since it was vmemmap usage.
>
> Yes.
>
> >
> > I will double check that this isn't working as expected, and i'll double
> > check for a build option as well.
> >
> > stupid question - it sorta seems like you'd want this as the default
> > setting for driver-managed hotplug memory blocks, but I suppose for
> > very small blocks there's problems (as described in the docs).
>
> The issue is that it is per-memblock. So you'll never have 1 GiB ranges
> of consecutive usable memory (e.g., 1 GiB hugetlb page).

Regardless of ZONE_MOVABLE or ZONE_NORMAL, right?

Thanks,
Yang

>
> >
> > :thinking: - is it silly to suggest maybe a per-driver memmap_on_memory
> > setting rather than just a global setting?  For CXL capacity, this seems
> > like a no-brainer since blocks can't be smaller than 256MB (per spec).
>
> I thought we had that? See MHP_MEMMAP_ON_MEMORY set by dax/kmem.
>
> IIRC, the global toggle must be enabled for the driver option to be considered.
>
> --
> Cheers,
>
> David / dhildenb
>