Re: [PATCH 0/4] Support dynamic (de)configuration of memory

[Sumanth Korikkar <sumanthk@linux.ibm.com>]

> > With the new interface, s390 will not add all possible hotplug memory in
> > advance, like before, to make it visible in sysfs for online/offline
> > actions. Instead, before memory block can be set online, it has to be
> > configured via a new interface in /sys/firmware/memory/memoryX/config,
> > which makes s390 similar to others.  i.e. Adding of hotpluggable memory is
> > controlled by the user instead of adding it at boottime.
> 
> Before I dig into the details, will onlining/offling still trigger
> hypervisor action, or does that now really happen when memory is
> added/removed?
> 
> That would be really nice, because it would remove the whole need for
> "standby" memory, and having to treat hotplugged memory differently under
> LPAR/z/VM than anywhere else (-> keep it offline).

With this approach, hypervisor actions are triggered only when memory is
actually added or removed.

Online and offline operations are common code memory hotplug actions and
the s390 memory notifier actions are none/minimal.

> > s390 kernel sysfs interface to configure/deconfigure memory with
> > memmap_on_memory (with upcoming lsmem changes):
> > * Initial memory layout:
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE                 SIZE   STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff   2G  online 0-15  yes        no
> > 0x80000000-0xffffffff   2G offline 16-31 no         yes
> 
> Could we instead modify "STATE" to reflect that it is "not added" / "not
> configured" / "disabled" etc?
> 
> Like
> 
> lsmem -o RANGE,SIZE,STATE,BLOCK,MEMMAP_ON_MEMORY
> RANGE                 SIZE    STATE BLOCK
> 0x00000000-0x7fffffff   2G   online 0-15
> 0x80000000-0xffffffff   2G disabled 16-31
> 
> Or is that an attempt to maintain backwards compatibility?

Mostly. Also, similar to lscpu output, where CPU status shows
CONFIGURED/STATE column.

Also, older scripts to get list of offline memory typically use:
lsmem | grep offline

and

chmem -e <SIZE> would work as usual, where <SIZE> specifies amount of
memory to set online.

chmem changes would look like:
chmem -c 128M -m 1 : configure memory with memmap-on-memory enabled
chmem -g 128M : deconfigure memory
chmem -e 128M : optionally configure (if supported by architecture) and
		always online memory
chmem -d 128M : offline and optionally deconfigure memory (if supported
		by architecture)

> > * Configure memory
> > echo 1 > /sys/firmware/memory/memory16/config
> 
> The granularity here is also memory_block_size_bytes(), correct?

Yes, correct.

> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE                  SIZE  STATE   BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff    2G  online  0-15  yes        no
> > 0x80000000-0x87ffffff  128M offline    16  yes        yes
> > 0x88000000-0xffffffff  1.9G offline 17-31  no         yes
> > 
> > * Deconfigure memory
> > echo 0 > /sys/firmware/memory/memory16/config
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE                 SIZE   STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff   2G  online 0-15  yes        no
> > 0x80000000-0xffffffff   2G offline 16-31 no         yes
> > 
> > * Enable memmap_on_memory and online it.
> > (Deconfigure first)
> > echo 0 > /sys/devices/system/memory/memory5/online
> > echo 0 > /sys/firmware/memory/memory5/config
> > 
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE                  SIZE  STATE  BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x27ffffff  640M  online 0-4   yes        no
> > 0x28000000-0x2fffffff  128M offline 5     no         no
> > 0x30000000-0x7fffffff  1.3G  online 6-15  yes        no
> > 0x80000000-0xffffffff    2G offline 16-31 no         yes
> > 
> > (Enable memmap_on_memory and online it)
> > echo 1 > /sys/firmware/memory/memory5/memmap_on_memory
> > echo 1 > /sys/firmware/memory/memory5/config
> > echo 1 > /sys/devices/system/memory/memory5/online
> 
> I guess the use for memmap_on_memory would now be limited to making hotplug
> more likely to succeed in OOM scenarios.

Yes. with memmap-on-memory enabled, mainly in OOM situations.

However, it also provides flexibility to the user to configure few
memory blocks with memmap-on-memory enabled and few with
memmap-on-memory disabled (When the user needs continuous physical
memory across memory blocks).

> > Patch 4 removes the MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers. It
> > is no longer needed.  Memory can be brought to accessible state before
> > adding memory now, with runtime (de)configuration of memory.
> 
> Nice.

Thank you David