Re: [1/7,v8] NUMA Hotplug Emulator: documentation

[Eric B Munson <emunson@mgebm.net>]

[-- Attachment #1: Type: text/plain, Size: 6409 bytes --]

Shaohui,

I have had some success.  I had run into confusion on the memory hotplug with 
which files to be using to online memory.  The latest patch sorted it out for me
and I can now online disabled memory in new nodes.  I still cannot online an offlined
cpu.  Of the 12 available thread, I have 8 activated on boot with the kernel command line:

mem=8G numa=possible=12 maxcpus=8 cpu_hpe=on

I can offline a CPU just fine according to the kernel:
root@bert:/sys/devices/system/cpu# echo 7 > release
(dmesg)
[  911.494852] offline cpu 7.
[  911.694323] CPU 7 is now offline

But when I try and re-add it I get an error:
root@bert:/sys/devices/system/cpu# echo 0 > probe
(dmesg)
Dec  8 10:41:55 bert kernel: [ 1190.095051] ------------[ cut here ]------------
Dec  8 10:41:55 bert kernel: [ 1190.095056] WARNING: at fs/sysfs/dir.c:451 sysfs_add_one+0xce/0x180()
Dec  8 10:41:55 bert kernel: [ 1190.095057] Hardware name: System Product Name
Dec  8 10:41:55 bert kernel: [ 1190.095058] sysfs: cannot create duplicate filename '/devices/system/cpu/cpu7'
Dec  8 10:41:55 bert kernel: [ 1190.095060] Modules linked in: nfs binfmt_misc lockd fscache nfs_acl auth_rpcgss sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek radeon snd_hda_intel snd_hda_codec snd_cmipci gameport snd_pcm ttm snd_opl3_lib drm_kms_helper snd_hwdep snd_mpu401_uart drm uvcvideo snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq xhci_hcd snd_timer videodev snd_seq_device snd psmouse i7core_edac i2c_algo_bit edac_core joydev v4l1_compat shpchp snd_page_alloc v4l2_compat_ioctl32 soundcore hwmon_vid asus_atk0110 max6650 serio_raw hid_microsoft usbhid hid firewire_ohci firewire_core crc_itu_t ahci sky2 libahci
Dec  8 10:41:55 bert kernel: [ 1190.095088] Pid: 2369, comm: bash Tainted: G        W   2.6.37-rc5-numa-test+ #3
Dec  8 10:41:55 bert kernel: [ 1190.095089] Call Trace:
Dec  8 10:41:55 bert kernel: [ 1190.095094]  [<ffffffff8105eb1f>] warn_slowpath_common+0x7f/0xc0
Dec  8 10:41:55 bert kernel: [ 1190.095096]  [<ffffffff8105ec16>] warn_slowpath_fmt+0x46/0x50
Dec  8 10:41:55 bert kernel: [ 1190.095098]  [<ffffffff811cf77e>] sysfs_add_one+0xce/0x180
Dec  8 10:41:55 bert kernel: [ 1190.095100]  [<ffffffff811cf8b1>] create_dir+0x81/0xd0
Dec  8 10:41:55 bert kernel: [ 1190.095102]  [<ffffffff811cf97d>] sysfs_create_dir+0x7d/0xd0
Dec  8 10:41:55 bert kernel: [ 1190.095106]  [<ffffffff815a2b3d>] ? sub_preempt_count+0x9d/0xd0
Dec  8 10:41:55 bert kernel: [ 1190.095109]  [<ffffffff812c9ffd>] kobject_add_internal+0xbd/0x200
Dec  8 10:41:55 bert kernel: [ 1190.095111]  [<ffffffff812ca258>] kobject_add_varg+0x38/0x60
Dec  8 10:41:55 bert kernel: [ 1190.095113]  [<ffffffff812ca2d3>] kobject_init_and_add+0x53/0x70
Dec  8 10:41:55 bert kernel: [ 1190.095117]  [<ffffffff8139475f>] sysdev_register+0x6f/0xf0
Dec  8 10:41:55 bert kernel: [ 1190.095121]  [<ffffffff81598f38>] register_cpu_node+0x32/0x88
Dec  8 10:41:55 bert kernel: [ 1190.095123]  [<ffffffff8158207e>] arch_register_cpu_node+0x3e/0x40
Dec  8 10:41:55 bert kernel: [ 1190.095127]  [<ffffffff8101220e>] arch_cpu_probe+0x10e/0x1f0
Dec  8 10:41:55 bert kernel: [ 1190.095129]  [<ffffffff813989d4>] cpu_probe_store+0x14/0x20
Dec  8 10:41:55 bert kernel: [ 1190.095131]  [<ffffffff81393ef0>] sysdev_class_store+0x20/0x30
Dec  8 10:41:55 bert kernel: [ 1190.095133]  [<ffffffff811cd925>] sysfs_write_file+0xe5/0x170
Dec  8 10:41:55 bert kernel: [ 1190.095137]  [<ffffffff811624c8>] vfs_write+0xc8/0x190
Dec  8 10:41:55 bert kernel: [ 1190.095139]  [<ffffffff81162e61>] sys_write+0x51/0x90
Dec  8 10:41:55 bert kernel: [ 1190.095142]  [<ffffffff8100c142>] system_call_fastpath+0x16/0x1b
Dec  8 10:41:55 bert kernel: [ 1190.095144] ---[ end trace f615c2a524d318ea ]---
Dec  8 10:41:55 bert kernel: [ 1190.095149] Pid: 2369, comm: bash Tainted: G        W   2.6.37-rc5-numa-test+ #3
Dec  8 10:41:55 bert kernel: [ 1190.095150] Call Trace:
Dec  8 10:41:55 bert kernel: [ 1190.095152]  [<ffffffff812ca09b>] kobject_add_internal+0x15b/0x200
Dec  8 10:41:55 bert kernel: [ 1190.095154]  [<ffffffff812ca258>] kobject_add_varg+0x38/0x60
Dec  8 10:41:55 bert kernel: [ 1190.095156]  [<ffffffff812ca2d3>] kobject_init_and_add+0x53/0x70
Dec  8 10:41:55 bert kernel: [ 1190.095158]  [<ffffffff8139475f>] sysdev_register+0x6f/0xf0
Dec  8 10:41:55 bert kernel: [ 1190.095160]  [<ffffffff81598f38>] register_cpu_node+0x32/0x88
Dec  8 10:41:55 bert kernel: [ 1190.095162]  [<ffffffff8158207e>] arch_register_cpu_node+0x3e/0x40
Dec  8 10:41:55 bert kernel: [ 1190.095164]  [<ffffffff8101220e>] arch_cpu_probe+0x10e/0x1f0
Dec  8 10:41:55 bert kernel: [ 1190.095166]  [<ffffffff813989d4>] cpu_probe_store+0x14/0x20
Dec  8 10:41:55 bert kernel: [ 1190.095168]  [<ffffffff81393ef0>] sysdev_class_store+0x20/0x30
Dec  8 10:41:55 bert kernel: [ 1190.095170]  [<ffffffff811cd925>] sysfs_write_file+0xe5/0x170
Dec  8 10:41:55 bert kernel: [ 1190.095172]  [<ffffffff811624c8>] vfs_write+0xc8/0x190
Dec  8 10:41:55 bert kernel: [ 1190.095174]  [<ffffffff81162e61>] sys_write+0x51/0x90
Dec  8 10:41:55 bert kernel: [ 1190.095176]  [<ffffffff8100c142>] system_call_fastpath+0x16/0x1b

Am I doing something wrong?

Thanks,
Eric


On Wed, 08 Dec 2010, Shaohui Zheng wrote:

> On Tue, Dec 07, 2010 at 11:24:20AM -0700, Eric B Munson wrote:
> > Shaohui,
> > 
> > The documentation patch seems to be stale, it needs to be updated to match the
> > new file names.
> > 
> Eric,
> 	the major change on the patchset is on the interface, for the v8 emulator,
> we accept David's per-node debugfs add_memory interface, we already included
> in the documentation patch. the change is very small, so it is not obvious.
> 
> This is the change on the documentation compare with v7:
> +3) Memory hotplug emulation:
> +
> +The emulator reserves memory before OS boots, the reserved memory region is
> +removed from e820 table. Each online node has an add_memory interface, and
> +memory can be hot-added via the per-ndoe add_memory debugfs interface.
> +
> +The difficulty of Memory Release is well-known, we have no plan for it until
> +now.
> +
> + - reserve memory thru a kernel boot paramter
> + 	mem=1024m
> +
> + - add a memory section to node 3
> +    # echo 0x40000000 > mem_hotplug/node3/add_memory
> +	OR
> +    # echo 1024m > mem_hotplug/node3/add_memory
> +
> 
> -- 
> Thanks & Regards,
> Shaohui
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]