Hello,

Starting linux-next20240827 build, running QEMU KVM instance with option to freeze the CPU at the very beginning and then continue results in below guest crash. Disabling CONFIG_KSM will make this issue go away. Same issue exists with todays linux-next as well.

SecCoreStartupWithStack(0xFFFCC000, 0x820000)
error: kvm run failed Bad address
RAX=0000000000232000 RBX=00000000fffdb101 RCX=000000000081fab0 RDX=00000000fffd7b03
RSI=0000000000807000 RDI=00000000fffd262b RBP=000000000081fc00 RSP=000000000081faa0
R8 =00000000fffd7af7 R9 =0000000000000071 R10=0000000000400000 R11=0000000000000000
R12=00000000fffcc094 R13=00000000fffcc000 R14=00000000ffdce000 R15=0000000000000600
RIP=00000000fffd1cb5 RFL=00000087 [--S--PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
CS =0038 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
FS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
GS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     00000000fffffed0 0000003f
IDT=     000000000081fd70 0000021f
CR0=80000033 CR2=0000000000000000 CR3=0000000000800000 CR4=00000660
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000500
Code=05 00 10 00 00 49 81 ee 00 10 00 00 48 3d 00 00 00 02 77 12 <41> 81 7e 28 5f 46 56 48 75 e1 49 3b 46 20 73 05 eb d9 4d 89 ee 48 8d 8d 10 ff ff ff ba 02

Host dmesg throws:

[  232.158038] BUG: Bad page state in process ksmd  pfn:404740a
[  232.164393] page: refcount:0 mapcount:1 mapping:0000000000000000 index:0x7f8687c0a pfn:0x404740a
[  232.164401] flags: 0x17ffffc0020819(locked|uptodate|dirty|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
[  232.164412] raw: 0017ffffc0020819 dead000000000100 dead000000000122 0000000000000000
[  232.164417] raw: 00000007f8687c0a 0000000000000000 0000000000000000 0000000000000000
[  232.164420] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  232.164423] Modules linked in: xt_CHECKSUM ipt_REJECT amd_atl intel_rapl_msr intel_rapl_common amd64_edac edac_mce_amd kvm_amd ipmi_ssif nls_iso8859_1 wmi_bmof rapl joydev input_leds ccp acpi_ipmi k10temp ipmi_si wmi mac_hid sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_devintf ipmi_msghandler msr drm efi_pstore autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 crct10dif_pclmul ahci crc32_pclmul tg3 ghash_clmulni_intel libahci i2c_piix4 i2c_smbus hid_generic usbhid hid aesni_intel crypto_simd cryptd
[  232.164556] CPU: 114 UID: 0 PID: 1594 Comm: ksmd Not tainted 6.11.0-rc5-next-20240829 #1
[  232.164564] Hardware name: AMD Corporation RUBY/RUBY, BIOS TRR100BD 12/11/2023
[  232.164568] Call Trace:
[  232.164572]  <TASK>
[  232.164580]  dump_stack_lvl+0x70/0x90
[  232.164593]  dump_stack+0x14/0x20
[  232.164601]  bad_page+0x71/0x100
[  232.164611]  free_page_is_bad_report+0x86/0x90
[  232.164618]  free_unref_page+0x3e4/0x5b0
[  232.164624]  ? srso_alias_return_thunk+0x5/0xfbef5
[  232.164632]  ? __mem_cgroup_uncharge+0x64/0x80
[  232.164641]  __folio_put+0xc4/0xf0
[  232.164649]  ksm_scan_thread+0x1279/0x23d0
[  232.164662]  ? try_to_wake_up+0x244/0x740
[  232.164675]  ? __pfx_ksm_scan_thread+0x10/0x10
[  232.164681]  kthread+0xe8/0x120
[  232.164687]  ? __pfx_kthread+0x10/0x10
[  232.164694]  ret_from_fork+0x40/0x60
[  232.164702]  ? __pfx_kthread+0x10/0x10
[  232.164707]  ret_from_fork_asm+0x1a/0x30
[  232.164721]  </TASK>
[  232.164724] Disabling lock debugging due to kernel taint
[  234.206074] BUG: Bad page state in process ksmd  pfn:18854a
[  234.212327] page: refcount:0 mapcount:1 mapping:0000000000000000 index:0x7f869274a pfn:0x18854a
[  234.212334] flags: 0x17ffffc0020819(locked|uptodate|dirty|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
[  234.212345] raw: 0017ffffc0020819 dead000000000100 dead000000000122 0000000000000000
[  234.212350] raw: 00000007f869274a 0000000000000000 0000000000000000 0000000000000000
[  234.212353] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  234.212356] Modules linked in: xt_CHECKSUM ipt_REJECT amd_atl intel_rapl_msr intel_rapl_common amd64_edac edac_mce_amd kvm_amd ipmi_ssif nls_iso8859_1 wmi_bmof rapl joydev input_leds ccp acpi_ipmi k10temp ipmi_si wmi mac_hid sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_devintf ipmi_msghandler msr drm efi_pstore autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 crct10dif_pclmul ahci crc32_pclmul tg3 ghash_clmulni_intel libahci i2c_piix4 i2c_smbus hid_generic usbhid hid aesni_intel crypto_simd cryptd
[  234.212490] CPU: 114 UID: 0 PID: 1594 Comm: ksmd Tainted: G    B              6.11.0-rc5-next-20240829 #1
[  234.212498] Tainted: [B]=BAD_PAGE
[  234.212502] Hardware name: AMD Corporation RUBY/RUBY, BIOS TRR100BD 12/11/2023
[  234.212505] Call Trace:
[  234.212510]  <TASK>
[  234.212517]  dump_stack_lvl+0x70/0x90
[  234.212531]  dump_stack+0x14/0x20
[  234.212538]  bad_page+0x71/0x100
[  234.212548]  free_page_is_bad_report+0x86/0x90
[  234.212556]  free_unref_page+0x3e4/0x5b0
[  234.212562]  ? srso_alias_return_thunk+0x5/0xfbef5
[  234.212569]  ? __mem_cgroup_uncharge+0x64/0x80
[  234.212579]  __folio_put+0xc4/0xf0
[  234.212587]  ksm_scan_thread+0x1279/0x23d0
[  234.212599]  ? try_to_wake_up+0x244/0x740
[  234.212612]  ? __pfx_ksm_scan_thread+0x10/0x10
[  234.212618]  kthread+0xe8/0x120
[  234.212625]  ? __pfx_kthread+0x10/0x10
[  234.212631]  ret_from_fork+0x40/0x60
[  234.212639]  ? __pfx_kthread+0x10/0x10
[  234.212645]  ret_from_fork_asm+0x1a/0x30
[  234.212658]  </TASK>
[  305.071553] ------------[ cut here ]------------
[  305.071561] WARNING: CPU: 191 PID: 3957 at mm/gup.c:144 try_grab_folio+0x7d/0xa0
[  305.071577] Modules linked in: xt_CHECKSUM ipt_REJECT amd_atl intel_rapl_msr intel_rapl_common amd64_edac edac_mce_amd kvm_amd ipmi_ssif nls_iso8859_1 wmi_bmof rapl joydev input_leds ccp acpi_ipmi k10temp ipmi_si wmi mac_hid sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_devintf ipmi_msghandler msr drm efi_pstore autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 crct10dif_pclmul ahci crc32_pclmul tg3 ghash_clmulni_intel libahci i2c_piix4 i2c_smbus hid_generic usbhid hid aesni_intel crypto_simd cryptd
[  305.071714] CPU: 191 UID: 0 PID: 3957 Comm: CPU 0/KVM Tainted: G    B              6.11.0-rc5-next-20240829 #1
[  305.071723] Tainted: [B]=BAD_PAGE
[  305.071726] Hardware name: AMD Corporation RUBY/RUBY, BIOS TRR100BD 12/11/2023
[  305.071730] RIP: 0010:try_grab_folio+0x7d/0xa0
[  305.071737] Code: 00 48 c1 e8 36 48 8b 3c c5 40 db 54 8e e8 bb 4e fe ff 31 c0 5d e9 3e b5 0b 01 f0 01 77 34 31 c0 e9 33 b5 0b 01 e9 2e b5 0b 01 <0f> 0b b8 f4 ff ff ff e9 22 b5 0b 01 89 f0 c1 e0 0a f0 01 47 34 eb
[  305.071742] RSP: 0018:ff57d354086837e0 EFLAGS: 00010246
[  305.071748] RAX: 0000000000000000 RBX: 00007f869274a000 RCX: dead000000000100
[  305.071753] RDX: 0000000000211052 RSI: 0000000000000001 RDI: ffe7c16546215280
[  305.071756] RBP: ff57d35408683838 R08: ffe7c16546215280 R09: 0000000000000000
[  305.071760] R10: 00007f8692220000 R11: 00007f8698020fff R12: 0000000000211052
[  305.071763] R13: ffe7c16546215280 R14: ff1ce69c31cadd78 R15: 800800018854a867
[  305.071767] FS:  00007fb48aa00640(0000) GS:ff1ce6da44380000(0000) knlGS:0000000000000000
[  305.071772] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  305.071775] CR2: 0000000000000000 CR3: 000800011a14c006 CR4: 0000000000771ef0
[  305.071780] PKRU: 00000000
[  305.071783] Call Trace:
[  305.071786]  <TASK>
[  305.071793]  ? show_regs+0x6d/0x80
[  305.071804]  ? __warn+0x91/0x140
[  305.071810]  ? try_grab_folio+0x7d/0xa0
[  305.071817]  ? report_bug+0x193/0x1a0
[  305.071828]  ? handle_bug+0x63/0xa0
[  305.071837]  ? exc_invalid_op+0x1d/0x80
[  305.071842]  ? asm_exc_invalid_op+0x1f/0x30
[  305.071856]  ? try_grab_folio+0x7d/0xa0
[  305.071863]  ? follow_page_pte+0x11d/0x650
[  305.071872]  __get_user_pages+0x463/0x15b0
[  305.071879]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.071884]  ? write_mmio+0x68/0x110
[  305.071899]  get_user_pages_unlocked+0xf0/0x360
[  305.071909]  hva_to_pfn+0x10f/0x4f0
[  305.071918]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.071923]  ? xas_load+0x1b/0x100
[  305.071933]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.071940]  __gfn_to_pfn_memslot+0x9e/0x100
[  305.071946]  kvm_faultin_pfn+0x11d/0x690
[  305.071958]  kvm_tdp_page_fault+0x9b/0xf0
[  305.071966]  kvm_mmu_do_page_fault+0x22d/0x270
[  305.071978]  kvm_mmu_page_fault+0x8b/0x7a0
[  305.071984]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.071991]  ? svm_interrupt_blocked+0xa0/0x110 [kvm_amd]
[  305.072011]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072016]  ? kvm_arch_vcpu_put+0x37/0x200
[  305.072024]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072028]  ? vcpu_put+0x26/0x60
[  305.072035]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072039]  ? kvm_arch_vcpu_ioctl_run+0x614/0x1760
[  305.072050]  npf_interception+0x99/0x180 [kvm_amd]
[  305.072061]  ? __pfx_npf_interception+0x10/0x10 [kvm_amd]
[  305.072073]  svm_invoke_exit_handler+0x17b/0x1b0 [kvm_amd]
[  305.072086]  svm_handle_exit+0xa5/0x1e0 [kvm_amd]
[  305.072097]  ? svm_vcpu_run+0x2cd/0x850 [kvm_amd]
[  305.072109]  kvm_arch_vcpu_ioctl_run+0xd65/0x1760
[  305.072118]  ? fire_user_return_notifiers+0x46/0x70
[  305.072127]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072132]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072137]  ? kvm_on_user_return+0x8e/0x100
[  305.072146]  kvm_vcpu_ioctl+0x321/0x950
[  305.072153]  ? do_syscall_64+0x7b/0x110
[  305.072161]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072166]  ? kvm_vcpu_ioctl+0x172/0x950
[  305.072171]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072175]  ? kvm_on_user_return+0x8e/0x100
[  305.072184]  __x64_sys_ioctl+0x99/0xd0
[  305.072194]  x64_sys_call+0x1227/0x2140
[  305.072201]  do_syscall_64+0x6f/0x110
[  305.072208]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072213]  ? do_syscall_64+0x7b/0x110
[  305.072219]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072224]  ? do_syscall_64+0x7b/0x110
[  305.072229]  ? syscall_exit_to_user_mode+0x57/0x1b0
[  305.072237]  ? srso_alias_return_thunk+0x5/0xfbef5
[  305.072242]  ? do_syscall_64+0x7b/0x110
[  305.072248]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[  305.072255] RIP: 0033:0x7fb49171a94f
[  305.072261] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <41> 89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25 28 00
[  305.072265] RSP: 002b:00007fb48a9ff6f0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[  305.072272] RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fb49171a94f
[  305.072275] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000000f
[  305.072279] RBP: 00005592558079e0 R08: 0000000000000000 R09: 0000000000000000
[  305.072282] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[  305.072285] R13: 0000000000000001 R14: 0000000000000071 R15: 0000000000000000
[  305.072296]  </TASK>
[  305.072299] ---[ end trace 0000000000000000 ]---
[  312.173980] BUG: Bad page state in process ksmd  pfn:4047c06
[  312.180332] page: refcount:0 mapcount:1 mapping:0000000000000000 index:0x7f88c4606 pfn:0x4047c06
[  312.180339] flags: 0x17ffffc0020819(locked|uptodate|dirty|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
[  312.180350] raw: 0017ffffc0020819 dead000000000100 dead000000000122 0000000000000000
[  312.180355] raw: 00000007f88c4606 0000000000000000 0000000000000000 0000000000000000
[  312.180358] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  312.180361] Modules linked in: xt_CHECKSUM ipt_REJECT amd_atl intel_rapl_msr intel_rapl_common amd64_edac edac_mce_amd kvm_amd ipmi_ssif nls_iso8859_1 wmi_bmof rapl joydev input_leds ccp acpi_ipmi k10temp ipmi_si wmi mac_hid sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_devintf ipmi_msghandler msr drm efi_pstore autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 crct10dif_pclmul ahci crc32_pclmul tg3 ghash_clmulni_intel libahci i2c_piix4 i2c_smbus hid_generic usbhid hid aesni_intel crypto_simd cryptd
[  312.180494] CPU: 114 UID: 0 PID: 1594 Comm: ksmd Tainted: G    B   W          6.11.0-rc5-next-20240829 #1
[  312.180503] Tainted: [B]=BAD_PAGE, [W]=WARN
[  312.180507] Hardware name: AMD Corporation RUBY/RUBY, BIOS TRR100BD 12/11/2023
[  312.180510] Call Trace:
[  312.180515]  <TASK>
[  312.180522]  dump_stack_lvl+0x70/0x90
[  312.180536]  dump_stack+0x14/0x20
[  312.180544]  bad_page+0x71/0x100
[  312.180554]  free_page_is_bad_report+0x86/0x90
[  312.180561]  free_unref_page+0x3e4/0x5b0
[  312.180567]  ? srso_alias_return_thunk+0x5/0xfbef5
[  312.180575]  ? __mem_cgroup_uncharge+0x64/0x80
[  312.180584]  __folio_put+0xc4/0xf0
[  312.180593]  ksm_scan_thread+0x1279/0x23d0
[  312.180605]  ? try_to_wake_up+0x244/0x740
[  312.180618]  ? __pfx_ksm_scan_thread+0x10/0x10
[  312.180624]  kthread+0xe8/0x120
[  312.180631]  ? __pfx_kthread+0x10/0x10
[  312.180637]  ret_from_fork+0x40/0x60
[  312.180645]  ? __pfx_kthread+0x10/0x10
[  312.180651]  ret_from_fork_asm+0x1a/0x30
[  312.180664]  </TASK>

Steps to recreate:

1. Start a QEMU KVM instance with -S option, sample below:

qemu-system-x86_64 \
-S \
-name guest=vm,debug-threads=on \
-blockdev node-name=file_ovmf_code,driver=file,filename=OVMF_CODE.fd,auto-read-only=on,discard=unmap \
-blockdev node-name=drive_ovmf_code,driver=raw,read-only=on,file=file_ovmf_code \
-blockdev node-name=file_ovmf_vars,driver=file,filename=vm1_22_04-server_qcow2_filesystem_VARS.raw,auto-read-only=on,discard=unmap \
-blockdev node-name=drive_ovmf_vars,driver=raw,read-only=off,file=file_ovmf_vars \
-machine q35,kernel_irqchip=split,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars,memory-backend=mem-machine_mem \
-cpu EPYC-Genoa,+svm,+x2apic \
-m 2048 \
-object memory-backend-ram,size=2048M,id=mem-machine_mem  \
-smp 1,maxcpus=1,cores=1,threads=1,dies=1,sockets=1  \
-kernel bzImage \
-append "root=/dev/sda rw console=ttyS0 net.ifnames=0 biosdevname=0 movable_node swiotlb=65536 " \
-drive id=disk0,file=22.04-server.qcow2,format=qcow2,if=none \
-device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true \
-device scsi-hd,drive=disk0 \
--enable-kvm \
--nographic \
-nic user,model=virtio-net-pci \
-vga none \
-monitor unix:qemu-monitor-socket,server,nowait
2. Now get onto qemu monitor and issue continue command, then the qemu instance crashes with above mentioned traces.

I have attached the dmesg and kconfig used.

Thanks,

Srikanth Aithal <sraithal@amd.com>