Memory space was fine.
I have big mem usage as i allocating 95% memory into huge pages 1G
so the rest is 32-64GB of memory for system. Anyway i had plenty at that time when this happened
It was on kernel Kernel: 6.18.8-pbk

Claude was able to remind me that issue:
walk_pgd_range BUG on PVE12 — Crash Details

  Server: PVE12 (10.10.42.12)
  Date: February 4, 2026, 21:42:16 UTC
  Kernel: 6.18.8-pbk (custom build, compiled January 30, 2026) - this is from https://prebuiltkernels.com/

  Error:
  BUG: unable to handle page fault for address: ff164aee00000000
  #PF: supervisor read access in kernel mode
  #PF: error_code(0x0000) - not-present page
  PGD 4602067 P4D 0
  Oops: 0000 [#1] SMP NOPTI
  CPU: 125  UID: 0  PID: 783442  Comm: qm  Not tainted 6.18.8-pbk #1 PREEMPT(voluntary)
  Hardware: TURIN2D24G-2L+/500W (AMD Turin platform)
  RIP: 0010:walk_pgd_range+0x6ff/0xbb0

  Call Trace:
  walk_pgd_range+0x6ff/0xbb0
   → __walk_page_range+0x8e/0x220
    → walk_page_vma+0x92/0xe0
     → smap_gather_stats.part.0+0x8c/0xd0
      → show_smaps_rollup+0x258/0x420
       → seq_read_iter → seq_read → vfs_read → ksys_read
        → __x64_sys_read → do_syscall_64 → entry_SYSCALL_64

  Root Cause Analysis:
  The qm process (Proxmox QEMU VM manager, PID 783442) triggered a kernel page fault while reading /proc/<pid>/smaps_rollup. The fault occurred in walk_pgd_range() when the kernel attempted to dereference address ff164aee00000000 (stored in R12), which was a not-present page. The corrupted
  pointer pattern suggests a stale or corrupted page table entry.

  Likely Causes (ranked):
  1. Race condition between KVM/QEMU VM memory operations and /proc/smaps_rollup reads
  2. Bug in vanilla kernel 6.18.x page table handling with VFIO passthrough (mlx5, vfio-pci modules were loaded)
  3. Possible instability introduced by custom -pbk kernel patches
  4. Memory corruption from VFIO PCI passthrough affecting page table structures

  Key Registers at Crash:
  - R12 (faulting pointer): ff164aee00000000
  - CR2 (fault address): ff164aee00000000
  - RSI: 000071afffffffff (end of VMA range)
  - R14: 000071afc0000000 (start of VMA range)

  Free Memory: Not checked during that session — no memory statistics were collected.

  Previous Kernel (boot -1): Same version 6.18.8-pbk — the system was rebooted after the crash and came back on the same kernel.


CleanShot 2026-03-05 at 12.25.32@2x.png

--

tel. 790 202 300

Tytus Rogalewski

Dolina Krzemowa 6A

83-010 Jagatowo

NIP: 9570976234



czw., 5 mar 2026 o 12:18 David Hildenbrand (Arm) <david@kernel.org> napisał(a):
On 3/5/26 09:11, Tytus Rogalewski wrote:
> Hi David,

Hi,

let me CC Jens.

@Jens, thread starts here:

https://marc.info/?l=linux-mm&m=176961245128225&w=2

For some reason, I cannot find that mail on lore, only my reply here:

https://lore.kernel.org/all/5948f3a6-8f30-4c45-9b86-2af9a6b37405@kernel.org/

>
> This is strange but the issue stopped when i changed io_uring to threads.
> Would that make any sense ? 

On the screenshot I can see that "Async IO" is changed from Default
(io_uring) to "threads". I assume that changes QEMU's behavior to not
use io_uring for I/O.

> We did also few other smaller things but honestly issues stopped. 
> Was it fixed or maybe Async IO could cause it ? (The qcow2 image is on
> fuse mounting).
>
> I am not certain but if that makes any sense, should i report this to
> someone working on io_uring or it should have nothing to do with that bug ?

Good question. It seems unrelated at first, but maybe it's related to
memory consumption with io_uring (below).

>
> CleanShot 2026-03-05 at 09.08.15@2x.png
>
> On all those kernels i seen no such bug for few weeks after changing
> async io.
>

What was the latest kernel you are running?

> CleanShot 2026-03-05 at 09.09.03@2x.png
>

Looking back at the original report, I can see that the system has extremely
little free memory. Maybe that gets eaten by io_uring somehow?

Do you still see that memory consumption when not using io_uring?


Let me copy the original information for Jens, maybe something could
give him a clue whether io_uring could be involved here.

"It happens on multiple servers (less on 6.18.6, more on 6.19-rc4+).
All servers are doing KVM with vfio GPU PCIE passthrough and it
happens when i am using HUGEPAGE 1GB + qemu Basically i am allocating
970GB into hugepages, leaving 37GB to kvm. In normal operation i have
about 20GB free space but when this issue occurs, all RAM is taken and
even when i have added 100GB swap, it was also consumed. It can work
for days or week without issue and

I did not seen that issue when i had hugepages disabled (on normal 2KB
pages allocation in kvm). And i am using hugepages as it is impossible
to boot VM with >200GB ram. When that issue happens, process ps hangs
and only top shows something but machine needs to be rebooted due to many
zombiee processes.


Hardware:
Motherboard: ASRockRack GENOA2D24G-2L
CPU: 2x AMD EPYC 9654 96-Core Processor
System ram: 1024 GB
GPUs: 8x RTX5090 vfio passthrough

root@pve14:~# uname -a
Linux pve14 6.18.6-pbk #1 SMP PREEMPT_DYNAMIC Mon Jan 19 20:59:46 UTC 2026 x86_64 GNU/Linux

[171053.341288] BUG: unable to handle page fault for address: ff469ae640000000
[171053.341310] #PF: supervisor read access in kernel mode
[171053.341319] #PF: error_code(0x0000) - not-present page
[171053.341328] PGD 4602067 P4D 0
[171053.341337] Oops: Oops: 0000 [#1] SMP NOPTI
[171053.341348] CPU: 16 UID: 0 PID: 3250869 Comm: qm Not tainted 6.18.6-pbk #1 PREEMPT(voluntary)
[171053.341362] Hardware name:  TURIN2D24G-2L+/500W/TURIN2D24G-2L+/500W, BIOS 10.20 05/05/2025
[171053.341373] RIP: 0010:walk_pgd_range+0x6ff/0xbb0
[171053.341386] Code: 08 49 39 dd 0f 84 8c 01 00 00 49 89 de 49 8d 9e 00 00 20 00 48 8b 75 b8 48 81 e3 00 00 e0 ff 48 8d 43 ff 48 39 f0 49 0f 43 dd <49> f7 04 24 9f ff ff ff 0f 84 e2 fd ff ff 48 8b 45 c0 41 c7 47 20
[171053.341406] RSP: 0018:ff59d95d70e6b748 EFLAGS: 00010287
[171053.341416] RAX: 00007a22401fffff RBX: 00007a2240200000 RCX: 0000000000000000
[171053.341425] RDX: 0000000000000000 RSI: 00007a227fffffff RDI: 800008dfc00002b7
[171053.341435] RBP: ff59d95d70e6b828 R08: 0000000000000080 R09: 0000000000000000
[171053.341444] R10: ffffffff8de588c0 R11: 0000000000000000 R12: ff469ae640000000
[171053.341454] R13: 00007a2280000000 R14: 00007a2240000000 R15: ff59d95d70e6b8a8
[171053.341464] FS:  00007d4e8ec94b80(0000) GS:ff4692876ae7e000(0000) knlGS:0000000000000000
[171053.341476] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[171053.341485] CR2: ff469ae640000000 CR3: 0000008241eed006 CR4: 0000000000f71ef0
[171053.341495] PKRU: 55555554
[171053.341501] Call Trace:
[171053.341508]  <TASK>
[171053.341518]  __walk_page_range+0x8e/0x220
[171053.341529]  ? sysvec_apic_timer_interrupt+0x57/0xc0
[171053.341541]  walk_page_vma+0x92/0xe0
[171053.341551]  smap_gather_stats.part.0+0x8c/0xd0
[171053.341563]  show_smaps_rollup+0x258/0x420
[171053.341577]  seq_read_iter+0x137/0x4c0
[171053.341588]  seq_read+0xf5/0x140
[171053.341596]  ? __x64_sys_move_mount+0x11/0x40
[171053.341607]  vfs_read+0xbb/0x350
[171053.341617]  ? do_syscall_64+0xb8/0xd00
[171053.341627]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.341637]  ? strncpy_from_user+0x27/0x130
[171053.341649]  ksys_read+0x69/0xf0
[171053.341658]  __x64_sys_read+0x19/0x30
[171053.341666]  x64_sys_call+0x2180/0x25a0
[171053.341855]  do_syscall_64+0x80/0xd00
[171053.342029]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.342198]  ? __x64_sys_ioctl+0x83/0x100
[171053.342367]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.342532]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.342696]  ? x64_sys_call+0xac0/0x25a0
[171053.342857]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.343019]  ? do_syscall_64+0xb8/0xd00
[171053.343181]  ? seq_read+0xf5/0x140
[171053.343341]  ? __x64_sys_move_mount+0x11/0x40
[171053.343504]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.343662]  ? vfs_read+0xbb/0x350
[171053.343819]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.343973]  ? ksys_read+0x69/0xf0
[171053.344126]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.344280]  ? generic_file_llseek+0x21/0x40
[171053.344432]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.344582]  ? kernfs_fop_llseek+0x7b/0xd0
[171053.344730]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.344873]  ? ksys_lseek+0x4f/0xd0
[171053.345010]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.345144]  ? __x64_sys_lseek+0x18/0x30
[171053.345275]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.345407]  ? x64_sys_call+0x1abe/0x25a0
[171053.345535]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.345665]  ? do_syscall_64+0xb8/0xd00
[171053.345792]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.345919]  ? irqentry_exit+0x43/0x50
[171053.346044]  ? srso_alias_return_thunk+0x5/0xfbef5
[171053.346169]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[171053.346292] RIP: 0033:0x7d4e8ed61687
[171053.346417] Code: 48 89 fa 4c 89 df e8 58 b3 00 00 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 1a 5b c3 0f 1f 84 00 00 00 00 00 48 8b 44 24 10 0f 05 <5b> c3 0f 1f 80 00 00 00 00 83 e2 39 83 fa 08 75 de e8 23 ff ff ff
[171053.346687] RSP: 002b:00007ffdd7c76000 EFLAGS: 00000202 ORIG_RAX: 0000000000000000
[171053.346828] RAX: ffffffffffffffda RBX: 00007d4e8ec94b80 RCX: 00007d4e8ed61687
[171053.346969] RDX: 0000000000002000 RSI: 000061ff84297ce0 RDI: 0000000000000006
[171053.347111] RBP: 0000000000002000 R08: 0000000000000000 R09: 0000000000000000
[171053.347253] R10: 0000000000000000 R11: 0000000000000202 R12: 000061ff84297ce0
[171053.347394] R13: 000061ff7d3d62a0 R14: 0000000000000006 R15: 000061ff842478c0
[171053.347542]  </TASK>
[171053.347684] Modules linked in: sctp ip6_udp_tunnel udp_tunnel nf_tables bridge stp llc softdog bonding sunrpc binfmt_misc nfnetlink_log amd_atl intel_rapl_msr intel_rapl_common nls_iso8859_1 amd64_edac edac_mce_amd kvm_amd snd_pcm snd_timer dax_hmem ipmi_ssif kvm cxl_acpi snd polyval_clmulni ghash_clmulni_intel cxl_port soundcore aesni_intel rapl cxl_core acpi_ipmi einj pcspkr ast ipmi_si spd5118 ipmi_devintf k10temp ccp ipmi_msghandler joydev input_leds mac_hid sch_fq_codel msr vhost_net vhost vhost_iotlb tap vfio_pci vfio_pci_core irqbypass vfio_iommu_type1 vfio iommufd efi_pstore nfnetlink dmi_sysfs autofs4 btrfs blake2b_generic xor raid6_pq mlx5_ib ib_uverbs macsec ib_core dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio nvme mlx5_core nvme_core cdc_ether nvme_keyring igb mlxfw psample usbnet nvme_auth i2c_algo_bit usbkbd mii hid_generic hkdf tls dca ahci i2c_piix4 libahci i2c_smbus usbmouse usbhid hid
[171053.349092] CR2: ff469ae640000000
[171053.349269] ---[ end trace 0000000000000000 ]---
[171054.248409] RIP: 0010:walk_pgd_range+0x6ff/0xbb0
[171054.248750] Code: 08 49 39 dd 0f 84 8c 01 00 00 49 89 de 49 8d 9e 00 00 20 00 48 8b 75 b8 48 81 e3 00 00 e0 ff 48 8d 43 ff 48 39 f0 49 0f 43 dd <49> f7 04 24 9f ff ff ff 0f 84 e2 fd ff ff 48 8b 45 c0 41 c7 47 20
[171054.249177] RSP: 0018:ff59d95d70e6b748 EFLAGS: 00010287
[171054.249392] RAX: 00007a22401fffff RBX: 00007a2240200000 RCX: 0000000000000000
[171054.249820] RDX: 0000000000000000 RSI: 00007a227fffffff RDI: 800008dfc00002b7
[171054.250036] RBP: ff59d95d70e6b828 R08: 0000000000000080 R09: 0000000000000000
[171054.250253] R10: ffffffff8de588c0 R11: 0000000000000000 R12: ff469ae640000000
[171054.250471] R13: 00007a2280000000 R14: 00007a2240000000 R15: ff59d95d70e6b8a8
[171054.250691] FS:  00007d4e8ec94b80(0000) GS:ff4692876ae7e000(0000) knlGS:0000000000000000
[171054.250914] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[171054.251137] CR2: ff469ae640000000 CR3: 0000008241eed006 CR4: 0000000000f71ef0
[171054.251375] PKRU: 55555554
[171054.251601] note: qm[3250869] exited with irqs disabled

--
Cheers,

David