On Thu, 27 Feb 2025 10:42:44 +0530 Ritesh Harjani (IBM) wrote: > Since only the swapon failed, I think you might still have the console > up right? So this is mostly a vmalloc allocation failure report? Correct. The machine stays usable via VNC in my case, even after the vmalloc allocation failure. > Though above are mainly the physical mem info printed, but vmalloc can > also fail sometimes (e.g. this report), it is nice if we can print how > much of vmalloc space is free out of vmalloc total in show_mem() here. > > Maybe linux-mm can tell if we should add this diff change for future? > > diff --git a/mm/show_mem.c b/mm/show_mem.c > index 43afb56abbd3..b3af59fced02 100644 > --- a/mm/show_mem.c > +++ b/mm/show_mem.c > @@ -14,6 +14,7 @@ > #include > #include > #include > +#include > > #include "internal.h" > #include "swap.h" > @@ -416,6 +417,8 @@ void __show_mem(unsigned int filter, nodemask_t *nodemask, int max_zone_idx) > printk("%lu pages RAM\n", total); > printk("%lu pages HighMem/MovableOnly\n", highmem); > printk("%lu pages reserved\n", reserved); > + printk("%lu pages Vmalloc Total\n", (unsigned long)VMALLOC_TOTAL >> PAGE_SHIFT); > + printk("%lu pages Vmalloc Used\n", vmalloc_nr_pages()); > #ifdef CONFIG_CMA > printk("%lu pages cma reserved\n", totalcma_pages); > #endif I applied your patch and now get this output: [...] vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000 modprobe: vmalloc error: size 1189956, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0 CPU: 1 UID: 0 PID: 1853 Comm: modprobe Tainted: G W 6.14.0-rc4-PMacG4-dirty #9 Tainted: [W]=WARN Hardware name: PowerMac3,6 7455 0x80010303 PowerMac Call Trace: [f427b850] [c14cfe28] dump_stack_lvl+0x70/0x8c (unreliable) [f427b870] [c04fba74] warn_alloc+0x154/0x2b8 [f427b930] [c04dea08] __vmalloc_node_range_noprof+0x154/0x958 [f427ba00] [c04df2f8] __vmalloc_node_noprof+0xec/0xf4 [f427ba40] [c062da4c] kernel_read_file+0x514/0x7a0 [f427baf0] [c01ee5d4] init_module_from_file+0xb0/0x108 [f427bbd0] [c01eed04] sys_finit_module+0x390/0x7a4 [f427bca0] [c001e574] system_call_exception+0x2dc/0x420 [f427bf30] [c00291ac] ret_from_syscall+0x0/0x2c --- interrupt: c00 at 0x321334 NIP: 00321334 LR: 005c9340 CTR: 002ad944 REGS: f427bf40 TRAP: 0c00 Tainted: G W (6.14.0-rc4-PMacG4-dirty) MSR: 0000d032 CR: 2822242c XER: 20000000 GPR00: 00000161 afb2c9e0 a7a03540 00000001 005d6a48 00000000 afb2c9b5 0000007f GPR08: 00000000 af3da4b8 00000000 00000008 00000003 005efe10 2422242c 00000000 GPR16: 00040000 00000000 00000000 00000000 00750610 00000000 00751a30 00000000 GPR24: 00750610 00000000 005d6a48 00751330 00040000 00000000 005f7aac 00750610 NIP [00321334] 0x321334 LR [005c9340] 0x5c9340 --- interrupt: c00 Mem-Info: active_anon:16715 inactive_anon:0 isolated_anon:0 active_file:25985 inactive_file:12645 isolated_file:0 unevictable:0 dirty:23 writeback:0 slab_reclaimable:2378 slab_unreclaimable:22874 mapped:14774 shmem:155 pagetables:355 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:388018 free_pcp:426 free_cma:0 Node 0 active_anon:66860kB inactive_anon:0kB active_file:103940kB inactive_file:50580kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:59096kB dirty:92kB writeback:0kB shmem:620kB writeback_tmp:0kB kernel_stack:3008kB pagetables:1420kB sec_pagetables:0kB all_unreclaimable? no DMA free:566884kB boost:0kB min:3380kB low:4224kB high:5068kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:28264kB unevictable:0kB writepending:80kB present:786432kB managed:716496kB mlocked:0kB bounce:0kB free_pcp:1704kB local_pcp:1304kB free_cma:0kB lowmem_reserve[]: 0 0 1184 0 DMA: 43*4kB (U) 29*8kB (UM) 21*16kB (UE) 41*32kB (UME) 9*64kB (UME) 6*128kB (UE) 3*256kB (UE) 3*512kB (UM) 6*1024kB (UME) 5*2048kB (UM) 133*4096kB (M) = 566852kB 38772 total pagecache pages 0 pages in swap cache Free swap = 8388604kB Total swap = 8388604kB 524288 pages RAM 327680 pages HighMem/MovableOnly 42060 pages reserved 16656 pages Vmalloc Total 6771 pages Vmalloc Used Memory allocations: 124 MiB 31813 mm/readahead.c:187 func:ractl_alloc_folio 88.6 MiB 7937 mm/slub.c:2423 func:alloc_slab_page 50.3 MiB 12883 ./include/linux/highmem.h:229 func:vma_alloc_zeroed_movable_folio 26.5 MiB 6781 mm/filemap.c:1970 func:__filemap_get_folio 14.5 MiB 3714 mm/memory.c:1063 func:folio_prealloc 12.3 MiB 3280 mm/execmem.c:44 func:execmem_vmalloc 9.22 MiB 2360 mm/kasan/shadow.c:304 func:kasan_populate_vmalloc_pte 5.44 MiB 348 lib/stackdepot.c:627 func:stack_depot_save_flags 4.40 MiB 531 mm/slub.c:2425 func:alloc_slab_page 4.01 MiB 1025 mm/swap_cgroup.c:140 func:swap_cgroup_swapon vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000 vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000 [...] Looks like there would be still Vmalloc pages free though? 6771 used vs. 16656 total. Also it seems more clear I get this vmalloc error at the module loading stage during bootup. For my original report the rather big btrfs module got pulled in at boot so I got the vmalloc failure instantly. Now the machine boots up ok, but as soon as some module gets loaded, e.g. nfs via mounting a remote directory I get the failure. This is the case for both INLINE_KASAN and OUTLINE_KASAN. Maybe for my original report the btrfs module was not loaded for my OUTLINE_KASAN boot so I got no hit early. Doesn't seem to be 100% deterministic what modules get loaded in what order. > But meanwhile below data can give more details about the vmalloc area. > > 1. cat /proc/vmallocinfo > 2. cat /proc/meminfo Did that, with "no_hash_pointers" used and for both INLINE and OUTLINE KASAN as suggested by Christophe. Both after I get the vmalloc failure after mounting a remote NFS directory. See attached files. For completeness also the OUTLINE_KASAN vmalloc failure: [...] vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000 modprobe: vmalloc error: size 832096, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0 CPU: 0 UID: 0 PID: 1857 Comm: modprobe Tainted: G W 6.14.0-rc4-PMacG4-dirty #10 Tainted: [W]=WARN Hardware name: PowerMac3,6 7455 0x80010303 PowerMac Call Trace: [f34db980] [c09e7f34] dump_stack_lvl+0x70/0x8c (unreliable) [f34db9a0] [c02887a4] warn_alloc+0x154/0x208 [f34dba60] [c027a308] __vmalloc_node_range_noprof+0x144/0x530 [f34dbb10] [c027a770] __vmalloc_node_noprof+0x7c/0x84 [f34dbb50] [c032a0d4] kernel_read_file+0x314/0x440 [f34dbbf0] [c0102918] init_module_from_file+0xb0/0x108 [f34dbcd0] [c0102d7c] sys_finit_module+0x1d0/0x37c [f34dbda0] [c00138b8] system_call_exception+0x154/0x1b8 [f34dbf30] [c001a1ac] ret_from_syscall+0x0/0x2c --- interrupt: c00 at 0x491334 NIP: 00491334 LR: 00739340 CTR: 0041d944 REGS: f34dbf40 TRAP: 0c00 Tainted: G W (6.14.0-rc4-PMacG4-dirty) MSR: 0000d032 CR: 2822242c XER: 20000000 GPR00: 00000161 af942800 a7fae540 00000001 00746a48 00000000 af9427d5 0000007f GPR08: 00000000 aea582d8 00000000 00000008 00000003 0075fe10 2422242c 00000000 GPR16: 00040000 00000000 00000000 00000000 00ee8610 00000000 00ee9a30 00000000 GPR24: 00ee8610 00000000 00746a48 00ee9330 00040000 00000000 00767aac 00ee8610 NIP [00491334] 0x491334 LR [00739340] 0x739340 --- interrupt: c00 Mem-Info: active_anon:16937 inactive_anon:0 isolated_anon:0 active_file:25969 inactive_file:11637 isolated_file:0 unevictable:0 dirty:21 writeback:0 slab_reclaimable:2396 slab_unreclaimable:23326 mapped:14784 shmem:155 pagetables:353 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:392772 free_pcp:777 free_cma:0 Node 0 active_anon:67748kB inactive_anon:0kB active_file:103876kB inactive_file:46548kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:59136kB dirty:84kB writeback:0kB shmem:620kB writeback_tmp:0kB kernel_stack:3216kB pagetables:1412kB sec_pagetables:0kB all_unreclaimable? no DMA free:576744kB boost:0kB min:3412kB low:4264kB high:5116kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:28408kB unevictable:0kB writepending:80kB present:786432kB managed:729040kB mlocked:0kB bounce:0kB free_pcp:3108kB local_pcp:1788kB free_cma:0kB lowmem_reserve[]: 0 0 1184 0 DMA: 0*4kB 2*8kB (UM) 5*16kB (UM) 36*32kB (UM) 15*64kB (UME) 2*128kB (ME) 3*256kB (UM) 2*512kB (UM) 3*1024kB (ME) 4*2048kB (UM) 137*4096kB (M) = 576672kB 37789 total pagecache pages 0 pages in swap cache Free swap = 8388604kB Total swap = 8388604kB 524288 pages RAM 327680 pages HighMem/MovableOnly 38924 pages reserved 16656 pages Vmalloc Total 5814 pages Vmalloc Used Memory allocations: 120 MiB 30794 mm/readahead.c:187 func:ractl_alloc_folio 90.4 MiB 8145 mm/slub.c:2423 func:alloc_slab_page 50.6 MiB 12941 ./include/linux/highmem.h:229 func:vma_alloc_zeroed_movable_folio 26.6 MiB 6819 mm/filemap.c:1970 func:__filemap_get_folio 15.0 MiB 3837 mm/memory.c:1063 func:folio_prealloc 8.71 MiB 2230 mm/kasan/shadow.c:304 func:kasan_populate_vmalloc_pte 8.34 MiB 2276 mm/execmem.c:44 func:execmem_vmalloc 5.14 MiB 329 lib/stackdepot.c:627 func:stack_depot_save_flags 4.42 MiB 531 mm/slub.c:2425 func:alloc_slab_page 4.01 MiB 1025 mm/swap_cgroup.c:140 func:swap_cgroup_swapon vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000 vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000 [...] Regards, Erhard