[PATCH] mm/vmalloc: Prevent RCU stall in decay_va_pool_node()

[PATCH] mm/vmalloc: Prevent RCU stall in decay_va_pool_node()

[Sechang Lim]

decay_va_pool_node() walks every per-pool free-list entry under
vmap_purge_lock and merges each vmap_area into a global RB-tree via
reclaim_list_global() without yielding. The outer loop has no
rescheduling point, so when many vmap areas are queued the function
can monopolize the CPU long enough to trigger an RCU self-detected
stall:

  rcu: INFO: rcu_preempt self-detected stall on CPU
  rcu:    2-...0: (6344 ticks this GP) idle=853c/1/0x4000000000000000 softirq=41536/41536 fqs=3211
  rcu:    (t=6528 jiffies g=37549 q=4652 ncpus=4)
  CPU: 2 UID: 0 PID: 1516 Comm: syz.5.318 Not tainted 7.0.0-rc7 #4 PREEMPT(full)
  Call Trace:
   <TASK>
   finish_task_switch.isra.0+0x23e/0x990     kernel/sched/core.c:5155
   context_switch                            kernel/sched/core.c:5301 [inline]
   __schedule+0xb3d/0x3680                   kernel/sched/core.c:6911
   preempt_schedule_common+0x44/0xc0         kernel/sched/core.c:7095
   preempt_schedule_thunk+0x16/0x30          arch/x86/entry/thunk.S:12
   __raw_spin_unlock                         include/linux/spinlock_api_smp.h:169 [inline]
   _raw_spin_unlock+0x43/0x50                kernel/locking/spinlock.c:186
   reclaim_list_global                       mm/vmalloc.c:2213 [inline]
   decay_va_pool_node+0xccf/0x1070           mm/vmalloc.c:2273
   __purge_vmap_area_lazy+0x136/0xc80        mm/vmalloc.c:2361
   _vm_unmap_aliases+0x469/0x6e0             mm/vmalloc.c:2996
   change_page_attr_set_clr+0x24d/0x4a0      arch/x86/mm/pat/set_memory.c:2082
   set_memory_rox+0xc2/0x110                 arch/x86/mm/pat/set_memory.c:2314
   create_trampoline                         arch/x86/kernel/ftrace.c:421 [inline]
   arch_ftrace_update_trampoline+0x79d/0xb50 arch/x86/kernel/ftrace.c:479
   ftrace_update_trampoline+0x45/0x360       kernel/trace/ftrace.c:8391
   __register_ftrace_function+0x238/0x340    kernel/trace/ftrace.c:365
   ftrace_startup+0x3b/0x370                 kernel/trace/ftrace.c:3098
   register_ftrace_function_nolock+0x5e/0x160 kernel/trace/ftrace.c:9162
   register_ftrace_function+0x32b/0x4c0      kernel/trace/ftrace.c:9189
   perf_ftrace_function_register             kernel/trace/trace_event_perf.c:494 [inline]
   perf_ftrace_event_register+0x159/0x240    kernel/trace/trace_event_perf.c:518
   perf_trace_event_open                     kernel/trace/trace_event_perf.c:184 [inline]
   perf_trace_event_init                     kernel/trace/trace_event_perf.c:206 [inline]
   perf_trace_event_init+0x17b/0xad0         kernel/trace/trace_event_perf.c:193
   perf_trace_init+0x176/0x290               kernel/trace/trace_event_perf.c:226
   perf_tp_event_init+0xa6/0x120             kernel/events/core.c:11270
   perf_try_init_event+0x103/0x930           kernel/events/core.c:13029
   perf_init_event                           kernel/events/core.c:13127 [inline]
   perf_event_alloc.part.0+0x11dd/0x4970     kernel/events/core.c:13402
   perf_event_alloc                          kernel/events/core.c:13283 [inline]
   __do_sys_perf_event_open+0x764/0x2eb0     kernel/events/core.c:13924
   do_syscall_x64                            arch/x86/entry/syscall_64.c:63 [inline]
   do_syscall_64+0xa9/0x580                  arch/x86/entry/syscall_64.c:94
   entry_SYSCALL_64_after_hwframe+0x76/0x7e
   </TASK>

Add cond_resched() at the bottom of the outer loop in
decay_va_pool_node(). At that point the per-pool spinlock has already
been released and the outer vmap_purge_lock is a mutex, so sleeping
is safe.

Found by Syzkaller.

Fixes: 72210662c5a2 ("mm: vmalloc: offload free_vmap_area_lock lock")
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
---
 mm/vmalloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 61caa55a4402..78e064a9c4c7 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -2268,6 +2268,8 @@ decay_va_pool_node(struct vmap_node *vn, bool full_decay)
 			WRITE_ONCE(vn->pool[i].len, pool_len);
 			spin_unlock(&vn->pool_lock);
 		}
+
+		cond_resched();
 	}
 
 	reclaim_list_global(&decay_list);
-- 
2.43.0