Realtime threads delayed due to kcompactd0

[Alexander Krabler <Alexander.Krabler@kuka.com>]

Hi all,

some of our realtime tasks get delayed from time to time due to activity of kcompactd0.
Out of nothing, realtime tasks go into uninterruptable sleep for some time.
This delay can be as much as 1.1ms, which is not acceptable for us.

Our hardware is an aarch64-based SOC with 8 A72 cores, kernel is 6.12.17 with PREEMPT_RT.
We have CONFIG_COMPACTION and CONFIG_MIGRATION enabled.

Here are some snippets from ftrace:
             kcompactd0-88    [001] 13112.100041: mm_compaction_begin:  zone_start=0x80000 migrate_pfn=0x80000 free_pfn=0xffe00 zone_end=0x100000, mode=sync
...      
             kcompactd0-88    [001] 13112.159782: mm_compaction_isolate_migratepages: range=(0x85800 ~ 0x85841) nr_scanned=65 nr_taken=32
             kcompactd0-88    [001] 13112.159810: mm_compaction_isolate_freepages: range=(0xddc40 ~ 0xddc48) nr_scanned=8 nr_taken=8
             kcompactd0-88    [001] 13112.160002: irq_handler_entry:    irq=11 name=arch_timer
             kcompactd0-88    [001] 13112.160012: irq_handler_exit:     irq=11 ret=handled
             kcompactd0-88    [001] 13112.160121: mm_compaction_migratepages: nr_migrated=32 nr_failed=0
             kcompactd0-88    [001] 13112.160122: mm_compaction_finished: node=0 zone=DMA      order=-1 ret=continue
             kcompactd0-88    [001] 13112.160185: mm_compaction_isolate_migratepages: range=(0x85841 ~ 0x85a00) nr_scanned=447 nr_taken=166
             kcompactd0-88    [001] 13112.160204: mm_compaction_isolate_freepages: range=(0xddc48 ~ 0xddd80) nr_scanned=312 nr_taken=196
              tRealtime-16499 [004] 13112.160511: sched_switch:         tRealtime:16499 [25] D ==> tKRC:16479 [39]
              tRealtime-16499 [004] 13112.160512: kernel_stack:         <stack trace >
=> __schedule (ffffcde843022d6c)
=> schedule (ffffcde843023464)
=> io_schedule (ffffcde8430235ec)
=> migration_entry_wait_on_locked (ffffcde8424a1ad8)
=> migration_entry_wait (ffffcde84254c400)
=> do_swap_page (ffffcde8424f7fac)
=> __handle_mm_fault (ffffcde8424f8b64)
=> handle_mm_fault (ffffcde8424f9bc0)
=> do_page_fault (ffffcde843030380)
=> do_translation_fault (ffffcde84303072c)
=> do_mem_abort (ffffcde84222f674)
=> el0_ia (ffffcde84301eb20)
=> el0t_64_sync_handler (ffffcde84301f020)
=> el0t_64_sync (ffffcde842211514)
             kcompactd0-88    [001] 13112.160557: sched_pi_setprio:     comm=kcompactd0 pid=88 oldprio=39 newprio=120
             kcompactd0-88    [001] 13112.160569: sched_waking:         comm=tKRC pid=16479 prio=39 target_cpu=004
             kcompactd0-88    [001] 13112.160986: sched_waking:         comm=tKRC pid=16479 prio=39 target_cpu=004
             kcompactd0-88    [001] 13112.161412: sched_waking:         comm=tOther pid=16520 prio=40 target_cpu=004
             kcompactd0-88    [001] 13112.161457: sched_pi_setprio:     comm=kcompactd0 pid=88 oldprio=40 newprio=120
             kcompactd0-88    [001] 13112.161465: sched_waking:         comm=tOther pid=16520 prio=40 target_cpu=004
             kcompactd0-88    [001] 13112.161654: sched_waking:         comm=tRealtime pid=16499 prio=25 target_cpu=004            

In our setup kcompactd0 gets enough CPU time (on core 1), however, it seems strange that it doesn't get the priority inherited from blocked realtime tasks.
(It does for short amounts of time, which seems to be due to the locks inside migration_entry_wait_on_locked.)

Is there anything we can do here?

Thanks,
Alexander