Re: [PATCH v7 2/2] mm/oom_kill: The OOM reaper traverses the VMA maple tree in reverse order

[Michal Hocko <mhocko@suse.com>]

On Thu 04-09-25 20:24:38, zhongjinji wrote:
> > On Wed 03-09-25 17:27:29, zhongjinji wrote:
> > > Although the oom_reaper is delayed and it gives the oom victim chance to
> > > clean up its address space this might take a while especially for
> > > processes with a large address space footprint. In those cases
> > > oom_reaper might start racing with the dying task and compete for shared
> > > resources - e.g. page table lock contention has been observed.
> > > 
> > > Reduce those races by reaping the oom victim from the other end of the
> > > address space.
> > > 
> > > It is also a significant improvement for process_mrelease(). When a process
> > > is killed, process_mrelease is used to reap the killed process and often
> > > runs concurrently with the dying task. The test data shows that after
> > > applying the patch, lock contention is greatly reduced during the procedure
> > > of reaping the killed process.
> > 
> > Thank you this is much better!
> > 
> > > Without the patch:
> > > |--99.74%-- oom_reaper
> > > |  |--76.67%-- unmap_page_range
> > > |  |  |--33.70%-- __pte_offset_map_lock
> > > |  |  |  |--98.46%-- _raw_spin_lock
> > > |  |  |--27.61%-- free_swap_and_cache_nr
> > > |  |  |--16.40%-- folio_remove_rmap_ptes
> > > |  |  |--12.25%-- tlb_flush_mmu
> > > |  |--12.61%-- tlb_finish_mmu
> > > 
> > > With the patch:
> > > |--98.84%-- oom_reaper
> > > |  |--53.45%-- unmap_page_range
> > > |  |  |--24.29%-- [hit in function]
> > > |  |  |--48.06%-- folio_remove_rmap_ptes
> > > |  |  |--17.99%-- tlb_flush_mmu
> > > |  |  |--1.72%-- __pte_offset_map_lock
> > > |  |--30.43%-- tlb_finish_mmu
> > 
> > Just curious. Do I read this correctly that the overall speedup is
> > mostly eaten by contention over tlb_finish_mmu?
> 
> Here is a more detailed perf report, which includes the execution times
> of some important functions. I believe it will address your concerns.
> 
> tlb_flush_mmu and tlb_finish_mmu perform similar tasks; they both mainly
> call free_pages_and_swap_cache, and its execution time is related to the
> number of anonymous pages being reclaimed.
> 
> In previous tests, the pte spinlock contention was so obvious that I
> overlooked other issues.
> 
> Without the patch
> |--99.50%-- oom_reaper
> |    |--0.50%-- [hit in function]
> |    |--71.06%-- unmap_page_range
> |    |    |--41.75%-- __pte_offset_map_lock
> |    |    |--23.23%-- folio_remove_rmap_ptes
> |    |    |--20.34%-- tlb_flush_mmu
> |    |    |           free_pages_and_swap_cache
> |    |    |--2.23%-- folio_mark_accessed
> |    |    |--1.19%-- free_swap_and_cache_nr
> |    |    |--1.13%-- __tlb_remove_folio_pages
> |    |    |--0.76%-- _raw_spin_lock
> |    |--16.02%-- tlb_finish_mmu
> |    |    |--26.08%-- [hit in function]
> |    |    |--72.97%-- free_pages_and_swap_cache
> |    |    |--0.67%-- free_pages
> |    |--2.27%-- folio_remove_rmap_ptes
> |    |--1.54%-- __tlb_remove_folio_pages
> |    |    |--83.47%-- [hit in function]
> |    |--0.51%-- __pte_offset_map_lock
> 
> Period (ms)           Symbol
> 79.180156             oom_reaper
> 56.321241             unmap_page_range
> 23.891714             __pte_offset_map_lock
> 20.711614             free_pages_and_swap_cache
> 12.831778             tlb_finish_mmu
> 11.443282             tlb_flush_mmu
> 
> With the patch
> |--99.54%-- oom_reaper
> |    |--0.29%-- [hit in function]
> |    |--57.91%-- unmap_page_range
> |    |    |--20.42%-- [hit in function]
> |    |    |--53.35%-- folio_remove_rmap_ptes
> |    |    |    |--5.85%-- [hit in function]
> |    |    |--10.49%-- __pte_offset_map_lock
> |    |    |    |--5.17%-- [hit in function]
> |    |    |--8.40%-- tlb_flush_mmu
> |    |    |--2.35%-- _raw_spin_lock
> |    |    |--1.89%-- folio_mark_accessed
> |    |    |--1.64%-- __tlb_remove_folio_pages
> |    |    |    |--57.95%-- [hit in function]
> |    |--36.34%-- tlb_finish_mmu
> |    |    |--14.70%-- [hit in function]
> |    |    |--84.85%-- free_pages_and_swap_cache
> |    |    |    |--2.32%-- [hit in function]
> |    |    |--0.37%-- free_pages
> |    |     --0.08%-- free_unref_page
> |    |--1.94%-- folio_remove_rmap_ptes
> |    |--1.68%-- __tlb_remove_folio_pages
> |    |--0.93%-- __pte_offset_map_lock
> |    |--0.43%-- folio_mark_accessed
> 
> Period (ms)           Symbol
> 49.580521             oom_reaper
> 28.781660             unmap_page_range
> 18.105898             tlb_finish_mmu
> 17.688397             free_pages_and_swap_cache
>  3.471721             __pte_offset_map_lock
>  2.412970             tlb_flush_mmu

yes, this break down gives much more insight. Percentage is quite
misleading as the base is different. Could you also provide cumulative
oom_reaper + exit_mmap(victim) time in both cases?


-- 
Michal Hocko
SUSE Labs