[QUESTION] oom killed the key system process triggered by a bad process alloc memory with MAP_LOCKED

[Yongqiang Liu <liuyongqiang13@huawei.com>]

Hi,

I was just wondering the process of  the patch "oom reaper: handle 
mlocked pages".

links:

https://lore.kernel.org/linux-mm/1454505240-23446-3-git-send-email-mhocko@kernel.org/

https://lore.kernel.org/linux-mm/20160223132157.GD14178@dhcp22.suse.cz/


In linux master, we found that when we start a process and alloc large 
memory with MAP_LOCKED, the oom will triggered and kill some system 
process such as sshd ,rsyslog etc.

...

[   45.110665] Out of memory: Killed process 2551 (oom) 
total-vm:1035324kB, anon-rss:1028196kB, file-rss:704kB, shmem-rss:0kB, 
UID:0 pgtables:2056kB oom_score_adj:1000
[   45.115303] Out of memory: Killed process 2554 (oom) 
total-vm:1025084kB, anon-rss:1015596kB, file-rss:388kB, shmem-rss:0kB, 
UID:0 pgtables:2032kB oom_score_adj:1000
[   45.115685] Out of memory: Killed process 2553 (oom) 
total-vm:953404kB, anon-rss:947748kB, file-rss:388kB, shmem-rss:0kB, 
UID:0 pgtables:1896kB oom_score_adj:1000
[   45.116031] Out of memory: Killed process 2552 (oom) 
total-vm:789564kB, anon-rss:783272kB, file-rss:388kB, shmem-rss:0kB, 
UID:0 pgtables:1576kB oom_score_adj:1000
[   45.117199] Out of memory: Killed process 2523 (sshd) 
total-vm:77052kB, anon-rss:804kB, file-rss:4672kB, shmem-rss:4kB, UID:0 
pgtables:196kB oom_score_adj:0
[   45.120936] Out of memory: Killed process 2526 (bash) 
total-vm:17792kB, anon-rss:1180kB, file-rss:2868kB, shmem-rss:0kB, UID:0 
pgtables:76kB oom_score_adj:0

...

the process demo is:

...

#define ALLOC_SIZE (10 * 1024 * 1024)

while (1) {

     addr = mmap(NULL, ALLOC_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | 
MAP_ANONYMOUS | MAP_LOCKED, -1, 0);

     if (addr != MAP_FAILED)

         memset(addr, 1, ALLOC_SIZE);

}

...

And we found that when the oom_reaper is done but the memory is still high:

[   45.115685] Out of memory: Killed process 2553 (oom) 
total-vm:953404kB, anon-rss:947748kB, file-rss:388kB, shmem-rss:0kB, 
UID:0 pgtables:1896kB oom_score_adj:1000
[   45.115739] oom_reaper: reaped process 2553 (oom), now 
anon-rss:947708kB, file-rss:0kB, shmem-rss:0kB

This is because the bad proccess which recieved SIGKILL is unlocking the 
mem to exit which needs more time. And the next oom is triggered to kill 
the other system process.

Kind regards,

Yongqiang Liu