On Wed, Jul 16, 2014 at 02:37:27PM -0400, Johannes Weiner wrote: > On Wed, Jul 16, 2014 at 02:10:07PM -0400, Naoya Horiguchi wrote: > > Hi, > > > > It seems that when a process in some memcg tries to allocate more than > > memcg.limit_in_bytes, oom happens instead of swaping out in > > mmotm-2014-07-09-17-08 (memcg.memsw.limit_in_bytes is large enough). > > It does work in v3.16-rc3, so I think latest patches changed something. > > I'm not familiar with memcg internally, so no idea about what caused it. > > Could you see the problem? > > There are a lot of changes in memory and swap accounting, but I can > not reproduce what you are describing: I set up a cgroup with a 100MB > memory limit and an unlimited memory+swap, then start a task in there > that faults 200MB worth of anonymous pages. The result is 100MB in > memory, 100MB in swap: Hmm, this is almost the same test condition. > cache 0 > rss 104267776 > rss_huge 0 > mapped_file 0 > writeback 0 > swap 105545728 > pgpgin 26367 > pgpgout 25950 > pgfault 26695 > pgmajfault 32 > inactive_anon 52285440 > active_anon 51982336 > inactive_file 0 > active_file 0 > unevictable 0 > hierarchical_memory_limit 104857600 > hierarchical_memsw_limit 18446744073709551615 > > Filename Type Size Used Priority > /swapfile file 8388604 109800 -1 > > Could you provide more detail on your configuration and test case? What I did is like this: [root@test_140715-0036 ~]# cat tmp_memcg_swap.sh cgcreate -g memory:test1 cgset -r memory.limit_in_bytes=0x1000000 test1 # 16 MB cgset -r memory.memsw.limit_in_bytes=0x4000000 test1 # 64 MB cgget -g memory:test1 swapon -s cgexec -g memory:test1 memhog -r1 20m # alloc 20 MB then output is like this: [root@test_140715-0036 ~]# bash tmp_memcg_swap.sh test1: memory.pressure_level: memory.kmem.max_usage_in_bytes: 0 memory.use_hierarchy: 1 memory.swappiness: 60 memory.memsw.failcnt: 0 memory.limit_in_bytes: 16777216 memory.memsw.max_usage_in_bytes: 0 memory.usage_in_bytes: 0 memory.memsw.limit_in_bytes: 67108864 memory.failcnt: 0 memory.kmem.limit_in_bytes: 18446744073709551615 memory.force_empty: memory.kmem.slabinfo: memory.memsw.usage_in_bytes: 0 memory.max_usage_in_bytes: 0 memory.numa_stat: total=0 N0=0 N1=0 N2=0 N3=0 file=0 N0=0 N1=0 N2=0 N3=0 anon=0 N0=0 N1=0 N2=0 N3=0 unevictable=0 N0=0 N1=0 N2=0 N3=0 hierarchical_total=0 N0=0 N1=0 N2=0 N3=0 hierarchical_file=0 N0=0 N1=0 N2=0 N3=0 hierarchical_anon=0 N0=0 N1=0 N2=0 N3=0 hierarchical_unevictable=0 N0=0 N1=0 N2=0 N3=0 memory.kmem.tcp.limit_in_bytes: 18446744073709551615 memory.oom_control: oom_kill_disable 0 under_oom 0 memory.kmem.tcp.max_usage_in_bytes: 0 memory.kmem.failcnt: 0 memory.kmem.usage_in_bytes: 0 memory.stat: cache 0 rss 0 rss_huge 0 mapped_file 0 writeback 0 swap 0 pgpgin 0 pgpgout 0 pgfault 0 pgmajfault 0 inactive_anon 0 active_anon 0 inactive_file 0 active_file 0 unevictable 0 hierarchical_memory_limit 16777216 hierarchical_memsw_limit 67108864 total_cache 0 total_rss 0 total_rss_huge 0 total_mapped_file 0 total_writeback 0 total_swap 0 total_pgpgin 0 total_pgpgout 0 total_pgfault 0 total_pgmajfault 0 total_inactive_anon 0 total_active_anon 0 total_inactive_file 0 total_active_file 0 total_unevictable 0 recent_rotated_anon 0 recent_rotated_file 0 recent_scanned_anon 0 recent_scanned_file 0 memory.kmem.tcp.usage_in_bytes: 0 memory.move_charge_at_immigrate: 0 memory.soft_limit_in_bytes: 18446744073709551615 memory.kmem.tcp.failcnt: 0 Filename Type Size Used Priority /root/page_table_walker/swapfile file 40956 0 -1 ..tmp_memcg_swap.sh: line 7: 1250 Killed cgexec -g memory:test1 memhog -r1 20m Kernel message is like this: [ 425.290261] memhog invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0 [ 425.291356] memhog cpuset=/ mems_allowed=0-3 [ 425.291992] CPU: 2 PID: 1234 Comm: memhog Not tainted 3.15.0-140716-1501-00003-gcb8370a1d76e #268 [ 425.293389] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 425.294300] ffff88011ca7a800 ffff8800795c7c50 ffffffff8172d5d7 ffff8800795c7c68 [ 425.295555] ffffffff8172d60e ffff88007d82d8b0 ffff8800795c7cb0 ffffffff817232a2 [ 425.296797] 0000000000000000 0000000081ca21e0 000000000000012b 00000000000000d0 [ 425.298059] Call Trace: [ 425.298425] [] __dump_stack+0x19/0x1b [ 425.299253] [] dump_stack+0x35/0x46 [ 425.299972] [] dump_header+0x7f/0xbd [ 425.300790] [] oom_kill_process+0x22c/0x3b0 [ 425.301697] [] ? mem_cgroup_iter+0x17b/0x1c0 [ 425.302780] [] mem_cgroup_out_of_memory+0x25c/0x290 [ 425.303787] [] mem_cgroup_oom_synchronize+0x177/0x1a0 [ 425.304808] [] ? memcg_charge_kmem+0xc0/0xc0 [ 425.305718] [] pagefault_out_of_memory+0x14/0x90 [ 425.306674] [] mm_fault_error+0x98/0xe3 [ 425.307522] [] __do_page_fault+0x482/0x520 [ 425.308362] [] ? __schedule+0x2d3/0x7d0 [ 425.309178] [] ? vfs_write+0x17c/0x1e0 [ 425.309937] [] do_page_fault+0xc/0x10 [ 425.310734] [] page_fault+0x22/0x30 [ 425.311525] Task in /test1 [ 425.311906] killed as a result of limit of /test1 [ 425.312748] [ 425.313082] memory: usage 16384kB, limit 16384kB, failcnt 42 [ 425.313879] memory+swap: usage 16384kB, limit 65536kB, failcnt 0 [ 425.314841] kmem: usage 0kB, limit 18014398509481983kB, failcnt 0 [ 425.315825] Memory cgroup stats for /test1: cache:16KB rss:16368KB rss_huge:0KB mapped_file:8KB writeback:0KB swap:0KB inactive_anon:8192KB active_anon:8176KB inactive_file:0KB active_file:0KB unevictable:0KB [ 425.318881] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [ 425.320130] [ 1234] 0 1234 7251 4401 22 0 0 memhog [ 425.321361] Memory cgroup out of memory: Kill process 1234 (memhog) score 299 or sacrifice child [ 425.322695] Killed process 1234 (memhog) total-vm:29004kB, anon-rss:16192kB, file-rss:1412kB I attached my .config. Thanks, Naoya Horiguchi