[PATCH] mm: limit a process RSS

[PATCH] mm: limit a process RSS

[Li RongQing]

we cannot limit a process RSS although there is ulimit -m,
not sure why and when ulimit -m is not working, make it work

similar requirement:
https://stackoverflow.com/questions/3360348/why-ulimit-cant-limit-resident-memory-successfully-and-how

Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
 mm/memory.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mm/memory.c b/mm/memory.c
index 5fcfc24904d1..50cf9399477c 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4140,6 +4140,9 @@ int handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
 		ret = __handle_mm_fault(vma, address, flags);
 
 	if (flags & FAULT_FLAG_USER) {
+		unsigned long total_rss = get_mm_rss(current->mm);
+		u64 rlimit;
+
 		mem_cgroup_oom_disable();
 		/*
 		 * The task may have entered a memcg OOM situation but
@@ -4149,6 +4152,17 @@ int handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
 		 */
 		if (task_in_memcg_oom(current) && !(ret & VM_FAULT_OOM))
 			mem_cgroup_oom_synchronize(false);
+
+		rlimit = current->signal->rlim[RLIMIT_RSS].rlim_cur;
+
+		if (unlikely(total_rss > (rlimit >> PAGE_SHIFT)) &&
+			(current->pid != 1)) {
+
+			pr_info("kill process %s rsslimit[%lluK] rss[%luK]\n",
+				current->comm, (rlimit >> 10),
+				total_rss << (PAGE_SHIFT - 10));
+			do_group_exit(SIGKILL);
+		}
 	}
 
 	return ret;
-- 
2.11.0

Re: [PATCH] mm: limit a process RSS

[Michal Hocko]

On Mon 02-04-18 15:45:19, Li RongQing wrote:
> we cannot limit a process RSS although there is ulimit -m,
> not sure why and when ulimit -m is not working, make it work

Could you be more specific about why do you need this functionality?
The RSS limit has never been implemented AFAIK and the main reason is
that the semantic is quite weak to be useful (e.g. the shared memory
accounting, resident memory that is not mapped etc.).

We have memory cgroup controller as an alternative.
-- 
Michal Hocko
SUSE Labs