From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAFF0C77B6F for ; Tue, 11 Apr 2023 13:45:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F432900002; Tue, 11 Apr 2023 09:45:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A4F16B0075; Tue, 11 Apr 2023 09:45:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAE38900002; Tue, 11 Apr 2023 09:45:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DBC416B0072 for ; Tue, 11 Apr 2023 09:45:51 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A9BF3120C6B for ; Tue, 11 Apr 2023 13:45:51 +0000 (UTC) X-FDA: 80669233302.19.548ACCB Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf13.hostedemail.com (Postfix) with ESMTP id 50D3D20008 for ; Tue, 11 Apr 2023 13:45:48 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jfK1PfIH; spf=pass (imf13.hostedemail.com: domain of ligang.bdlg@bytedance.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=ligang.bdlg@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681220749; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=qZX6YeaID3c9+hbjD4dUk+FdkX7cttToQ9GXc0PnhLA=; b=O8SBf6DF8JKitXJZFxeXgHuq5SWpcjqTZsRCbWs7MfAbgJJp0iEmMIwr5JSsV+aSWbgiX1 L2YOstIP02hl4UwHaJeMJkurRctHEW57cEpAMASc9QeRx7CemSm5ZSVBTdl2dqMqAWhln8 MbLDExP9ADs3ph+dnHhGrFou94NDsZM= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jfK1PfIH; spf=pass (imf13.hostedemail.com: domain of ligang.bdlg@bytedance.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=ligang.bdlg@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681220749; a=rsa-sha256; cv=none; b=kOvnJS0z2i7N6x0IxN7Azm3SLLzYNovOeB7noVlGIulYcd1Y3kxIHoeagNPG4Fc0RJw93V +rlZXjEJ6IRBLGgggAjU5RWYvGJhiIHYs1QAd+nHvdSjyo0iW4XGjdSIVRc8/FptVxXZ6Y fjeRlGWvcwL3SlsfaODmOUjXgHG/Dzs= Received: by mail-pj1-f51.google.com with SMTP id jx2-20020a17090b46c200b002469a9ff94aso6150087pjb.3 for ; Tue, 11 Apr 2023 06:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1681220747; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=qZX6YeaID3c9+hbjD4dUk+FdkX7cttToQ9GXc0PnhLA=; b=jfK1PfIHWSTtmaeGN8AuiadBEVxrrwgMKykaBvz1P42IvsNcoSAoBwwD5A1VQ5YVv6 sBxeCJ2QdUzFTz2W+fe8GWtxUePVjcsou3w0GKVLGJ40XSjhwMEbJIFGTDncib47HfqB H+hjaSJDzFb24HyF6+VJlx9VSft5PY7C0Taqf0bemWiiHgIgm/6Blr8f5+Abua7+giCH wwrkv8EKd2TW3LHz1vMPKVJaMkFcmdk/mWcwciBSE8r3cfWvc/R5ZNObeCUVUF9i/Qgw yMzjxKlG2RR/YZ+H8hBWUIq9BVshqi7oA0wv0wdEZ+U4Np4Gp5fc+KcakjusXRIywwIf ecwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681220747; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qZX6YeaID3c9+hbjD4dUk+FdkX7cttToQ9GXc0PnhLA=; b=eZZJpeG0RL/6DjslsMQcoB2a/Jkglu+29OxCFBLjKsaglyCP+0qEvEcPsXkiH6PAXx E2dtvryUqLoMftAJRBX225IX77YSvrmDadklNrlohby/RRkaHGPbXjaAAc1ZGd0eSTXz N99sLqHKZZB8+9Pp0klYItFDtQOQsXrnhTKck07NEWK6Zhe1a2Jq42oM8LUbLfFhO2Gs fyoJoC2Ch04Lg1B1/3O+hbpIEcB0T7mTdlIg02slSx+7dlOj0kX1AZFTsUjKK1Qyw9k6 lZ2p8vtq32ogLEGQu6+A/1bWdKTkbPLONjixXPCvGb5fBP5YdUbzQfo0hAfChbDwLgOU qlPA== X-Gm-Message-State: AAQBX9e/if4FYda7eVun2isFiG+xYbzwGTjLr9VHX5gPAtiWU6lGgI3m Ljk+R1XuJvqzOsLt46uJ5Q6jbQ== X-Google-Smtp-Source: AKy350byq01H5QC3Rko1z3BRFn8kAzbNwtspEBqlTUMi8xkEhrEMoZL7JgBlJ0hxCykMWQvXBtJASg== X-Received: by 2002:a17:90b:4b06:b0:246:bb61:4a56 with SMTP id lx6-20020a17090b4b0600b00246bb614a56mr4361207pjb.27.1681220746944; Tue, 11 Apr 2023 06:45:46 -0700 (PDT) Received: from C02FT5A6MD6R.bytedance.net ([61.213.176.13]) by smtp.gmail.com with ESMTPSA id x4-20020a170902820400b001a633a9322dsm4757460pln.212.2023.04.11.06.45.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Apr 2023 06:45:46 -0700 (PDT) From: Gang Li To: Waiman Long , Michal Hocko Cc: Gang Li , cgroups@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, Zefan Li , linux-kernel@vger.kernel.org Subject: [PATCH v5] mm: oom: introduce cpuset oom Date: Tue, 11 Apr 2023 21:45:38 +0800 Message-Id: <20230411134539.45046-1-ligang.bdlg@bytedance.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 50D3D20008 X-Stat-Signature: 94t8ssu83ypup4jfsx31tkthun61r5wn X-HE-Tag: 1681220748-232826 X-HE-Meta: U2FsdGVkX18m7tv/+6Na6m5gdr9jEMazp5v4B/tdIrAgbpxjUFRckeX+U+WmvaA4CB38SXROvB8JL1OdT6OBUV0GaePpbnKGdGjU0zmsro60/2fm2P63TBO3XuCTHvbcAvbLFAwUSpmAK8HmRkeapfabGt5WDoWA5D6cZqgD1ZC5O21Rxg9iKpJhJfgd/YBoSS/58aq3YIrYH0Q7xKliMq7TOlIQYEHeSQnUmhUBrkmmTy//tSxIM1QabBAFhVaJ7w//p/zGn7n4PVPRXiigdt4FQkCLOH8IYSQPWp42aaR6mZEjANmhLvZ9P+9yfOPGvazLMGCZ5dsXJx7HMVvWcS5LP2Npuk9xo4fb4gvyUyG+iGz8wTyXc4//tuxh6PelIFZqJqw7BnUmIuFnu7WW7k6zdf1GxPRn7myqW0cWmQuVkQB2o0nD+VzSIudHd3QaDmZguu1buLaKBr1E6Qd8DMIroAojULr2Eq/OUL+OSo/cZQ4+7M5uPhtdPLM1oT2gouFR9nLj5t+rzgc9gGJXisJKiL/DhHcwfVmtucvtqqFfZ2e8tK5jHt8ZwnsGCt42bcYTOmG3DhfG6+rJyA/+Hn2iiqEAyAR0zZ+ou6ZuChPpO2+h1CU0UFqsJ3ac/ejUNjXyJw47sdjAkg3L8keO9gLVMlRlBlE9FzRCdB9moIxivv2Quitwwb96tp82m9QRwBfQlGURQZyYAaz3Hc3OK3junBPsDHySVx57Gjk1JL5/kpeAlp2p1NArF88P+loPffx3vGoNelK4latSslQ2H4IkG8n+Oarr1sydsvrDIJ3gS3dlAKn4R/GJBz0mP3h9Yx3L65F0piSaM5LtVcMy80eQTYBfWK3SU9WWvFKIPb35TRoAmL4QPCYGu/zvLAk31CTM1xUHuQVoCy1eEZFSfy4JJKlPpJl4suafL/s5SUeEAtMBRO4aL9/deQ+0JaLNyumkS+hrAShMbxX8v50 i6WqOUv9 fJqBG0jeOgQ39F6z5WSEtMH9WZvuMgaNeU7XQnG56gMPfczhqnjk6LzyFOguxzpuZ5BRP9gG5jNd3PSu6pAh37IvPkGpQc+sy+pKN2yS0eUYXTIrlPw+BCEQ5C+jP3UM8sKPlCetQFREedmvmHO3jLiyf/VKJTNRQLOvSYKQt5i+Z0YKvN4mZmj4NhMSgOw8uu8lMVTCdbJEW1S2t2bsuvK7tQTZOIXkgWHNy7LMTZEFAZZdw5l7wBW/dZbfmNXWlCnkF6xuUuh6tGUpd05sGIPu4mHMhgSPCb+ntUOh7uU6AB0NpjF+cmTtZhogSPeR7YN0pkHFQZ0PWj46mc1cQ51rvMIy7U6o00WazxPBSLvnmOC6rpQ9yxOUQyvsw7jP44NhPiXC8mquRHDn23UuzZ9ZqqSZ0UPzLfEZnQDZpCYI/2aDdvvFovQi2xuZq0/+5z9Nx4+cc0gl7hw8SByvsSd94Cvbv6YscY9fR9im4DMYvrvCnvxjn0aPAJCd0hs7bAIu12ZxhDfztw3RYuqgUdeKJP6Q/gkdPToKvMby7mb+qba40r2yr706Xbnin01pZ2Xkm1092Hh+I0SW0Mz0z7Tdpmg6PljmQtvmuMnN8QImTo5Lxg68QmCer9f7rOqJuFYvEcpYiM1tON+HVUjvK2Vd98yi6ermXhUkjvctT8tYgPpBdvkHpHbyxrN5MuvdcaULKI64yFujLEvbcUAyOF1bHpRUuoeq+KzmdY3ycFdSdnebvwmGdk2hHS15COfr4+QXJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Cpusets constrain the CPU and Memory placement of tasks. `CONSTRAINT_CPUSET` type in oom has existed for a long time, but has never been utilized. When a process in cpuset which constrain memory placement triggers oom, it may kill a completely irrelevant process on other numa nodes, which will not release any memory for this cpuset. We can easily achieve node aware oom by using `CONSTRAINT_CPUSET` and selecting victim from cpuset the allocating process belongs to. Example: Create two processes named mem_on_node0 and mem_on_node1 constrained by cpusets respectively. These two processes alloc memory on their own node. Now node0 has run out of memory, OOM will be invokled by mem_on_node0. Before this patch: Since `CONSTRAINT_CPUSET` do nothing, the victim will be selected from the entire system. Therefore, the OOM is highly likely to kill mem_on_node1, which will not free any memory for mem_on_node0. This is a useless kill. ``` [ 2786.519080] mem_on_node0 invoked oom-killer [ 2786.885738] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [ 2787.181724] [ 13432] 0 13432 787016 786745 6344704 0 0 mem_on_node1 [ 2787.189115] [ 13457] 0 13457 787002 785504 6340608 0 0 mem_on_node0 [ 2787.216534] oom-kill:constraint=CONSTRAINT_CPUSET,nodemask=(null),cpuset=test,mems_allowed=0 [ 2787.229991] Out of memory: Killed process 13432 (mem_on_node1) ``` After this patch: The victim will be selected only in mem_on_node0's own cpuset. This will prevent useless kill and protect innocent victims. ``` [ 395.922444] mem_on_node0 invoked oom-killer [ 396.239777] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [ 396.246128] [ 2614] 0 2614 1311294 1144192 9224192 0 0 mem_on_node0 [ 396.252655] oom-kill:constraint=CONSTRAINT_CPUSET,nodemask=(null),cpuset=test,mems_allowed=0 [ 396.264068] Out of memory: Killed process 2614 (mem_on_node0) ``` Suggested-by: Michal Hocko Cc: Cc: Cc: Cc: Waiman Long Cc: Zefan Li Signed-off-by: Gang Li --- Changes in v5: - Select victim in the cpuset the allocating process belongs to. Changes in v4: - https://lore.kernel.org/all/20230411065816.9798-1-ligang.bdlg@bytedance.com/ - Modify comments and documentation. Changes in v3: - https://lore.kernel.org/all/20230410025056.22103-1-ligang.bdlg@bytedance.com/ - Provide more details about the use case, testing, implementation. - Document the userspace visible change in Documentation. - Rename cpuset_cgroup_scan_tasks() to cpuset_scan_tasks() and add a doctext comment about its purpose and how it should be used. - Take cpuset_rwsem to ensure that cpusets are stable. Changes in v2: - https://lore.kernel.org/all/20230404115509.14299-1-ligang.bdlg@bytedance.com/ - Select victim from all cpusets with the same mems_allowed as the current cpuset. v1: - https://lore.kernel.org/all/20220921064710.89663-1-ligang.bdlg@bytedance.com/ - Introduce cpuset oom. --- .../admin-guide/cgroup-v1/cpusets.rst | 15 ++++++-- Documentation/admin-guide/cgroup-v2.rst | 4 +++ include/linux/cpuset.h | 6 ++++ kernel/cgroup/cpuset.c | 34 +++++++++++++++++++ mm/oom_kill.c | 4 +++ 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v1/cpusets.rst b/Documentation/admin-guide/cgroup-v1/cpusets.rst index 5d844ed4df69..57bc15782d56 100644 --- a/Documentation/admin-guide/cgroup-v1/cpusets.rst +++ b/Documentation/admin-guide/cgroup-v1/cpusets.rst @@ -25,7 +25,8 @@ Written by Simon.Derr@bull.net 1.6 What is memory spread ? 1.7 What is sched_load_balance ? 1.8 What is sched_relax_domain_level ? - 1.9 How do I use cpusets ? + 1.9 What is cpuset oom ? + 1.10 How do I use cpusets ? 2. Usage Examples and Syntax 2.1 Basic Usage 2.2 Adding/removing cpus @@ -607,8 +608,18 @@ If your situation is: - The latency is required even it sacrifices cache hit rate etc. then increasing 'sched_relax_domain_level' would benefit you. +1.9 What is cpuset oom ? +-------------------------- +If there is no available memory to allocate on the nodes specified by +cpuset.mems, then an OOM (Out-Of-Memory) will be invoked. + +Since the victim selection is a heuristic algorithm, we cannot select +the "perfect" victim. So just select a process from the cpuset the +allocating process belongs to. + +Cpuset oom works in both cgroup v1 and v2. -1.9 How do I use cpusets ? +1.10 How do I use cpusets ? -------------------------- In order to minimize the impact of cpusets on critical kernel diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index f67c0829350b..5db84fb4f1cc 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2199,6 +2199,10 @@ Cpuset Interface Files a need to change "cpuset.mems" with active tasks, it shouldn't be done frequently. + When a process invokes oom due to the constraint of cpuset.mems, + the victim will be selected from cpuset the allocating process + belongs to. + cpuset.mems.effective A read-only multiple values file which exists on all cpuset-enabled cgroups. diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index 980b76a1237e..75465bf58f74 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -171,6 +171,8 @@ static inline void set_mems_allowed(nodemask_t nodemask) task_unlock(current); } +int cpuset_scan_tasks(int (*fn)(struct task_struct *, void *), void *arg); + #else /* !CONFIG_CPUSETS */ static inline bool cpusets_enabled(void) { return false; } @@ -287,6 +289,10 @@ static inline bool read_mems_allowed_retry(unsigned int seq) return false; } +static inline int cpuset_scan_tasks(int (*fn)(struct task_struct *, void *), void *arg) +{ + return 0; +} #endif /* !CONFIG_CPUSETS */ #endif /* _LINUX_CPUSET_H */ diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bc4dcfd7bee5..624454368605 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -4013,6 +4013,40 @@ void cpuset_print_current_mems_allowed(void) rcu_read_unlock(); } +/** + * cpuset_scan_tasks - specify the oom scan range + * @fn: callback function to select oom victim + * @arg: argument for callback function, usually a pointer to struct oom_control + * + * Description: This function is used to specify the oom scan range. Return 0 if + * no task is selected, otherwise return 1. The selected task will be stored in + * arg->chosen. This function can only be called in cpuset oom context. + * + * The selection algorithm is heuristic, therefore requires constant iteration + * based on user feedback. Currently, we just scan the current cpuset. + */ +int cpuset_scan_tasks(int (*fn)(struct task_struct *, void *), void *arg) +{ + int ret = 0; + struct css_task_iter it; + struct task_struct *task; + + /* + * Situation gets complex with overlapping nodemasks in different cpusets. + * TODO: Maybe we should calculate the "distance" between different mems_allowed. + * + * But for now, let's make it simple. Just scan current cpuset. + */ + rcu_read_lock(); + css_task_iter_start(&(task_cs(current)->css), CSS_TASK_ITER_PROCS, &it); + while (!ret && (task = css_task_iter_next(&it))) + ret = fn(task, arg); + css_task_iter_end(&it); + rcu_read_unlock(); + + return ret; +} + /* * Collection of memory_pressure is suppressed unless * this flag is enabled by writing "1" to the special diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 044e1eed720e..228257788d9e 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -367,6 +367,8 @@ static void select_bad_process(struct oom_control *oc) if (is_memcg_oom(oc)) mem_cgroup_scan_tasks(oc->memcg, oom_evaluate_task, oc); + else if (oc->constraint == CONSTRAINT_CPUSET) + cpuset_scan_tasks(oom_evaluate_task, oc); else { struct task_struct *p; @@ -427,6 +429,8 @@ static void dump_tasks(struct oom_control *oc) if (is_memcg_oom(oc)) mem_cgroup_scan_tasks(oc->memcg, dump_task, oc); + else if (oc->constraint == CONSTRAINT_CPUSET) + cpuset_scan_tasks(dump_task, oc); else { struct task_struct *p; -- 2.20.1