From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00916C433F5 for ; Thu, 12 May 2022 04:47:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 914236B0074; Thu, 12 May 2022 00:47:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89CD96B0075; Thu, 12 May 2022 00:47:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 716946B0078; Thu, 12 May 2022 00:47:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 605B56B0074 for ; Thu, 12 May 2022 00:47:11 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 314CC30822 for ; Thu, 12 May 2022 04:47:11 +0000 (UTC) X-FDA: 79455856662.12.18A5F75 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf27.hostedemail.com (Postfix) with ESMTP id 3BEBE400A7 for ; Thu, 12 May 2022 04:47:08 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id v10so3548650pgl.11 for ; Wed, 11 May 2022 21:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=/pTgPhGOi/9d6Q/oqwaxhfFkmjeH4PVid+OZ0dZHdSw=; b=HvhsyDv1TuH2yCE9PXc22mXXi4RP3ENuXk44vilMbhfXZAh3usDN+q7UYC4Qg+hkbD ZW2CKXaScrpCanbEd7qxTPf1kXPgl9QX8PbgfaG4a5jGRByg/9RvP66UvSti/F/eerne 9zZlmrit2DsqjneYnmSf5DYy2mgQT4UVh++EH5K5BS7KJ8FBUg++vX3SFWV0debV1ZGZ hQulKyeZYRi6+uBqZiVSot2SpRBP/ubpK3I7T7r3VT5dOzEHe1FjUbIYr5VsapkkoWXL XKFqIcpMeCvJy5i5+IKJlG02P/U1Fg7p5E8+zKZGAkfwuK2GfX+G3gQbW7uvquoMjGln ZY5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=/pTgPhGOi/9d6Q/oqwaxhfFkmjeH4PVid+OZ0dZHdSw=; b=54yRWvTlInTNdmdjL+5CsmpPu3Yi1EFc2hBsGt325hSW4FGpeNFDlW3RtClvI2DSI1 GM8tpolQz5x/v0J3GDA2aVrJJXozaMH2DWVrGQR+6dhHQO4yCsPts69PxaxlX1YdLpMF fU1XvasA+3yCptVnM63bvzqJKyckkIqHyLXznUsBDFEl3TQgZUlnDZZNY45CXOkuovUN gdTk6vexWzX81Zfr7zYWs/4YhdLyzrC8vuAJAzHOKtWZVyQ7zEiXr9Id2acYtOynvrRE b0uJf2MfITswnao9VzQMZxyOxmNYjZxOLJl2JaoUjyGVzFWS0xlDp1XJaK6MfpAAlA/n PpLQ== X-Gm-Message-State: AOAM530rLsP7cRbSCdE/7tnF5NH5fKn7oKI9ghcP+kwXfR6rmi9CR4N8 9rUw9ldtDENJ6wGa1nubJQVMJA== X-Google-Smtp-Source: ABdhPJxVJxgprVu7WxtZqidhH+r8VEa6XOk7al5eXrOJrdSve9mLiQ3pEAj3JyiqY7GAMCjs7fxFtg== X-Received: by 2002:a63:d842:0:b0:3c6:ab6b:5f3c with SMTP id k2-20020a63d842000000b003c6ab6b5f3cmr16180665pgj.148.1652330829556; Wed, 11 May 2022 21:47:09 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.253]) by smtp.gmail.com with ESMTPSA id 5-20020a170902e9c500b0015edc07dcf3sm2790824plk.21.2022.05.11.21.46.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 May 2022 21:47:08 -0700 (PDT) From: Gang Li To: akpm@linux-foundation.org Cc: songmuchun@bytedance.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, ebiederm@xmission.com, keescook@chromium.org, viro@zeniv.linux.org.uk, rostedt@goodmis.org, mingo@redhat.com, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, david@redhat.com, imbrenda@linux.ibm.com, apopple@nvidia.com, adobriyan@gmail.com, stephen.s.brennan@oracle.com, ohoono.kwon@samsung.com, haolee.swjtu@gmail.com, kaleshsingh@google.com, zhengqi.arch@bytedance.com, peterx@redhat.com, shy828301@gmail.com, surenb@google.com, ccross@google.com, vincent.whitchurch@axis.com, tglx@linutronix.de, bigeasy@linutronix.de, fenghua.yu@intel.com, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-perf-users@vger.kernel.org, Gang Li Subject: [PATCH 0/5 v1] mm, oom: Introduce per numa node oom for CONSTRAINT_MEMORY_POLICY Date: Thu, 12 May 2022 12:46:29 +0800 Message-Id: <20220512044634.63586-1-ligang.bdlg@bytedance.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 3BEBE400A7 X-Stat-Signature: xrxdkqq8ic1rsb4wd9gsz9zd5nwiejgu X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=HvhsyDv1; spf=pass (imf27.hostedemail.com: domain of ligang.bdlg@bytedance.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=ligang.bdlg@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam09 X-HE-Tag: 1652330828-383111 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: TLDR: If a mempolicy is in effect(oc->constraint == CONSTRAINT_MEMORY_POLICY), out_of_memory() will select victim on specific node to kill. So that kernel can avoid accidental killing on NUMA system. Problem: Before this patch series, oom will only kill the process with the highest memory usage. by selecting process with the highest oom_badness on the entire system to kill. This works fine on UMA system, but may have some accidental killing on NUMA system. As shown below, if process c.out is bind to Node1 and keep allocating pages from Node1, a.out will be killed first. But killing a.out did't free any mem on Node1, so c.out will be killed then. A lot of our AMD machines have 8 numa nodes. In these systems, there is a greater chance of triggering this problem. OOM before patches: ``` Per-node process memory usage (in MBs) PID Node 0 Node 1 Total ----------- ---------- ------------- ---------- 3095 a.out 3073.34 0.11 3073.45(Killed first. Maximum memory consumption) 3199 b.out 501.35 1500.00 2001.35 3805 c.out 1.52 (grow)2248.00 2249.52(Killed then. Node1 is full) ----------- ---------- ------------- ---------- Total 3576.21 3748.11 7324.31 ``` Solution: We store per node rss in mm_rss_stat for each process. If a page allocation with mempolicy in effect(oc->constraint == CONSTRAINT_MEMORY_POLICY) triger oom. We will calculate oom_badness with rss counter for the corresponding node. Then select the process with the highest oom_badness on the corresponding node to kill. OOM after patches: ``` Per-node process memory usage (in MBs) PID Node 0 Node 1 Total ----------- ---------- ------------- ---------- 3095 a.out 3073.34 0.11 3073.45 3199 b.out 501.35 1500.00 2001.35 3805 c.out 1.52 (grow)2248.00 2249.52(killed) ----------- ---------- ------------- ---------- Total 3576.21 3748.11 7324.31 ``` Gang Li (5): mm: add a new parameter `node` to `get/add/inc/dec_mm_counter` mm: add numa_count field for rss_stat mm: add numa fields for tracepoint rss_stat mm: enable per numa node rss_stat count mm, oom: enable per numa node oom for CONSTRAINT_MEMORY_POLICY arch/s390/mm/pgtable.c | 4 +- fs/exec.c | 2 +- fs/proc/base.c | 6 +- fs/proc/task_mmu.c | 14 ++-- include/linux/mm.h | 59 ++++++++++++----- include/linux/mm_types_task.h | 16 +++++ include/linux/oom.h | 2 +- include/trace/events/kmem.h | 27 ++++++-- kernel/events/uprobes.c | 6 +- kernel/fork.c | 70 +++++++++++++++++++- mm/huge_memory.c | 13 ++-- mm/khugepaged.c | 4 +- mm/ksm.c | 2 +- mm/madvise.c | 2 +- mm/memory.c | 116 ++++++++++++++++++++++++---------- mm/migrate.c | 2 + mm/migrate_device.c | 2 +- mm/oom_kill.c | 59 ++++++++++++----- mm/rmap.c | 16 ++--- mm/swapfile.c | 4 +- mm/userfaultfd.c | 2 +- 21 files changed, 317 insertions(+), 111 deletions(-) -- 2.20.1