From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00BDAC433EF for ; Thu, 12 May 2022 22:31:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B5FA6B0073; Thu, 12 May 2022 18:31:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43E916B0074; Thu, 12 May 2022 18:31:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DE646B0078; Thu, 12 May 2022 18:31:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1B0ED6B0073 for ; Thu, 12 May 2022 18:31:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id CDB87120837 for ; Thu, 12 May 2022 22:31:46 +0000 (UTC) X-FDA: 79458539412.26.25D3C0B Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by imf02.hostedemail.com (Postfix) with ESMTP id 79E1680003 for ; Thu, 12 May 2022 22:31:38 +0000 (UTC) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-2ef5380669cso72720617b3.9 for ; Thu, 12 May 2022 15:31:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AJGrVRrH+dOEhiElDtPJnom+nsrzVSlx3m1P3MXcyt8=; b=HKHBQnUdowSNLBhbkOBsBEX5C+iK1w0Yf+bcFPhOqlRiMP6A0Mz5ltaYanBi+i05iG cin6l4liXxnPvewh1T7T7VdF/b2gZubu8BFpF1WFRdvGC4vbHWLBnyEHMDVH+/ycYupI sjKR4L02x/5+BSW+jam05AzZ2NmQlqkBhti6Uq78H0zr3uNSRVTWjzYDMTHeVMumdU/f 6Swv4843WGA/Mnd9MNpjok50X0vmtCH9JqfJWCHJy2aQ5P6XU2iRMYw+xQW3GDRas9BS PaouOeajWzpDJoWV+wRBWGsBQvWe3KgQtzreixaQBUYxzKZaZERpu/imjAsk5CAWC9cB OUbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AJGrVRrH+dOEhiElDtPJnom+nsrzVSlx3m1P3MXcyt8=; b=uUEFeBX94uURVlX5hZmPInNw+j/RLy6wTg00IGX/jq+pQjINWYSD7cOSU2B1xvh44d LcLrmUl8pP9iO3YSKZO+3prcP0YZrYTmNthCo1syZX+SJd4MSp6QDAByclT8qpjQjVYR 8edRdlMNEPGr77cp+TT6ShYZAHP2Oap2ZdOfJAy0z6RZoipPkSjQX1N/NNYC/8HM47+j cdtOZJ7Ma0fQV1k0MiZaNgaxg2267T45ase92YY9rCoVGwOLPTbl+oYTzZpe920fLRUq zLqmwiNLktar6LnsM92ZWoLpN+e/h9AVm/VdTwm84A5TsmHpF5k6YYis7nMcGplAnF4r WbCQ== X-Gm-Message-State: AOAM531Ysz4WyHUK+38dqhK2CW62927btO1GbbBCqMWAfLIr4knyMQAu q54kQdFG4X8uIlTGt6W3NnO9DiuLZnzLoN+IbZ90ZQ== X-Google-Smtp-Source: ABdhPJwzefOUbhnD4vlu7+gan+V9STOahhU9EhcwGhhFaV8YI/mjlx7OYk8OXREHs4uUiLanalOYjb8ajWHHjC3d7gc= X-Received: by 2002:a81:4fd0:0:b0:2fb:2c3e:6cbc with SMTP id d199-20020a814fd0000000b002fb2c3e6cbcmr2716004ywb.180.1652394705360; Thu, 12 May 2022 15:31:45 -0700 (PDT) MIME-Version: 1.0 References: <20220512044634.63586-1-ligang.bdlg@bytedance.com> In-Reply-To: <20220512044634.63586-1-ligang.bdlg@bytedance.com> From: Suren Baghdasaryan Date: Thu, 12 May 2022 15:31:34 -0700 Message-ID: Subject: Re: [PATCH 0/5 v1] mm, oom: Introduce per numa node oom for CONSTRAINT_MEMORY_POLICY To: Gang Li , Michal Hocko Cc: Andrew Morton , Muchun Song , hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, "Eric W. Biederman" , Kees Cook , Al Viro , Steven Rostedt , Ingo Molnar , Peter Zijlstra , acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, David Hildenbrand , imbrenda@linux.ibm.com, apopple@nvidia.com, Alexey Dobriyan , stephen.s.brennan@oracle.com, ohoono.kwon@samsung.com, haolee.swjtu@gmail.com, Kalesh Singh , zhengqi.arch@bytedance.com, Peter Xu , Yang Shi , Colin Cross , vincent.whitchurch@axis.com, Thomas Gleixner , bigeasy@linutronix.de, fenghua.yu@intel.com, linux-s390@vger.kernel.org, LKML , linux-mm , linux-fsdevel , linux-perf-users@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: 1js15zspaydfwms7a7g3tqqzyp7sofay X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 79E1680003 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HKHBQnUd; spf=pass (imf02.hostedemail.com: domain of surenb@google.com designates 209.85.128.170 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-HE-Tag: 1652394698-147549 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 11, 2022 at 9:47 PM Gang Li wrote: > > TLDR: > If a mempolicy is in effect(oc->constraint == CONSTRAINT_MEMORY_POLICY), out_of_memory() will > select victim on specific node to kill. So that kernel can avoid accidental killing on NUMA system. > > Problem: > Before this patch series, oom will only kill the process with the highest memory usage. > by selecting process with the highest oom_badness on the entire system to kill. > > This works fine on UMA system, but may have some accidental killing on NUMA system. > > As shown below, if process c.out is bind to Node1 and keep allocating pages from Node1, > a.out will be killed first. But killing a.out did't free any mem on Node1, so c.out > will be killed then. > > A lot of our AMD machines have 8 numa nodes. In these systems, there is a greater chance > of triggering this problem. > > OOM before patches: > ``` > Per-node process memory usage (in MBs) > PID Node 0 Node 1 Total > ----------- ---------- ------------- ---------- > 3095 a.out 3073.34 0.11 3073.45(Killed first. Maximum memory consumption) > 3199 b.out 501.35 1500.00 2001.35 > 3805 c.out 1.52 (grow)2248.00 2249.52(Killed then. Node1 is full) > ----------- ---------- ------------- ---------- > Total 3576.21 3748.11 7324.31 > ``` > > Solution: > We store per node rss in mm_rss_stat for each process. > > If a page allocation with mempolicy in effect(oc->constraint == CONSTRAINT_MEMORY_POLICY) > triger oom. We will calculate oom_badness with rss counter for the corresponding node. Then > select the process with the highest oom_badness on the corresponding node to kill. > > OOM after patches: > ``` > Per-node process memory usage (in MBs) > PID Node 0 Node 1 Total > ----------- ---------- ------------- ---------- > 3095 a.out 3073.34 0.11 3073.45 > 3199 b.out 501.35 1500.00 2001.35 > 3805 c.out 1.52 (grow)2248.00 2249.52(killed) > ----------- ---------- ------------- ---------- > Total 3576.21 3748.11 7324.31 > ``` You included lots of people but missed Michal Hocko. CC'ing him and please include him in the future postings. > > Gang Li (5): > mm: add a new parameter `node` to `get/add/inc/dec_mm_counter` > mm: add numa_count field for rss_stat > mm: add numa fields for tracepoint rss_stat > mm: enable per numa node rss_stat count > mm, oom: enable per numa node oom for CONSTRAINT_MEMORY_POLICY > > arch/s390/mm/pgtable.c | 4 +- > fs/exec.c | 2 +- > fs/proc/base.c | 6 +- > fs/proc/task_mmu.c | 14 ++-- > include/linux/mm.h | 59 ++++++++++++----- > include/linux/mm_types_task.h | 16 +++++ > include/linux/oom.h | 2 +- > include/trace/events/kmem.h | 27 ++++++-- > kernel/events/uprobes.c | 6 +- > kernel/fork.c | 70 +++++++++++++++++++- > mm/huge_memory.c | 13 ++-- > mm/khugepaged.c | 4 +- > mm/ksm.c | 2 +- > mm/madvise.c | 2 +- > mm/memory.c | 116 ++++++++++++++++++++++++---------- > mm/migrate.c | 2 + > mm/migrate_device.c | 2 +- > mm/oom_kill.c | 59 ++++++++++++----- > mm/rmap.c | 16 ++--- > mm/swapfile.c | 4 +- > mm/userfaultfd.c | 2 +- > 21 files changed, 317 insertions(+), 111 deletions(-) > > -- > 2.20.1 >