From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D017AC11F68 for ; Wed, 30 Jun 2021 18:01:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 54C5461492 for ; Wed, 30 Jun 2021 18:01:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54C5461492 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5BF898D01BC; Wed, 30 Jun 2021 14:01:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56F908D01A2; Wed, 30 Jun 2021 14:01:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40FFA8D01BC; Wed, 30 Jun 2021 14:01:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0080.hostedemail.com [216.40.44.80]) by kanga.kvack.org (Postfix) with ESMTP id 1DFAA8D01A2 for ; Wed, 30 Jun 2021 14:01:12 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DEE8B1802EB1E for ; Wed, 30 Jun 2021 18:01:11 +0000 (UTC) X-FDA: 78311156742.20.30DB4AA Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by imf30.hostedemail.com (Postfix) with ESMTP id 90064E00304E for ; Wed, 30 Jun 2021 18:01:11 +0000 (UTC) Received: by mail-lf1-f46.google.com with SMTP id a15so6708706lfr.6 for ; Wed, 30 Jun 2021 11:01:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=f7Zc20M89DiLlQ+moMDDwd+VTvIiSSibcrF+4SaGp1M=; b=t+qcpe2VpHTQAZMvMfUa8hTVYiG9UEnIuclYa6oqSZoPe++uiGLJdbJf1lNa6xQp9s bMdc7XO7sqeSUfBvUjN3t2rtxSNNrnwlu7290pttzrctk62ECG9urOqo+Oy/W6gwdete oltlaUhK3LUJ17DlVRL2QtaNDMmK7qwIei2X8JXddo8NGdF2F7AIBzzrnrqwbmzRkprv CD1FNIzEe9EPf03lrjJjBkF39Iw49KiH1U2+fvsonnIGIdQ1oM3aXELMD0rK1/mw9uZM OU/5seZRhsrFdJ+ULPu03FEV0V2+nGrDMzNSCM55pKJfBfv2wKLuOKWPhUaEITVaup+z /C0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=f7Zc20M89DiLlQ+moMDDwd+VTvIiSSibcrF+4SaGp1M=; b=eLadjKLuaNlYgK+P6ICE7E2MnQs2333fwfymqiiDKdXa+EpDidDB6x7E8s2m5nkxkc 8dKJt9+ZXlxGLRNDyeAdZ5wPkwopii6PiqnR8cp9MhNmEfJbIhVGxyPAxGrTFl4iWS+c AgHCZwJvez4VaK5Le2v4cv4hgvO1UOe0XBj6tC2G8A4wuqUubFBL7ieSGA6n/G8x0cdH WHs0skiHAbtG7h49EJUCB0nn3jQYPQztkVe1iOob4j+B9RuPMJgMDGTayLPiCHc0SJ72 hs/dqac0ahnzqo3vfU27X2N8TEQK6GAHW4lXHP3mts5Apq03nOls/hQXqyFhmpv6n7y7 Fenw== X-Gm-Message-State: AOAM532uk5juBiHn9JU3n8zCPdx8KRfbcgeTfV7SQ1QKXhBDfrGjzIJZ V2Bg3aHyPIsNwVHlsc6k/c6YAgn0jiIaZM/xIbt+Dg== X-Google-Smtp-Source: ABdhPJyzv8U11rrq4ziEq24flRmWXao2NiiFFqLQu8OaqSR5WpQxTM+m2nhk+h3b3tTZNyFZcKI+/0zP7z3IhqmJPR8= X-Received: by 2002:a05:6512:3155:: with SMTP id s21mr27115898lfi.358.1625076067603; Wed, 30 Jun 2021 11:01:07 -0700 (PDT) MIME-Version: 1.0 References: <20210623192822.3072029-1-surenb@google.com> In-Reply-To: <20210623192822.3072029-1-surenb@google.com> From: Shakeel Butt Date: Wed, 30 Jun 2021 11:00:56 -0700 Message-ID: Subject: Re: [PATCH 1/1] mm: introduce process_reap system call To: Suren Baghdasaryan Cc: Andrew Morton , Michal Hocko , Michal Hocko , David Rientjes , Matthew Wilcox , Johannes Weiner , Roman Gushchin , Rik van Riel , Minchan Kim , Christian Brauner , Christoph Hellwig , Oleg Nesterov , David Hildenbrand , Jann Horn , Tim Murray , Linux API , Linux MM , LKML , kernel-team Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 90064E00304E Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=t+qcpe2V; spf=pass (imf30.hostedemail.com: domain of shakeelb@google.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: hzc6oiocqoq61xm45xz8yzdwgqgxawbp X-HE-Tag: 1625076071-111998 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Suren, On Wed, Jun 23, 2021 at 12:28 PM Suren Baghdasaryan wrote: > > In modern systems it's not unusual to have a system component monitoring > memory conditions of the system and tasked with keeping system memory > pressure under control. One way to accomplish that is to kill > non-essential processes to free up memory for more important ones. > Examples of this are Facebook's OOM killer daemon called oomd and > Android's low memory killer daemon called lmkd. > For such system component it's important to be able to free memory > quickly and efficiently. Unfortunately the time process takes to free > up its memory after receiving a SIGKILL might vary based on the state > of the process (uninterruptible sleep), size and OPP level of the core > the process is running. A mechanism to free resources of the target > process in a more predictable way would improve system's ability to > control its memory pressure. > Introduce process_reap system call that reclaims memory of a dying process > from the context of the caller. This way the memory in freed in a more > controllable way with CPU affinity and priority of the caller. The workload > of freeing the memory will also be charged to the caller. > The operation is allowed only on a dying process. > > Previously I proposed a number of alternatives to accomplish this: > - https://lore.kernel.org/patchwork/patch/1060407 extending > pidfd_send_signal to allow memory reaping using oom_reaper thread; > - https://lore.kernel.org/patchwork/patch/1338196 extending > pidfd_send_signal to reap memory of the target process synchronously from > the context of the caller; > - https://lore.kernel.org/patchwork/patch/1344419/ to add MADV_DONTNEED > support for process_madvise implementing synchronous memory reaping. > > The end of the last discussion culminated with suggestion to introduce a > dedicated system call (https://lore.kernel.org/patchwork/patch/1344418/#1553875) > The reasoning was that the new variant of process_madvise > a) does not work on an address range > b) is destructive > c) doesn't share much code at all with the rest of process_madvise > From the userspace point of view it was awkward and inconvenient to provide > memory range for this operation that operates on the entire address space. > Using special flags or address values to specify the entire address space > was too hacky. > > The API is as follows, > > int process_reap(int pidfd, unsigned int flags); > > DESCRIPTION > The process_reap() system call is used to free the memory of a > dying process. > > The pidfd selects the process referred to by the PID file > descriptor. > (See pidofd_open(2) for further information) *pidfd_open > > The flags argument is reserved for future use; currently, this > argument must be specified as 0. > > RETURN VALUE > On success, process_reap() returns 0. On error, -1 is returned > and errno is set to indicate the error. > > Signed-off-by: Suren Baghdasaryan Thanks for continuously pushing this. One question I have is how do you envision this syscall to be used for the cgroup based workloads. Traverse the target tree, read pids from cgroup.procs files, pidfd_open them, send SIGKILL and then process_reap them. Is that right? Orthogonal to this patch I wonder if we should have an optimized way to reap processes from a cgroup. Something similar to cgroup.kill (or maybe overload cgroup.kill with reaping as well). [...] > + > +SYSCALL_DEFINE2(process_reap, int, pidfd, unsigned int, flags) > +{ > + struct pid *pid; > + struct task_struct *task; > + struct mm_struct *mm = NULL; > + unsigned int f_flags; > + long ret = 0; > + > + if (flags != 0) > + return -EINVAL; > + > + pid = pidfd_get_pid(pidfd, &f_flags); > + if (IS_ERR(pid)) > + return PTR_ERR(pid); > + > + task = get_pid_task(pid, PIDTYPE_PID); > + if (!task) { > + ret = -ESRCH; > + goto put_pid; > + } > + > + /* > + * If the task is dying and in the process of releasing its memory > + * then get its mm. > + */ > + task_lock(task); > + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) { task_will_free_mem() is fine here but I think in parallel we should optimize this function. At the moment it is traversing all the processes on the machine. It is very normal to have tens of thousands of processes on big machines, so it would be really costly when reaping a bunch of processes.