From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB43AC4742C for ; Mon, 2 Nov 2020 20:29:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2712120719 for ; Mon, 2 Nov 2020 20:29:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jPO0UrA+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2712120719 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 54BAE6B0036; Mon, 2 Nov 2020 15:29:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D4FE6B005C; Mon, 2 Nov 2020 15:29:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 377926B0068; Mon, 2 Nov 2020 15:29:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id F38836B0036 for ; Mon, 2 Nov 2020 15:29:37 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 93145181AEF07 for ; Mon, 2 Nov 2020 20:29:37 +0000 (UTC) X-FDA: 77440618794.08.name71_2c0488a272b2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 640701819E76C for ; Mon, 2 Nov 2020 20:29:37 +0000 (UTC) X-HE-Tag: name71_2c0488a272b2 X-Filterd-Recvd-Size: 8911 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Mon, 2 Nov 2020 20:29:36 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id k18so10737420wmj.5 for ; Mon, 02 Nov 2020 12:29:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ECOxljNhNX/s1sQ6iseERJZz03J3Iz4bJCTWa11aECk=; b=jPO0UrA+g8o1Z94W0ABEZZV+bkPHrnTbfSRspSkgKKEvySDQKARqABAqhAPSZhyPPu egtFdNIu1i1Ue/xxkt0NhC2mEbVb59sN81GRBmSf7cnvvmDcVV7B20mnMXionPtPiRCZ 7Tp/ALKy6GXTAPbbS/mcYUxxtVngeYdwZH6xE9u6gnIbdPkes8jzXHjkGPlJhtRCRHOJ YrIFb90mCDqzB55cBoXZjHARhiQUBIaVH+2iAnGHyYW22ppskRea8zUb97CO2UBPXWKk mnHySbK9GOyi0eQ0GKdGsfg2KDJXiBW+Q+pgSC7Z82uxMbeIpAOYvh8NJmnTn0s9+Ha6 8wMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ECOxljNhNX/s1sQ6iseERJZz03J3Iz4bJCTWa11aECk=; b=Pck6VzFR23Msd2qfvpfSfTf7TYHWwaMDuk7tM+UPMEB05w8FQ1G1dIdAF6vdX/7Gbv 20tQZR37tJ2ZvHXmFfRWp4MlNZFaYqHE7gwKEHPEI9Gm8Hho6mMT4LVFl2MC5HKEkQrg rkQE6/zb2+Rof/RFvJEuLnRnX2mh5smmVqXxieCSEMyW5h5DdzpNKpWFs9GtvSnN/iqF MqqnT/anXW9DPBKyaM4MbIiBbkWhKAVR+DdPyMe05ygJXxpl6ocX5LQfJz6A5Zm2BQHc 84PWtgfmBYO2vRgudICtC2PSBtbWHNpvo0+ukwnrMDOa6y3Nh670dZ2fNIPvhopDXo/r 0Nfw== X-Gm-Message-State: AOAM532wBpPunfhNmnJ2asnQF09g806XOOfS4IiK8QZUpHB0JGmKrTmt HuWrGPNwW0MRH+2+7RS/W5Use/0WCIxKupeqTRI5Eg== X-Google-Smtp-Source: ABdhPJyBqPT7hImxXae117HZTa3xYjk6cLTEcHGERmKQOGcQAtw23SsJ9EEH1gW1YbRM9hOa//4svSCGGim8n//N9+I= X-Received: by 2002:a1c:ba0b:: with SMTP id k11mr8999674wmf.37.1604348975030; Mon, 02 Nov 2020 12:29:35 -0800 (PST) MIME-Version: 1.0 References: <20201014120937.GC4440@dhcp22.suse.cz> <20201015092030.GB22589@dhcp22.suse.cz> In-Reply-To: From: Suren Baghdasaryan Date: Mon, 2 Nov 2020 12:29:24 -0800 Message-ID: Subject: Re: [RFC]: userspace memory reaping To: Michal Hocko Cc: linux-api@vger.kernel.org, linux-mm , Andrew Morton , David Rientjes , Matthew Wilcox , Johannes Weiner , Roman Gushchin , Rik van Riel , Minchan Kim , Christian Brauner , Oleg Nesterov , Tim Murray , kernel-team , LKML , Mel Gorman Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 15, 2020 at 12:25 PM Suren Baghdasaryan wrote: > > On Thu, Oct 15, 2020 at 2:20 AM Michal Hocko wrote: > > > > On Wed 14-10-20 09:57:20, Suren Baghdasaryan wrote: > > > On Wed, Oct 14, 2020 at 5:09 AM Michal Hocko wrote: > > [...] > > > > > > The need is similar to why oom-reaper was introduced - when a process > > > > > > is being killed to free memory we want to make sure memory is freed > > > > > > even if the victim is in uninterruptible sleep or is busy and reaction > > > > > > to SIGKILL is delayed by an unpredictable amount of time. I > > > > > > experimented with enabling process_madvise(MADV_DONTNEED) operation > > > > > > and using it to force memory reclaim of the target process after > > > > > > sending SIGKILL. Unfortunately this approach requires the caller to > > > > > > read proc/pid/maps to extract the list of VMAs to pass as an input to > > > > > > process_madvise(). > > > > > > > > Well I would argue that this is not really necessary. You can simply > > > > call process_madvise with the full address range and let the kernel > > > > operated only on ranges which are safe to tear down asynchronously. > > > > Sure that would require some changes to the existing code to not fail > > > > on those ranges if they contain incompatible vmas but that should be > > > > possible. If we are worried about backward compatibility then a > > > > dedicated flag could override. > > > > > > > > > > IIUC this is very similar to the last option I proposed. I think this > > > is doable if we treat it as a special case. process_madvise() return > > > value not being able to handle a large range would still be a problem. > > > Maybe we can return MAX_INT in those cases? > > > > madvise is documented to return > > On success, madvise() returns zero. On error, it returns -1 and > > errno is set appropriately. > > [...] > > NOTES > > Linux notes > > The Linux implementation requires that the address addr be > > page-aligned, and allows length to be zero. If there are some > > parts of the specified address range that are not mapped, the > > Linux version of madvise() ignores them and applies the call to > > the rest (but returns ENOMEM from the system call, as it should). > > > > I have learned about ENOMEM case only now. And it seems this is indeed > > what we are implementing. So if we want to add a new mode to > > opportunistically attempt madvise on the whole given range without a > > failure then we need a specific flag for that. Advice is a number rather > > than a bitmask but (ab)using the top bit or use negative number space > > (e.g. -MADV_DONTNEED) for that sounds possible albeit bit hackish. > > process_madvise() has an additional flag parameter. Why not have a > separate flag to denote that we want to just skip VMA gaps and proceed > without error? Something like MADVF_SKIP_GAPS? > > > > > [...] > > > > I do have a vague recollection that we have discussed a kill(2) based > > > > approach as well in the past. Essentially SIG_KILL_SYNC which would > > > > not only send the signal but it would start a teardown of resources > > > > owned by the task - at least those we can remove safely. The interface > > > > would be much more simple and less tricky to use. You just make your > > > > userspace oom killer or potentially other users call SIG_KILL_SYNC which > > > > will be more expensive but you would at least know that as many > > > > resources have been freed as the kernel can afford at the moment. > > > > > > Correct, my early RFC here > > > https://patchwork.kernel.org/project/linux-mm/patch/20190411014353.113252-3-surenb@google.com > > > was using a new flag for pidfd_send_signal() to request mm reaping by > > > oom-reaper kthread. IIUC you propose to have a new SIG_KILL_SYNC > > > signal instead of a new pidfd_send_signal() flag and otherwise a very > > > similar solution. Is my understanding correct? > > > > Well, I think you shouldn't focus too much on the oom-reaper aspect > > of it. Sure it can be used for that but I believe that a new signal > > should provide a sync behavior. People more familiar with the process > > management would be better off defining what is possible for a new sync > > signal. Ideally not only pro-active process destruction but also sync > > waiting until the target process is released so that you know that once > > kill syscall returns the process is gone. > > If your suggestion is for SIG_KILL_SYNC to perform victim's resource > cleanup in the context of the caller while the victim is in > uninterruptible sleep that would definitely be useful. I assume there > are some resources which can't be reclaimed until the process itself > wakes up and handles the SIGKILL. If so, I hope kill(SIG_KILL_SYNC) > would not have to wait for the victim to wake up and handle the > signal. This would really complicate the userspace in cases when we > just want to reclaim whatever we can without victim's involvement and > continue. For cases when waiting is required waitid() with P_PIDFD can > be used. > Would this semantic work? > To follow up on this. Should I post an RFC implementing SIGKILL_SYNC which in addition to sending a kill signal would also reap the victim's mm in the context of the caller? Maybe having some code will get the discussion moving forward? > > > > -- > > Michal Hocko > > SUSE Labs