From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18203C433EF for ; Thu, 9 Dec 2021 03:00:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F3D86B0071; Wed, 8 Dec 2021 21:59:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A1DC6B0073; Wed, 8 Dec 2021 21:59:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61B046B0074; Wed, 8 Dec 2021 21:59:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 5095F6B0071 for ; Wed, 8 Dec 2021 21:59:58 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 167B784781 for ; Thu, 9 Dec 2021 02:59:48 +0000 (UTC) X-FDA: 78896750856.30.4EFFC63 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf18.hostedemail.com (Postfix) with ESMTP id EAA6D1C0009 for ; Thu, 9 Dec 2021 02:59:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1639018787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wfb3U3niqbzfyllKy5x8hcf0lVNrqUlULt6ivQ8k0u8=; b=fNsN3pn2Xanb1ETWQIcab+8oGR4YW2CuBuAg2FqN8JK4VtkbM0Pj1LGj16U6yxOGjgjghW 8nLIBqHUXnhCEMqejUXkK3CBrsyI+h6P+V2wUVw3wiOH8j/5J7V+rYDayQidCHXQUk7tPM +NECuUzSygQrPsEEHeffB/Tc/bnRSq4= Received: from mail-yb1-f200.google.com (mail-yb1-f200.google.com [209.85.219.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-598-41GHIvhTPtOCjXuCgP37pA-1; Wed, 08 Dec 2021 21:59:46 -0500 X-MC-Unique: 41GHIvhTPtOCjXuCgP37pA-1 Received: by mail-yb1-f200.google.com with SMTP id l75-20020a25254e000000b005f763be2fecso8089316ybl.7 for ; Wed, 08 Dec 2021 18:59:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wfb3U3niqbzfyllKy5x8hcf0lVNrqUlULt6ivQ8k0u8=; b=oK/EWSfnWiJE5tk+GtZnQA23LUUHthDTwemL5oqmoJ2StpfSP+MvqLT9LH1Vdsg9Qk SUOhajbYaNYniQJxf2qbYZTziGL3Ebu+ytYlaN8PcuGt0w8/9Q55A62BXN3E9G6rfROC wfU+h9A4ePkZAjstrTF+YLIaK9b09/PmOj3Ku8+h8JFKFFOnuuALmxG2mz3fg/oJ7e/v LermftYsVPURRpaiFLNe3MfN3dz6aB5xs7jR9T+TTApjVo6aVmLfOzrUM8/I6KVkdKfL aMcTvpZyWuXrWu/gm19h1KIfm9Jk65ysMz25wxSCcIjzP+G30ei683CeIv2Mxmxur73i +uRw== X-Gm-Message-State: AOAM530PETLLizmHnWT77EgZdR2V0pEYdlg4QWq1QpjbgFLYd3uSQsyK F32KRXzOEJhBvpn/xBw0jCfAa1vShPimQ0BnK3REx7o6mMqpX72zSGOV4mamOs6SrrLt8XcjQ/w LRTLaoC8EjFIHwdtdzC0K50aiCwQ= X-Received: by 2002:a25:d792:: with SMTP id o140mr3166115ybg.493.1639018785518; Wed, 08 Dec 2021 18:59:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJzBg0mmEpQ4xuCvkM0lS1TYLQJICuVFphZCyXkaM7g/28leqI7DNxQZ+/xxr0+/aZMzpmIyIOAygxl8cdrYg5Y= X-Received: by 2002:a25:d792:: with SMTP id o140mr3166080ybg.493.1639018785247; Wed, 08 Dec 2021 18:59:45 -0800 (PST) MIME-Version: 1.0 References: <20211207214902.772614-1-jsavitz@redhat.com> <20211207154759.3f3fe272349c77e0c4aca36f@linux-foundation.org> In-Reply-To: From: Joel Savitz Date: Wed, 8 Dec 2021 21:59:29 -0500 Message-ID: Subject: Re: [PATCH] mm/oom_kill: wake futex waiters before annihilating victim shared mutex To: Michal Hocko Cc: Andrew Morton , linux-kernel , Waiman Long , linux-mm@kvack.org, Nico Pache , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Darren Hart , Davidlohr Bueso , =?UTF-8?Q?Andr=C3=A9_Almeida?= X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EAA6D1C0009 X-Stat-Signature: hzbitzb89poz38qhde9ptd8rod6miatm Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fNsN3pn2; spf=none (imf18.hostedemail.com: domain of jsavitz@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=jsavitz@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-HE-Tag: 1639018786-28969 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 8, 2021 at 11:05 AM Michal Hocko wrote: > > On Wed 08-12-21 10:01:44, Michal Hocko wrote: > > On Tue 07-12-21 15:47:59, Andrew Morton wrote: > > > (cc's added) > > > > Extend CC to have all futex maintainers on board. > > > > > On Tue, 7 Dec 2021 16:49:02 -0500 Joel Savitz wrote: > > > > > > > In the case that two or more processes share a futex located within > > > > a shared mmaped region, such as a process that shares a lock between > > > > itself and a number of child processes, we have observed that when > > > > a process holding the lock is oom killed, at least one waiter is never > > > > alerted to this new development and simply continues to wait. > > > > > > Well dang. Is there any way of killing off that waiting process, or do > > > we have a resource leak here? > > > > > > > This is visible via pthreads by checking the __owner field of the > > > > pthread_mutex_t structure within a waiting process, perhaps with gdb. > > > > > > > > We identify reproduction of this issue by checking a waiting process of > > > > a test program and viewing the contents of the pthread_mutex_t, taking note > > > > of the value in the owner field, and then checking dmesg to see if the > > > > owner has already been killed. > > > > > > > > This issue can be tricky to reproduce, but with the modifications of > > > > this small patch, I have found it to be impossible to reproduce. There > > > > may be additional considerations that I have not taken into account in > > > > this patch and I welcome any comments and criticism. > > > > Why does OOM killer need a special handling. All the oom killer does is > > to send a fatal signal to the victim. Why is this any different from > > sending SIGKILL from the userspace? > > I have had a closer look and I guess I can see what you are trying to > achieve. futex_exit_release is normally called from exit_mm context. You > are likely seeing a situation when the oom victim is blocked and cannot > exit. That is certainly possible but it shouldn't be a permanent state. > So I would be more interested about your particular issue and how long > the task has been stuck unable to exit. Before applying this patch I never saw a task eventually exit during the reproduction of this system state. Every task in this waiting-on-a-dead-owner situation state appeared to be permanently blocked until user intervention killed it manually. > > Whether this is safe to be called from the oom killer context I cannot > really judge. That would be a question to Futex folks. I am also very interested in feedback from the Futex folks. This is the first fix for the bug that I have found but I am not sure whether this introduces other issues due to the context. > -- > Michal Hocko > SUSE Labs > Best, Joel Savitz