From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA8FDC433F5 for ; Tue, 21 Sep 2021 18:55:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91FF1611CE for ; Tue, 21 Sep 2021 18:55:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 91FF1611CE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id DB2A3900003; Tue, 21 Sep 2021 14:55:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6168900002; Tue, 21 Sep 2021 14:55:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4FC2900003; Tue, 21 Sep 2021 14:55:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id B1D06900002 for ; Tue, 21 Sep 2021 14:55:27 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C95572DD7C for ; Tue, 21 Sep 2021 18:55:25 +0000 (UTC) X-FDA: 78612483810.11.357CF4B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 78F8A6001988 for ; Tue, 21 Sep 2021 18:55:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 449AB61186; Tue, 21 Sep 2021 18:55:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1632250524; bh=4nfKbEEvnuIK73VTBrjV7lsPLdW7d5cR0/3cZDpflDE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=XQ+uwo9Ce+qeUe704wH19OCWWpxZTbGGZupNJ9QAWTxokV6Pl54kFG8CGUjfSin5P 2qTK4wVyhoeLepL7/wCyWf9SYbghTbeZ9lU9z7iIS+ZxtmF1wiEpq6H4d9beaaP3bG FCxw6qzVjGkphbM53CO0re/aquN3en7e4TUSt7NY= Date: Tue, 21 Sep 2021 11:55:23 -0700 From: Andrew Morton To: Vasily Averin Cc: Tetsuo Handa , Michal Hocko , Johannes Weiner , Vladimir Davydov , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel@openvz.org, "Uladzislau Rezki (Sony)" Subject: Re: [PATCH mm] vmalloc: back off when the current task is OOM-killed Message-Id: <20210921115523.8606cea0b2f0a5ca4e79cbd0@linux-foundation.org> In-Reply-To: References: <20210919163126.431674722b8db218453dc18c@linux-foundation.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=XQ+uwo9C; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: qxr34x5onimjysob9wzp95o7r8cabbxs X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 78F8A6001988 X-HE-Tag: 1632250525-342452 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 20 Sep 2021 13:59:35 +0300 Vasily Averin wrote: > On 9/20/21 4:22 AM, Tetsuo Handa wrote: > > On 2021/09/20 8:31, Andrew Morton wrote: > >> On Fri, 17 Sep 2021 11:06:49 +0300 Vasily Averin wrote: > >> > >>> Huge vmalloc allocation on heavy loaded node can lead to a global > >>> memory shortage. A task called vmalloc can have the worst badness > >>> and be chosen by OOM-killer, however received fatal signal and > >>> oom victim mark does not interrupt allocation cycle. Vmalloc will > >>> continue allocating pages over and over again, exacerbating the crisis > >>> and consuming the memory freed up by another killed tasks. > >>> > >>> This patch allows OOM-killer to break vmalloc cycle, makes OOM more > >>> effective and avoid host panic. > >>> > >>> Unfortunately it is not 100% safe. Previous attempt to break vmalloc > >>> cycle was reverted by commit b8c8a338f75e ("Revert "vmalloc: back off when > >>> the current task is killed"") due to some vmalloc callers did not handled > >>> failures properly. Found issues was resolved, however, there may > >>> be other similar places. > >> > >> Well that was lame of us. > >> > >> I believe that at least one of the kernel testbots can utilize fault > >> injection. If we were to wire up vmalloc (as we have done with slab > >> and pagealloc) then this will help to locate such buggy vmalloc callers. > > Andrew, could you please clarify how we can do it? > Do you mean we can use exsiting allocation fault injection infrastructure to trigger > such kind of issues? Unfortunately I found no ways to reach this goal. > It allows to emulate single faults with small probability, however it is not enough, > we need to completely disable all vmalloc allocations. I don't see why there's a problem? You're saying "there might still be vmalloc() callers which don't correctly handle allocation failures", yes? I'm suggesting that we use fault injection to cause a small proportion of vmalloc() calls to artificially fail, so such buggy callers will eventually be found and fixed. Why does such a scheme require that *all* vmalloc() calls fail?