From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E86EC47254 for ; Tue, 5 May 2020 15:03:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 39FAB20757 for ; Tue, 5 May 2020 15:03:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NedESV1m" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39FAB20757 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B44B58E0006; Tue, 5 May 2020 11:03:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1BF68E0003; Tue, 5 May 2020 11:03:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A311E8E0006; Tue, 5 May 2020 11:03:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 8A8688E0003 for ; Tue, 5 May 2020 11:03:31 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3B2A5180ACEF0 for ; Tue, 5 May 2020 15:03:31 +0000 (UTC) X-FDA: 76782984222.20.air29_194fae5b0108 X-HE-Tag: air29_194fae5b0108 X-Filterd-Recvd-Size: 6389 Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Tue, 5 May 2020 15:03:30 +0000 (UTC) Received: by mail-lf1-f68.google.com with SMTP id h26so1633145lfg.6 for ; Tue, 05 May 2020 08:03:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=G5HI11tS+yKnZwbaq8TtL0vEO4XhsA9qTOxtb1zDBVc=; b=NedESV1mf2KEGbnGwBXjkNGi7R9SVO/NUVnBR/MIIA4ksHkYG9Y0BOJUSO7cPfF101 DhqLvXmcnRAqIon/UhQwhmdhaRJTFMRbqv7rnUrFiiLplm7OYMLhdbiEBbtARuNhVV8D LTzgdGvjzWUJX1UQL/oIjMDuzzQetiMjcVkKOHZidnk7VXhouvnvChgf89k1EWSMWRjT FhIwRnMbbr/4G57qwJzkX9wXI0g0aGrRDYcsPodwxKY6npl93JABKfVo0PFuu+LIkLAx gWRTkTusvXUAXg8/Xmo6bq3CqbG9PEAZT5/liY82/n294mGhWG9jm1kwV8U5Yd9K6Gur 9WEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=G5HI11tS+yKnZwbaq8TtL0vEO4XhsA9qTOxtb1zDBVc=; b=Muu03J6doP6CPRgmYYPqfvyt6wnTOHL8dXJWN4JYaomdVKq/vxZEk+LQ4kw1ZxvesC uyDTydQzYhvyv6rhBpa0qXI1wv5euOxAoO3V2NpMnKKP5UWFortxqF2WxVxvOu1IlGic XjblmxvTyEyOJk+H5/KmAspJJqd1KSOolZ9+TwtDgnn50OqLuayH8UJR68nbLtX5hPfy Qk1phLTB5f86gQ+7/kVVFmvcW6oo7bocAj5O/4XOHdqoRsKWma2v5YGc870sXhlpxDjU 4Z5Id+2DrONp5ZKVA0qa9f1wr1XdveOdrsF7SSi9m/nYuwLAi/OqbhtjPYKfEZDY9Xzx /exg== X-Gm-Message-State: AGi0PuZjU/yVaMfjVMcs3CIz5BdhEkXCqlfXyeM1XvogOEPVII5Pb5To HsUPiG3uoVR0xET4GMpet3ddA9hnlZ5FUozUCAuyuw== X-Google-Smtp-Source: APiQypIYjYUvEtVGNd27c1+JwM3UC4JdspvxdSp2uB6/Zn/IP6HBQYr4Yxw+YKakR94PGo55lnU3ndTA9L+bkjrGVfM= X-Received: by 2002:a19:c85:: with SMTP id 127mr1421149lfm.189.1588691008811; Tue, 05 May 2020 08:03:28 -0700 (PDT) MIME-Version: 1.0 References: <20200430182712.237526-1-shakeelb@google.com> <20200504065600.GA22838@dhcp22.suse.cz> <20200504141136.GR22838@dhcp22.suse.cz> <20200504150052.GT22838@dhcp22.suse.cz> <20200504160613.GU22838@dhcp22.suse.cz> <20200505071324.GB16322@dhcp22.suse.cz> In-Reply-To: <20200505071324.GB16322@dhcp22.suse.cz> From: Shakeel Butt Date: Tue, 5 May 2020 08:03:17 -0700 Message-ID: Subject: Re: [PATCH] memcg: oom: ignore oom warnings from memory.max To: Michal Hocko Cc: Johannes Weiner , Roman Gushchin , Greg Thelen , Andrew Morton , Linux MM , Cgroups , LKML Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 5, 2020 at 12:13 AM Michal Hocko wrote: > > On Mon 04-05-20 12:23:51, Shakeel Butt wrote: > [...] > > *Potentially* useful for debugging versus actually beneficial for > > "sweep before tear down" use-case. > > I definitely do not want to prevent you from achieving what you > want/need. Let's get back to your argument on why you cannot use > memory.high for this purpose and what is the actual difference from > memory.max on the "sweep before removal". You've said > > : Yes that would work but remote charging concerns me. Remote charging > : can still happen after the memcg is offlined and at the moment, high > : reclaim does not work for remote memcg and the usage can go till max > : or global pressure. This is most probably a misconfiguration and we > : might not receive the warnings in the log ever. Setting memory.max to > : 0 will definitely give such warnings. > > So essentially the only reason you are not using memory.high which would > effectively achieve the same level of reclaim for your usecase is that > potential future remote charges could get unnoticed. Yes. > I have proposed to > warn when charging to an offline memcg because that looks like a sign of > bug to me. Instead of a bug, I would say misconfiguration but there is at least a genuine user i.e. buffer_head. It can be allocated in reclaim and trigger remote charging but it should be fine as the page it is attached to will possibly get freed soon. So, I don't think we want to warn for all remote charges to an offlined memcg. > Having the hard limit clamped to 0 (or some other small > value) would complain loud by the oom report and no eligible tasks > message but it will unlikely help to stop such a usage because, well, > there is nothing reclaimable and we force the charge in that case. So > you are effectively in the memory.high like situation. Yes, effectively it will be similar to memory.high but at least we will get early warnings. Now rethinking about the remote charging of buffer_head to an offlined memcg with memory.max=0. It seems like it is special in the sense that it is using __GFP_NOFAIL and will skip the oom-killer and thus warnings. Maybe the right approach is, as you suggested, always warn for charging an offline memcg unless (__GFP_NOFAIL|__GFP_RETRY_MAYFAIL). Though I am not sure if this is doable without code duplication. > > So instead of potentially removing a useful information can we focus on > the remote charging side of the problem and deal with it in a sensible > way? That would make memory.high usable for your usecase and I still > believe that this is what you should be using in the first place. We talked about this at LSFMM'19 and I think the decision was to not fix high reclaim for remote memcg until it will be an actual issue. I suppose now we can treat it as an actual issue. There are a couple of open questions: 1) Should the remote chargers be throttled and do the high reclaim? 2) There can be multiple remote charges to multiple memcgs in a single kernel entry. Should we handle such scenarios? Shakeel