From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12B7BC433FE for ; Mon, 7 Mar 2022 18:31:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6ABC68D0002; Mon, 7 Mar 2022 13:31:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 65AC08D0001; Mon, 7 Mar 2022 13:31:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FB2C8D0002; Mon, 7 Mar 2022 13:31:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 3FD198D0001 for ; Mon, 7 Mar 2022 13:31:46 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 0C22B61CBE for ; Mon, 7 Mar 2022 18:31:46 +0000 (UTC) X-FDA: 79218433812.12.2A3F9F9 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf28.hostedemail.com (Postfix) with ESMTP id A0412C000B for ; Mon, 7 Mar 2022 18:31:45 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id 184-20020a6215c1000000b004f6dc47ec08so4450844pfv.21 for ; Mon, 07 Mar 2022 10:31:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BBupMD3/tVCuAYdiY+YqGDwvU5b3Do4EmcmiCaFj50U=; b=Qmu/VDQ43GmcvvTRFwWJEIiF5SJPgh3dgT+TSfg0vLR61qunFWUK3/OaVaTawdXmD8 naDoLszpSb3BHMQFxFsU9L25tR4CxwC7D7xKWgvO04Xu0VFwDGmbrTt+ECaPmfvIfsiZ fCoGyX+E98GyrcCfXZr9qXyIj3U+ldG80xKB94Kv/60XtkzssV3TPCeYGsitqpepiyY1 4Ab2htXEUlWXy+vNfmfRPO9CeFI5WdOshut+fZHzzwrQdl3x7T9OMst8WhaQLy0Le58F 6aBgHzVQzIsuADMBv5GfZR+7rmMWejJI07u8N6cdcSk5gkL+9SNSFQDqhTrmW7JwEzpf 0Y7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BBupMD3/tVCuAYdiY+YqGDwvU5b3Do4EmcmiCaFj50U=; b=hMzvtYw/YORA5z+/1gBM1huUa965Cc2RxtlVXxXUimkTHex2mQjXkJv1njTZibnj27 3fGLjzOVmu/HPUErXTKWTTnBYr22EGkWGpkONn7kUYP+bRP820j/QieZnCA4vJtS3+U2 BtTFH5qtwQm0FtCoxNBuH48BLSTf8Nl+pL50UvkQAV/CHvz2DI10hwT6T3CsLfB22ITH EgIg1mdklkPCNoR1zgvwLxiv15IYBwwDW0iwS/thEs9krkz2J76TDaVhiycqq0pNL/PP F9ZJO7MDVNmTIcEyjKAvnTUSjgnfofCddtPGXyQ4/CtGJBmZFAos7Fq237hMtTUliWp1 W1bg== X-Gm-Message-State: AOAM532KJEGlSKoZzLvh4P+jv3gnNYKkgC2mp54+s4pbgEIeI91Zz/Ms afZk1KoxlMNlv4RQX0isNOSqhXvlThsg9w== X-Google-Smtp-Source: ABdhPJzk1yw87jvTE9aVLoUL/4YSI7k/S69NV2L0Ol3O29VSTFRPb+yzT9k/sP0Owunk5r3nunB3YkiGA0VZBQ== X-Received: from shakeelb.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:28b]) (user=shakeelb job=sendgmr) by 2002:a17:902:d4c2:b0:151:d590:a13d with SMTP id o2-20020a170902d4c200b00151d590a13dmr11323099plg.85.1646677904363; Mon, 07 Mar 2022 10:31:44 -0800 (PST) Date: Mon, 7 Mar 2022 18:31:41 +0000 In-Reply-To: Message-Id: <20220307183141.npa4627fpbsbgwvv@google.com> Mime-Version: 1.0 References: <5df21376-7dd1-bf81-8414-32a73cea45dd@google.com> Subject: Re: [RFC] Mechanism to induce memory reclaim From: Shakeel Butt To: Michal Hocko Cc: David Rientjes , Andrew Morton , Johannes Weiner , Yu Zhao , Dave Hansen , linux-mm@kvack.org, Yosry Ahmed , Wei Xu , Greg Thelen Content-Type: text/plain; charset="UTF-8"; format=flowed; delsp=yes X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: ke3qw9hwxck99kyg1wr3ymaoszgmjixc Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="Qmu/VDQ4"; spf=pass (imf28.hostedemail.com: domain of 3kE8mYggKCN8TIBLFFMCHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--shakeelb.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3kE8mYggKCN8TIBLFFMCHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--shakeelb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Queue-Id: A0412C000B X-HE-Tag: 1646677905-604232 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Mar 07, 2022 at 03:41:45PM +0100, Michal Hocko wrote: > On Sun 06-03-22 15:11:23, David Rientjes wrote: > [...] > > Some questions to get discussion going: > > > > - Overall feedback or suggestions for the proposal in general? > Do we really need this interface? What would be usecases which cannot > use an existing interfaces we have for that? Most notably memcg and > their high limit? Let me take a stab at this. The specific reasons why high limit is not a good interface to implement proactive reclaim: 1) It can cause allocations from the target application to get throttled. 2) It leaves a state (high limit) in the kernel which needs to be reset by the userspace part of proactive reclaimer. If I remember correctly, Facebook actually tried to use high limit to implement the proactive reclaim but due to exactly these limitations [1] they went the route [2] aligned with this proposal. To further explain why the above limitations are pretty bad: The proactive reclaimers usually use feedback loop to decide how much to squeeze from the target applications without impacting their performance or impacting within a tolerable range. The metrics used for the feedback loop are either refaults or PSI and these metrics becomes messy due to application getting throttled due to high limit. For (2), the high limit interface is a very awkward interface to use to do proactive reclaim. If the userspace proactive reclaimer fails/crashed due to whatever reason during triggering the reclaim in an application, it can leave the application in a bad state (memory pressure state and throttled) for a long time. [1] https://lore.kernel.org/all/20200928210216.GA378894@cmpxchg.org/ [2] https://dl.acm.org/doi/10.1145/3503222.3507731 (Section 3.3)