From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A778C4727F for ; Tue, 22 Sep 2020 04:21:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 84A9A23A1E for ; Tue, 22 Sep 2020 04:21:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eF/EbBSe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84A9A23A1E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B5FC4900021; Tue, 22 Sep 2020 00:21:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1051900012; Tue, 22 Sep 2020 00:21:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A261C900021; Tue, 22 Sep 2020 00:21:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 8C6AC900012 for ; Tue, 22 Sep 2020 00:21:29 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4B7748249980 for ; Tue, 22 Sep 2020 04:21:29 +0000 (UTC) X-FDA: 77289398298.13.ants00_43088012714a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 2A7D918140B67 for ; Tue, 22 Sep 2020 04:21:29 +0000 (UTC) X-HE-Tag: ants00_43088012714a X-Filterd-Recvd-Size: 7901 Received: from mail-il1-f194.google.com (mail-il1-f194.google.com [209.85.166.194]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 04:21:28 +0000 (UTC) Received: by mail-il1-f194.google.com with SMTP id x18so13567471ila.7 for ; Mon, 21 Sep 2020 21:21:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZNNbuALh04//uifYVPdDjh++fb2lT/Xqcs4an3gHVio=; b=eF/EbBSe9yFBqTjdwzbFsunrkCPqbYXgfxHDSEkrckP5A1JHkh3XKpgKh1/IGo/vpm PxlwoZMqDyyio84Vt96+iIDsRTiLJBgiQsTvGsDCpKt/pgm36IvvjTbwaqIKrGYde6Hi 0BHrqmtRg+GVQzDb9NRZEk/zfFZn8kaIVBnLwClJLRYsQc+uNdTo9V0vEmmil93p8bpC gB9+qh4AB+tp0jmSgoo6Fcn9UmBpXDRPaykudNaQMqKbTqnFcAEL0jv3aGMx7vnanVue GIlYCRrZPBKR9PHUH6u1kyqfeUZu/yQM1gKgTq9owy3HhmFOH2sUvEg8zRosVx5M7MpM vbyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZNNbuALh04//uifYVPdDjh++fb2lT/Xqcs4an3gHVio=; b=FKIIH4bLH/s0fREXd4k6y5+rGH7/dFChmDtDWo8hLTp62Psq9VnXIAjj6c5kW7E9To 4HrNCR90s6pExcrWycLWqNDRbu7PnD5eBJ/Nn1bSTtnKvAc8Vu2xb3e0TpAwnIIgXImZ 4AiqOceg94jcyVZAOkk+4ewLAWsDSVrZqsxoy9KCbDi7Pe5rkshwLnyOoJCbYVn/1m9e IyXHibvb0UkpmzOPPzK7wrAu6opmJgbXfjlLn4y5YKGcH8rpcA5iMxIOVdbVzZWqYDOm 1rxHM5XyU0V90F+xTRF7WF9DeyN31ITb+z8VjDxO04nZcd8lfVK02XvrA8J7rfyNJPOu wwIQ== X-Gm-Message-State: AOAM533tzLThPk7ulSXRdwEcXipmLloSt+F79s8TAX5MsTnDCgFqB9lT kq8Wwy/LRCek3/9mGlEhHxuNsRQoGMbQvuApsMY= X-Google-Smtp-Source: ABdhPJzDl8MdjPVNRJmS4FRLkv0hUY6IYs3el3g0ioVtIQKA0mfz3YZi4KUDwxEL6Q++VW2yZEYb9pNfWRlG4ObXVfo= X-Received: by 2002:a92:c7b0:: with SMTP id f16mr2847131ilk.137.1600748488148; Mon, 21 Sep 2020 21:21:28 -0700 (PDT) MIME-Version: 1.0 References: <20200921080255.15505-1-zangchunxin@bytedance.com> <20200921081200.GE12990@dhcp22.suse.cz> <20200921110505.GH12990@dhcp22.suse.cz> <20200921113646.GJ12990@dhcp22.suse.cz> In-Reply-To: <20200921113646.GJ12990@dhcp22.suse.cz> From: Yafang Shao Date: Tue, 22 Sep 2020 12:20:52 +0800 Message-ID: Subject: Re: [PATCH] mm/memcontrol: Add the drop_cache interface for cgroup v2 To: Michal Hocko Cc: zangchunxin@bytedance.com, Johannes Weiner , Vladimir Davydov , Andrew Morton , Tejun Heo , lizefan@huawei.com, Jonathan Corbet , Alexei Starovoitov , Daniel Borkmann , kafai@fb.com, Song Liu , Yonghong Song , andriin@fb.com, john.fastabend@gmail.com, kpsingh@chromium.org, Cgroups , linux-doc@vger.kernel.org, Linux MM , LKML , netdev , bpf@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 21, 2020 at 7:36 PM Michal Hocko wrote: > > On Mon 21-09-20 19:23:01, Yafang Shao wrote: > > On Mon, Sep 21, 2020 at 7:05 PM Michal Hocko wrote: > > > > > > On Mon 21-09-20 18:55:40, Yafang Shao wrote: > > > > On Mon, Sep 21, 2020 at 4:12 PM Michal Hocko wrote: > > > > > > > > > > On Mon 21-09-20 16:02:55, zangchunxin@bytedance.com wrote: > > > > > > From: Chunxin Zang > > > > > > > > > > > > In the cgroup v1, we have 'force_mepty' interface. This is very > > > > > > useful for userspace to actively release memory. But the cgroup > > > > > > v2 does not. > > > > > > > > > > > > This patch reuse cgroup v1's function, but have a new name for > > > > > > the interface. Because I think 'drop_cache' may be is easier to > > > > > > understand :) > > > > > > > > > > This should really explain a usecase. Global drop_caches is a terrible > > > > > interface and it has caused many problems in the past. People have > > > > > learned to use it as a remedy to any problem they might see and cause > > > > > other problems without realizing that. This is the reason why we even > > > > > log each attempt to drop caches. > > > > > > > > > > I would rather not repeat the same mistake on the memcg level unless > > > > > there is a very strong reason for it. > > > > > > > > > > > > > I think we'd better add these comments above the function > > > > mem_cgroup_force_empty() to explain why we don't want to expose this > > > > interface in cgroup2, otherwise people will continue to send this > > > > proposal without any strong reason. > > > > > > I do not mind people sending this proposal. "V1 used to have an > > > interface, we need it in v2 as well" is not really viable without > > > providing more reasoning on the specific usecase. > > > > > > _Any_ patch should have a proper justification. This is nothing really > > > new to the process and I am wondering why this is coming as a surprise. > > > > > > > Container users always want to drop cache in a specific container, > > because they used to use drop_caches to fix memory pressure issues. > > This is exactly the kind of problems we have seen in the past. There > should be zero reason to addre potential reclaim problems by dropping > page cache on the floor. There is a huge cargo cult about this > procedure and I have seen numerous reports when people complained about > performance afterwards just to learn that the dropped page cache was one > of the resons for that. > > > Although drop_caches can cause some unexpected issues, it could also > > fix some issues. > > "Some issues" is way too general. We really want to learn about those > issues and address them properly. > One use case in our production environment is that some of our tasks become very latency sensitive from 7am to 10am, so before these tasks become active we will use drop_caches to drop page caches generated by other tasks at night to avoid these tasks triggering direct reclaim. The best way to do it is to fix the latency in direct reclaim, but it will take great effort. while drop_caches give us an easier way to achieve the same goal. IOW, drop_caches give the users an option to achieve their goal before they find a better solution. > > So container users want to use it in containers as > > well. > > If this feature is not implemented in cgroup, they will ask you why > > but there is no explanation in the kernel. > > There is no usecase that would really require it so far. > > > Regarding the memory.high, it is not perfect as well, because you have > > to set it to 0 to drop_caches, and the processes in the containers > > have to reclaim pages as well because they reach the memory.high, but > > memory.force_empty won't make other processes to reclaim. > > Since 536d3bf261a2 ("mm: memcontrol: avoid workload stalls when lowering > memory.high") the limit is set after the reclaim so the race window when > somebody would be pushed to high limit reclaim is reduced. But I do > agree this is just a workaround. > > > That doesn't mean I agree to add this interface, while I really mean > > that if we discard one feature we'd better explain why. > > We need to understand why somebody wants an interface because once it is > added it will have to be maintained for ever. > -- > Michal Hocko > SUSE Labs -- Thanks Yafang