From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C2E3CA9EB9 for ; Wed, 23 Oct 2019 17:57:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 202D02086D for ; Wed, 23 Oct 2019 17:57:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20150623.gappssmtp.com header.i=@cmpxchg-org.20150623.gappssmtp.com header.b="uo+vdPpQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 202D02086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92CB46B0003; Wed, 23 Oct 2019 13:57:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DBA16B0006; Wed, 23 Oct 2019 13:57:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CA976B0007; Wed, 23 Oct 2019 13:57:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 56ACB6B0003 for ; Wed, 23 Oct 2019 13:57:28 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 08E255DD8 for ; Wed, 23 Oct 2019 17:57:28 +0000 (UTC) X-FDA: 76075806576.13.cause37_410922b4dfc3c X-HE-Tag: cause37_410922b4dfc3c X-Filterd-Recvd-Size: 5853 Received: from mail-qk1-f193.google.com (mail-qk1-f193.google.com [209.85.222.193]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 23 Oct 2019 17:57:26 +0000 (UTC) Received: by mail-qk1-f193.google.com with SMTP id e66so20596449qkf.13 for ; Wed, 23 Oct 2019 10:57:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=wxnBd+F8Nf7wSyQxckHqdi1Fetryll1VVy13XOaIum4=; b=uo+vdPpQUGBdDsiOxiU5xzOq3ciq7/h61CPNrbHH6bEnPeZWi70jwzdXokMQ61oSK8 rTXtFa0lcLXkvqhPULewxxxUtDFV4OdEKkF+3MtGAq3oD4t33ui3rlQmryCA5dHrhbH8 1L/cOeBr+WvWi8jookl/mPz73fiJY5q2vVKtTsEiemt5411aG6pJeH0tzuqwBPlB2WVx GkS5UYatQVBGvg4CBUNn/DBINBMX4y6slxramY0g2OZMbSZhGy9Ery80tGLSo5VV6+z0 zsvnWlfB8EpmVZSz4gPBC5XrCPkY+2/OWD0KhmrzKkgZHDoG8tL1Vn0iasFo8DSuLlM/ 84Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=wxnBd+F8Nf7wSyQxckHqdi1Fetryll1VVy13XOaIum4=; b=UWxKf2ZBJNaLgJSj6qolf7fLnXh61d4aIeW7bQC7/2KW845IuZE+IEEg9H20DF6VbQ ZOqLJgJzzMVx8nL4itL8j/KWBnI7ojiGr4FQvVrn15rf+Yqy3vhC4rEgSSwtR1NiHmaM E1j4GRXI4PsG+ZLGxy+/D7NjbaWMYAxdbsbS5iKejKLUKwV0c7W3ACvafPCoGZk8hkRE cPI2HiR/jI+rltGH1hFuTmYrtOX2UUawod+nD4gGFvsVQTM5Q0Fcig2aIEG0F/3brpqu kHWpGQEZpgOM7Z/iUJGMO5pjnqviohuGaPByyTbvAq1Ju6hnENwIPGUYXuZnQEbnuo9R lLrw== X-Gm-Message-State: APjAAAUCEWV/H51gSFHKlmyOfWSL4iGWmUOqJW9IMJVm6oKBNxx8UQ/b P/y/Bj4D2RY2kbtgPH4p3qs/ng== X-Google-Smtp-Source: APXvYqxYbgcO74tp3k4i+lmIz9XorffsmYUz6wGRkYQdXBMa1aSlDHfIgcUcydeXFQhdMxs6OJ36wQ== X-Received: by 2002:a05:620a:140c:: with SMTP id d12mr2216973qkj.375.1571853446084; Wed, 23 Oct 2019 10:57:26 -0700 (PDT) Received: from localhost ([2620:10d:c091:500::2:c4de]) by smtp.gmail.com with ESMTPSA id 81sm15001008qkd.73.2019.10.23.10.57.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Oct 2019 10:57:25 -0700 (PDT) Date: Wed, 23 Oct 2019 13:57:24 -0400 From: Johannes Weiner To: Michal Hocko Cc: Andrew Morton , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH 2/2] mm: memcontrol: try harder to set a new memory.high Message-ID: <20191023175724.GD366316@cmpxchg.org> References: <20191022201518.341216-1-hannes@cmpxchg.org> <20191022201518.341216-2-hannes@cmpxchg.org> <20191023065949.GD754@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191023065949.GD754@dhcp22.suse.cz> User-Agent: Mutt/1.12.2 (2019-09-21) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 23, 2019 at 08:59:49AM +0200, Michal Hocko wrote: > On Tue 22-10-19 16:15:18, Johannes Weiner wrote: > > Setting a memory.high limit below the usage makes almost no effort to > > shrink the cgroup to the new target size. > > > > While memory.high is a "soft" limit that isn't supposed to cause OOM > > situations, we should still try harder to meet a user request through > > persistent reclaim. > > > > For example, after setting a 10M memory.high on an 800M cgroup full of > > file cache, the usage shrinks to about 350M: > > > > + cat /cgroup/workingset/memory.current > > 841568256 > > + echo 10M > > + cat /cgroup/workingset/memory.current > > 355729408 > > > > This isn't exactly what the user would expect to happen. Setting the > > value a few more times eventually whittles the usage down to what we > > are asking for: > > > > + echo 10M > > + cat /cgroup/workingset/memory.current > > 104181760 > > + echo 10M > > + cat /cgroup/workingset/memory.current > > 31801344 > > + echo 10M > > + cat /cgroup/workingset/memory.current > > 10440704 > > > > To improve this, add reclaim retry loops to the memory.high write() > > callback, similar to what we do for memory.max, to make a reasonable > > effort that the usage meets the requested size after the call returns. > > That suggests that the reclaim couldn't meet the given reclaim target > but later attempts just made it through. Is this due to amount of dirty > pages or what prevented the reclaim to do its job? > > While I am not against the reclaim retry loop I would like to understand > the underlying issue. Because if this is really about dirty memory then > we should probably be more pro-active in flushing it. Otherwise the > retry might not be of any help. All the pages in my test case are clean cache. But they are active, and they need to go through the inactive list before reclaiming. The inactive list size is designed to pre-age just enough pages for regular reclaim targets, i.e. pages in the SWAP_CLUSTER_MAX ballpark, In this case, the reclaim goal for a single invocation is 790M and the inactive list is a small funnel to put all that through, and we need several iterations to accomplish that. But 790M is not a reasonable reclaim target to ask of a single reclaim invocation. And it wouldn't be reasonable to optimize the reclaim code for it. So asking for the full size but retrying is not a bad choice here: we express our intent, and benefit if reclaim becomes better at handling larger requests, but we also acknowledge that some of the deltas we can encounter in memory_high_write() are just too ridiculously big for a single reclaim invocation to manage.