From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 676B5C433E6 for ; Wed, 15 Jul 2020 07:00:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 272E6206F4 for ; Wed, 15 Jul 2020 07:00:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n3ujhdyr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 272E6206F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AED386B0003; Wed, 15 Jul 2020 03:00:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A759C6B0005; Wed, 15 Jul 2020 03:00:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93E146B0006; Wed, 15 Jul 2020 03:00:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0233.hostedemail.com [216.40.44.233]) by kanga.kvack.org (Postfix) with ESMTP id 787836B0003 for ; Wed, 15 Jul 2020 03:00:07 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 05AC98249980 for ; Wed, 15 Jul 2020 07:00:07 +0000 (UTC) X-FDA: 77039410854.15.humor30_2c1321e26ef7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id C97551814B0C8 for ; Wed, 15 Jul 2020 07:00:06 +0000 (UTC) X-HE-Tag: humor30_2c1321e26ef7 X-Filterd-Recvd-Size: 7086 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 15 Jul 2020 07:00:06 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id ch3so2622958pjb.5 for ; Wed, 15 Jul 2020 00:00:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=yBPygJDADyMNow8RF1wBXyoTGHqGjUzbugs5/7dNL0M=; b=n3ujhdyrkJ3XIm1Z2m0bjNTlOXVZRaAKbX073AtF4CoZfp+JL5moww96IYfa2Zp6CO Z6Hpm2uebEQCM39VwWuSv++0g/CcDGivEP6viEe3torjfdNt+1bgd5ow3ddxRujlYM1r Kiul0VFbezJQl35UeARAsbD+a2xyGDc6at7km2HIaGUEGtwGicYcaBJW2GGB5fUWLhel brUXOz/U12156GNtfE9BR15WhuZAUbuq85QdwqByA23LPsYe1f108hICSjufwSdbH/Vy 5NHSLei0PXIsxhmxPDQ06sW0iedaGzmzOl6dLXc4qRJnfKeJNvQd4chCnVs7uO8C8FuJ i8/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=yBPygJDADyMNow8RF1wBXyoTGHqGjUzbugs5/7dNL0M=; b=HbaOLicv1PeKfp54Piz5QxfmRmVrlI/QD93/Vl0shq5f6TdEY2h6VmwMFdElhzQxlO xN1b+DQ5rTv0BGU0ABjp3sFtkvhR2OnofLruyaUjXBdrU0Sh/R1KaEjpneZjo/4OHhCZ 5plPGS/knGdpDPm5dD2Mz9IVzIVgUNNksvIbUaYUGp27/u8Ssk8Y8jNyMnUZ9TyXX7NF 6hRgXHoKu7XjjTD0C2OHkXWp7sVVJYqo+Lvh5PNcZTvK2+fqdAs09HXoxbZx1L106O/5 iQu5xJOnf55xD/TybGejGUnKeA8OSoAd9qf2sNAIxQBnWC6pKylpDDzqcXh/uFoOtyI8 2gog== X-Gm-Message-State: AOAM530BaI07v8epJ479N8boVNwBbjAvJUrUOXWx/ERjrZ9oozByxrqm s4A7/doIJcnHOjqBTv3F8esM8w== X-Google-Smtp-Source: ABdhPJyPnCMJ8yxvUFrjJQm7SHOtB4nUtjz+pspNUf7GK2HE55bWBAiVOa+UnCEl+v+Er1qQDKLSUQ== X-Received: by 2002:a17:902:c086:: with SMTP id j6mr6867497pld.293.1594796405104; Wed, 15 Jul 2020 00:00:05 -0700 (PDT) Received: from [2620:15c:17:3:4a0f:cfff:fe51:6667] ([2620:15c:17:3:4a0f:cfff:fe51:6667]) by smtp.gmail.com with ESMTPSA id cv7sm1029072pjb.9.2020.07.15.00.00.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 00:00:04 -0700 (PDT) Date: Wed, 15 Jul 2020 00:00:03 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Andrew Morton , Yang Shi cc: Michal Hocko , Shakeel Butt , Yang Shi , Roman Gushchin , Greg Thelen , Johannes Weiner , Vladimir Davydov , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [patch] mm, memcg: provide a stat to describe reclaimable memory In-Reply-To: Message-ID: References: User-Agent: Alpine 2.23 (DEB 453 2020-06-18) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: C97551814B0C8 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 14 Jul 2020, David Rientjes wrote: > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst > --- a/Documentation/admin-guide/cgroup-v2.rst > +++ b/Documentation/admin-guide/cgroup-v2.rst > @@ -1314,6 +1314,18 @@ PAGE_SIZE multiple when read back. > Part of "slab" that cannot be reclaimed on memory > pressure. > > + avail > + An estimate of how much memory can be made available for > + starting new applications, similar to MemAvailable from > + /proc/meminfo (Documentation/filesystems/proc.rst). > + > + This is derived by assuming that half of page cahce and > + reclaimable slab can be uncharged without significantly > + impacting the workload, similar to MemAvailable. It also > + factors in the amount of lazy freeable memory (MADV_FREE) and > + compound pages that can be split and uncharged under memory > + pressure. > + > pgfault > Total number of page faults incurred > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -1350,6 +1350,35 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) > return false; > } > > +/* > + * Returns an estimate of the amount of available memory that can be reclaimed > + * for a memcg, in pages. > + */ > +static unsigned long mem_cgroup_avail(struct mem_cgroup *memcg) > +{ > + long deferred, lazyfree; > + > + /* > + * Deferred pages are charged anonymous pages that are on the LRU but > + * are unmapped. These compound pages are split under memory pressure. > + */ > + deferred = max_t(long, memcg_page_state(memcg, NR_ACTIVE_ANON) + > + memcg_page_state(memcg, NR_INACTIVE_ANON) - > + memcg_page_state(memcg, NR_ANON_MAPPED), 0); > + /* > + * Lazyfree pages are charged clean anonymous pages that are on the file > + * LRU and can be reclaimed under memory pressure. > + */ > + lazyfree = max_t(long, memcg_page_state(memcg, NR_ACTIVE_FILE) + > + memcg_page_state(memcg, NR_INACTIVE_FILE) - > + memcg_page_state(memcg, NR_FILE_PAGES), 0); > + > + /* Using same heuristic as si_mem_available() */ > + return (unsigned long)deferred + (unsigned long)lazyfree + > + (memcg_page_state(memcg, NR_FILE_PAGES) + > + memcg_page_state(memcg, NR_SLAB_RECLAIMABLE)) / 2; > +} > + > static char *memory_stat_format(struct mem_cgroup *memcg) > { > struct seq_buf s; > @@ -1417,6 +1446,12 @@ static char *memory_stat_format(struct mem_cgroup *memcg) > seq_buf_printf(&s, "slab_unreclaimable %llu\n", > (u64)memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE) * > PAGE_SIZE); > + /* > + * All values in this buffer are read individually, no implied > + * consistency amongst them. > + */ > + seq_buf_printf(&s, "avail %llu\n", > + (u64)mem_cgroup_avail(memcg) * PAGE_SIZE); > > /* Accumulated memory events */ > > An alternative to this would also be to change from an "available" metric to an "anon_reclaimable" metric since both the deferred split queues and lazy freeable memory would pertain to anon. This would no longer attempt to mimic MemAvailable and leave any such calculation to userspace (anon_reclaimable + (file + slab_reclaimable) / 2). With this route, care would need to be taken to clearly indicate that anon_reclaimable is not necessarily a subset of the "anon" metric since reclaimable memory from compound pages on deferred split queues is not mapped, so it doesn't show up in NR_ANON_MAPPED. I'm indifferent to either approach and would be happy to switch to anon_reclaimable if others agree and doesn't foresee any extensibility issues.