From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f181.google.com (mail-ob0-f181.google.com [209.85.214.181]) by kanga.kvack.org (Postfix) with ESMTP id 91D1D828E1 for ; Thu, 28 Jan 2016 20:29:44 -0500 (EST) Received: by mail-ob0-f181.google.com with SMTP id ba1so51459185obb.3 for ; Thu, 28 Jan 2016 17:29:44 -0800 (PST) Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com. [173.37.86.74]) by mx.google.com with ESMTPS id dp7si10062610obb.40.2016.01.28.17.29.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Jan 2016 17:29:43 -0800 (PST) Subject: Re: computing drop-able caches References: <56AAA77D.7090000@cisco.com> <20160128235815.GA5953@cmpxchg.org> <56AABA79.3030103@cisco.com> From: Daniel Walker Message-ID: <56AAC085.9060509@cisco.com> Date: Thu, 28 Jan 2016 17:29:41 -0800 MIME-Version: 1.0 In-Reply-To: <56AABA79.3030103@cisco.com> Content-Type: multipart/alternative; boundary="------------050209090902070508010707" Sender: owner-linux-mm@kvack.org List-ID: To: Johannes Weiner Cc: Alexander Viro , Michal Hocko , Andrew Morton , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Khalid Mughal (khalidm)" , "xe-kernel@external.cisco.com" , Rik van Riel This is a multi-part message in MIME format. --------------050209090902070508010707 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 01/28/2016 05:03 PM, Daniel Walker wrote: > On 01/28/2016 03:58 PM, Johannes Weiner wrote: >> On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote: >>> "Currently there is no way to figure out the droppable pagecache size >>> from the meminfo output. The MemFree size can shrink during normal >>> system operation, when some of the memory pages get cached and is >>> reflected in "Cached" field. Similarly for file operations some of >>> the buffer memory gets cached and it is reflected in "Buffers" field. >>> The kernel automatically reclaims all this cached & buffered memory, >>> when it is needed elsewhere on the system. The only way to manually >>> reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. " >> [...] >> >>> The point of the whole exercise is to get a better idea of free >>> memory for >>> our employer. Does it make sense to do this for computing free memory? >> /proc/meminfo::MemAvailable was added for this purpose. See the doc >> text in Documentation/filesystem/proc.txt. >> >> It's an approximation, however, because this question is not easy to >> answer. Pages might be in various states and uses that can make them >> unreclaimable. > > > Khalid was telling me that our internal sources rejected MemAvailable > because it was not accurate enough. It says in the description, > "The estimate takes into account that the system needs some page cache > to function well". I suspect that's part of the inaccuracy. I asked > Khalid to respond with more details on this. > Some quotes, " [regarding MemAvaiable] This new metric purportedly helps usrespace assess available memory. But, its again based on heuristic, it takes 1/2 of page cache as reclaimable.. Somewhat arbitrary choice. Maybe appropriate for desktops, where page cache is mainly used as page cache, not as a first class store which is the case on embedded systems. Our systems are swap less, they have little secondary storage, they use in-memory databases/filesystems/shared memories/ etc. which are all setup on page caches).. This metric as it is implemented in 3.14 leads to a totally mis-leading picture of available memory" Daniel --------------050209090902070508010707 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 7bit
On 01/28/2016 05:03 PM, Daniel Walker wrote:
On 01/28/2016 03:58 PM, Johannes Weiner wrote:
On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
"Currently there is no way to figure out the droppable pagecache size
from the meminfo output. The MemFree size can shrink during normal
system operation, when some of the memory pages get cached and is
reflected in "Cached" field. Similarly for file operations some of
the buffer memory gets cached and it is reflected in "Buffers" field.
The kernel automatically reclaims all this cached & buffered memory,
when it is needed elsewhere on the system. The only way to manually
reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
[...]

The point of the whole exercise is to get a better idea of free memory for
our employer. Does it make sense to do this for computing free memory?
/proc/meminfo::MemAvailable was added for this purpose. See the doc
text in Documentation/filesystem/proc.txt.

It's an approximation, however, because this question is not easy to
answer. Pages might be in various states and uses that can make them
unreclaimable.


Khalid was telling me that our internal sources rejected MemAvailable because it was not accurate enough. It says in the description,
"The estimate takes into account that the system needs some page cache to function well". I suspect that's part of the inaccuracy. I asked Khalid to respond with more details on this.


Some quotes,

"
[regarding MemAvaiable]

This new metric purportedly helps usrespace assess available memory. But,
its again based on heuristic, it takes 1/2 of page cache as reclaimable..

Somewhat arbitrary choice. Maybe appropriate for desktops, where page
cache is mainly used as page cache, not as a first class store which is
the case on embedded systems. Our systems are swap less, they have little
secondary storage, they use in-memory databases/filesystems/shared memories/
etc. which are all setup on page caches).. This metric as it is implemented
in 3.14 leads to a totally mis-leading picture of available memory"

Daniel
--------------050209090902070508010707-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org