From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
To: linux-kernel@vger.kernel.org
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>,
Alexey Dobriyan <adobriyan@gmail.com>,
Al Viro <viro@zeniv.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>,
Brendan Gregg <bgregg@netflix.com>,
carmenjackson@google.com, Christian Hansen <chansen3@cisco.com>,
Colin Ian King <colin.king@canonical.com>,
dancol@google.com, David Howells <dhowells@redhat.com>,
fmayer@google.com, joaodias@google.com, joelaf@google.com,
Jonathan Corbet <corbet@lwn.net>,
Kees Cook <keescook@chromium.org>,
kernel-team@android.com, Kirill Tkhai <ktkhai@virtuozzo.com>,
Konstantin Khlebnikov <khlebnikov@yandex-team.ru>,
linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-mm@kvack.org, Michal Hocko <mhocko@suse.com>,
Mike Rapoport <rppt@linux.ibm.com>,
minchan@google.com, minchan@kernel.org, namhyung@google.com,
sspatil@google.com, surenb@google.com,
Thomas Gleixner <tglx@linutronix.de>,
timmurray@google.com, tkjos@google.com, vdavydov.dev@gmail.com,
Vlastimil Babka <vbabka@suse.cz>,
wvw@google.com
Subject: [PATCH v1 2/2] doc: Update documentation for page_idle virtual address indexing
Date: Mon, 22 Jul 2019 17:32:05 -0400 [thread overview]
Message-ID: <20190722213205.140845-2-joel@joelfernandes.org> (raw)
In-Reply-To: <20190722213205.140845-1-joel@joelfernandes.org>
This patch updates the documentation with the new page_idle tracking
feature which uses virtual address indexing.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
.../admin-guide/mm/idle_page_tracking.rst | 41 +++++++++++++++----
1 file changed, 34 insertions(+), 7 deletions(-)
diff --git a/Documentation/admin-guide/mm/idle_page_tracking.rst b/Documentation/admin-guide/mm/idle_page_tracking.rst
index df9394fb39c2..70d3bf6f1f8c 100644
--- a/Documentation/admin-guide/mm/idle_page_tracking.rst
+++ b/Documentation/admin-guide/mm/idle_page_tracking.rst
@@ -19,10 +19,14 @@ It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.
User API
========
+There are 2 ways to access the idle page tracking API. One uses physical
+address indexing, another uses a simpler virtual address indexing scheme.
-The idle page tracking API is located at ``/sys/kernel/mm/page_idle``.
-Currently, it consists of the only read-write file,
-``/sys/kernel/mm/page_idle/bitmap``.
+Physical address indexing
+-------------------------
+The idle page tracking API for physical address indexing using page frame
+numbers (PFN) is located at ``/sys/kernel/mm/page_idle``. Currently, it
+consists of the only read-write file, ``/sys/kernel/mm/page_idle/bitmap``.
The file implements a bitmap where each bit corresponds to a memory page. The
bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
@@ -74,6 +78,29 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more
information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and
``/proc/kpagecgroup``.
+Virtual address indexing
+------------------------
+The idle page tracking API for virtual address indexing using virtual page
+frame numbers (VFN) is located at ``/proc/<pid>/page_idle``. It is a bitmap
+that follows the same semantics as ``/sys/kernel/mm/page_idle/bitmap``
+except that it uses virtual instead of physical frame numbers.
+
+This idle page tracking API can be simpler to use than physical address
+indexing, since the ``pagemap`` for a process does not need to be looked up to
+mark or read a page's idle bit. It is also more accurate than physical address
+indexing since in physical address indexing, address space changes can occur
+between reading the ``pagemap`` and reading the ``bitmap``. In virtual address
+indexing, the process's ``mmap_sem`` is held for the duration of the access.
+
+To estimate the amount of pages that are not used by a workload one should:
+
+ 1. Mark all the workload's pages as idle by setting corresponding bits in
+ ``/proc/<pid>/page_idle``.
+
+ 2. Wait until the workload accesses its working set.
+
+ 3. Read ``/proc/<pid>/page_idle`` and count the number of bits set.
+
.. _impl_details:
Implementation Details
@@ -99,10 +126,10 @@ When a dirty page is written to swap or disk as a result of memory reclaim or
exceeding the dirty memory limit, it is not marked referenced.
The idle memory tracking feature adds a new page flag, the Idle flag. This flag
-is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the
-:ref:`User API <user_api>`
-section), and cleared automatically whenever a page is referenced as defined
-above.
+is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` for physical
+addressing or by writing to ``/proc/<pid>/page_idle`` for virtual
+addressing (see the :ref:`User API <user_api>` section), and cleared
+automatically whenever a page is referenced as defined above.
When a page is marked idle, the Accessed bit must be cleared in all PTEs it is
mapped to, otherwise we will not be able to detect accesses to the page coming
--
2.22.0.657.g960e92d24f-goog
next prev parent reply other threads:[~2019-07-22 21:32 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-22 21:32 [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing Joel Fernandes (Google)
2019-07-22 21:32 ` Joel Fernandes (Google) [this message]
2019-07-22 22:06 ` Andrew Morton
2019-07-23 14:43 ` Joel Fernandes
2019-07-24 19:33 ` Joel Fernandes
2019-07-23 6:05 ` Michal Hocko
2019-07-23 14:34 ` Joel Fernandes
2019-07-23 6:13 ` Minchan Kim
2019-07-23 14:20 ` Joel Fernandes
2019-07-24 4:28 ` Minchan Kim
2019-07-24 14:10 ` Joel Fernandes
2019-07-25 8:15 ` Konstantin Khlebnikov
2019-07-26 0:06 ` Joel Fernandes
2019-07-26 11:16 ` Konstantin Khlebnikov
2019-07-26 12:54 ` Joel Fernandes
2019-07-23 8:43 ` Konstantin Khlebnikov
2019-07-23 10:10 ` Konstantin Khlebnikov
2019-07-23 13:47 ` Joel Fernandes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190722213205.140845-2-joel@joelfernandes.org \
--to=joel@joelfernandes.org \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=bgregg@netflix.com \
--cc=carmenjackson@google.com \
--cc=chansen3@cisco.com \
--cc=colin.king@canonical.com \
--cc=corbet@lwn.net \
--cc=dancol@google.com \
--cc=dhowells@redhat.com \
--cc=fmayer@google.com \
--cc=joaodias@google.com \
--cc=joelaf@google.com \
--cc=keescook@chromium.org \
--cc=kernel-team@android.com \
--cc=khlebnikov@yandex-team.ru \
--cc=ktkhai@virtuozzo.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=minchan@google.com \
--cc=minchan@kernel.org \
--cc=namhyung@google.com \
--cc=rppt@linux.ibm.com \
--cc=sspatil@google.com \
--cc=surenb@google.com \
--cc=tglx@linutronix.de \
--cc=timmurray@google.com \
--cc=tkjos@google.com \
--cc=vbabka@suse.cz \
--cc=vdavydov.dev@gmail.com \
--cc=viro@zeniv.linux.org.uk \
--cc=wvw@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox