[RFC PATCH] docs: hugetlbpage.rst: add free surplus huge pages description

[Jinjiang Tu <tujinjiang@huawei.com>]

When echo 0 > /proc/sys/vm/nr_hugepages is concurrent with freeing in-use
huge pages to the huge page pool, some free huge pages may fail to be
destroyed and accounted as surplus. The counts are like below:

  HugePages_Total: 1024
  HugePages_Free: 1024
  HugePages_Surp: 1024

When set_max_huge_pages() decrease the pool size, it first return free
pages to the buddy allocator, and then account other pages as surplus.
Between the two steps, the hugetlb_lock is released to free memory and
require the hugetlb_lock again. If another process free huge pages to the
pool between the two steps, these free huge pages will be accounted as
surplus.

Besides, Free surplus huge pages come from failing to restore vmemmap.

Once the two situation occurs, users couldn't directly shrink the huge
page pool via echo 0 > nr_hugepages, should use one of the two ways to
destroy these free surplus huge pages:
 1) echo $nr_surplus > nr_hugepages to convert the surplus free huge pages
to persistent free huge pages first, and then echo 0 > nr_hugepages to
destroy these huge pages.
 2) allocate these free surplus huge pages, and will try to destroy them
when freeing them.

However, there is no documentation to describe it, users may be confused
and don't know how to handle in such case. So update the documention.

Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
---
 Documentation/admin-guide/mm/hugetlbpage.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst
index 67a941903fd2..0456cefae039 100644
--- a/Documentation/admin-guide/mm/hugetlbpage.rst
+++ b/Documentation/admin-guide/mm/hugetlbpage.rst
@@ -239,6 +239,17 @@ this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is
 increased sufficiently, or the surplus huge pages go out of use and are freed--
 no more surplus huge pages will be allowed to be allocated.
 
+Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` may be
+concurrent with freeing in-use huge pages to the huge page pool, leading to some
+huge pages are still in the huge page pool and accounted as surplus. Besides,
+When the feature of freeing unused vmemmap pages associated with each hugetlb page
+is enabled, free huge page may be accounted as surplus too. In such two cases, users
+couldn't directly shrink the huge page pool via echo 0 to ``nr_hugepages``, should
+echo $nr_surplus to ``nr_hugepages`` to convert the surplus free huge pages to
+persistent free huge pages first, and then echo 0 to ``nr_hugepages`` to destroy
+these huge pages. Another way to destroy is allocating these free surplus huge
+pages and these huge pages will be tried to destroy when they are freed.
+
 With support for multiple huge page pools at run-time available, much of
 the huge page userspace interface in ``/proc/sys/vm`` has been duplicated in
 sysfs.
-- 
2.43.0