[PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jiaqi Yan]

Correctable memory errors are very common on servers with large
amount of memory, and are corrected by ECC, but with two
pain points to users:
1. Correction usually happens on the fly and adds latency overhead
2. Not-fully-proved theory states excessive correctable memory
   errors can develop into uncorrectable memory error.

Soft offline is kernel's additional solution for memory pages
having (excessive) corrected memory errors. Impacted page is migrated
to healthy page if it is in use, then the original page is discarded
for any future use.

The actual policy on whether (and when) to soft offline should be
maintained by userspace, especially in case of HugeTLB hugepages.
Soft-offline dissolves a hugepage, either in-use or free, into
chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
If userspace has not acknowledged such behavior, it may be surprised
when later mmap hugepages MAP_FAILED due to lack of hugepages.
In addition, discarding the entire 1G memory page only because of
corrected memory errors sounds very costly and kernel better not
doing under the hood. But today there are at least 2 such cases:
1. GHES driver sees both GHES_SEV_CORRECTED and
   CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
2. RAS Correctable Errors Collector counts correctable errors per
   PFN and when the counter for a PFN reaches threshold
In both cases, userspace has no control of the soft offline performed
by kernel's memory failure recovery.

This patch series give userspace the control of soft-offlining
HugeTLB pages: kernel only soft offlines hugepage if userspace has
opt-ed in for that specific hugepage size, and exposed to userspace
by a new sysfs entry called softoffline_corrected_errors under
/sys/kernel/mm/hugepages/hugepages-${size}kB directory:
* When softoffline_corrected_errors=0, skip soft offlining for all
  hugepages of size ${size}kB.
* When softoffline_corrected_errors=1, soft offline as before this
  patch series.
By default softoffline_corrected_errors is 1.

This patch set is based at
commit a52b4f11a2e1 ("selftest mm/mseal read-only elf memory segment").

Jiaqi Yan (3):
  mm/memory-failure: userspace controls soft-offlining hugetlb pages
  selftest/mm: test softoffline_corrected_errors behaviors
  docs: hugetlbpage.rst: add softoffline_corrected_errors

 Documentation/admin-guide/mm/hugetlbpage.rst  |  15 +-
 include/linux/hugetlb.h                       |  17 ++
 mm/hugetlb.c                                  |  34 +++
 mm/memory-failure.c                           |   7 +
 tools/testing/selftests/mm/.gitignore         |   1 +
 tools/testing/selftests/mm/Makefile           |   1 +
 .../selftests/mm/hugetlb-soft-offline.c       | 262 ++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   4 +
 8 files changed, 340 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/mm/hugetlb-soft-offline.c

-- 
2.45.1.288.g0e0cd299f1-goog

[PATCH v1 1/3] mm/memory-failure: userspace controls soft-offlining hugetlb pages

[Jiaqi Yan]

Correctable memory errors are very common on servers with large
amount of memory, and are corrected by ECC. Soft offline is kernel's
additional recovery handling for memory pages having (excessive)
corrected memory errors. Impacted page is migrated to a healthy page
if mapped/inuse; the original page is discarded for any future use.

The actual policy on whether (and when) to soft offline should be
maintained by userspace, especially in case of HugeTLB hugepages.
Soft-offline dissolves a hugepage, either in-use or free, into
chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
If userspace has not acknowledged such behavior, it may be surprised
when later mmap hugepages MAP_FAILED due to lack of hugepages.
In addition, discarding the entire 1G memory page only because of
corrected memory errors sounds very costly and kernel better not
doing under the hood. But today there are at least 2 such cases:
1. GHES driver sees both GHES_SEV_CORRECTED and
   CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
2. RAS Correctable Errors Collector counts correctable errors per
   PFN and when the counter for a PFN reaches threshold
In both cases, userspace has no control of the soft offline performed
by kernel's memory failure recovery.

This commit gives userspace the control of soft-offlining HugeTLB
pages: kernel only soft offlines hugepage if userspace has opt-ed in
in for that specific hugepage size. The interface to userspace is a
new sysfs entry called softoffline_corrected_errors under the
/sys/kernel/mm/hugepages/hugepages-${size}kB directory:
* When softoffline_corrected_errors=0, skip soft offlining for all
  hugepages of size ${size}kB.
* When softoffline_corrected_errors=1, soft offline as before this
  patch series.

So the granularity of the control is per hugepage size, and is kept
in corresponding hstate. By default softoffline_corrected_errors is
1 to preserve existing behavior in kernel.

Signed-off-by: Jiaqi Yan <jiaqiyan@google.com>
---
 include/linux/hugetlb.h | 17 +++++++++++++++++
 mm/hugetlb.c            | 34 ++++++++++++++++++++++++++++++++++
 mm/memory-failure.c     |  7 +++++++
 3 files changed, 58 insertions(+)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 2b3c3a404769..55f9e9593cce 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -685,6 +685,7 @@ struct hstate {
 	int next_nid_to_free;
 	unsigned int order;
 	unsigned int demote_order;
+	unsigned int softoffline_corrected_errors;
 	unsigned long mask;
 	unsigned long max_huge_pages;
 	unsigned long nr_huge_pages;
@@ -1029,6 +1030,16 @@ void hugetlb_unregister_node(struct node *node);
  */
 bool is_raw_hwpoison_page_in_hugepage(struct page *page);
 
+/*
+ * For certain hugepage size, when a hugepage has corrected memory error(s):
+ * - Return 0 if userspace wants to disable soft offlining the hugepage.
+ * - Return > 0 if userspace allows soft offlining the hugepage.
+ */
+static inline int hugetlb_softoffline_corrected_errors(struct folio *folio)
+{
+	return folio_hstate(folio)->softoffline_corrected_errors;
+}
+
 #else	/* CONFIG_HUGETLB_PAGE */
 struct hstate {};
 
@@ -1226,6 +1237,12 @@ static inline bool hugetlbfs_pagecache_present(
 {
 	return false;
 }
+
+static inline int hugetlb_softoffline_corrected_errors(struct folio *folio)
+{
+	return 1;
+}
+
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 static inline spinlock_t *huge_pte_lock(struct hstate *h,
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 6be78e7d4f6e..a184e28ce592 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4325,6 +4325,38 @@ static ssize_t demote_size_store(struct kobject *kobj,
 }
 HSTATE_ATTR(demote_size);
 
+static ssize_t softoffline_corrected_errors_show(struct kobject *kobj,
+						 struct kobj_attribute *attr,
+						 char *buf)
+{
+	struct hstate *h = kobj_to_hstate(kobj, NULL);
+
+	return sysfs_emit(buf, "%d\n", h->softoffline_corrected_errors);
+}
+
+static ssize_t softoffline_corrected_errors_store(struct kobject *kobj,
+						  struct kobj_attribute *attr,
+						  const char *buf,
+						  size_t count)
+{
+	int err;
+	unsigned long input;
+	struct hstate *h = kobj_to_hstate(kobj, NULL);
+
+	err = kstrtoul(buf, 10, &input);
+	if (err)
+		return err;
+
+	/* softoffline_corrected_errors is either 0 or 1. */
+	if (input > 1)
+		return -EINVAL;
+
+	h->softoffline_corrected_errors = input;
+
+	return count;
+}
+HSTATE_ATTR(softoffline_corrected_errors);
+
 static struct attribute *hstate_attrs[] = {
 	&nr_hugepages_attr.attr,
 	&nr_overcommit_hugepages_attr.attr,
@@ -4334,6 +4366,7 @@ static struct attribute *hstate_attrs[] = {
 #ifdef CONFIG_NUMA
 	&nr_hugepages_mempolicy_attr.attr,
 #endif
+	&softoffline_corrected_errors_attr.attr,
 	NULL,
 };
 
@@ -4655,6 +4688,7 @@ void __init hugetlb_add_hstate(unsigned int order)
 	h = &hstates[hugetlb_max_hstate++];
 	mutex_init(&h->resize_lock);
 	h->order = order;
+	h->softoffline_corrected_errors = 1;
 	h->mask = ~(huge_page_size(h) - 1);
 	for (i = 0; i < MAX_NUMNODES; ++i)
 		INIT_LIST_HEAD(&h->hugepage_freelists[i]);
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 16ada4fb02b7..7094fc4c62e2 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2776,6 +2776,13 @@ int soft_offline_page(unsigned long pfn, int flags)
 		return -EIO;
 	}
 
+	if (PageHuge(page) &&
+	    !hugetlb_softoffline_corrected_errors(page_folio(page))) {
+		pr_info("soft offline: %#lx: hugetlb page is ignored\n", pfn);
+		put_ref_page(pfn, flags);
+		return -EINVAL;
+	}
+
 	mutex_lock(&mf_mutex);
 
 	if (PageHWPoison(page)) {
-- 
2.45.1.288.g0e0cd299f1-goog

[PATCH v1 2/3] selftest/mm: test softoffline_corrected_errors behaviors

[Jiaqi Yan]

Add regression and new tests when hugepage has correctable memory
errors, and how userspace wants to deal with it:
* if softoffline_corrected_errors=0, mapped hugepage is soft offlined
* if softoffline_corrected_errors=1, mapped hugepage is intact

Free hugepages case is not explicitly covered by the tests.

Hugepage having corrected memory errors is emulated with
MADV_SOFT_OFFLINE.

Signed-off-by: Jiaqi Yan <jiaqiyan@google.com>
---
 tools/testing/selftests/mm/.gitignore         |   1 +
 tools/testing/selftests/mm/Makefile           |   1 +
 .../selftests/mm/hugetlb-soft-offline.c       | 262 ++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   4 +
 4 files changed, 268 insertions(+)
 create mode 100644 tools/testing/selftests/mm/hugetlb-soft-offline.c

diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 0b9ab987601c..064e7b125643 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -6,6 +6,7 @@ hugepage-shm
 hugepage-vmemmap
 hugetlb-madvise
 hugetlb-read-hwpoison
+hugetlb-soft-offline
 khugepaged
 map_hugetlb
 map_populate
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 3b49bc3d0a3b..d166067d75ef 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -42,6 +42,7 @@ TEST_GEN_FILES += gup_test
 TEST_GEN_FILES += hmm-tests
 TEST_GEN_FILES += hugetlb-madvise
 TEST_GEN_FILES += hugetlb-read-hwpoison
+TEST_GEN_FILES += hugetlb-soft-offline
 TEST_GEN_FILES += hugepage-mmap
 TEST_GEN_FILES += hugepage-mremap
 TEST_GEN_FILES += hugepage-shm
diff --git a/tools/testing/selftests/mm/hugetlb-soft-offline.c b/tools/testing/selftests/mm/hugetlb-soft-offline.c
new file mode 100644
index 000000000000..8d1d7d4a84d8
--- /dev/null
+++ b/tools/testing/selftests/mm/hugetlb-soft-offline.c
@@ -0,0 +1,262 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test soft offline behavior for HugeTLB pages:
+ * - if softoffline_corrected_errors = 0, hugepages should stay intact and soft
+ *   offlining failed with EINVAL.
+ * - if softoffline_corrected_errors > 0, a hugepage should be dissolved and
+ *   nr_hugepages should be reduced by 1.
+ *
+ * Before running, make sure more than 2 hugepages of default_hugepagesz
+ * are allocated. For example, if /proc/meminfo/Hugepagesize is 2048kB:
+ *   echo 8 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
+ */
+
+#define _GNU_SOURCE
+#include <errno.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <linux/magic.h>
+#include <linux/memfd.h>
+#include <sys/mman.h>
+#include <sys/statfs.h>
+#include <sys/types.h>
+
+#ifndef MADV_SOFT_OFFLINE
+#define MADV_SOFT_OFFLINE 101
+#endif
+
+#define PREFIX " ... "
+#define EPREFIX " !!! "
+
+enum test_status {
+	TEST_PASS = 0,
+	TEST_FAILED = 1,
+	// From ${ksft_skip} in run_vmtests.sh.
+	TEST_SKIPPED = 4,
+};
+
+static enum test_status do_soft_offline(int fd, size_t len, int expect_ret)
+{
+	char *filemap = NULL;
+	char *hwp_addr = NULL;
+	const unsigned long pagesize = getpagesize();
+	int ret = 0;
+	enum test_status status = TEST_SKIPPED;
+
+	if (ftruncate(fd, len) < 0) {
+		perror(EPREFIX "ftruncate to len failed");
+		return status;
+	}
+
+	filemap = mmap(NULL, len, PROT_READ | PROT_WRITE,
+		       MAP_SHARED | MAP_POPULATE, fd, 0);
+	if (filemap == MAP_FAILED) {
+		perror(EPREFIX "mmap failed");
+		goto untruncate;
+	}
+
+	memset(filemap, 0xab, len);
+	printf(PREFIX "Allocated %#lx bytes of hugetlb pages\n", len);
+
+	hwp_addr = filemap + len / 2;
+	ret = madvise(hwp_addr, pagesize, MADV_SOFT_OFFLINE);
+	printf(PREFIX "MADV_SOFT_OFFLINE %p ret=%d, errno=%d\n",
+	       hwp_addr, ret, errno);
+	if (ret != 0)
+		perror(EPREFIX "madvise failed");
+
+	if (errno == expect_ret)
+		status = TEST_PASS;
+	else {
+		printf(EPREFIX "MADV_SOFT_OFFLINE should ret %d\n", expect_ret);
+		status = TEST_FAILED;
+	}
+
+	munmap(filemap, len);
+untruncate:
+	if (ftruncate(fd, 0) < 0)
+		perror(EPREFIX "ftruncate back to 0 failed");
+
+	return status;
+}
+
+static int set_softoffline_corrected_errors(unsigned long hugepage_size, int value)
+{
+	char cmd[256] = {0};
+	FILE *cmdfile = NULL;
+
+	if (value != 0 && value != 1)
+		return -EINVAL;
+
+	sprintf(cmd,
+		"echo %d > /sys/kernel/mm/hugepages/hugepages-%ldkB/softoffline_corrected_errors",
+		value, hugepage_size);
+	cmdfile = popen(cmd, "r");
+
+	if (cmdfile == NULL)
+		perror(EPREFIX "failed to set softoffline_corrected_errors");
+	else
+		printf(PREFIX
+		       "softoffline_corrected_errors=%d for %ldkB hugepages\n",
+		       value, hugepage_size);
+
+	pclose(cmdfile);
+	return 0;
+}
+
+static int read_nr_hugepages(unsigned long hugepage_size,
+			     unsigned long *nr_hugepages)
+{
+	char buffer[256] = {0};
+	char cmd[256] = {0};
+
+	sprintf(cmd, "cat /sys/kernel/mm/hugepages/hugepages-%ldkB/nr_hugepages",
+		hugepage_size);
+	FILE *cmdfile = popen(cmd, "r");
+
+	if (!fgets(buffer, sizeof(buffer), cmdfile)) {
+		perror(EPREFIX "failed to read nr_hugepages");
+		pclose(cmdfile);
+		return -1;
+	}
+
+	*nr_hugepages = atoll(buffer);
+	pclose(cmdfile);
+	return 0;
+}
+
+static int create_hugetlbfs_file(struct statfs *file_stat)
+{
+	int fd;
+
+	fd = memfd_create("hugetlb_tmp", MFD_HUGETLB);
+	if (fd < 0) {
+		perror(EPREFIX "could not open hugetlbfs file");
+		return -1;
+	}
+
+	memset(file_stat, 0, sizeof(*file_stat));
+	if (fstatfs(fd, file_stat)) {
+		perror(EPREFIX "fstatfs failed");
+		goto close;
+	}
+	if (file_stat->f_type != HUGETLBFS_MAGIC) {
+		printf(EPREFIX "not hugetlbfs file\n");
+		goto close;
+	}
+
+	return fd;
+close:
+	close(fd);
+	return -1;
+}
+
+static enum test_status test_soft_offline(void)
+{
+	int fd;
+	struct statfs file_stat;
+	unsigned long hugepagesize_kb = 0;
+	unsigned long nr_hugepages_before = 0;
+	unsigned long nr_hugepages_after = 0;
+	enum test_status status = TEST_SKIPPED;
+
+	printf("Test Soft Offline When softoffline_corrected_errors=1\n");
+
+	fd = create_hugetlbfs_file(&file_stat);
+	if (fd < 0) {
+		printf(EPREFIX "Failed to create hugetlbfs file\n");
+		return status;
+	}
+
+	hugepagesize_kb = file_stat.f_bsize / 1024;
+	printf(PREFIX "Hugepagesize is %ldkB\n", hugepagesize_kb);
+
+	if (set_softoffline_corrected_errors(hugepagesize_kb, 1))
+		return TEST_FAILED;
+
+	if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_before) != 0)
+		return TEST_FAILED;
+
+	printf(PREFIX "Before MADV_SOFT_OFFLINE nr_hugepages=%ld\n",
+		nr_hugepages_before);
+
+	status = do_soft_offline(fd, 2 * file_stat.f_bsize, /*expect_ret=*/0);
+
+	if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_after) != 0)
+		return TEST_FAILED;
+
+	printf(PREFIX "After MADV_SOFT_OFFLINE nr_hugepages=%ld\n",
+		nr_hugepages_after);
+
+	if (nr_hugepages_before != nr_hugepages_after + 1) {
+		printf(EPREFIX "MADV_SOFT_OFFLINE should reduced 1 hugepage\n");
+		return TEST_FAILED;
+	}
+
+	return status;
+}
+
+static enum test_status test_disable_soft_offline(void)
+{
+	int fd;
+	struct statfs file_stat;
+	unsigned long hugepagesize_kb = 0;
+	unsigned long nr_hugepages_before = 0;
+	unsigned long nr_hugepages_after = 0;
+	enum test_status status = TEST_SKIPPED;
+
+	printf("Test Soft Offline When softoffline_corrected_errors=0\n");
+
+	fd = create_hugetlbfs_file(&file_stat);
+	if (fd < 0) {
+		printf(EPREFIX "Failed to create hugetlbfs file\n");
+		return status;
+	}
+
+	hugepagesize_kb = file_stat.f_bsize / 1024;
+	printf(PREFIX "Hugepagesize is %ldkB\n", hugepagesize_kb);
+
+	if (set_softoffline_corrected_errors(hugepagesize_kb, 0))
+		return TEST_FAILED;
+
+	if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_before) != 0)
+		return TEST_FAILED;
+
+	printf(PREFIX "Before MADV_SOFT_OFFLINE nr_hugepages=%ld\n",
+		nr_hugepages_before);
+
+	status = do_soft_offline(fd, 2 * file_stat.f_bsize, /*expect_ret=*/EINVAL);
+
+	if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_after) != 0)
+		return TEST_FAILED;
+
+	printf(PREFIX "After MADV_SOFT_OFFLINE nr_hugepages=%ld\n",
+		nr_hugepages_after);
+
+	if (nr_hugepages_before != nr_hugepages_after) {
+		printf(EPREFIX "MADV_SOFT_OFFLINE reduced %lu hugepages\n",
+			nr_hugepages_before - nr_hugepages_after);
+		return TEST_FAILED;
+	}
+
+	return status;
+}
+
+int main(void)
+{
+	enum test_status status;
+
+	status = test_soft_offline();
+	if (status != TEST_PASS)
+		return status;
+
+	status = test_disable_soft_offline();
+	if (status != TEST_PASS)
+		return status;
+
+	printf("Test Soft Offline All Good!\n");
+	return TEST_PASS;
+}
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 3157204b9047..91db9971ba69 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -332,6 +332,10 @@ CATEGORY="hugetlb" run_test ./charge_reserved_hugetlb.sh -cgroup-v2
 CATEGORY="hugetlb" run_test ./hugetlb_reparenting_test.sh -cgroup-v2
 if $RUN_DESTRUCTIVE; then
 CATEGORY="hugetlb" run_test ./hugetlb-read-hwpoison
+nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages)
+echo 8 > /proc/sys/vm/nr_hugepages
+CATEGORY="hugetlb" run_test ./hugetlb-soft-offline
+echo "$nr_hugepages_tmp" > /proc/sys/vm/nr_hugepages
 fi
 
 if [ $VADDR64 -ne 0 ]; then
-- 
2.45.1.288.g0e0cd299f1-goog

[PATCH v1 3/3] docs: hugetlbpage.rst: add softoffline_corrected_errors

[Jiaqi Yan]

Add the documentation for what softoffline_corrected_errors
sysfs interface is for.

Signed-off-by: Jiaqi Yan <jiaqiyan@google.com>
---
 Documentation/admin-guide/mm/hugetlbpage.rst | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst
index f34a0d798d5b..7969ae47f5f1 100644
--- a/Documentation/admin-guide/mm/hugetlbpage.rst
+++ b/Documentation/admin-guide/mm/hugetlbpage.rst
@@ -244,7 +244,8 @@ will exist, of the form::
 
 Inside each of these directories, the set of files contained in ``/proc``
 will exist.  In addition, two additional interfaces for demoting huge
-pages may exist::
+pages, and one additional interface for handling corrected memory errors,
+may exist::
 
         demote
         demote_size
@@ -254,6 +255,7 @@ pages may exist::
 	free_hugepages
 	resv_hugepages
 	surplus_hugepages
+	softoffline_corrected_errors
 
 The demote interfaces provide the ability to split a huge page into
 smaller huge pages.  For example, the x86 architecture supports both
@@ -276,6 +278,17 @@ demote
         actually demoted, compare the value of nr_hugepages before and after
         writing to the demote interface.  demote is a write only interface.
 
+The interface for handling corrected memory errors is
+
+softoffline_corrected_errors
+	allow userspace to control how to deal with hugepages that have
+	corrected memory errors.  When setting to 1, kernel attempts to soft
+	offline the hugepage whenever it thinks needed.  If soft offlinging a
+	huge page succeeds, for in-use hugepage, page content is migrated to a
+	new hugepage; however, regardless of in-use or free, capacity of the
+	hugepages will reduce by 1.  When setting to 0, kernel won't attempt to
+	soft offline the hugepage of the specific size. Its default value is 1.
+
 The interfaces which are the same as in ``/proc`` (all except demote and
 demote_size) function as described above for the default huge page-sized case.
 
-- 
2.45.1.288.g0e0cd299f1-goog

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Miaohe Lin]

On 2024/6/1 5:34, Jiaqi Yan wrote:
> Correctable memory errors are very common on servers with large
> amount of memory, and are corrected by ECC, but with two
> pain points to users:
> 1. Correction usually happens on the fly and adds latency overhead
> 2. Not-fully-proved theory states excessive correctable memory
>    errors can develop into uncorrectable memory error.

Thanks for your patch.

> 
> Soft offline is kernel's additional solution for memory pages
> having (excessive) corrected memory errors. Impacted page is migrated
> to healthy page if it is in use, then the original page is discarded
> for any future use.
> 
> The actual policy on whether (and when) to soft offline should be
> maintained by userspace, especially in case of HugeTLB hugepages.
> Soft-offline dissolves a hugepage, either in-use or free, into
> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
> If userspace has not acknowledged such behavior, it may be surprised
> when later mmap hugepages MAP_FAILED due to lack of hugepages.

For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
capacity won't be modified in that case. So I assume you're referring to the
free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
But if we don't do that, we might encounter uncorrectable memory error later
which will be more severe? Will it be better to add a way to compensate the
capacity?

> In addition, discarding the entire 1G memory page only because of
> corrected memory errors sounds very costly and kernel better not
> doing under the hood. But today there are at least 2 such cases:
> 1. GHES driver sees both GHES_SEV_CORRECTED and
>    CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
> 2. RAS Correctable Errors Collector counts correctable errors per
>    PFN and when the counter for a PFN reaches threshold
> In both cases, userspace has no control of the soft offline performed
> by kernel's memory failure recovery.

Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
-rlN` and then decide whether to soft offline the page according to it. But for
GHES driver, I think it has to be done in the kernel. So add a control in /sys/
seems like a good idea.

> 
> This patch series give userspace the control of soft-offlining
> HugeTLB pages: kernel only soft offlines hugepage if userspace has
> opt-ed in for that specific hugepage size, and exposed to userspace
> by a new sysfs entry called softoffline_corrected_errors under
> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
> * When softoffline_corrected_errors=0, skip soft offlining for all
>   hugepages of size ${size}kB.
> * When softoffline_corrected_errors=1, soft offline as before this

Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?

Thanks.
.

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jiaqi Yan]

On Tue, Jun 4, 2024 at 12:19 AM Miaohe Lin <linmiaohe@huawei.com> wrote:
>
> On 2024/6/1 5:34, Jiaqi Yan wrote:
> > Correctable memory errors are very common on servers with large
> > amount of memory, and are corrected by ECC, but with two
> > pain points to users:
> > 1. Correction usually happens on the fly and adds latency overhead
> > 2. Not-fully-proved theory states excessive correctable memory
> >    errors can develop into uncorrectable memory error.
>
> Thanks for your patch.

Thanks Miaohe, sorry I missed your message (Gmail mistakenly put it in
my spam folder).

>
> >
> > Soft offline is kernel's additional solution for memory pages
> > having (excessive) corrected memory errors. Impacted page is migrated
> > to healthy page if it is in use, then the original page is discarded
> > for any future use.
> >
> > The actual policy on whether (and when) to soft offline should be
> > maintained by userspace, especially in case of HugeTLB hugepages.
> > Soft-offline dissolves a hugepage, either in-use or free, into
> > chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
> > If userspace has not acknowledged such behavior, it may be surprised
> > when later mmap hugepages MAP_FAILED due to lack of hugepages.
>
> For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
> capacity won't be modified in that case. So I assume you're referring to the

I don't think so.

For in-use hugetlb folio case, after migrate_pages, kernel will
dissolve_free_hugetlb_folio the src hugetlb folio. At this point
refcount of src hugetlb folio should be zero already, and
remove_hugetlb_folio will reduce the hugetlb pool capacity (both
nr_hugepages and free_hugepages) accordingly.

For the free hugetlb folio case, dissolving also happens. But CE on
free pages should be very rare (since no one is accessing except
patrol scrubber).

One of my test cases in patch 2/3 validates my point: the test case
MADV_SOFT_OFFLINE a mapped page and at the point soft offline
succeeds, both nr_hugepages and nr_freepages are reduced by 1.

> free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
> But if we don't do that, we might encounter uncorrectable memory error later

If your concern is more correctable error will develop into more
severe uncorrectable, your concern is absolutely valid. There is a
tradeoff between reliability vs performance (availability of hugetlb
pages), but IMO should be decided by userspace.

> which will be more severe? Will it be better to add a way to compensate the
> capacity?

Corner cases: What if finding physically contiguous memory takes too
long? What if we can't find any physically contiguous memory to
compensate? (then hugetlb pool will still need to be reduced).

If we treat "compensate" as an improvement to the overall soft offline
process, it is something we can do in future and it is something
orthogonal to this control API, right? I think if userspace explicitly
tells kernel to soft offline, then they are also well-prepared for the
corner cases above.

>
> > In addition, discarding the entire 1G memory page only because of
> > corrected memory errors sounds very costly and kernel better not
> > doing under the hood. But today there are at least 2 such cases:
> > 1. GHES driver sees both GHES_SEV_CORRECTED and
> >    CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
> > 2. RAS Correctable Errors Collector counts correctable errors per
> >    PFN and when the counter for a PFN reaches threshold
> > In both cases, userspace has no control of the soft offline performed
> > by kernel's memory failure recovery.
>
> Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
> -rlN` and then decide whether to soft offline the page according to it. But for
> GHES driver, I think it has to be done in the kernel. So add a control in /sys/
> seems like a good idea.

Thanks.

>
> >
> > This patch series give userspace the control of soft-offlining
> > HugeTLB pages: kernel only soft offlines hugepage if userspace has
> > opt-ed in for that specific hugepage size, and exposed to userspace
> > by a new sysfs entry called softoffline_corrected_errors under
> > /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
> > * When softoffline_corrected_errors=0, skip soft offlining for all
> >   hugepages of size ${size}kB.
> > * When softoffline_corrected_errors=1, soft offline as before this
>
> Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?

"soft_offline_enabled" is less optimal as it can't be extended to
support something like "soft offline this PFN if something repeatedly
requested soft offline this exact PFN x times". (although I don't
think we need it).

softoffline_corrected_errors is one char less, but if you insist,
soft_offline_corrected_errors also works for me.

>
> Thanks.
> .
>

Re: [PATCH v1 1/3] mm/memory-failure: userspace controls soft-offlining hugetlb pages

[Jiaqi Yan]

+CC Jane.

On Fri, May 31, 2024 at 2:34 PM Jiaqi Yan <jiaqiyan@google.com> wrote:
>
> Correctable memory errors are very common on servers with large
> amount of memory, and are corrected by ECC. Soft offline is kernel's
> additional recovery handling for memory pages having (excessive)
> corrected memory errors. Impacted page is migrated to a healthy page
> if mapped/inuse; the original page is discarded for any future use.
>
> The actual policy on whether (and when) to soft offline should be
> maintained by userspace, especially in case of HugeTLB hugepages.
> Soft-offline dissolves a hugepage, either in-use or free, into
> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
> If userspace has not acknowledged such behavior, it may be surprised
> when later mmap hugepages MAP_FAILED due to lack of hugepages.
> In addition, discarding the entire 1G memory page only because of
> corrected memory errors sounds very costly and kernel better not
> doing under the hood. But today there are at least 2 such cases:
> 1. GHES driver sees both GHES_SEV_CORRECTED and
>    CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
> 2. RAS Correctable Errors Collector counts correctable errors per
>    PFN and when the counter for a PFN reaches threshold
> In both cases, userspace has no control of the soft offline performed
> by kernel's memory failure recovery.
>
> This commit gives userspace the control of soft-offlining HugeTLB
> pages: kernel only soft offlines hugepage if userspace has opt-ed in
> in for that specific hugepage size. The interface to userspace is a
> new sysfs entry called softoffline_corrected_errors under the
> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
> * When softoffline_corrected_errors=0, skip soft offlining for all
>   hugepages of size ${size}kB.
> * When softoffline_corrected_errors=1, soft offline as before this
>   patch series.
>
> So the granularity of the control is per hugepage size, and is kept
> in corresponding hstate. By default softoffline_corrected_errors is
> 1 to preserve existing behavior in kernel.
>
> Signed-off-by: Jiaqi Yan <jiaqiyan@google.com>
> ---
>  include/linux/hugetlb.h | 17 +++++++++++++++++
>  mm/hugetlb.c            | 34 ++++++++++++++++++++++++++++++++++
>  mm/memory-failure.c     |  7 +++++++
>  3 files changed, 58 insertions(+)
>
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 2b3c3a404769..55f9e9593cce 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -685,6 +685,7 @@ struct hstate {
>         int next_nid_to_free;
>         unsigned int order;
>         unsigned int demote_order;
> +       unsigned int softoffline_corrected_errors;
>         unsigned long mask;
>         unsigned long max_huge_pages;
>         unsigned long nr_huge_pages;
> @@ -1029,6 +1030,16 @@ void hugetlb_unregister_node(struct node *node);
>   */
>  bool is_raw_hwpoison_page_in_hugepage(struct page *page);
>
> +/*
> + * For certain hugepage size, when a hugepage has corrected memory error(s):
> + * - Return 0 if userspace wants to disable soft offlining the hugepage.
> + * - Return > 0 if userspace allows soft offlining the hugepage.
> + */
> +static inline int hugetlb_softoffline_corrected_errors(struct folio *folio)
> +{
> +       return folio_hstate(folio)->softoffline_corrected_errors;
> +}
> +
>  #else  /* CONFIG_HUGETLB_PAGE */
>  struct hstate {};
>
> @@ -1226,6 +1237,12 @@ static inline bool hugetlbfs_pagecache_present(
>  {
>         return false;
>  }
> +
> +static inline int hugetlb_softoffline_corrected_errors(struct folio *folio)
> +{
> +       return 1;
> +}
> +
>  #endif /* CONFIG_HUGETLB_PAGE */
>
>  static inline spinlock_t *huge_pte_lock(struct hstate *h,
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 6be78e7d4f6e..a184e28ce592 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -4325,6 +4325,38 @@ static ssize_t demote_size_store(struct kobject *kobj,
>  }
>  HSTATE_ATTR(demote_size);
>
> +static ssize_t softoffline_corrected_errors_show(struct kobject *kobj,
> +                                                struct kobj_attribute *attr,
> +                                                char *buf)
> +{
> +       struct hstate *h = kobj_to_hstate(kobj, NULL);
> +
> +       return sysfs_emit(buf, "%d\n", h->softoffline_corrected_errors);
> +}
> +
> +static ssize_t softoffline_corrected_errors_store(struct kobject *kobj,
> +                                                 struct kobj_attribute *attr,
> +                                                 const char *buf,
> +                                                 size_t count)
> +{
> +       int err;
> +       unsigned long input;
> +       struct hstate *h = kobj_to_hstate(kobj, NULL);
> +
> +       err = kstrtoul(buf, 10, &input);
> +       if (err)
> +               return err;
> +
> +       /* softoffline_corrected_errors is either 0 or 1. */
> +       if (input > 1)
> +               return -EINVAL;
> +
> +       h->softoffline_corrected_errors = input;
> +
> +       return count;
> +}
> +HSTATE_ATTR(softoffline_corrected_errors);
> +
>  static struct attribute *hstate_attrs[] = {
>         &nr_hugepages_attr.attr,
>         &nr_overcommit_hugepages_attr.attr,
> @@ -4334,6 +4366,7 @@ static struct attribute *hstate_attrs[] = {
>  #ifdef CONFIG_NUMA
>         &nr_hugepages_mempolicy_attr.attr,
>  #endif
> +       &softoffline_corrected_errors_attr.attr,
>         NULL,
>  };
>
> @@ -4655,6 +4688,7 @@ void __init hugetlb_add_hstate(unsigned int order)
>         h = &hstates[hugetlb_max_hstate++];
>         mutex_init(&h->resize_lock);
>         h->order = order;
> +       h->softoffline_corrected_errors = 1;
>         h->mask = ~(huge_page_size(h) - 1);
>         for (i = 0; i < MAX_NUMNODES; ++i)
>                 INIT_LIST_HEAD(&h->hugepage_freelists[i]);
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 16ada4fb02b7..7094fc4c62e2 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -2776,6 +2776,13 @@ int soft_offline_page(unsigned long pfn, int flags)
>                 return -EIO;
>         }
>
> +       if (PageHuge(page) &&
> +           !hugetlb_softoffline_corrected_errors(page_folio(page))) {
> +               pr_info("soft offline: %#lx: hugetlb page is ignored\n", pfn);
> +               put_ref_page(pfn, flags);
> +               return -EINVAL;
> +       }
> +
>         mutex_lock(&mf_mutex);
>
>         if (PageHWPoison(page)) {
> --
> 2.45.1.288.g0e0cd299f1-goog
>

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jane Chu]

On 6/7/2024 3:22 PM, Jiaqi Yan wrote:

> On Tue, Jun 4, 2024 at 12:19 AM Miaohe Lin <linmiaohe@huawei.com> wrote:
>> On 2024/6/1 5:34, Jiaqi Yan wrote:
>>> Correctable memory errors are very common on servers with large
>>> amount of memory, and are corrected by ECC, but with two
>>> pain points to users:
>>> 1. Correction usually happens on the fly and adds latency overhead
>>> 2. Not-fully-proved theory states excessive correctable memory
>>>     errors can develop into uncorrectable memory error.
>> Thanks for your patch.
> Thanks Miaohe, sorry I missed your message (Gmail mistakenly put it in
> my spam folder).
>
>>> Soft offline is kernel's additional solution for memory pages
>>> having (excessive) corrected memory errors. Impacted page is migrated
>>> to healthy page if it is in use, then the original page is discarded
>>> for any future use.
>>>
>>> The actual policy on whether (and when) to soft offline should be
>>> maintained by userspace, especially in case of HugeTLB hugepages.
>>> Soft-offline dissolves a hugepage, either in-use or free, into
>>> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
>>> If userspace has not acknowledged such behavior, it may be surprised
>>> when later mmap hugepages MAP_FAILED due to lack of hugepages.
>> For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
>> capacity won't be modified in that case. So I assume you're referring to the
> I don't think so.
>
> For in-use hugetlb folio case, after migrate_pages, kernel will
> dissolve_free_hugetlb_folio the src hugetlb folio. At this point
> refcount of src hugetlb folio should be zero already, and
> remove_hugetlb_folio will reduce the hugetlb pool capacity (both
> nr_hugepages and free_hugepages) accordingly.
>
> For the free hugetlb folio case, dissolving also happens. But CE on
> free pages should be very rare (since no one is accessing except
> patrol scrubber).
>
> One of my test cases in patch 2/3 validates my point: the test case
> MADV_SOFT_OFFLINE a mapped page and at the point soft offline
> succeeds, both nr_hugepages and nr_freepages are reduced by 1.
>
>> free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
>> But if we don't do that, we might encounter uncorrectable memory error later
> If your concern is more correctable error will develop into more
> severe uncorrectable, your concern is absolutely valid. There is a
> tradeoff between reliability vs performance (availability of hugetlb
> pages), but IMO should be decided by userspace.
>
>> which will be more severe? Will it be better to add a way to compensate the
>> capacity?
> Corner cases: What if finding physically contiguous memory takes too
> long? What if we can't find any physically contiguous memory to
> compensate? (then hugetlb pool will still need to be reduced).
>
> If we treat "compensate" as an improvement to the overall soft offline
> process, it is something we can do in future and it is something
> orthogonal to this control API, right? I think if userspace explicitly
> tells kernel to soft offline, then they are also well-prepared for the
> corner cases above.
>
>>> In addition, discarding the entire 1G memory page only because of
>>> corrected memory errors sounds very costly and kernel better not
>>> doing under the hood. But today there are at least 2 such cases:
>>> 1. GHES driver sees both GHES_SEV_CORRECTED and
>>>     CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
>>> 2. RAS Correctable Errors Collector counts correctable errors per
>>>     PFN and when the counter for a PFN reaches threshold
>>> In both cases, userspace has no control of the soft offline performed
>>> by kernel's memory failure recovery.
>> Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
>> -rlN` and then decide whether to soft offline the page according to it. But for
>> GHES driver, I think it has to be done in the kernel. So add a control in /sys/
>> seems like a good idea.
> Thanks.
>
>>> This patch series give userspace the control of soft-offlining
>>> HugeTLB pages: kernel only soft offlines hugepage if userspace has
>>> opt-ed in for that specific hugepage size, and exposed to userspace
>>> by a new sysfs entry called softoffline_corrected_errors under
>>> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
>>> * When softoffline_corrected_errors=0, skip soft offlining for all
>>>    hugepages of size ${size}kB.
>>> * When softoffline_corrected_errors=1, soft offline as before this
>> Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?
> "soft_offline_enabled" is less optimal as it can't be extended to
> support something like "soft offline this PFN if something repeatedly
> requested soft offline this exact PFN x times". (although I don't
> think we need it).

The "x time" thing is a threshold thing, and if your typical application 
needs to have a say about performance(and maintaining physically 
contiguous memory) over RAS, shouldn't that be baked into the driver 
rather than hugetlbfs ?

Also, I am not comfortable with this being hugetlbfs specific. What is 
the objection to creating a "soft_offline_enabled" switch that is 
applicable to any user page size?

thanks,

-jane

>
> softoffline_corrected_errors is one char less, but if you insist,
> soft_offline_corrected_errors also works for me.
>
>> Thanks.
>> .
>>

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jiaqi Yan]

Thanks for your feedback, Jane!

On Mon, Jun 10, 2024 at 12:41 PM Jane Chu <jane.chu@oracle.com> wrote:
>
> On 6/7/2024 3:22 PM, Jiaqi Yan wrote:
>
> > On Tue, Jun 4, 2024 at 12:19 AM Miaohe Lin <linmiaohe@huawei.com> wrote:
> >> On 2024/6/1 5:34, Jiaqi Yan wrote:
> >>> Correctable memory errors are very common on servers with large
> >>> amount of memory, and are corrected by ECC, but with two
> >>> pain points to users:
> >>> 1. Correction usually happens on the fly and adds latency overhead
> >>> 2. Not-fully-proved theory states excessive correctable memory
> >>>     errors can develop into uncorrectable memory error.
> >> Thanks for your patch.
> > Thanks Miaohe, sorry I missed your message (Gmail mistakenly put it in
> > my spam folder).
> >
> >>> Soft offline is kernel's additional solution for memory pages
> >>> having (excessive) corrected memory errors. Impacted page is migrated
> >>> to healthy page if it is in use, then the original page is discarded
> >>> for any future use.
> >>>
> >>> The actual policy on whether (and when) to soft offline should be
> >>> maintained by userspace, especially in case of HugeTLB hugepages.
> >>> Soft-offline dissolves a hugepage, either in-use or free, into
> >>> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
> >>> If userspace has not acknowledged such behavior, it may be surprised
> >>> when later mmap hugepages MAP_FAILED due to lack of hugepages.
> >> For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
> >> capacity won't be modified in that case. So I assume you're referring to the
> > I don't think so.
> >
> > For in-use hugetlb folio case, after migrate_pages, kernel will
> > dissolve_free_hugetlb_folio the src hugetlb folio. At this point
> > refcount of src hugetlb folio should be zero already, and
> > remove_hugetlb_folio will reduce the hugetlb pool capacity (both
> > nr_hugepages and free_hugepages) accordingly.
> >
> > For the free hugetlb folio case, dissolving also happens. But CE on
> > free pages should be very rare (since no one is accessing except
> > patrol scrubber).
> >
> > One of my test cases in patch 2/3 validates my point: the test case
> > MADV_SOFT_OFFLINE a mapped page and at the point soft offline
> > succeeds, both nr_hugepages and nr_freepages are reduced by 1.
> >
> >> free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
> >> But if we don't do that, we might encounter uncorrectable memory error later
> > If your concern is more correctable error will develop into more
> > severe uncorrectable, your concern is absolutely valid. There is a
> > tradeoff between reliability vs performance (availability of hugetlb
> > pages), but IMO should be decided by userspace.
> >
> >> which will be more severe? Will it be better to add a way to compensate the
> >> capacity?
> > Corner cases: What if finding physically contiguous memory takes too
> > long? What if we can't find any physically contiguous memory to
> > compensate? (then hugetlb pool will still need to be reduced).
> >
> > If we treat "compensate" as an improvement to the overall soft offline
> > process, it is something we can do in future and it is something
> > orthogonal to this control API, right? I think if userspace explicitly
> > tells kernel to soft offline, then they are also well-prepared for the
> > corner cases above.
> >
> >>> In addition, discarding the entire 1G memory page only because of
> >>> corrected memory errors sounds very costly and kernel better not
> >>> doing under the hood. But today there are at least 2 such cases:
> >>> 1. GHES driver sees both GHES_SEV_CORRECTED and
> >>>     CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
> >>> 2. RAS Correctable Errors Collector counts correctable errors per
> >>>     PFN and when the counter for a PFN reaches threshold
> >>> In both cases, userspace has no control of the soft offline performed
> >>> by kernel's memory failure recovery.
> >> Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
> >> -rlN` and then decide whether to soft offline the page according to it. But for
> >> GHES driver, I think it has to be done in the kernel. So add a control in /sys/
> >> seems like a good idea.
> > Thanks.
> >
> >>> This patch series give userspace the control of soft-offlining
> >>> HugeTLB pages: kernel only soft offlines hugepage if userspace has
> >>> opt-ed in for that specific hugepage size, and exposed to userspace
> >>> by a new sysfs entry called softoffline_corrected_errors under
> >>> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
> >>> * When softoffline_corrected_errors=0, skip soft offlining for all
> >>>    hugepages of size ${size}kB.
> >>> * When softoffline_corrected_errors=1, soft offline as before this
> >> Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?
> > "soft_offline_enabled" is less optimal as it can't be extended to
> > support something like "soft offline this PFN if something repeatedly
> > requested soft offline this exact PFN x times". (although I don't
> > think we need it).
>
> The "x time" thing is a threshold thing, and if your typical application
> needs to have a say about performance(and maintaining physically
> contiguous memory) over RAS, shouldn't that be baked into the driver
> rather than hugetlbfs ?

I mostly agree, only that I want to point out the threshold has
already been maintained by some firmware. For example, CPER has
something like the following defined in UEFI Spec Table N.5: Section
Descriptor:

  Bit 3 - Error threshold exceeded: If set, OS may choose to discontinue
  use of this resource.

In this case, I think "enable_soft_offline" is a better name for "OS
choose to discontinue use of this page" (enable_soft_offline=1) or not
(enable_soft_offline=0). WDYT?

>
> Also, I am not comfortable with this being hugetlbfs specific. What is
> the objection to creating a "soft_offline_enabled" switch that is
> applicable to any user page size?

I have no objection to making the "soft_offline_enabled" switch to
apply to anything (hugetlb, transparent hugepage, raw page, etc). The
only reason my current patch is hugetlb specific is because
softoffline behavior is very disruptive in the hugetlb 1G page case,
and I want to start with a limited scope in my first attempt.

If Miaohe, you, and other people are fine with making it applicable to
any user pages, maybe a better interface for this could be at
something like /sys/devices/system/memory/enable_soft_offline
(location-wise close to /sys/devices/system/memory/soft_offline_page)?

>
> thanks,
>
> -jane
>
> >
> > softoffline_corrected_errors is one char less, but if you insist,
> > soft_offline_corrected_errors also works for me.
> >
> >> Thanks.
> >> .
> >>

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jane Chu]

On 6/10/2024 3:55 PM, Jiaqi Yan wrote:

> Thanks for your feedback, Jane!
>
> On Mon, Jun 10, 2024 at 12:41 PM Jane Chu <jane.chu@oracle.com> wrote:
>> On 6/7/2024 3:22 PM, Jiaqi Yan wrote:
>>
>>> On Tue, Jun 4, 2024 at 12:19 AM Miaohe Lin <linmiaohe@huawei.com> wrote:
>>>> On 2024/6/1 5:34, Jiaqi Yan wrote:
>>>>> Correctable memory errors are very common on servers with large
>>>>> amount of memory, and are corrected by ECC, but with two
>>>>> pain points to users:
>>>>> 1. Correction usually happens on the fly and adds latency overhead
>>>>> 2. Not-fully-proved theory states excessive correctable memory
>>>>>      errors can develop into uncorrectable memory error.
>>>> Thanks for your patch.
>>> Thanks Miaohe, sorry I missed your message (Gmail mistakenly put it in
>>> my spam folder).
>>>
>>>>> Soft offline is kernel's additional solution for memory pages
>>>>> having (excessive) corrected memory errors. Impacted page is migrated
>>>>> to healthy page if it is in use, then the original page is discarded
>>>>> for any future use.
>>>>>
>>>>> The actual policy on whether (and when) to soft offline should be
>>>>> maintained by userspace, especially in case of HugeTLB hugepages.
>>>>> Soft-offline dissolves a hugepage, either in-use or free, into
>>>>> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
>>>>> If userspace has not acknowledged such behavior, it may be surprised
>>>>> when later mmap hugepages MAP_FAILED due to lack of hugepages.
>>>> For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
>>>> capacity won't be modified in that case. So I assume you're referring to the
>>> I don't think so.
>>>
>>> For in-use hugetlb folio case, after migrate_pages, kernel will
>>> dissolve_free_hugetlb_folio the src hugetlb folio. At this point
>>> refcount of src hugetlb folio should be zero already, and
>>> remove_hugetlb_folio will reduce the hugetlb pool capacity (both
>>> nr_hugepages and free_hugepages) accordingly.
>>>
>>> For the free hugetlb folio case, dissolving also happens. But CE on
>>> free pages should be very rare (since no one is accessing except
>>> patrol scrubber).
>>>
>>> One of my test cases in patch 2/3 validates my point: the test case
>>> MADV_SOFT_OFFLINE a mapped page and at the point soft offline
>>> succeeds, both nr_hugepages and nr_freepages are reduced by 1.
>>>
>>>> free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
>>>> But if we don't do that, we might encounter uncorrectable memory error later
>>> If your concern is more correctable error will develop into more
>>> severe uncorrectable, your concern is absolutely valid. There is a
>>> tradeoff between reliability vs performance (availability of hugetlb
>>> pages), but IMO should be decided by userspace.
>>>
>>>> which will be more severe? Will it be better to add a way to compensate the
>>>> capacity?
>>> Corner cases: What if finding physically contiguous memory takes too
>>> long? What if we can't find any physically contiguous memory to
>>> compensate? (then hugetlb pool will still need to be reduced).
>>>
>>> If we treat "compensate" as an improvement to the overall soft offline
>>> process, it is something we can do in future and it is something
>>> orthogonal to this control API, right? I think if userspace explicitly
>>> tells kernel to soft offline, then they are also well-prepared for the
>>> corner cases above.
>>>
>>>>> In addition, discarding the entire 1G memory page only because of
>>>>> corrected memory errors sounds very costly and kernel better not
>>>>> doing under the hood. But today there are at least 2 such cases:
>>>>> 1. GHES driver sees both GHES_SEV_CORRECTED and
>>>>>      CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
>>>>> 2. RAS Correctable Errors Collector counts correctable errors per
>>>>>      PFN and when the counter for a PFN reaches threshold
>>>>> In both cases, userspace has no control of the soft offline performed
>>>>> by kernel's memory failure recovery.
>>>> Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
>>>> -rlN` and then decide whether to soft offline the page according to it. But for
>>>> GHES driver, I think it has to be done in the kernel. So add a control in /sys/
>>>> seems like a good idea.
>>> Thanks.
>>>
>>>>> This patch series give userspace the control of soft-offlining
>>>>> HugeTLB pages: kernel only soft offlines hugepage if userspace has
>>>>> opt-ed in for that specific hugepage size, and exposed to userspace
>>>>> by a new sysfs entry called softoffline_corrected_errors under
>>>>> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
>>>>> * When softoffline_corrected_errors=0, skip soft offlining for all
>>>>>     hugepages of size ${size}kB.
>>>>> * When softoffline_corrected_errors=1, soft offline as before this
>>>> Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?
>>> "soft_offline_enabled" is less optimal as it can't be extended to
>>> support something like "soft offline this PFN if something repeatedly
>>> requested soft offline this exact PFN x times". (although I don't
>>> think we need it).
>> The "x time" thing is a threshold thing, and if your typical application
>> needs to have a say about performance(and maintaining physically
>> contiguous memory) over RAS, shouldn't that be baked into the driver
>> rather than hugetlbfs ?
> I mostly agree, only that I want to point out the threshold has
> already been maintained by some firmware. For example, CPER has
> something like the following defined in UEFI Spec Table N.5: Section
> Descriptor:
>
>    Bit 3 - Error threshold exceeded: If set, OS may choose to discontinue
>    use of this resource.
>
> In this case, I think "enable_soft_offline" is a better name for "OS
> choose to discontinue use of this page" (enable_soft_offline=1) or not
> (enable_soft_offline=0). WDYT?

Yes, as long as enable_soft_offline=1 is the default. Out of thought, I 
suppose the CE count and threshold can be retrieved by the GHES driver? 
I haven't checked. If so, maybe another way is to implement a per task 
CE threshold: add a new field.ce_thresholdto the tsak struct, add a 
function to prctl(2) for a user thread to specify a CE threshold, also a 
function to retrieve the firmware defined default CE threshold, and let 
soft_offline_page() check against the task->ce_threshold to decide 
whether to offline the page. If you want to apply the CE threshold to 
patrol scrub triggered soft offline, than you could define a 
global/system wide CE threshold. That said, this might be an overblown 
to what you need, I'm just letting it out there for the sake of brain 
storming.

>
>> Also, I am not comfortable with this being hugetlbfs specific. What is
>> the objection to creating a "soft_offline_enabled" switch that is
>> applicable to any user page size?
> I have no objection to making the "soft_offline_enabled" switch to
> apply to anything (hugetlb, transparent hugepage, raw page, etc). The
> only reason my current patch is hugetlb specific is because
> softoffline behavior is very disruptive in the hugetlb 1G page case,
> and I want to start with a limited scope in my first attempt.
>
> If Miaohe, you, and other people are fine with making it applicable to
> any user pages, maybe a better interface for this could be at
> something like /sys/devices/system/memory/enable_soft_offline
> (location-wise close to /sys/devices/system/memory/soft_offline_page)?

Or, you could use /proc/sys/vm/enable_soft_offline, side by side with 
the existing 'memory_failure_early_kill' and 'memory_failure_recovery' 
switches.

You could also make 'enable_soft_offline' a per process option, similar 
to 'PR_MCE_KILL_EARLY' in prctl(2).*
*

thanks,

-jane

>
>> thanks,
>>
>> -jane
>>
>>> softoffline_corrected_errors is one char less, but if you insist,
>>> soft_offline_corrected_errors also works for me.
>>>
>>>> Thanks.
>>>> .
>>>>

Re: [PATCH v1 0/3] Userspace controls soft-offline HugeTLB pages

[Jiaqi Yan]

On Tue, Jun 11, 2024 at 10:55 AM Jane Chu <jane.chu@oracle.com> wrote:
>
> On 6/10/2024 3:55 PM, Jiaqi Yan wrote:
>
> > Thanks for your feedback, Jane!
> >
> > On Mon, Jun 10, 2024 at 12:41 PM Jane Chu <jane.chu@oracle.com> wrote:
> >> On 6/7/2024 3:22 PM, Jiaqi Yan wrote:
> >>
> >>> On Tue, Jun 4, 2024 at 12:19 AM Miaohe Lin <linmiaohe@huawei.com> wrote:
> >>>> On 2024/6/1 5:34, Jiaqi Yan wrote:
> >>>>> Correctable memory errors are very common on servers with large
> >>>>> amount of memory, and are corrected by ECC, but with two
> >>>>> pain points to users:
> >>>>> 1. Correction usually happens on the fly and adds latency overhead
> >>>>> 2. Not-fully-proved theory states excessive correctable memory
> >>>>>      errors can develop into uncorrectable memory error.
> >>>> Thanks for your patch.
> >>> Thanks Miaohe, sorry I missed your message (Gmail mistakenly put it in
> >>> my spam folder).
> >>>
> >>>>> Soft offline is kernel's additional solution for memory pages
> >>>>> having (excessive) corrected memory errors. Impacted page is migrated
> >>>>> to healthy page if it is in use, then the original page is discarded
> >>>>> for any future use.
> >>>>>
> >>>>> The actual policy on whether (and when) to soft offline should be
> >>>>> maintained by userspace, especially in case of HugeTLB hugepages.
> >>>>> Soft-offline dissolves a hugepage, either in-use or free, into
> >>>>> chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
> >>>>> If userspace has not acknowledged such behavior, it may be surprised
> >>>>> when later mmap hugepages MAP_FAILED due to lack of hugepages.
> >>>> For in use hugetlb folio case, migrate_pages() is called. The hugetlb pool
> >>>> capacity won't be modified in that case. So I assume you're referring to the
> >>> I don't think so.
> >>>
> >>> For in-use hugetlb folio case, after migrate_pages, kernel will
> >>> dissolve_free_hugetlb_folio the src hugetlb folio. At this point
> >>> refcount of src hugetlb folio should be zero already, and
> >>> remove_hugetlb_folio will reduce the hugetlb pool capacity (both
> >>> nr_hugepages and free_hugepages) accordingly.
> >>>
> >>> For the free hugetlb folio case, dissolving also happens. But CE on
> >>> free pages should be very rare (since no one is accessing except
> >>> patrol scrubber).
> >>>
> >>> One of my test cases in patch 2/3 validates my point: the test case
> >>> MADV_SOFT_OFFLINE a mapped page and at the point soft offline
> >>> succeeds, both nr_hugepages and nr_freepages are reduced by 1.
> >>>
> >>>> free hugetlb folio case? The Hugetlb pool capacity is reduced in that case.
> >>>> But if we don't do that, we might encounter uncorrectable memory error later
> >>> If your concern is more correctable error will develop into more
> >>> severe uncorrectable, your concern is absolutely valid. There is a
> >>> tradeoff between reliability vs performance (availability of hugetlb
> >>> pages), but IMO should be decided by userspace.
> >>>
> >>>> which will be more severe? Will it be better to add a way to compensate the
> >>>> capacity?
> >>> Corner cases: What if finding physically contiguous memory takes too
> >>> long? What if we can't find any physically contiguous memory to
> >>> compensate? (then hugetlb pool will still need to be reduced).
> >>>
> >>> If we treat "compensate" as an improvement to the overall soft offline
> >>> process, it is something we can do in future and it is something
> >>> orthogonal to this control API, right? I think if userspace explicitly
> >>> tells kernel to soft offline, then they are also well-prepared for the
> >>> corner cases above.
> >>>
> >>>>> In addition, discarding the entire 1G memory page only because of
> >>>>> corrected memory errors sounds very costly and kernel better not
> >>>>> doing under the hood. But today there are at least 2 such cases:
> >>>>> 1. GHES driver sees both GHES_SEV_CORRECTED and
> >>>>>      CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
> >>>>> 2. RAS Correctable Errors Collector counts correctable errors per
> >>>>>      PFN and when the counter for a PFN reaches threshold
> >>>>> In both cases, userspace has no control of the soft offline performed
> >>>>> by kernel's memory failure recovery.
> >>>> Userspace can figure out the hugetlb folio pfn range by using `page-types -b huge
> >>>> -rlN` and then decide whether to soft offline the page according to it. But for
> >>>> GHES driver, I think it has to be done in the kernel. So add a control in /sys/
> >>>> seems like a good idea.
> >>> Thanks.
> >>>
> >>>>> This patch series give userspace the control of soft-offlining
> >>>>> HugeTLB pages: kernel only soft offlines hugepage if userspace has
> >>>>> opt-ed in for that specific hugepage size, and exposed to userspace
> >>>>> by a new sysfs entry called softoffline_corrected_errors under
> >>>>> /sys/kernel/mm/hugepages/hugepages-${size}kB directory:
> >>>>> * When softoffline_corrected_errors=0, skip soft offlining for all
> >>>>>     hugepages of size ${size}kB.
> >>>>> * When softoffline_corrected_errors=1, soft offline as before this
> >>>> Will it be better to be called as "soft_offline_corrected_errors" or simplify "soft_offline_enabled"?
> >>> "soft_offline_enabled" is less optimal as it can't be extended to
> >>> support something like "soft offline this PFN if something repeatedly
> >>> requested soft offline this exact PFN x times". (although I don't
> >>> think we need it).
> >> The "x time" thing is a threshold thing, and if your typical application
> >> needs to have a say about performance(and maintaining physically
> >> contiguous memory) over RAS, shouldn't that be baked into the driver
> >> rather than hugetlbfs ?
> > I mostly agree, only that I want to point out the threshold has
> > already been maintained by some firmware. For example, CPER has
> > something like the following defined in UEFI Spec Table N.5: Section
> > Descriptor:
> >
> >    Bit 3 - Error threshold exceeded: If set, OS may choose to discontinue
> >    use of this resource.
> >
> > In this case, I think "enable_soft_offline" is a better name for "OS
> > choose to discontinue use of this page" (enable_soft_offline=1) or not
> > (enable_soft_offline=0). WDYT?
>
> Yes, as long as enable_soft_offline=1 is the default. Out of thought, I

For sure, like this patcheset, I will ensure enable_soft_offline keeps
the "default on" behavior.

> suppose the CE count and threshold can be retrieved by the GHES driver?

Unfortunately GHES doesn't have visibility to CE count and threshold
value (RAS Correctable Errors Collector does by itself) . GHES driver
only knows it from the CPER_SEC_ERROR_THRESHOLD_EXCEEDED bit in the
CPER reported by some firmware.

> I haven't checked. If so, maybe another way is to implement a per task
> CE threshold: add a new field.ce_thresholdto the tsak struct, add a
> function to prctl(2) for a user thread to specify a CE threshold, also a
> function to retrieve the firmware defined default CE threshold, and let
> soft_offline_page() check against the task->ce_threshold to decide
> whether to offline the page. If you want to apply the CE threshold to
> patrol scrub triggered soft offline, than you could define a
> global/system wide CE threshold. That said, this might be an overblown
> to what you need, I'm just letting it out there for the sake of brain
> storming.

Thanks for your great idea! But yeah, it sounds like an overkill for
the current problem. I think starting with OS-wide control of whether
to soft offline any page already gives much better flexibility to
userspace.

>
> >
> >> Also, I am not comfortable with this being hugetlbfs specific. What is
> >> the objection to creating a "soft_offline_enabled" switch that is
> >> applicable to any user page size?
> > I have no objection to making the "soft_offline_enabled" switch to
> > apply to anything (hugetlb, transparent hugepage, raw page, etc). The
> > only reason my current patch is hugetlb specific is because
> > softoffline behavior is very disruptive in the hugetlb 1G page case,
> > and I want to start with a limited scope in my first attempt.
> >
> > If Miaohe, you, and other people are fine with making it applicable to
> > any user pages, maybe a better interface for this could be at
> > something like /sys/devices/system/memory/enable_soft_offline
> > (location-wise close to /sys/devices/system/memory/soft_offline_page)?
>
> Or, you could use /proc/sys/vm/enable_soft_offline, side by side with
> the existing 'memory_failure_early_kill' and 'memory_failure_recovery'
> switches.

I was actually looking into this better option, but your reply beats me ;)

>
> You could also make 'enable_soft_offline' a per process option, similar
> to 'PR_MCE_KILL_EARLY' in prctl(2).*
> *
>
> thanks,
>
> -jane
>
> >
> >> thanks,
> >>
> >> -jane
> >>
> >>> softoffline_corrected_errors is one char less, but if you insist,
> >>> soft_offline_corrected_errors also works for me.
> >>>
> >>>> Thanks.
> >>>> .
> >>>>