[PATCH 0/5] fix offline memcgroup still hold in memory

[PATCH 0/5] fix offline memcgroup still hold in memory

[Xiongchun Duan]

we find that in huge memory system frequent creat creation and deletion
memcgroup make the system leave lots of offline memcgroup.we had seen 100000 
unrelease offline memcgroup in our system(512G memory).

this memcgroup hold because some memory page still charged.
so we try to Multiple interval call force_empty to reclaim this memory page.

after applying those patchs,in our system,the unrelease offline memcgroup
was reduced from 100000 to 100.


Xiongchun Duan (5):
  Memcgroup: force empty after memcgroup offline
  Memcgroup: Add timer to trigger workqueue
  Memcgroup:add a global work
  Memcgroup:Implement force empty work function
  Memcgroup:add cgroup fs to show offline memcgroup status

 Documentation/cgroup-v1/memory.txt |   7 +-
 Documentation/sysctl/kernel.txt    |  10 ++
 include/linux/memcontrol.h         |  11 ++
 kernel/sysctl.c                    |   9 ++
 mm/memcontrol.c                    | 271 +++++++++++++++++++++++++++++++++++++
 5 files changed, 306 insertions(+), 2 deletions(-)

-- 
1.8.3.1

[PATCH 1/5] Memcgroup: force empty after memcgroup offline

[Xiongchun Duan]

After memcgroup offline,if page still charge in memcgroup, this memcgroup will hold
in memory.in some system which has many memory(such 256G) will hold more than
100000 offline memcgroup. this memory can't be free as soon as possible.
Using workqueue and timer to repeatedly trigger offline memcgroup force empty
memory will solve this problem. the reason why need repeatedly trigger is
that force_empty fail to reclaim page when this page is locked.

Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
---
 Documentation/cgroup-v1/memory.txt |  7 +++++--
 Documentation/sysctl/kernel.txt    | 10 ++++++++++
 include/linux/memcontrol.h         |  6 ++++++
 kernel/sysctl.c                    |  9 +++++++++
 mm/memcontrol.c                    | 17 +++++++++++++++++
 5 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/Documentation/cgroup-v1/memory.txt b/Documentation/cgroup-v1/memory.txt
index 3682e99..bdba86f 100644
--- a/Documentation/cgroup-v1/memory.txt
+++ b/Documentation/cgroup-v1/memory.txt
@@ -452,11 +452,14 @@ About use_hierarchy, see Section 6.
 
 5.1 force_empty
   memory.force_empty interface is provided to make cgroup's memory usage empty.
-  When writing anything to this
+  When writing o or 1 or >18 to this
 
   # echo 0 > memory.force_empty
 
-  the cgroup will be reclaimed and as many pages reclaimed as possible.
+  the cgroup will be reclaimed and as many pages reclaimed as possible
+  synchronously.
+  writing 2 to 18 to this, the cgroup will delay the memory reclaim to css offline.
+  if memory reclaim fail one call, will delay to workqueue to recalaim as many as value.
 
   The typical use case for this interface is before calling rmdir().
   Because rmdir() moves all pages to parent, some out-of-use page caches can be
diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index c0527d8..fc0b9b1 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -99,6 +99,7 @@ show up in /proc/sys/kernel:
 - unknown_nmi_panic
 - watchdog
 - watchdog_thresh
+- cgroup_default_retry
 - version
 
 ==============================================================
@@ -1137,3 +1138,12 @@ The softlockup threshold is (2 * watchdog_thresh). Setting this
 tunable to zero will disable lockup detection altogether.
 
 ==============================================================
+
+cgroup_default_retry:
+
+This value can be used to control the default of memory cgroup reclaim
+times . The default value is 0 .
+
+the max value is 16 the min value is 0.
+
+==============================================================
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 83ae11c..d6fbb77 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -311,9 +311,15 @@ struct mem_cgroup {
 	struct list_head event_list;
 	spinlock_t event_list_lock;
 
+	int max_retry;
+	int current_retry;
+
 	struct mem_cgroup_per_node *nodeinfo[0];
 	/* WARNING: nodeinfo must be the last member here */
 };
+extern int sysctl_cgroup_default_retry;
+extern int sysctl_cgroup_default_retry_min;
+extern int sysctl_cgroup_default_retry_max;
 
 /*
  * size of first charge trial. "32" comes from vmscan.c's magic value.
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index ba4d9e8..b6dbb10 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -1252,6 +1252,15 @@ static int sysrq_sysctl_handler(struct ctl_table *table, int write,
 		.extra2		= &one,
 	},
 #endif
+	{
+		.procname       = "cgroup_default_retry",
+		.data           = &sysctl_cgroup_default_retry,
+		.maxlen         = sizeof(unsigned int),
+		.mode           = 0644,
+		.proc_handler   = proc_dointvec_minmax,
+		.extra1         = &sysctl_cgroup_default_retry_min,
+		.extra2         = &sysctl_cgroup_default_retry_max,
+	},
 	{ }
 };
 
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index af7f18b..2b13c2b 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -77,6 +77,10 @@
 struct cgroup_subsys memory_cgrp_subsys __read_mostly;
 EXPORT_SYMBOL(memory_cgrp_subsys);
 
+int sysctl_cgroup_default_retry __read_mostly;
+int sysctl_cgroup_default_retry_min;
+int sysctl_cgroup_default_retry_max = 16;
+
 struct mem_cgroup *root_mem_cgroup __read_mostly;
 
 #define MEM_CGROUP_RECLAIM_RETRIES	5
@@ -2911,10 +2915,21 @@ static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of,
 					    char *buf, size_t nbytes,
 					    loff_t off)
 {
+	unsigned long val;
+	ssize_t ret;
 	struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
 
 	if (mem_cgroup_is_root(memcg))
 		return -EINVAL;
+
+	buf = strstrip(buf);
+	ret = kstrtoul(buf, 10, &val);
+	if (ret < 0)
+		return ret;
+	if (val > 1 && val < 18) {
+		memcg->max_retry = val - 1;
+		return nbytes;
+	}
 	return mem_cgroup_force_empty(memcg) ?: nbytes;
 }
 
@@ -4521,6 +4536,8 @@ static struct mem_cgroup *mem_cgroup_alloc(void)
 
 	if (cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_nosocket)
 		static_branch_inc(&memcg_sockets_enabled_key);
+	memcg->max_retry = sysctl_cgroup_default_retry;
+	memcg->current_retry  = 0;
 
 	return &memcg->css;
 fail:
-- 
1.8.3.1

Re: [PATCH 1/5] Memcgroup: force empty after memcgroup offline

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 10701 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: x86_64-randconfig-x005-201903 (attached as .config)
compiler: gcc-8 (Debian 8.2.0-14) 8.2.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

>> kernel/sysctl.c:1257:22: error: 'sysctl_cgroup_default_retry' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .data           = &sysctl_cgroup_default_retry,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default
>> kernel/sysctl.c:1261:22: error: 'sysctl_cgroup_default_retry_min' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .extra1         = &sysctl_cgroup_default_retry_min,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default
>> kernel/sysctl.c:1262:22: error: 'sysctl_cgroup_default_retry_max' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .extra2         = &sysctl_cgroup_default_retry_max,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default

vim +1257 kernel/sysctl.c

   977	
   978	#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86)
   979		{
   980			.procname       = "unknown_nmi_panic",
   981			.data           = &unknown_nmi_panic,
   982			.maxlen         = sizeof (int),
   983			.mode           = 0644,
   984			.proc_handler   = proc_dointvec,
   985		},
   986	#endif
   987	#if defined(CONFIG_X86)
   988		{
   989			.procname	= "panic_on_unrecovered_nmi",
   990			.data		= &panic_on_unrecovered_nmi,
   991			.maxlen		= sizeof(int),
   992			.mode		= 0644,
   993			.proc_handler	= proc_dointvec,
   994		},
   995		{
   996			.procname	= "panic_on_io_nmi",
   997			.data		= &panic_on_io_nmi,
   998			.maxlen		= sizeof(int),
   999			.mode		= 0644,
  1000			.proc_handler	= proc_dointvec,
  1001		},
  1002	#ifdef CONFIG_DEBUG_STACKOVERFLOW
  1003		{
  1004			.procname	= "panic_on_stackoverflow",
  1005			.data		= &sysctl_panic_on_stackoverflow,
  1006			.maxlen		= sizeof(int),
  1007			.mode		= 0644,
  1008			.proc_handler	= proc_dointvec,
  1009		},
  1010	#endif
  1011		{
  1012			.procname	= "bootloader_type",
  1013			.data		= &bootloader_type,
  1014			.maxlen		= sizeof (int),
  1015			.mode		= 0444,
  1016			.proc_handler	= proc_dointvec,
  1017		},
  1018		{
  1019			.procname	= "bootloader_version",
  1020			.data		= &bootloader_version,
  1021			.maxlen		= sizeof (int),
  1022			.mode		= 0444,
  1023			.proc_handler	= proc_dointvec,
  1024		},
  1025		{
  1026			.procname	= "io_delay_type",
  1027			.data		= &io_delay_type,
  1028			.maxlen		= sizeof(int),
  1029			.mode		= 0644,
  1030			.proc_handler	= proc_dointvec,
  1031		},
  1032	#endif
  1033	#if defined(CONFIG_MMU)
  1034		{
  1035			.procname	= "randomize_va_space",
  1036			.data		= &randomize_va_space,
  1037			.maxlen		= sizeof(int),
  1038			.mode		= 0644,
  1039			.proc_handler	= proc_dointvec,
  1040		},
  1041	#endif
  1042	#if defined(CONFIG_S390) && defined(CONFIG_SMP)
  1043		{
  1044			.procname	= "spin_retry",
  1045			.data		= &spin_retry,
  1046			.maxlen		= sizeof (int),
  1047			.mode		= 0644,
  1048			.proc_handler	= proc_dointvec,
  1049		},
  1050	#endif
  1051	#if	defined(CONFIG_ACPI_SLEEP) && defined(CONFIG_X86)
  1052		{
  1053			.procname	= "acpi_video_flags",
  1054			.data		= &acpi_realmode_flags,
  1055			.maxlen		= sizeof (unsigned long),
  1056			.mode		= 0644,
  1057			.proc_handler	= proc_doulongvec_minmax,
  1058		},
  1059	#endif
  1060	#ifdef CONFIG_SYSCTL_ARCH_UNALIGN_NO_WARN
  1061		{
  1062			.procname	= "ignore-unaligned-usertrap",
  1063			.data		= &no_unaligned_warning,
  1064			.maxlen		= sizeof (int),
  1065		 	.mode		= 0644,
  1066			.proc_handler	= proc_dointvec,
  1067		},
  1068	#endif
  1069	#ifdef CONFIG_IA64
  1070		{
  1071			.procname	= "unaligned-dump-stack",
  1072			.data		= &unaligned_dump_stack,
  1073			.maxlen		= sizeof (int),
  1074			.mode		= 0644,
  1075			.proc_handler	= proc_dointvec,
  1076		},
  1077	#endif
  1078	#ifdef CONFIG_DETECT_HUNG_TASK
  1079		{
  1080			.procname	= "hung_task_panic",
  1081			.data		= &sysctl_hung_task_panic,
  1082			.maxlen		= sizeof(int),
  1083			.mode		= 0644,
  1084			.proc_handler	= proc_dointvec_minmax,
  1085			.extra1		= &zero,
  1086			.extra2		= &one,
  1087		},
  1088		{
  1089			.procname	= "hung_task_check_count",
  1090			.data		= &sysctl_hung_task_check_count,
  1091			.maxlen		= sizeof(int),
  1092			.mode		= 0644,
  1093			.proc_handler	= proc_dointvec_minmax,
  1094			.extra1		= &zero,
  1095		},
  1096		{
  1097			.procname	= "hung_task_timeout_secs",
  1098			.data		= &sysctl_hung_task_timeout_secs,
  1099			.maxlen		= sizeof(unsigned long),
  1100			.mode		= 0644,
  1101			.proc_handler	= proc_dohung_task_timeout_secs,
  1102			.extra2		= &hung_task_timeout_max,
  1103		},
  1104		{
  1105			.procname	= "hung_task_check_interval_secs",
  1106			.data		= &sysctl_hung_task_check_interval_secs,
  1107			.maxlen		= sizeof(unsigned long),
  1108			.mode		= 0644,
  1109			.proc_handler	= proc_dohung_task_timeout_secs,
  1110			.extra2		= &hung_task_timeout_max,
  1111		},
  1112		{
  1113			.procname	= "hung_task_warnings",
  1114			.data		= &sysctl_hung_task_warnings,
  1115			.maxlen		= sizeof(int),
  1116			.mode		= 0644,
  1117			.proc_handler	= proc_dointvec_minmax,
  1118			.extra1		= &neg_one,
  1119		},
  1120	#endif
  1121	#ifdef CONFIG_RT_MUTEXES
  1122		{
  1123			.procname	= "max_lock_depth",
  1124			.data		= &max_lock_depth,
  1125			.maxlen		= sizeof(int),
  1126			.mode		= 0644,
  1127			.proc_handler	= proc_dointvec,
  1128		},
  1129	#endif
  1130		{
  1131			.procname	= "poweroff_cmd",
  1132			.data		= &poweroff_cmd,
  1133			.maxlen		= POWEROFF_CMD_PATH_LEN,
  1134			.mode		= 0644,
  1135			.proc_handler	= proc_dostring,
  1136		},
  1137	#ifdef CONFIG_KEYS
  1138		{
  1139			.procname	= "keys",
  1140			.mode		= 0555,
  1141			.child		= key_sysctls,
  1142		},
  1143	#endif
  1144	#ifdef CONFIG_PERF_EVENTS
  1145		/*
  1146		 * User-space scripts rely on the existence of this file
  1147		 * as a feature check for perf_events being enabled.
  1148		 *
  1149		 * So it's an ABI, do not remove!
  1150		 */
  1151		{
  1152			.procname	= "perf_event_paranoid",
  1153			.data		= &sysctl_perf_event_paranoid,
  1154			.maxlen		= sizeof(sysctl_perf_event_paranoid),
  1155			.mode		= 0644,
  1156			.proc_handler	= proc_dointvec,
  1157		},
  1158		{
  1159			.procname	= "perf_event_mlock_kb",
  1160			.data		= &sysctl_perf_event_mlock,
  1161			.maxlen		= sizeof(sysctl_perf_event_mlock),
  1162			.mode		= 0644,
  1163			.proc_handler	= proc_dointvec,
  1164		},
  1165		{
  1166			.procname	= "perf_event_max_sample_rate",
  1167			.data		= &sysctl_perf_event_sample_rate,
  1168			.maxlen		= sizeof(sysctl_perf_event_sample_rate),
  1169			.mode		= 0644,
  1170			.proc_handler	= perf_proc_update_handler,
  1171			.extra1		= &one,
  1172		},
  1173		{
  1174			.procname	= "perf_cpu_time_max_percent",
  1175			.data		= &sysctl_perf_cpu_time_max_percent,
  1176			.maxlen		= sizeof(sysctl_perf_cpu_time_max_percent),
  1177			.mode		= 0644,
  1178			.proc_handler	= perf_cpu_time_max_percent_handler,
  1179			.extra1		= &zero,
  1180			.extra2		= &one_hundred,
  1181		},
  1182		{
  1183			.procname	= "perf_event_max_stack",
  1184			.data		= &sysctl_perf_event_max_stack,
  1185			.maxlen		= sizeof(sysctl_perf_event_max_stack),
  1186			.mode		= 0644,
  1187			.proc_handler	= perf_event_max_stack_handler,
  1188			.extra1		= &zero,
  1189			.extra2		= &six_hundred_forty_kb,
  1190		},
  1191		{
  1192			.procname	= "perf_event_max_contexts_per_stack",
  1193			.data		= &sysctl_perf_event_max_contexts_per_stack,
  1194			.maxlen		= sizeof(sysctl_perf_event_max_contexts_per_stack),
  1195			.mode		= 0644,
  1196			.proc_handler	= perf_event_max_stack_handler,
  1197			.extra1		= &zero,
  1198			.extra2		= &one_thousand,
  1199		},
  1200	#endif
  1201		{
  1202			.procname	= "panic_on_warn",
  1203			.data		= &panic_on_warn,
  1204			.maxlen		= sizeof(int),
  1205			.mode		= 0644,
  1206			.proc_handler	= proc_dointvec_minmax,
  1207			.extra1		= &zero,
  1208			.extra2		= &one,
  1209		},
  1210	#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
  1211		{
  1212			.procname	= "timer_migration",
  1213			.data		= &sysctl_timer_migration,
  1214			.maxlen		= sizeof(unsigned int),
  1215			.mode		= 0644,
  1216			.proc_handler	= timer_migration_handler,
  1217			.extra1		= &zero,
  1218			.extra2		= &one,
  1219		},
  1220	#endif
  1221	#ifdef CONFIG_BPF_SYSCALL
  1222		{
  1223			.procname	= "unprivileged_bpf_disabled",
  1224			.data		= &sysctl_unprivileged_bpf_disabled,
  1225			.maxlen		= sizeof(sysctl_unprivileged_bpf_disabled),
  1226			.mode		= 0644,
  1227			/* only handle a transition from default "0" to "1" */
  1228			.proc_handler	= proc_dointvec_minmax,
  1229			.extra1		= &one,
  1230			.extra2		= &one,
  1231		},
  1232	#endif
  1233	#if defined(CONFIG_TREE_RCU) || defined(CONFIG_PREEMPT_RCU)
  1234		{
  1235			.procname	= "panic_on_rcu_stall",
  1236			.data		= &sysctl_panic_on_rcu_stall,
  1237			.maxlen		= sizeof(sysctl_panic_on_rcu_stall),
  1238			.mode		= 0644,
  1239			.proc_handler	= proc_dointvec_minmax,
  1240			.extra1		= &zero,
  1241			.extra2		= &one,
  1242		},
  1243	#endif
  1244	#ifdef CONFIG_STACKLEAK_RUNTIME_DISABLE
  1245		{
  1246			.procname	= "stack_erasing",
  1247			.data		= NULL,
  1248			.maxlen		= sizeof(int),
  1249			.mode		= 0600,
  1250			.proc_handler	= stack_erasing_sysctl,
  1251			.extra1		= &zero,
  1252			.extra2		= &one,
  1253		},
  1254	#endif
  1255		{
  1256			.procname       = "cgroup_default_retry",
> 1257			.data           = &sysctl_cgroup_default_retry,
  1258			.maxlen         = sizeof(unsigned int),
  1259			.mode           = 0644,
  1260			.proc_handler   = proc_dointvec_minmax,
> 1261			.extra1         = &sysctl_cgroup_default_retry_min,
> 1262			.extra2         = &sysctl_cgroup_default_retry_max,
  1263		},
  1264		{ }
  1265	};
  1266	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 29505 bytes --]

Re: [PATCH 1/5] Memcgroup: force empty after memcgroup offline

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 10701 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: x86_64-randconfig-x005-201903 (attached as .config)
compiler: gcc-8 (Debian 8.2.0-14) 8.2.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

>> kernel/sysctl.c:1257:22: error: 'sysctl_cgroup_default_retry' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .data           = &sysctl_cgroup_default_retry,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default
>> kernel/sysctl.c:1261:22: error: 'sysctl_cgroup_default_retry_min' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .extra1         = &sysctl_cgroup_default_retry_min,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default
>> kernel/sysctl.c:1262:22: error: 'sysctl_cgroup_default_retry_max' undeclared here (not in a function); did you mean 'sysctl_rmem_default'?
      .extra2         = &sysctl_cgroup_default_retry_max,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         sysctl_rmem_default

vim +1257 kernel/sysctl.c

   977	
   978	#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86)
   979		{
   980			.procname       = "unknown_nmi_panic",
   981			.data           = &unknown_nmi_panic,
   982			.maxlen         = sizeof (int),
   983			.mode           = 0644,
   984			.proc_handler   = proc_dointvec,
   985		},
   986	#endif
   987	#if defined(CONFIG_X86)
   988		{
   989			.procname	= "panic_on_unrecovered_nmi",
   990			.data		= &panic_on_unrecovered_nmi,
   991			.maxlen		= sizeof(int),
   992			.mode		= 0644,
   993			.proc_handler	= proc_dointvec,
   994		},
   995		{
   996			.procname	= "panic_on_io_nmi",
   997			.data		= &panic_on_io_nmi,
   998			.maxlen		= sizeof(int),
   999			.mode		= 0644,
  1000			.proc_handler	= proc_dointvec,
  1001		},
  1002	#ifdef CONFIG_DEBUG_STACKOVERFLOW
  1003		{
  1004			.procname	= "panic_on_stackoverflow",
  1005			.data		= &sysctl_panic_on_stackoverflow,
  1006			.maxlen		= sizeof(int),
  1007			.mode		= 0644,
  1008			.proc_handler	= proc_dointvec,
  1009		},
  1010	#endif
  1011		{
  1012			.procname	= "bootloader_type",
  1013			.data		= &bootloader_type,
  1014			.maxlen		= sizeof (int),
  1015			.mode		= 0444,
  1016			.proc_handler	= proc_dointvec,
  1017		},
  1018		{
  1019			.procname	= "bootloader_version",
  1020			.data		= &bootloader_version,
  1021			.maxlen		= sizeof (int),
  1022			.mode		= 0444,
  1023			.proc_handler	= proc_dointvec,
  1024		},
  1025		{
  1026			.procname	= "io_delay_type",
  1027			.data		= &io_delay_type,
  1028			.maxlen		= sizeof(int),
  1029			.mode		= 0644,
  1030			.proc_handler	= proc_dointvec,
  1031		},
  1032	#endif
  1033	#if defined(CONFIG_MMU)
  1034		{
  1035			.procname	= "randomize_va_space",
  1036			.data		= &randomize_va_space,
  1037			.maxlen		= sizeof(int),
  1038			.mode		= 0644,
  1039			.proc_handler	= proc_dointvec,
  1040		},
  1041	#endif
  1042	#if defined(CONFIG_S390) && defined(CONFIG_SMP)
  1043		{
  1044			.procname	= "spin_retry",
  1045			.data		= &spin_retry,
  1046			.maxlen		= sizeof (int),
  1047			.mode		= 0644,
  1048			.proc_handler	= proc_dointvec,
  1049		},
  1050	#endif
  1051	#if	defined(CONFIG_ACPI_SLEEP) && defined(CONFIG_X86)
  1052		{
  1053			.procname	= "acpi_video_flags",
  1054			.data		= &acpi_realmode_flags,
  1055			.maxlen		= sizeof (unsigned long),
  1056			.mode		= 0644,
  1057			.proc_handler	= proc_doulongvec_minmax,
  1058		},
  1059	#endif
  1060	#ifdef CONFIG_SYSCTL_ARCH_UNALIGN_NO_WARN
  1061		{
  1062			.procname	= "ignore-unaligned-usertrap",
  1063			.data		= &no_unaligned_warning,
  1064			.maxlen		= sizeof (int),
  1065		 	.mode		= 0644,
  1066			.proc_handler	= proc_dointvec,
  1067		},
  1068	#endif
  1069	#ifdef CONFIG_IA64
  1070		{
  1071			.procname	= "unaligned-dump-stack",
  1072			.data		= &unaligned_dump_stack,
  1073			.maxlen		= sizeof (int),
  1074			.mode		= 0644,
  1075			.proc_handler	= proc_dointvec,
  1076		},
  1077	#endif
  1078	#ifdef CONFIG_DETECT_HUNG_TASK
  1079		{
  1080			.procname	= "hung_task_panic",
  1081			.data		= &sysctl_hung_task_panic,
  1082			.maxlen		= sizeof(int),
  1083			.mode		= 0644,
  1084			.proc_handler	= proc_dointvec_minmax,
  1085			.extra1		= &zero,
  1086			.extra2		= &one,
  1087		},
  1088		{
  1089			.procname	= "hung_task_check_count",
  1090			.data		= &sysctl_hung_task_check_count,
  1091			.maxlen		= sizeof(int),
  1092			.mode		= 0644,
  1093			.proc_handler	= proc_dointvec_minmax,
  1094			.extra1		= &zero,
  1095		},
  1096		{
  1097			.procname	= "hung_task_timeout_secs",
  1098			.data		= &sysctl_hung_task_timeout_secs,
  1099			.maxlen		= sizeof(unsigned long),
  1100			.mode		= 0644,
  1101			.proc_handler	= proc_dohung_task_timeout_secs,
  1102			.extra2		= &hung_task_timeout_max,
  1103		},
  1104		{
  1105			.procname	= "hung_task_check_interval_secs",
  1106			.data		= &sysctl_hung_task_check_interval_secs,
  1107			.maxlen		= sizeof(unsigned long),
  1108			.mode		= 0644,
  1109			.proc_handler	= proc_dohung_task_timeout_secs,
  1110			.extra2		= &hung_task_timeout_max,
  1111		},
  1112		{
  1113			.procname	= "hung_task_warnings",
  1114			.data		= &sysctl_hung_task_warnings,
  1115			.maxlen		= sizeof(int),
  1116			.mode		= 0644,
  1117			.proc_handler	= proc_dointvec_minmax,
  1118			.extra1		= &neg_one,
  1119		},
  1120	#endif
  1121	#ifdef CONFIG_RT_MUTEXES
  1122		{
  1123			.procname	= "max_lock_depth",
  1124			.data		= &max_lock_depth,
  1125			.maxlen		= sizeof(int),
  1126			.mode		= 0644,
  1127			.proc_handler	= proc_dointvec,
  1128		},
  1129	#endif
  1130		{
  1131			.procname	= "poweroff_cmd",
  1132			.data		= &poweroff_cmd,
  1133			.maxlen		= POWEROFF_CMD_PATH_LEN,
  1134			.mode		= 0644,
  1135			.proc_handler	= proc_dostring,
  1136		},
  1137	#ifdef CONFIG_KEYS
  1138		{
  1139			.procname	= "keys",
  1140			.mode		= 0555,
  1141			.child		= key_sysctls,
  1142		},
  1143	#endif
  1144	#ifdef CONFIG_PERF_EVENTS
  1145		/*
  1146		 * User-space scripts rely on the existence of this file
  1147		 * as a feature check for perf_events being enabled.
  1148		 *
  1149		 * So it's an ABI, do not remove!
  1150		 */
  1151		{
  1152			.procname	= "perf_event_paranoid",
  1153			.data		= &sysctl_perf_event_paranoid,
  1154			.maxlen		= sizeof(sysctl_perf_event_paranoid),
  1155			.mode		= 0644,
  1156			.proc_handler	= proc_dointvec,
  1157		},
  1158		{
  1159			.procname	= "perf_event_mlock_kb",
  1160			.data		= &sysctl_perf_event_mlock,
  1161			.maxlen		= sizeof(sysctl_perf_event_mlock),
  1162			.mode		= 0644,
  1163			.proc_handler	= proc_dointvec,
  1164		},
  1165		{
  1166			.procname	= "perf_event_max_sample_rate",
  1167			.data		= &sysctl_perf_event_sample_rate,
  1168			.maxlen		= sizeof(sysctl_perf_event_sample_rate),
  1169			.mode		= 0644,
  1170			.proc_handler	= perf_proc_update_handler,
  1171			.extra1		= &one,
  1172		},
  1173		{
  1174			.procname	= "perf_cpu_time_max_percent",
  1175			.data		= &sysctl_perf_cpu_time_max_percent,
  1176			.maxlen		= sizeof(sysctl_perf_cpu_time_max_percent),
  1177			.mode		= 0644,
  1178			.proc_handler	= perf_cpu_time_max_percent_handler,
  1179			.extra1		= &zero,
  1180			.extra2		= &one_hundred,
  1181		},
  1182		{
  1183			.procname	= "perf_event_max_stack",
  1184			.data		= &sysctl_perf_event_max_stack,
  1185			.maxlen		= sizeof(sysctl_perf_event_max_stack),
  1186			.mode		= 0644,
  1187			.proc_handler	= perf_event_max_stack_handler,
  1188			.extra1		= &zero,
  1189			.extra2		= &six_hundred_forty_kb,
  1190		},
  1191		{
  1192			.procname	= "perf_event_max_contexts_per_stack",
  1193			.data		= &sysctl_perf_event_max_contexts_per_stack,
  1194			.maxlen		= sizeof(sysctl_perf_event_max_contexts_per_stack),
  1195			.mode		= 0644,
  1196			.proc_handler	= perf_event_max_stack_handler,
  1197			.extra1		= &zero,
  1198			.extra2		= &one_thousand,
  1199		},
  1200	#endif
  1201		{
  1202			.procname	= "panic_on_warn",
  1203			.data		= &panic_on_warn,
  1204			.maxlen		= sizeof(int),
  1205			.mode		= 0644,
  1206			.proc_handler	= proc_dointvec_minmax,
  1207			.extra1		= &zero,
  1208			.extra2		= &one,
  1209		},
  1210	#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
  1211		{
  1212			.procname	= "timer_migration",
  1213			.data		= &sysctl_timer_migration,
  1214			.maxlen		= sizeof(unsigned int),
  1215			.mode		= 0644,
  1216			.proc_handler	= timer_migration_handler,
  1217			.extra1		= &zero,
  1218			.extra2		= &one,
  1219		},
  1220	#endif
  1221	#ifdef CONFIG_BPF_SYSCALL
  1222		{
  1223			.procname	= "unprivileged_bpf_disabled",
  1224			.data		= &sysctl_unprivileged_bpf_disabled,
  1225			.maxlen		= sizeof(sysctl_unprivileged_bpf_disabled),
  1226			.mode		= 0644,
  1227			/* only handle a transition from default "0" to "1" */
  1228			.proc_handler	= proc_dointvec_minmax,
  1229			.extra1		= &one,
  1230			.extra2		= &one,
  1231		},
  1232	#endif
  1233	#if defined(CONFIG_TREE_RCU) || defined(CONFIG_PREEMPT_RCU)
  1234		{
  1235			.procname	= "panic_on_rcu_stall",
  1236			.data		= &sysctl_panic_on_rcu_stall,
  1237			.maxlen		= sizeof(sysctl_panic_on_rcu_stall),
  1238			.mode		= 0644,
  1239			.proc_handler	= proc_dointvec_minmax,
  1240			.extra1		= &zero,
  1241			.extra2		= &one,
  1242		},
  1243	#endif
  1244	#ifdef CONFIG_STACKLEAK_RUNTIME_DISABLE
  1245		{
  1246			.procname	= "stack_erasing",
  1247			.data		= NULL,
  1248			.maxlen		= sizeof(int),
  1249			.mode		= 0600,
  1250			.proc_handler	= stack_erasing_sysctl,
  1251			.extra1		= &zero,
  1252			.extra2		= &one,
  1253		},
  1254	#endif
  1255		{
  1256			.procname       = "cgroup_default_retry",
> 1257			.data           = &sysctl_cgroup_default_retry,
  1258			.maxlen         = sizeof(unsigned int),
  1259			.mode           = 0644,
  1260			.proc_handler   = proc_dointvec_minmax,
> 1261			.extra1         = &sysctl_cgroup_default_retry_min,
> 1262			.extra2         = &sysctl_cgroup_default_retry_max,
  1263		},
  1264		{ }
  1265	};
  1266	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 29505 bytes --]

Re: [PATCH 1/5] Memcgroup: force empty after memcgroup offline

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 10489 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: i386-randconfig-s2-01210338 (attached as .config)
compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

>> kernel/sysctl.c:1257:22: error: 'sysctl_cgroup_default_retry' undeclared here (not in a function)
      .data           = &sysctl_cgroup_default_retry,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
>> kernel/sysctl.c:1261:22: error: 'sysctl_cgroup_default_retry_min' undeclared here (not in a function)
      .extra1         = &sysctl_cgroup_default_retry_min,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> kernel/sysctl.c:1262:22: error: 'sysctl_cgroup_default_retry_max' undeclared here (not in a function)
      .extra2         = &sysctl_cgroup_default_retry_max,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/sysctl_cgroup_default_retry +1257 kernel/sysctl.c

   977	
   978	#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86)
   979		{
   980			.procname       = "unknown_nmi_panic",
   981			.data           = &unknown_nmi_panic,
   982			.maxlen         = sizeof (int),
   983			.mode           = 0644,
   984			.proc_handler   = proc_dointvec,
   985		},
   986	#endif
   987	#if defined(CONFIG_X86)
   988		{
   989			.procname	= "panic_on_unrecovered_nmi",
   990			.data		= &panic_on_unrecovered_nmi,
   991			.maxlen		= sizeof(int),
   992			.mode		= 0644,
   993			.proc_handler	= proc_dointvec,
   994		},
   995		{
   996			.procname	= "panic_on_io_nmi",
   997			.data		= &panic_on_io_nmi,
   998			.maxlen		= sizeof(int),
   999			.mode		= 0644,
  1000			.proc_handler	= proc_dointvec,
  1001		},
  1002	#ifdef CONFIG_DEBUG_STACKOVERFLOW
  1003		{
  1004			.procname	= "panic_on_stackoverflow",
  1005			.data		= &sysctl_panic_on_stackoverflow,
  1006			.maxlen		= sizeof(int),
  1007			.mode		= 0644,
  1008			.proc_handler	= proc_dointvec,
  1009		},
  1010	#endif
  1011		{
  1012			.procname	= "bootloader_type",
  1013			.data		= &bootloader_type,
  1014			.maxlen		= sizeof (int),
  1015			.mode		= 0444,
  1016			.proc_handler	= proc_dointvec,
  1017		},
  1018		{
  1019			.procname	= "bootloader_version",
  1020			.data		= &bootloader_version,
  1021			.maxlen		= sizeof (int),
  1022			.mode		= 0444,
  1023			.proc_handler	= proc_dointvec,
  1024		},
  1025		{
  1026			.procname	= "io_delay_type",
  1027			.data		= &io_delay_type,
  1028			.maxlen		= sizeof(int),
  1029			.mode		= 0644,
  1030			.proc_handler	= proc_dointvec,
  1031		},
  1032	#endif
  1033	#if defined(CONFIG_MMU)
  1034		{
  1035			.procname	= "randomize_va_space",
  1036			.data		= &randomize_va_space,
  1037			.maxlen		= sizeof(int),
  1038			.mode		= 0644,
  1039			.proc_handler	= proc_dointvec,
  1040		},
  1041	#endif
  1042	#if defined(CONFIG_S390) && defined(CONFIG_SMP)
  1043		{
  1044			.procname	= "spin_retry",
  1045			.data		= &spin_retry,
  1046			.maxlen		= sizeof (int),
  1047			.mode		= 0644,
  1048			.proc_handler	= proc_dointvec,
  1049		},
  1050	#endif
  1051	#if	defined(CONFIG_ACPI_SLEEP) && defined(CONFIG_X86)
  1052		{
  1053			.procname	= "acpi_video_flags",
  1054			.data		= &acpi_realmode_flags,
  1055			.maxlen		= sizeof (unsigned long),
  1056			.mode		= 0644,
  1057			.proc_handler	= proc_doulongvec_minmax,
  1058		},
  1059	#endif
  1060	#ifdef CONFIG_SYSCTL_ARCH_UNALIGN_NO_WARN
  1061		{
  1062			.procname	= "ignore-unaligned-usertrap",
  1063			.data		= &no_unaligned_warning,
  1064			.maxlen		= sizeof (int),
  1065		 	.mode		= 0644,
  1066			.proc_handler	= proc_dointvec,
  1067		},
  1068	#endif
  1069	#ifdef CONFIG_IA64
  1070		{
  1071			.procname	= "unaligned-dump-stack",
  1072			.data		= &unaligned_dump_stack,
  1073			.maxlen		= sizeof (int),
  1074			.mode		= 0644,
  1075			.proc_handler	= proc_dointvec,
  1076		},
  1077	#endif
  1078	#ifdef CONFIG_DETECT_HUNG_TASK
  1079		{
  1080			.procname	= "hung_task_panic",
  1081			.data		= &sysctl_hung_task_panic,
  1082			.maxlen		= sizeof(int),
  1083			.mode		= 0644,
  1084			.proc_handler	= proc_dointvec_minmax,
  1085			.extra1		= &zero,
  1086			.extra2		= &one,
  1087		},
  1088		{
  1089			.procname	= "hung_task_check_count",
  1090			.data		= &sysctl_hung_task_check_count,
  1091			.maxlen		= sizeof(int),
  1092			.mode		= 0644,
  1093			.proc_handler	= proc_dointvec_minmax,
  1094			.extra1		= &zero,
  1095		},
  1096		{
  1097			.procname	= "hung_task_timeout_secs",
  1098			.data		= &sysctl_hung_task_timeout_secs,
  1099			.maxlen		= sizeof(unsigned long),
  1100			.mode		= 0644,
  1101			.proc_handler	= proc_dohung_task_timeout_secs,
  1102			.extra2		= &hung_task_timeout_max,
  1103		},
  1104		{
  1105			.procname	= "hung_task_check_interval_secs",
  1106			.data		= &sysctl_hung_task_check_interval_secs,
  1107			.maxlen		= sizeof(unsigned long),
  1108			.mode		= 0644,
  1109			.proc_handler	= proc_dohung_task_timeout_secs,
  1110			.extra2		= &hung_task_timeout_max,
  1111		},
  1112		{
  1113			.procname	= "hung_task_warnings",
  1114			.data		= &sysctl_hung_task_warnings,
  1115			.maxlen		= sizeof(int),
  1116			.mode		= 0644,
  1117			.proc_handler	= proc_dointvec_minmax,
  1118			.extra1		= &neg_one,
  1119		},
  1120	#endif
  1121	#ifdef CONFIG_RT_MUTEXES
  1122		{
  1123			.procname	= "max_lock_depth",
  1124			.data		= &max_lock_depth,
  1125			.maxlen		= sizeof(int),
  1126			.mode		= 0644,
  1127			.proc_handler	= proc_dointvec,
  1128		},
  1129	#endif
  1130		{
  1131			.procname	= "poweroff_cmd",
  1132			.data		= &poweroff_cmd,
  1133			.maxlen		= POWEROFF_CMD_PATH_LEN,
  1134			.mode		= 0644,
  1135			.proc_handler	= proc_dostring,
  1136		},
  1137	#ifdef CONFIG_KEYS
  1138		{
  1139			.procname	= "keys",
  1140			.mode		= 0555,
  1141			.child		= key_sysctls,
  1142		},
  1143	#endif
  1144	#ifdef CONFIG_PERF_EVENTS
  1145		/*
  1146		 * User-space scripts rely on the existence of this file
  1147		 * as a feature check for perf_events being enabled.
  1148		 *
  1149		 * So it's an ABI, do not remove!
  1150		 */
  1151		{
  1152			.procname	= "perf_event_paranoid",
  1153			.data		= &sysctl_perf_event_paranoid,
  1154			.maxlen		= sizeof(sysctl_perf_event_paranoid),
  1155			.mode		= 0644,
  1156			.proc_handler	= proc_dointvec,
  1157		},
  1158		{
  1159			.procname	= "perf_event_mlock_kb",
  1160			.data		= &sysctl_perf_event_mlock,
  1161			.maxlen		= sizeof(sysctl_perf_event_mlock),
  1162			.mode		= 0644,
  1163			.proc_handler	= proc_dointvec,
  1164		},
  1165		{
  1166			.procname	= "perf_event_max_sample_rate",
  1167			.data		= &sysctl_perf_event_sample_rate,
  1168			.maxlen		= sizeof(sysctl_perf_event_sample_rate),
  1169			.mode		= 0644,
  1170			.proc_handler	= perf_proc_update_handler,
  1171			.extra1		= &one,
  1172		},
  1173		{
  1174			.procname	= "perf_cpu_time_max_percent",
  1175			.data		= &sysctl_perf_cpu_time_max_percent,
  1176			.maxlen		= sizeof(sysctl_perf_cpu_time_max_percent),
  1177			.mode		= 0644,
  1178			.proc_handler	= perf_cpu_time_max_percent_handler,
  1179			.extra1		= &zero,
  1180			.extra2		= &one_hundred,
  1181		},
  1182		{
  1183			.procname	= "perf_event_max_stack",
  1184			.data		= &sysctl_perf_event_max_stack,
  1185			.maxlen		= sizeof(sysctl_perf_event_max_stack),
  1186			.mode		= 0644,
  1187			.proc_handler	= perf_event_max_stack_handler,
  1188			.extra1		= &zero,
  1189			.extra2		= &six_hundred_forty_kb,
  1190		},
  1191		{
  1192			.procname	= "perf_event_max_contexts_per_stack",
  1193			.data		= &sysctl_perf_event_max_contexts_per_stack,
  1194			.maxlen		= sizeof(sysctl_perf_event_max_contexts_per_stack),
  1195			.mode		= 0644,
  1196			.proc_handler	= perf_event_max_stack_handler,
  1197			.extra1		= &zero,
  1198			.extra2		= &one_thousand,
  1199		},
  1200	#endif
  1201		{
  1202			.procname	= "panic_on_warn",
  1203			.data		= &panic_on_warn,
  1204			.maxlen		= sizeof(int),
  1205			.mode		= 0644,
  1206			.proc_handler	= proc_dointvec_minmax,
  1207			.extra1		= &zero,
  1208			.extra2		= &one,
  1209		},
  1210	#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
  1211		{
  1212			.procname	= "timer_migration",
  1213			.data		= &sysctl_timer_migration,
  1214			.maxlen		= sizeof(unsigned int),
  1215			.mode		= 0644,
  1216			.proc_handler	= timer_migration_handler,
  1217			.extra1		= &zero,
  1218			.extra2		= &one,
  1219		},
  1220	#endif
  1221	#ifdef CONFIG_BPF_SYSCALL
  1222		{
  1223			.procname	= "unprivileged_bpf_disabled",
  1224			.data		= &sysctl_unprivileged_bpf_disabled,
  1225			.maxlen		= sizeof(sysctl_unprivileged_bpf_disabled),
  1226			.mode		= 0644,
  1227			/* only handle a transition from default "0" to "1" */
  1228			.proc_handler	= proc_dointvec_minmax,
  1229			.extra1		= &one,
  1230			.extra2		= &one,
  1231		},
  1232	#endif
  1233	#if defined(CONFIG_TREE_RCU) || defined(CONFIG_PREEMPT_RCU)
  1234		{
  1235			.procname	= "panic_on_rcu_stall",
  1236			.data		= &sysctl_panic_on_rcu_stall,
  1237			.maxlen		= sizeof(sysctl_panic_on_rcu_stall),
  1238			.mode		= 0644,
  1239			.proc_handler	= proc_dointvec_minmax,
  1240			.extra1		= &zero,
  1241			.extra2		= &one,
  1242		},
  1243	#endif
  1244	#ifdef CONFIG_STACKLEAK_RUNTIME_DISABLE
  1245		{
  1246			.procname	= "stack_erasing",
  1247			.data		= NULL,
  1248			.maxlen		= sizeof(int),
  1249			.mode		= 0600,
  1250			.proc_handler	= stack_erasing_sysctl,
  1251			.extra1		= &zero,
  1252			.extra2		= &one,
  1253		},
  1254	#endif
  1255		{
  1256			.procname       = "cgroup_default_retry",
> 1257			.data           = &sysctl_cgroup_default_retry,
  1258			.maxlen         = sizeof(unsigned int),
  1259			.mode           = 0644,
  1260			.proc_handler   = proc_dointvec_minmax,
> 1261			.extra1         = &sysctl_cgroup_default_retry_min,
> 1262			.extra2         = &sysctl_cgroup_default_retry_max,
  1263		},
  1264		{ }
  1265	};
  1266	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 29617 bytes --]

Re: [PATCH 1/5] Memcgroup: force empty after memcgroup offline

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 10489 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: i386-randconfig-s2-01210338 (attached as .config)
compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

>> kernel/sysctl.c:1257:22: error: 'sysctl_cgroup_default_retry' undeclared here (not in a function)
      .data           = &sysctl_cgroup_default_retry,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
>> kernel/sysctl.c:1261:22: error: 'sysctl_cgroup_default_retry_min' undeclared here (not in a function)
      .extra1         = &sysctl_cgroup_default_retry_min,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> kernel/sysctl.c:1262:22: error: 'sysctl_cgroup_default_retry_max' undeclared here (not in a function)
      .extra2         = &sysctl_cgroup_default_retry_max,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/sysctl_cgroup_default_retry +1257 kernel/sysctl.c

   977	
   978	#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86)
   979		{
   980			.procname       = "unknown_nmi_panic",
   981			.data           = &unknown_nmi_panic,
   982			.maxlen         = sizeof (int),
   983			.mode           = 0644,
   984			.proc_handler   = proc_dointvec,
   985		},
   986	#endif
   987	#if defined(CONFIG_X86)
   988		{
   989			.procname	= "panic_on_unrecovered_nmi",
   990			.data		= &panic_on_unrecovered_nmi,
   991			.maxlen		= sizeof(int),
   992			.mode		= 0644,
   993			.proc_handler	= proc_dointvec,
   994		},
   995		{
   996			.procname	= "panic_on_io_nmi",
   997			.data		= &panic_on_io_nmi,
   998			.maxlen		= sizeof(int),
   999			.mode		= 0644,
  1000			.proc_handler	= proc_dointvec,
  1001		},
  1002	#ifdef CONFIG_DEBUG_STACKOVERFLOW
  1003		{
  1004			.procname	= "panic_on_stackoverflow",
  1005			.data		= &sysctl_panic_on_stackoverflow,
  1006			.maxlen		= sizeof(int),
  1007			.mode		= 0644,
  1008			.proc_handler	= proc_dointvec,
  1009		},
  1010	#endif
  1011		{
  1012			.procname	= "bootloader_type",
  1013			.data		= &bootloader_type,
  1014			.maxlen		= sizeof (int),
  1015			.mode		= 0444,
  1016			.proc_handler	= proc_dointvec,
  1017		},
  1018		{
  1019			.procname	= "bootloader_version",
  1020			.data		= &bootloader_version,
  1021			.maxlen		= sizeof (int),
  1022			.mode		= 0444,
  1023			.proc_handler	= proc_dointvec,
  1024		},
  1025		{
  1026			.procname	= "io_delay_type",
  1027			.data		= &io_delay_type,
  1028			.maxlen		= sizeof(int),
  1029			.mode		= 0644,
  1030			.proc_handler	= proc_dointvec,
  1031		},
  1032	#endif
  1033	#if defined(CONFIG_MMU)
  1034		{
  1035			.procname	= "randomize_va_space",
  1036			.data		= &randomize_va_space,
  1037			.maxlen		= sizeof(int),
  1038			.mode		= 0644,
  1039			.proc_handler	= proc_dointvec,
  1040		},
  1041	#endif
  1042	#if defined(CONFIG_S390) && defined(CONFIG_SMP)
  1043		{
  1044			.procname	= "spin_retry",
  1045			.data		= &spin_retry,
  1046			.maxlen		= sizeof (int),
  1047			.mode		= 0644,
  1048			.proc_handler	= proc_dointvec,
  1049		},
  1050	#endif
  1051	#if	defined(CONFIG_ACPI_SLEEP) && defined(CONFIG_X86)
  1052		{
  1053			.procname	= "acpi_video_flags",
  1054			.data		= &acpi_realmode_flags,
  1055			.maxlen		= sizeof (unsigned long),
  1056			.mode		= 0644,
  1057			.proc_handler	= proc_doulongvec_minmax,
  1058		},
  1059	#endif
  1060	#ifdef CONFIG_SYSCTL_ARCH_UNALIGN_NO_WARN
  1061		{
  1062			.procname	= "ignore-unaligned-usertrap",
  1063			.data		= &no_unaligned_warning,
  1064			.maxlen		= sizeof (int),
  1065		 	.mode		= 0644,
  1066			.proc_handler	= proc_dointvec,
  1067		},
  1068	#endif
  1069	#ifdef CONFIG_IA64
  1070		{
  1071			.procname	= "unaligned-dump-stack",
  1072			.data		= &unaligned_dump_stack,
  1073			.maxlen		= sizeof (int),
  1074			.mode		= 0644,
  1075			.proc_handler	= proc_dointvec,
  1076		},
  1077	#endif
  1078	#ifdef CONFIG_DETECT_HUNG_TASK
  1079		{
  1080			.procname	= "hung_task_panic",
  1081			.data		= &sysctl_hung_task_panic,
  1082			.maxlen		= sizeof(int),
  1083			.mode		= 0644,
  1084			.proc_handler	= proc_dointvec_minmax,
  1085			.extra1		= &zero,
  1086			.extra2		= &one,
  1087		},
  1088		{
  1089			.procname	= "hung_task_check_count",
  1090			.data		= &sysctl_hung_task_check_count,
  1091			.maxlen		= sizeof(int),
  1092			.mode		= 0644,
  1093			.proc_handler	= proc_dointvec_minmax,
  1094			.extra1		= &zero,
  1095		},
  1096		{
  1097			.procname	= "hung_task_timeout_secs",
  1098			.data		= &sysctl_hung_task_timeout_secs,
  1099			.maxlen		= sizeof(unsigned long),
  1100			.mode		= 0644,
  1101			.proc_handler	= proc_dohung_task_timeout_secs,
  1102			.extra2		= &hung_task_timeout_max,
  1103		},
  1104		{
  1105			.procname	= "hung_task_check_interval_secs",
  1106			.data		= &sysctl_hung_task_check_interval_secs,
  1107			.maxlen		= sizeof(unsigned long),
  1108			.mode		= 0644,
  1109			.proc_handler	= proc_dohung_task_timeout_secs,
  1110			.extra2		= &hung_task_timeout_max,
  1111		},
  1112		{
  1113			.procname	= "hung_task_warnings",
  1114			.data		= &sysctl_hung_task_warnings,
  1115			.maxlen		= sizeof(int),
  1116			.mode		= 0644,
  1117			.proc_handler	= proc_dointvec_minmax,
  1118			.extra1		= &neg_one,
  1119		},
  1120	#endif
  1121	#ifdef CONFIG_RT_MUTEXES
  1122		{
  1123			.procname	= "max_lock_depth",
  1124			.data		= &max_lock_depth,
  1125			.maxlen		= sizeof(int),
  1126			.mode		= 0644,
  1127			.proc_handler	= proc_dointvec,
  1128		},
  1129	#endif
  1130		{
  1131			.procname	= "poweroff_cmd",
  1132			.data		= &poweroff_cmd,
  1133			.maxlen		= POWEROFF_CMD_PATH_LEN,
  1134			.mode		= 0644,
  1135			.proc_handler	= proc_dostring,
  1136		},
  1137	#ifdef CONFIG_KEYS
  1138		{
  1139			.procname	= "keys",
  1140			.mode		= 0555,
  1141			.child		= key_sysctls,
  1142		},
  1143	#endif
  1144	#ifdef CONFIG_PERF_EVENTS
  1145		/*
  1146		 * User-space scripts rely on the existence of this file
  1147		 * as a feature check for perf_events being enabled.
  1148		 *
  1149		 * So it's an ABI, do not remove!
  1150		 */
  1151		{
  1152			.procname	= "perf_event_paranoid",
  1153			.data		= &sysctl_perf_event_paranoid,
  1154			.maxlen		= sizeof(sysctl_perf_event_paranoid),
  1155			.mode		= 0644,
  1156			.proc_handler	= proc_dointvec,
  1157		},
  1158		{
  1159			.procname	= "perf_event_mlock_kb",
  1160			.data		= &sysctl_perf_event_mlock,
  1161			.maxlen		= sizeof(sysctl_perf_event_mlock),
  1162			.mode		= 0644,
  1163			.proc_handler	= proc_dointvec,
  1164		},
  1165		{
  1166			.procname	= "perf_event_max_sample_rate",
  1167			.data		= &sysctl_perf_event_sample_rate,
  1168			.maxlen		= sizeof(sysctl_perf_event_sample_rate),
  1169			.mode		= 0644,
  1170			.proc_handler	= perf_proc_update_handler,
  1171			.extra1		= &one,
  1172		},
  1173		{
  1174			.procname	= "perf_cpu_time_max_percent",
  1175			.data		= &sysctl_perf_cpu_time_max_percent,
  1176			.maxlen		= sizeof(sysctl_perf_cpu_time_max_percent),
  1177			.mode		= 0644,
  1178			.proc_handler	= perf_cpu_time_max_percent_handler,
  1179			.extra1		= &zero,
  1180			.extra2		= &one_hundred,
  1181		},
  1182		{
  1183			.procname	= "perf_event_max_stack",
  1184			.data		= &sysctl_perf_event_max_stack,
  1185			.maxlen		= sizeof(sysctl_perf_event_max_stack),
  1186			.mode		= 0644,
  1187			.proc_handler	= perf_event_max_stack_handler,
  1188			.extra1		= &zero,
  1189			.extra2		= &six_hundred_forty_kb,
  1190		},
  1191		{
  1192			.procname	= "perf_event_max_contexts_per_stack",
  1193			.data		= &sysctl_perf_event_max_contexts_per_stack,
  1194			.maxlen		= sizeof(sysctl_perf_event_max_contexts_per_stack),
  1195			.mode		= 0644,
  1196			.proc_handler	= perf_event_max_stack_handler,
  1197			.extra1		= &zero,
  1198			.extra2		= &one_thousand,
  1199		},
  1200	#endif
  1201		{
  1202			.procname	= "panic_on_warn",
  1203			.data		= &panic_on_warn,
  1204			.maxlen		= sizeof(int),
  1205			.mode		= 0644,
  1206			.proc_handler	= proc_dointvec_minmax,
  1207			.extra1		= &zero,
  1208			.extra2		= &one,
  1209		},
  1210	#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
  1211		{
  1212			.procname	= "timer_migration",
  1213			.data		= &sysctl_timer_migration,
  1214			.maxlen		= sizeof(unsigned int),
  1215			.mode		= 0644,
  1216			.proc_handler	= timer_migration_handler,
  1217			.extra1		= &zero,
  1218			.extra2		= &one,
  1219		},
  1220	#endif
  1221	#ifdef CONFIG_BPF_SYSCALL
  1222		{
  1223			.procname	= "unprivileged_bpf_disabled",
  1224			.data		= &sysctl_unprivileged_bpf_disabled,
  1225			.maxlen		= sizeof(sysctl_unprivileged_bpf_disabled),
  1226			.mode		= 0644,
  1227			/* only handle a transition from default "0" to "1" */
  1228			.proc_handler	= proc_dointvec_minmax,
  1229			.extra1		= &one,
  1230			.extra2		= &one,
  1231		},
  1232	#endif
  1233	#if defined(CONFIG_TREE_RCU) || defined(CONFIG_PREEMPT_RCU)
  1234		{
  1235			.procname	= "panic_on_rcu_stall",
  1236			.data		= &sysctl_panic_on_rcu_stall,
  1237			.maxlen		= sizeof(sysctl_panic_on_rcu_stall),
  1238			.mode		= 0644,
  1239			.proc_handler	= proc_dointvec_minmax,
  1240			.extra1		= &zero,
  1241			.extra2		= &one,
  1242		},
  1243	#endif
  1244	#ifdef CONFIG_STACKLEAK_RUNTIME_DISABLE
  1245		{
  1246			.procname	= "stack_erasing",
  1247			.data		= NULL,
  1248			.maxlen		= sizeof(int),
  1249			.mode		= 0600,
  1250			.proc_handler	= stack_erasing_sysctl,
  1251			.extra1		= &zero,
  1252			.extra2		= &one,
  1253		},
  1254	#endif
  1255		{
  1256			.procname       = "cgroup_default_retry",
> 1257			.data           = &sysctl_cgroup_default_retry,
  1258			.maxlen         = sizeof(unsigned int),
  1259			.mode           = 0644,
  1260			.proc_handler   = proc_dointvec_minmax,
> 1261			.extra1         = &sysctl_cgroup_default_retry_min,
> 1262			.extra2         = &sysctl_cgroup_default_retry_max,
  1263		},
  1264		{ }
  1265	};
  1266	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 29617 bytes --]

[PATCH 2/5] Memcgroup: Add timer to trigger workqueue

[Xiongchun Duan]

Add timer to trigger workqueue which will scan offline memcgroup and call trigger
memcgroup force_empty worker to force_empty itself.

Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
---
 include/linux/memcontrol.h |  1 +
 mm/memcontrol.c            | 23 +++++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index d6fbb77..0a29f7f 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -313,6 +313,7 @@ struct mem_cgroup {
 
 	int max_retry;
 	int current_retry;
+	unsigned long timer_jiffies;
 
 	struct mem_cgroup_per_node *nodeinfo[0];
 	/* WARNING: nodeinfo must be the last member here */
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 2b13c2b..4db08b7 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -81,6 +81,8 @@
 int sysctl_cgroup_default_retry_min;
 int sysctl_cgroup_default_retry_max = 16;
 
+struct timer_list empty_trigger;
+
 struct mem_cgroup *root_mem_cgroup __read_mostly;
 
 #define MEM_CGROUP_RECLAIM_RETRIES	5
@@ -2933,6 +2935,11 @@ static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of,
 	return mem_cgroup_force_empty(memcg) ?: nbytes;
 }
 
+static void add_force_empty_list(struct mem_cgroup *memcg)
+{
+
+}
+
 static u64 mem_cgroup_hierarchy_read(struct cgroup_subsys_state *css,
 				     struct cftype *cft)
 {
@@ -4566,11 +4573,26 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
 	return 0;
 }
 
+void empty_timer_trigger(struct timer_list *t)
+{
+
+}
+
 static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 {
 	struct mem_cgroup *memcg = mem_cgroup_from_css(css);
 	struct mem_cgroup_event *event, *tmp;
 
+	if (memcg->max_retry != 0) {
+		memcg->current_retry = 1;
+		mem_cgroup_force_empty(memcg);
+		if (page_counter_read(&memcg->memory) &&
+				memcg->max_retry != 1) {
+			memcg->timer_jiffies = jiffies + HZ;
+			add_force_empty_list(memcg);
+		}
+	}
+
 	/*
 	 * Unregister events and notify userspace.
 	 * Notify userspace about cgroup removing only after rmdir of cgroup
@@ -6368,6 +6390,7 @@ static int __init mem_cgroup_init(void)
 	memcg_kmem_cache_wq = alloc_workqueue("memcg_kmem_cache", 0, 1);
 	BUG_ON(!memcg_kmem_cache_wq);
 #endif
+	timer_setup(&empty_trigger, empty_timer_trigger, 0);
 
 	cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
 				  memcg_hotplug_cpu_dead);
-- 
1.8.3.1

Re: [PATCH 2/5] Memcgroup: Add timer to trigger workqueue

[Michal Hocko]

On Sat 19-01-19 22:30:18, Xiongchun Duan wrote:
> Add timer to trigger workqueue which will scan offline memcgroup and call trigger
> memcgroup force_empty worker to force_empty itself.

This requires much more explanation but it looks like a complete hack to
me. Why do we need a timer at all? Why is the hardcoded timeout a
generic enough?

> Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
> ---
>  include/linux/memcontrol.h |  1 +
>  mm/memcontrol.c            | 23 +++++++++++++++++++++++
>  2 files changed, 24 insertions(+)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index d6fbb77..0a29f7f 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -313,6 +313,7 @@ struct mem_cgroup {
>  
>  	int max_retry;
>  	int current_retry;
> +	unsigned long timer_jiffies;
>  
>  	struct mem_cgroup_per_node *nodeinfo[0];
>  	/* WARNING: nodeinfo must be the last member here */
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 2b13c2b..4db08b7 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -81,6 +81,8 @@
>  int sysctl_cgroup_default_retry_min;
>  int sysctl_cgroup_default_retry_max = 16;
>  
> +struct timer_list empty_trigger;
> +
>  struct mem_cgroup *root_mem_cgroup __read_mostly;
>  
>  #define MEM_CGROUP_RECLAIM_RETRIES	5
> @@ -2933,6 +2935,11 @@ static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of,
>  	return mem_cgroup_force_empty(memcg) ?: nbytes;
>  }
>  
> +static void add_force_empty_list(struct mem_cgroup *memcg)
> +{
> +
> +}
> +
>  static u64 mem_cgroup_hierarchy_read(struct cgroup_subsys_state *css,
>  				     struct cftype *cft)
>  {
> @@ -4566,11 +4573,26 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
>  	return 0;
>  }
>  
> +void empty_timer_trigger(struct timer_list *t)
> +{
> +
> +}
> +
>  static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
>  {
>  	struct mem_cgroup *memcg = mem_cgroup_from_css(css);
>  	struct mem_cgroup_event *event, *tmp;
>  
> +	if (memcg->max_retry != 0) {
> +		memcg->current_retry = 1;
> +		mem_cgroup_force_empty(memcg);
> +		if (page_counter_read(&memcg->memory) &&
> +				memcg->max_retry != 1) {
> +			memcg->timer_jiffies = jiffies + HZ;
> +			add_force_empty_list(memcg);
> +		}
> +	}
> +
>  	/*
>  	 * Unregister events and notify userspace.
>  	 * Notify userspace about cgroup removing only after rmdir of cgroup
> @@ -6368,6 +6390,7 @@ static int __init mem_cgroup_init(void)
>  	memcg_kmem_cache_wq = alloc_workqueue("memcg_kmem_cache", 0, 1);
>  	BUG_ON(!memcg_kmem_cache_wq);
>  #endif
> +	timer_setup(&empty_trigger, empty_timer_trigger, 0);
>  
>  	cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
>  				  memcg_hotplug_cpu_dead);
> -- 
> 1.8.3.1

-- 
Michal Hocko
SUSE Labs

[PATCH 3/5] Memcgroup:add a global work

[Xiongchun Duan]

Add a global work to scan offline cgroup and trigger
offline cgroup force empty.

Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
---
 mm/memcontrol.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 4db08b7..fad1aae 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -82,6 +82,7 @@
 int sysctl_cgroup_default_retry_max = 16;
 
 struct timer_list empty_trigger;
+struct work_struct timer_poll_work;
 
 struct mem_cgroup *root_mem_cgroup __read_mostly;
 
@@ -320,6 +321,7 @@ void memcg_put_cache_ids(void)
 EXPORT_SYMBOL(memcg_kmem_enabled_key);
 
 struct workqueue_struct *memcg_kmem_cache_wq;
+struct workqueue_struct *memcg_force_empty_wq;
 
 static int memcg_shrinker_map_size;
 static DEFINE_MUTEX(memcg_shrinker_map_mutex);
@@ -4573,11 +4575,16 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
 	return 0;
 }
 
-void empty_timer_trigger(struct timer_list *t)
+static void trigger_force_empty(struct work_struct *work)
 {
 
 }
 
+static void empty_timer_trigger(struct timer_list *t)
+{
+	queue_work(memcg_force_empty_wq, &timer_poll_work);
+}
+
 static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 {
 	struct mem_cgroup *memcg = mem_cgroup_from_css(css);
@@ -6390,6 +6397,9 @@ static int __init mem_cgroup_init(void)
 	memcg_kmem_cache_wq = alloc_workqueue("memcg_kmem_cache", 0, 1);
 	BUG_ON(!memcg_kmem_cache_wq);
 #endif
+	memcg_force_empty_wq = alloc_workqueue("memcg_force_empty_wq", 0, 1);
+	BUG_ON(!memcg_force_empty_wq);
+	INIT_WORK(&timer_poll_work, trigger_force_empty);
 	timer_setup(&empty_trigger, empty_timer_trigger, 0);
 
 	cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
-- 
1.8.3.1

Re: [PATCH 3/5] Memcgroup:add a global work

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: i386-randconfig-x010-201903 (attached as .config)
compiler: gcc-8 (Debian 8.2.0-14) 8.2.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

   mm/memcontrol.c: In function 'empty_timer_trigger':
>> mm/memcontrol.c:4585:13: error: 'memcg_force_empty_wq' undeclared (first use in this function); did you mean 'mem_cgroup_force_empty'?
     queue_work(memcg_force_empty_wq, &timer_poll_work);
                ^~~~~~~~~~~~~~~~~~~~
                mem_cgroup_force_empty
   mm/memcontrol.c:4585:13: note: each undeclared identifier is reported only once for each function it appears in
   mm/memcontrol.c: In function 'mem_cgroup_init':
   mm/memcontrol.c:6400:2: error: 'memcg_force_empty_wq' undeclared (first use in this function); did you mean 'mem_cgroup_force_empty'?
     memcg_force_empty_wq = alloc_workqueue("memcg_force_empty_wq", 0, 1);
     ^~~~~~~~~~~~~~~~~~~~
     mem_cgroup_force_empty

vim +4585 mm/memcontrol.c

  4582	
  4583	static void empty_timer_trigger(struct timer_list *t)
  4584	{
> 4585		queue_work(memcg_force_empty_wq, &timer_poll_work);
  4586	}
  4587	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26860 bytes --]

Re: [PATCH 3/5] Memcgroup:add a global work

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: i386-randconfig-x010-201903 (attached as .config)
compiler: gcc-8 (Debian 8.2.0-14) 8.2.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

   mm/memcontrol.c: In function 'empty_timer_trigger':
>> mm/memcontrol.c:4585:13: error: 'memcg_force_empty_wq' undeclared (first use in this function); did you mean 'mem_cgroup_force_empty'?
     queue_work(memcg_force_empty_wq, &timer_poll_work);
                ^~~~~~~~~~~~~~~~~~~~
                mem_cgroup_force_empty
   mm/memcontrol.c:4585:13: note: each undeclared identifier is reported only once for each function it appears in
   mm/memcontrol.c: In function 'mem_cgroup_init':
   mm/memcontrol.c:6400:2: error: 'memcg_force_empty_wq' undeclared (first use in this function); did you mean 'mem_cgroup_force_empty'?
     memcg_force_empty_wq = alloc_workqueue("memcg_force_empty_wq", 0, 1);
     ^~~~~~~~~~~~~~~~~~~~
     mem_cgroup_force_empty

vim +4585 mm/memcontrol.c

  4582	
  4583	static void empty_timer_trigger(struct timer_list *t)
  4584	{
> 4585		queue_work(memcg_force_empty_wq, &timer_poll_work);
  4586	}
  4587	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26860 bytes --]

Re: [PATCH 3/5] Memcgroup:add a global work

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: x86_64-randconfig-r0-01212129 (attached as .config)
compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   mm/memcontrol.c: In function 'empty_timer_trigger':
>> mm/memcontrol.c:4585:13: error: 'memcg_force_empty_wq' undeclared (first use in this function)
     queue_work(memcg_force_empty_wq, &timer_poll_work);
                ^~~~~~~~~~~~~~~~~~~~
   mm/memcontrol.c:4585:13: note: each undeclared identifier is reported only once for each function it appears in
   mm/memcontrol.c: In function 'mem_cgroup_init':
   mm/memcontrol.c:6400:2: error: 'memcg_force_empty_wq' undeclared (first use in this function)
     memcg_force_empty_wq = alloc_workqueue("memcg_force_empty_wq", 0, 1);
     ^~~~~~~~~~~~~~~~~~~~

vim +/memcg_force_empty_wq +4585 mm/memcontrol.c

  4582	
  4583	static void empty_timer_trigger(struct timer_list *t)
  4584	{
> 4585		queue_work(memcg_force_empty_wq, &timer_poll_work);
  4586	}
  4587	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 38367 bytes --]

Re: [PATCH 3/5] Memcgroup:add a global work

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]

Hi Xiongchun,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0-rc2 next-20190116]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xiongchun-Duan/Memcgroup-force-empty-after-memcgroup-offline/20190122-014721
config: x86_64-randconfig-r0-01212129 (attached as .config)
compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   mm/memcontrol.c: In function 'empty_timer_trigger':
>> mm/memcontrol.c:4585:13: error: 'memcg_force_empty_wq' undeclared (first use in this function)
     queue_work(memcg_force_empty_wq, &timer_poll_work);
                ^~~~~~~~~~~~~~~~~~~~
   mm/memcontrol.c:4585:13: note: each undeclared identifier is reported only once for each function it appears in
   mm/memcontrol.c: In function 'mem_cgroup_init':
   mm/memcontrol.c:6400:2: error: 'memcg_force_empty_wq' undeclared (first use in this function)
     memcg_force_empty_wq = alloc_workqueue("memcg_force_empty_wq", 0, 1);
     ^~~~~~~~~~~~~~~~~~~~

vim +/memcg_force_empty_wq +4585 mm/memcontrol.c

  4582	
  4583	static void empty_timer_trigger(struct timer_list *t)
  4584	{
> 4585		queue_work(memcg_force_empty_wq, &timer_poll_work);
  4586	}
  4587	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 38367 bytes --]

[PATCH 4/5] Memcgroup:Implement force empty work function

[Xiongchun Duan]

Implement force empty work function and add trigger by global work.
force_empty_list : offline cgroup wait for trigger force empty.
empty_fail_list: offline cgroup which had been trigger for too
many time will not auto retrigger.

Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
---
 include/linux/memcontrol.h |  4 +++
 mm/memcontrol.c            | 81 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 0a29f7f..064192e 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -315,6 +315,10 @@ struct mem_cgroup {
 	int current_retry;
 	unsigned long timer_jiffies;
 
+	struct list_head force_empty_node;
+	struct list_head empty_fail_node;
+	struct work_struct force_empty_work;
+
 	struct mem_cgroup_per_node *nodeinfo[0];
 	/* WARNING: nodeinfo must be the last member here */
 };
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index fad1aae..21b4432 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -86,6 +86,10 @@
 
 struct mem_cgroup *root_mem_cgroup __read_mostly;
 
+static DEFINE_MUTEX(offline_cgroup_mutex);
+static LIST_HEAD(force_empty_list);
+static LIST_HEAD(empty_fail_list);
+
 #define MEM_CGROUP_RECLAIM_RETRIES	5
 
 /* Socket memory accounting disabled? */
@@ -2939,9 +2943,52 @@ static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of,
 
 static void add_force_empty_list(struct mem_cgroup *memcg)
 {
+	struct list_head *pos, *n;
+	struct mem_cgroup *pos_memcg;
+	unsigned long tmp = memcg->timer_jiffies;
+
+	mutex_lock(&offline_cgroup_mutex);
+	list_for_each_safe(pos, n, &force_empty_list) {
+		pos_memcg = container_of(pos,
+				struct mem_cgroup, force_empty_node);
+		if	(time_after(tmp, pos_memcg->timer_jiffies))
+			tmp = pos_memcg->timer_jiffies;
+		if (time_after(pos_memcg->timer_jiffies, memcg->timer_jiffies))
+			break;
+	}
+	list_add_tail(&memcg->force_empty_node, pos);
+	mutex_unlock(&offline_cgroup_mutex);
+	mod_timer(&empty_trigger, tmp);
 
 }
 
+static void mem_cgroup_force_empty_delay(struct work_struct *work)
+{
+	unsigned int order;
+	struct mem_cgroup *memcg = container_of(work,
+			struct mem_cgroup, force_empty_work);
+
+	if (page_counter_read(&memcg->memory)) {
+		mem_cgroup_force_empty(memcg);
+		memcg->current_retry += 1;
+		if (page_counter_read(&memcg->memory)) {
+			if (memcg->current_retry >= memcg->max_retry) {
+				if (list_empty(&memcg->empty_fail_node)) {
+					mutex_lock(&offline_cgroup_mutex);
+					list_add(&memcg->empty_fail_node,
+							&empty_fail_list);
+					mutex_unlock(&offline_cgroup_mutex);
+				}
+			} else {
+				order = 1 << (memcg->current_retry - 1);
+				memcg->timer_jiffies = jiffies + HZ * order;
+				add_force_empty_list(memcg);
+			}
+		}
+	}
+	css_put(&memcg->css);
+}
+
 static u64 mem_cgroup_hierarchy_read(struct cgroup_subsys_state *css,
 				     struct cftype *cft)
 {
@@ -4545,6 +4592,9 @@ static struct mem_cgroup *mem_cgroup_alloc(void)
 
 	if (cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_nosocket)
 		static_branch_inc(&memcg_sockets_enabled_key);
+
+	INIT_LIST_HEAD(&memcg->force_empty_node);
+	INIT_LIST_HEAD(&memcg->empty_fail_node);
 	memcg->max_retry = sysctl_cgroup_default_retry;
 	memcg->current_retry  = 0;
 
@@ -4577,7 +4627,26 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
 
 static void trigger_force_empty(struct work_struct *work)
 {
+	struct list_head *pos, *n;
+	struct mem_cgroup *memcg;
 
+	mutex_lock(&offline_cgroup_mutex);
+	list_for_each_safe(pos, n, &force_empty_list) {
+		memcg = container_of(pos, struct mem_cgroup,
+				force_empty_node);
+		if (time_after(jiffies, memcg->timer_jiffies)) {
+			if (atomic_long_add_unless(&memcg->css.refcnt.count,
+						1, 0) == 0) {
+				continue;
+			} else if (!queue_work(memcg_force_empty_wq,
+						&memcg->force_empty_work)) {
+				css_put(&memcg->css);
+			} else {
+				list_del_init(&memcg->force_empty_node);
+			}
+		}
+	}
+	mutex_unlock(&offline_cgroup_mutex);
 }
 
 static void empty_timer_trigger(struct timer_list *t)
@@ -4595,6 +4664,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 		mem_cgroup_force_empty(memcg);
 		if (page_counter_read(&memcg->memory) &&
 				memcg->max_retry != 1) {
+			INIT_WORK(&memcg->force_empty_work,
+					mem_cgroup_force_empty_delay);
 			memcg->timer_jiffies = jiffies + HZ;
 			add_force_empty_list(memcg);
 		}
@@ -4626,6 +4697,16 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 static void mem_cgroup_css_released(struct cgroup_subsys_state *css)
 {
 	struct mem_cgroup *memcg = mem_cgroup_from_css(css);
+	if (!list_empty(&memcg->force_empty_node)) {
+		mutex_lock(&offline_cgroup_mutex);
+		list_del_init(&memcg->force_empty_node);
+		mutex_unlock(&offline_cgroup_mutex);
+	}
+	if (!list_empty(&memcg->empty_fail_node)) {
+		mutex_lock(&offline_cgroup_mutex);
+		list_del_init(&memcg->empty_fail_node);
+		mutex_unlock(&offline_cgroup_mutex);
+	}
 
 	invalidate_reclaim_iterators(memcg);
 }
-- 
1.8.3.1

[PATCH 5/5] Memcgroup:add cgroup fs to show offline memcgroup status

[Xiongchun Duan]

Add cgroups_wait_empty proc file to show wait force empty memcgroup Add
cgroup_empty_fail file to show memcgroup which had try many time still did
not release. you can echo 0 > /proc/cgroup_empty_fail to manualy trigger
force empty this memcgroup

Signed-off-by: Xiongchun Duan <duanxiongchun@bytedance.com>
---
 mm/memcontrol.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 140 insertions(+)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 21b4432..1529549 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -57,6 +57,7 @@
 #include <linux/sort.h>
 #include <linux/fs.h>
 #include <linux/seq_file.h>
+#include <linux/proc_fs.h>
 #include <linux/vmpressure.h>
 #include <linux/mm_inline.h>
 #include <linux/swap_cgroup.h>
@@ -6440,6 +6441,140 @@ void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages)
 	refill_stock(memcg, nr_pages);
 }
 
+#ifdef CONFIG_PROC_FS
+static void print_memcg_header(struct seq_file *m)
+{
+	seq_puts(m, "address,css_ref,mem_ref,current_retry,max_retry\n");
+}
+
+static void memcgroup_show(struct mem_cgroup *memcg,
+		struct seq_file *m, bool header)
+{
+	if (header)
+		print_memcg_header(m);
+	seq_printf(m, "%p,%lu,%lu,%d,%d\n", memcg,
+			atomic_long_read(&memcg->css.refcnt.count),
+			page_counter_read(&memcg->memory),
+			memcg->current_retry, memcg->max_retry);
+}
+
+void *fail_start(struct seq_file *m, loff_t *pos)
+{
+	mutex_lock(&offline_cgroup_mutex);
+	return seq_list_start(&empty_fail_list, *pos);
+}
+
+void *fail_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	return seq_list_next(p, &empty_fail_list, pos);
+}
+
+void fail_stop(struct seq_file *m, void *p)
+{
+	mutex_unlock(&offline_cgroup_mutex);
+}
+
+static int fail_show(struct seq_file *m, void *p)
+{
+	struct mem_cgroup *memcg = list_entry(p, struct mem_cgroup,
+			empty_fail_node);
+	if (p == empty_fail_list.next)
+		memcgroup_show(memcg, m, true);
+	else
+		memcgroup_show(memcg, m, false);
+
+	return 0;
+}
+
+static const struct seq_operations fail_list_op = {
+	.start = fail_start,
+	.next = fail_next,
+	.stop = fail_stop,
+	.show = fail_show,
+};
+
+static int fail_list_open(struct inode *inode, struct file *file)
+{
+	return seq_open(file, &fail_list_op);
+}
+
+ssize_t fail_list_write(struct file *file, const char __user *buffer,
+	size_t count, loff_t *ppos)
+{
+	struct list_head *pos, *n;
+	struct mem_cgroup *memcg;
+
+	mutex_lock(&offline_cgroup_mutex);
+	list_for_each_safe(pos, n, &empty_fail_list) {
+		memcg = container_of(pos, struct mem_cgroup, empty_fail_node);
+		if (atomic_long_add_unless(&memcg->css.refcnt.count,
+					1, 0) == 0) {
+			continue;
+		} else if (!queue_work(memcg_force_empty_wq,
+					&memcg->force_empty_work)) {
+			css_put(&memcg->css);
+		}
+	}
+	mutex_unlock(&offline_cgroup_mutex);
+	return count;
+}
+
+static const struct file_operations proc_fail_list_operations = {
+	.open = fail_list_open,
+	.read = seq_read,
+	.write = fail_list_write,
+	.llseek = seq_lseek,
+	.release = seq_release,
+};
+
+void *empty_start(struct seq_file *m, loff_t *pos)
+{
+	mutex_lock(&offline_cgroup_mutex);
+	return seq_list_start(&force_empty_list, *pos);
+}
+
+void *empty_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	return seq_list_next(p, &force_empty_list, pos);
+}
+
+void empty_stop(struct seq_file *m, void *p)
+{
+	mutex_unlock(&offline_cgroup_mutex);
+}
+
+static int empty_show(struct seq_file *m, void *p)
+{
+	struct mem_cgroup *memcg = list_entry(p,
+			struct mem_cgroup, force_empty_node);
+	if (p == force_empty_list.next)
+		memcgroup_show(memcg, m, true);
+	else
+		memcgroup_show(memcg, m, false);
+
+	return 0;
+}
+
+static const struct seq_operations empty_list_op = {
+	.start = empty_start,
+	.next = empty_next,
+	.stop = empty_stop,
+	.show = empty_show,
+};
+
+static int empty_list_open(struct inode *inode, struct file *file)
+{
+	return seq_open(file, &empty_list_op);
+}
+
+static const struct file_operations proc_empty_list_operations = {
+	.open = empty_list_open,
+	.read = seq_read,
+	.llseek = seq_lseek,
+	.release = seq_release,
+};
+#endif
+
 static int __init cgroup_memory(char *s)
 {
 	char *token;
@@ -6483,6 +6618,11 @@ static int __init mem_cgroup_init(void)
 	INIT_WORK(&timer_poll_work, trigger_force_empty);
 	timer_setup(&empty_trigger, empty_timer_trigger, 0);
 
+#ifdef CONFIG_PROC_FS
+	proc_create("cgroups_wait_empty", 0, NULL, &proc_empty_list_operations);
+	proc_create("cgroups_empty_fail", 0, NULL, &proc_fail_list_operations);
+#endif
+
 	cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
 				  memcg_hotplug_cpu_dead);
 
-- 
1.8.3.1

Re: [PATCH 0/5] fix offline memcgroup still hold in memory

[Michal Hocko]

On Sat 19-01-19 22:30:16, Xiongchun Duan wrote:
> we find that in huge memory system frequent creat creation and deletion
> memcgroup make the system leave lots of offline memcgroup.we had seen 100000 
> unrelease offline memcgroup in our system(512G memory).
> 
> this memcgroup hold because some memory page still charged.
> so we try to Multiple interval call force_empty to reclaim this memory page.
> 
> after applying those patchs,in our system,the unrelease offline memcgroup
> was reduced from 100000 to 100.

I've stopped at patch 3 because all these patches really ask for much
more justification. Please note that each patch should explain the
problem, the proposed solution on highlevel and explain why it is done
so. The cover letter should also explain the underlying problem.
What does prevent those memcgs to be reclaim completely on force_empty?

Besides that, I do agree that the current implementation of force_empty
is rather suboptimal. There is a hardcoded retry loop counter which
doesn't really make much sense to me. The context is interruptible and
there is no real reason to have a retry count limit. If a userspace want
to implement a policy it can do it from userspace - e.g.

timeout 2m echo 0 > force_empty

Last but not least a force_empty on offline has been proposed recently
http://lkml.kernel.org/r/1547061285-100329-1-git-send-email-yang.shi@linux.alibaba.com
I haven't followed up on that discussion yet but it is always better to
check what where arguments there and explain why this approach is any
better/different.

> Xiongchun Duan (5):
>   Memcgroup: force empty after memcgroup offline
>   Memcgroup: Add timer to trigger workqueue
>   Memcgroup:add a global work
>   Memcgroup:Implement force empty work function
>   Memcgroup:add cgroup fs to show offline memcgroup status
> 
>  Documentation/cgroup-v1/memory.txt |   7 +-
>  Documentation/sysctl/kernel.txt    |  10 ++
>  include/linux/memcontrol.h         |  11 ++
>  kernel/sysctl.c                    |   9 ++
>  mm/memcontrol.c                    | 271 +++++++++++++++++++++++++++++++++++++
>  5 files changed, 306 insertions(+), 2 deletions(-)
> 
> -- 
> 1.8.3.1

-- 
Michal Hocko
SUSE Labs