[PATCH 4.19.y 1/2] mm: hide incomplete nr_indirectly_reclaimable in /proc/zoneinfo

[PATCH 4.19.y 1/2] mm: hide incomplete nr_indirectly_reclaimable in /proc/zoneinfo

[Konstantin Khlebnikov]

From: Roman Gushchin <guro@fb.com>

[ commit c29f9010a35604047f96a7e9d6cbabfa36d996d1 from 4.14.y ]

Yongqin reported that /proc/zoneinfo format is broken in 4.14
due to commit 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable
in /proc/vmstat")

Node 0, zone      DMA
  per-node stats
      nr_inactive_anon 403
      nr_active_anon 89123
      nr_inactive_file 128887
      nr_active_file 47377
      nr_unevictable 2053
      nr_slab_reclaimable 7510
      nr_slab_unreclaimable 10775
      nr_isolated_anon 0
      nr_isolated_file 0
      <...>
      nr_vmscan_write 0
      nr_vmscan_immediate_reclaim 0
      nr_dirtied   6022
      nr_written   5985
                   74240
      ^^^^^^^^^^
  pages free     131656

The problem is caused by the nr_indirectly_reclaimable counter,
which is hidden from the /proc/vmstat, but not from the
/proc/zoneinfo. Let's fix this inconsistency and hide the
counter from /proc/zoneinfo exactly as from /proc/vmstat.

BTW, in 4.19+ the counter has been renamed and exported by
the commit b29940c1abd7 ("mm: rename and change semantics of
nr_indirectly_reclaimable_bytes"), so there is no such a problem
anymore.

Cc: <stable@vger.kernel.org> # 4.19.y
Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
Reported-by: Yongqin Liu <yongqin.liu@linaro.org>
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
 mm/vmstat.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mm/vmstat.c b/mm/vmstat.c
index 72ef3936d15d..7b8937cb2876 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1550,6 +1550,10 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat,
 	if (is_zone_first_populated(pgdat, zone)) {
 		seq_printf(m, "\n  per-node stats");
 		for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
+			/* Skip hidden vmstat items. */
+			if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
+					 NR_VM_NUMA_STAT_ITEMS] == '\0')
+				continue;
 			seq_printf(m, "\n      %-12s %lu",
 				vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
 				NR_VM_NUMA_STAT_ITEMS],

[PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

[Konstantin Khlebnikov]

This fixes /sys/devices/system/node/node*/vmstat format:

...
nr_dirtied 6613155
nr_written 5796802
 11089216
...

In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
change semantics of nr_indirectly_reclaimable_bytes").

Cc: <stable@vger.kernel.org> # 4.19.y
Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Cc: Roman Gushchin <guro@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
---
 drivers/base/node.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/base/node.c b/drivers/base/node.c
index 1ac4c36e13bb..c3968e2d0a98 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -197,11 +197,16 @@ static ssize_t node_read_vmstat(struct device *dev,
 			     sum_zone_numa_state(nid, i));
 #endif
 
-	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++)
+	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
+		/* Skip hidden vmstat items. */
+		if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
+				 NR_VM_NUMA_STAT_ITEMS] == '\0')
+			continue;
 		n += sprintf(buf+n, "%s %lu\n",
 			     vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
 			     NR_VM_NUMA_STAT_ITEMS],
 			     node_page_state(pgdat, i));
+	}
 
 	return n;
 }

Re: [PATCH 4.19.y 1/2] mm: hide incomplete nr_indirectly_reclaimable in /proc/zoneinfo

[Vlastimil Babka]

On 4/9/19 7:05 PM, Konstantin Khlebnikov wrote:
> From: Roman Gushchin <guro@fb.com>
> 
> [ commit c29f9010a35604047f96a7e9d6cbabfa36d996d1 from 4.14.y ]
> 
> Yongqin reported that /proc/zoneinfo format is broken in 4.14
> due to commit 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable
> in /proc/vmstat")
> 
> Node 0, zone      DMA
>   per-node stats
>       nr_inactive_anon 403
>       nr_active_anon 89123
>       nr_inactive_file 128887
>       nr_active_file 47377
>       nr_unevictable 2053
>       nr_slab_reclaimable 7510
>       nr_slab_unreclaimable 10775
>       nr_isolated_anon 0
>       nr_isolated_file 0
>       <...>
>       nr_vmscan_write 0
>       nr_vmscan_immediate_reclaim 0
>       nr_dirtied   6022
>       nr_written   5985
>                    74240
>       ^^^^^^^^^^
>   pages free     131656
> 
> The problem is caused by the nr_indirectly_reclaimable counter,
> which is hidden from the /proc/vmstat, but not from the
> /proc/zoneinfo. Let's fix this inconsistency and hide the
> counter from /proc/zoneinfo exactly as from /proc/vmstat.
> 
> BTW, in 4.19+ the counter has been renamed and exported by

This was actually 4.20+ and this mistake is why we initially forgot
about 4.19 stable in [1]

[1]
https://lore.kernel.org/linux-mm/20181030174649.16778-1-guro@fb.com/

> the commit b29940c1abd7 ("mm: rename and change semantics of
> nr_indirectly_reclaimable_bytes"), so there is no such a problem
> anymore.
> 
> Cc: <stable@vger.kernel.org> # 4.19.y
> Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
> Reported-by: Yongqin Liu <yongqin.liu@linaro.org>
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>

Thanks.

> ---
>  mm/vmstat.c |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 72ef3936d15d..7b8937cb2876 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1550,6 +1550,10 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat,
>  	if (is_zone_first_populated(pgdat, zone)) {
>  		seq_printf(m, "\n  per-node stats");
>  		for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
> +			/* Skip hidden vmstat items. */
> +			if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
> +					 NR_VM_NUMA_STAT_ITEMS] == '\0')
> +				continue;
>  			seq_printf(m, "\n      %-12s %lu",
>  				vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
>  				NR_VM_NUMA_STAT_ITEMS],
> 

Re: [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

[Vlastimil Babka]

On 4/9/19 7:05 PM, Konstantin Khlebnikov wrote:
> This fixes /sys/devices/system/node/node*/vmstat format:
> 
> ...
> nr_dirtied 6613155
> nr_written 5796802
>  11089216
> ...
> 
> In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
> change semantics of nr_indirectly_reclaimable_bytes").
> 
> Cc: <stable@vger.kernel.org> # 4.19.y

So given the same circumstances as patch 1/2, shouldn't this also
include 4.14.y ?

> Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> Cc: Roman Gushchin <guro@fb.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> ---
>  drivers/base/node.c |    7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index 1ac4c36e13bb..c3968e2d0a98 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -197,11 +197,16 @@ static ssize_t node_read_vmstat(struct device *dev,
>  			     sum_zone_numa_state(nid, i));
>  #endif
>  
> -	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++)
> +	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
> +		/* Skip hidden vmstat items. */
> +		if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
> +				 NR_VM_NUMA_STAT_ITEMS] == '\0')
> +			continue;
>  		n += sprintf(buf+n, "%s %lu\n",
>  			     vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
>  			     NR_VM_NUMA_STAT_ITEMS],
>  			     node_page_state(pgdat, i));
> +	}
>  
>  	return n;
>  }
> 

Re: [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

[Konstantin Khlebnikov]

On 09.04.2019 21:22, Vlastimil Babka wrote:
> On 4/9/19 7:05 PM, Konstantin Khlebnikov wrote:
>> This fixes /sys/devices/system/node/node*/vmstat format:
>>
>> ...
>> nr_dirtied 6613155
>> nr_written 5796802
>>   11089216
>> ...
>>
>> In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
>> change semantics of nr_indirectly_reclaimable_bytes").
>>
>> Cc: <stable@vger.kernel.org> # 4.19.y
> 
> So given the same circumstances as patch 1/2, shouldn't this also
> include 4.14.y ?

Oh, yes. Second patch should be applied for 4.14.y too.

> 
>> Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
>> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
>> Cc: Roman Gushchin <guro@fb.com>
>> Cc: Vlastimil Babka <vbabka@suse.cz>
>> ---
>>   drivers/base/node.c |    7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/base/node.c b/drivers/base/node.c
>> index 1ac4c36e13bb..c3968e2d0a98 100644
>> --- a/drivers/base/node.c
>> +++ b/drivers/base/node.c
>> @@ -197,11 +197,16 @@ static ssize_t node_read_vmstat(struct device *dev,
>>   			     sum_zone_numa_state(nid, i));
>>   #endif
>>   
>> -	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++)
>> +	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
>> +		/* Skip hidden vmstat items. */
>> +		if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
>> +				 NR_VM_NUMA_STAT_ITEMS] == '\0')
>> +			continue;
>>   		n += sprintf(buf+n, "%s %lu\n",
>>   			     vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
>>   			     NR_VM_NUMA_STAT_ITEMS],
>>   			     node_page_state(pgdat, i));
>> +	}
>>   
>>   	return n;
>>   }
>>
> 

Patch "[PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs" has been added to the 4.14-stable tree

[gregkh]

This is a note to let you know that I've just added the patch titled

    [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

to the 4.14-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     mm-hide-incomplete-nr_indirectly_reclaimable-in-sysfs.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@vger.kernel.org> know about it.


From khlebnikov@yandex-team.ru  Thu Apr 18 17:53:53 2019
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Date: Tue, 09 Apr 2019 20:05:43 +0300
Subject: [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs
To: stable@vger.kernel.org
Cc: linux-mm@kvack.org, Roman Gushchin <guro@fb.com>, Vlastimil Babka <vbabka@suse.cz>
Message-ID: <155482954368.2823.12386748649541618609.stgit@buzz>

From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>

In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
change semantics of nr_indirectly_reclaimable_bytes").

This fixes /sys/devices/system/node/node*/vmstat format:

...
nr_dirtied 6613155
nr_written 5796802
 11089216
...

Cc: <stable@vger.kernel.org> # 4.19.y
Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Cc: Roman Gushchin <guro@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/base/node.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -197,11 +197,16 @@ static ssize_t node_read_vmstat(struct d
 			     sum_zone_numa_state(nid, i));
 #endif
 
-	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++)
+	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
+		/* Skip hidden vmstat items. */
+		if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
+				 NR_VM_NUMA_STAT_ITEMS] == '\0')
+			continue;
 		n += sprintf(buf+n, "%s %lu\n",
 			     vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
 			     NR_VM_NUMA_STAT_ITEMS],
 			     node_page_state(pgdat, i));
+	}
 
 	return n;
 }


Patches currently in stable-queue which might be from khlebnikov@yandex-team.ru are

queue-4.14/mm-hide-incomplete-nr_indirectly_reclaimable-in-sysfs.patch

Patch "[PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs" has been added to the 4.19-stable tree

[gregkh]

This is a note to let you know that I've just added the patch titled

    [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

to the 4.19-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     mm-hide-incomplete-nr_indirectly_reclaimable-in-sysfs.patch
and it can be found in the queue-4.19 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@vger.kernel.org> know about it.


From khlebnikov@yandex-team.ru  Thu Apr 18 17:53:53 2019
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Date: Tue, 09 Apr 2019 20:05:43 +0300
Subject: [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs
To: stable@vger.kernel.org
Cc: linux-mm@kvack.org, Roman Gushchin <guro@fb.com>, Vlastimil Babka <vbabka@suse.cz>
Message-ID: <155482954368.2823.12386748649541618609.stgit@buzz>

From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>

In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
change semantics of nr_indirectly_reclaimable_bytes").

This fixes /sys/devices/system/node/node*/vmstat format:

...
nr_dirtied 6613155
nr_written 5796802
 11089216
...

Cc: <stable@vger.kernel.org> # 4.19.y
Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Cc: Roman Gushchin <guro@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/base/node.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -197,11 +197,16 @@ static ssize_t node_read_vmstat(struct d
 			     sum_zone_numa_state(nid, i));
 #endif
 
-	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++)
+	for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) {
+		/* Skip hidden vmstat items. */
+		if (*vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
+				 NR_VM_NUMA_STAT_ITEMS] == '\0')
+			continue;
 		n += sprintf(buf+n, "%s %lu\n",
 			     vmstat_text[i + NR_VM_ZONE_STAT_ITEMS +
 			     NR_VM_NUMA_STAT_ITEMS],
 			     node_page_state(pgdat, i));
+	}
 
 	return n;
 }


Patches currently in stable-queue which might be from khlebnikov@yandex-team.ru are

queue-4.19/mm-hide-incomplete-nr_indirectly_reclaimable-in-sysfs.patch
queue-4.19/sched-core-fix-buffer-overflow-in-cgroup2-property-c.patch

Re: [PATCH 4.19.y 1/2] mm: hide incomplete nr_indirectly_reclaimable in /proc/zoneinfo

[Greg KH]

On Tue, Apr 09, 2019 at 08:05:41PM +0300, Konstantin Khlebnikov wrote:
> From: Roman Gushchin <guro@fb.com>
> 
> [ commit c29f9010a35604047f96a7e9d6cbabfa36d996d1 from 4.14.y ]
> 
> Yongqin reported that /proc/zoneinfo format is broken in 4.14
> due to commit 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable
> in /proc/vmstat")
> 
> Node 0, zone      DMA
>   per-node stats
>       nr_inactive_anon 403
>       nr_active_anon 89123
>       nr_inactive_file 128887
>       nr_active_file 47377
>       nr_unevictable 2053
>       nr_slab_reclaimable 7510
>       nr_slab_unreclaimable 10775
>       nr_isolated_anon 0
>       nr_isolated_file 0
>       <...>
>       nr_vmscan_write 0
>       nr_vmscan_immediate_reclaim 0
>       nr_dirtied   6022
>       nr_written   5985
>                    74240
>       ^^^^^^^^^^
>   pages free     131656
> 
> The problem is caused by the nr_indirectly_reclaimable counter,
> which is hidden from the /proc/vmstat, but not from the
> /proc/zoneinfo. Let's fix this inconsistency and hide the
> counter from /proc/zoneinfo exactly as from /proc/vmstat.
> 
> BTW, in 4.19+ the counter has been renamed and exported by
> the commit b29940c1abd7 ("mm: rename and change semantics of
> nr_indirectly_reclaimable_bytes"), so there is no such a problem
> anymore.
> 
> Cc: <stable@vger.kernel.org> # 4.19.y
> Fixes: 7aaf77272358 ("mm: don't show nr_indirectly_reclaimable in /proc/vmstat")
> Reported-by: Yongqin Liu <yongqin.liu@linaro.org>
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> ---
>  mm/vmstat.c |    4 ++++
>  1 file changed, 4 insertions(+)

Both of these now queued up, thanks!

greg k-h

Re: [PATCH 4.19.y 2/2] mm: hide incomplete nr_indirectly_reclaimable in sysfs

[Greg KH]

On Tue, Apr 09, 2019 at 08:05:43PM +0300, Konstantin Khlebnikov wrote:
> This fixes /sys/devices/system/node/node*/vmstat format:
> 
> ...
> nr_dirtied 6613155
> nr_written 5796802
>  11089216
> ...
> 
> In upstream branch this fixed by commit b29940c1abd7 ("mm: rename and
> change semantics of nr_indirectly_reclaimable_bytes").

We are running at almost 100% for the times we add a patch to the tree
that is not in Linus's tree, for a fixup being needed for it
after-the-fact.

{sigh}

Now queued up, thanks.

greg k-h