* [PATCH v2] mm: show free pages per each migrate type
@ 2015-04-09 2:19 Neil Zhang
2015-04-09 20:47 ` Andrew Morton
2015-04-11 2:50 ` David Rientjes
0 siblings, 2 replies; 10+ messages in thread
From: Neil Zhang @ 2015-04-09 2:19 UTC (permalink / raw)
To: linux-mm, linux-kernel; +Cc: akpm, Neil Zhang
show detailed free pages per each migrate type in show_free_areas.
After apply this patch, the log printed out will be changed from
[ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
[ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
to
[ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
[ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
[ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
[ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
[ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
[ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
[ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
[ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
[ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
[ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
[ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
[ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
[ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
[ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
[ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
[ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
Signed-off-by: Neil Zhang <neilzhang1123@hotmail.com>
---
mm/internal.h | 2 ++
mm/page_alloc.c | 55 ++++++++++++++++++++++++++-----------------------------
mm/vmstat.c | 13 -------------
3 files changed, 28 insertions(+), 42 deletions(-)
diff --git a/mm/internal.h b/mm/internal.h
index a96da5b..5cb3079 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -14,6 +14,8 @@
#include <linux/fs.h>
#include <linux/mm.h>
+extern char * const migratetype_names[MIGRATE_TYPES];
+
void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *start_vma,
unsigned long floor, unsigned long ceiling);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 40e2942..2d70892 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3170,32 +3170,18 @@ out:
#define K(x) ((x) << (PAGE_SHIFT-10))
-static void show_migration_types(unsigned char type)
-{
- static const char types[MIGRATE_TYPES] = {
- [MIGRATE_UNMOVABLE] = 'U',
- [MIGRATE_RECLAIMABLE] = 'E',
- [MIGRATE_MOVABLE] = 'M',
- [MIGRATE_RESERVE] = 'R',
+char * const migratetype_names[MIGRATE_TYPES] = {
+ "Unmovable",
+ "Reclaimable",
+ "Movable",
+ "Reserve",
#ifdef CONFIG_CMA
- [MIGRATE_CMA] = 'C',
+ "CMA",
#endif
#ifdef CONFIG_MEMORY_ISOLATION
- [MIGRATE_ISOLATE] = 'I',
+ "Isolate",
#endif
- };
- char tmp[MIGRATE_TYPES + 1];
- char *p = tmp;
- int i;
-
- for (i = 0; i < MIGRATE_TYPES; i++) {
- if (type & (1 << i))
- *p++ = types[i];
- }
-
- *p = '\0';
- printk("(%s) ", tmp);
-}
+};
/*
* Show free area list (used inside shift_scroll-lock stuff)
@@ -3327,7 +3313,7 @@ void show_free_areas(unsigned int filter)
for_each_populated_zone(zone) {
unsigned long nr[MAX_ORDER], flags, order, total = 0;
- unsigned char types[MAX_ORDER];
+ unsigned long nr_free[MAX_ORDER][MIGRATE_TYPES], mtype;
if (skip_free_areas_node(filter, zone_to_nid(zone)))
continue;
@@ -3337,24 +3323,35 @@ void show_free_areas(unsigned int filter)
spin_lock_irqsave(&zone->lock, flags);
for (order = 0; order < MAX_ORDER; order++) {
struct free_area *area = &zone->free_area[order];
+ struct list_head *curr;
int type;
nr[order] = area->nr_free;
total += nr[order] << order;
- types[order] = 0;
for (type = 0; type < MIGRATE_TYPES; type++) {
+ nr_free[order][type] = 0;
if (!list_empty(&area->free_list[type]))
- types[order] |= 1 << type;
+ list_for_each(curr, &area->free_list[type])
+ nr_free[order][type]++;
}
}
spin_unlock_irqrestore(&zone->lock, flags);
- for (order = 0; order < MAX_ORDER; order++) {
+ for (order = 0; order < MAX_ORDER; order++)
printk("%lu*%lukB ", nr[order], K(1UL) << order);
- if (nr[order])
- show_migration_types(types[order]);
- }
printk("= %lukB\n", K(total));
+
+ printk("%12s: ", "orders");
+ for (order = 0; order < MAX_ORDER; order++)
+ printk("%6lu ", order);
+ printk("\n");
+
+ for (mtype = 0; mtype < MIGRATE_TYPES; mtype++) {
+ printk("%12s: ", migratetype_names[mtype]);
+ for (order = 0; order < MAX_ORDER; order++)
+ printk("%6lu ", nr_free[order][mtype]);
+ printk("\n");
+ }
}
hugetlb_show_meminfo();
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 4f5cd97..699eeb3 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -897,19 +897,6 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
#endif
#ifdef CONFIG_PROC_FS
-static char * const migratetype_names[MIGRATE_TYPES] = {
- "Unmovable",
- "Reclaimable",
- "Movable",
- "Reserve",
-#ifdef CONFIG_CMA
- "CMA",
-#endif
-#ifdef CONFIG_MEMORY_ISOLATION
- "Isolate",
-#endif
-};
-
static void frag_show_print(struct seq_file *m, pg_data_t *pgdat,
struct zone *zone)
{
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] mm: show free pages per each migrate type
2015-04-09 2:19 [PATCH v2] mm: show free pages per each migrate type Neil Zhang
@ 2015-04-09 20:47 ` Andrew Morton
2015-04-10 4:16 ` ZhangNeil
2015-04-11 2:50 ` David Rientjes
1 sibling, 1 reply; 10+ messages in thread
From: Andrew Morton @ 2015-04-09 20:47 UTC (permalink / raw)
To: Neil Zhang; +Cc: linux-mm, linux-kernel
On Thu, 9 Apr 2015 10:19:10 +0800 Neil Zhang <neilzhang1123@hotmail.com> wrote:
> show detailed free pages per each migrate type in show_free_areas.
>
> After apply this patch, the log printed out will be changed from
>
> [ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
> [ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
>
> to
>
> [ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
> [ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> [ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
> [ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
> [ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
> [ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
> [ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
> [ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> [ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
> [ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
> [ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
> [ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
Thanks. The proposed output does indeed look a lot better.
The columns don't line up, but I guess we can live with that ;)
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3327,7 +3313,7 @@ void show_free_areas(unsigned int filter)
>
> for_each_populated_zone(zone) {
> unsigned long nr[MAX_ORDER], flags, order, total = 0;
> - unsigned char types[MAX_ORDER];
> + unsigned long nr_free[MAX_ORDER][MIGRATE_TYPES], mtype;
>
> if (skip_free_areas_node(filter, zone_to_nid(zone)))
> continue;
nr_free[][] is an 8x11 array of 8, I think? That's 704 bytes of stack,
and show_free_areas() is called from very deep call stacks - from the
oom-killer, for example. We shouldn't do this.
I think we can eliminate nr_free[][]:
> + for (mtype = 0; mtype < MIGRATE_TYPES; mtype++) {
> + printk("%12s: ", migratetype_names[mtype]);
> + for (order = 0; order < MAX_ORDER; order++)
> + printk("%6lu ", nr_free[order][mtype]);
> + printk("\n");
> + }
In the above loop, take zone->lock and calculate the nr_free for this
particular order/mtype, then release zone->lock.
That will be slower, but show_free_areas() doesn't need to be fast.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] mm: show free pages per each migrate type
2015-04-09 20:47 ` Andrew Morton
@ 2015-04-10 4:16 ` ZhangNeil
2015-04-10 4:24 ` Andrew Morton
0 siblings, 1 reply; 10+ messages in thread
From: ZhangNeil @ 2015-04-10 4:16 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel
----------------------------------------
> Date: Thu, 9 Apr 2015 13:47:01 -0700
> From: akpm@linux-foundation.org
> To: neilzhang1123@hotmail.com
> CC: linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2] mm: show free pages per each migrate type
>
> On Thu, 9 Apr 2015 10:19:10 +0800 Neil Zhang <neilzhang1123@hotmail.com> wrote:
>
>> show detailed free pages per each migrate type in show_free_areas.
>>
>> After apply this patch, the log printed out will be changed from
>>
>> [ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
>> [ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
>>
>> to
>>
>> [ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
>> [ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
>> [ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
>> [ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
>> [ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
>> [ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
>> [ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
>> [ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
>> [ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
>> [ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
>> [ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
>> [ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
>
> Thanks. The proposed output does indeed look a lot better.
>
> The columns don't line up, but I guess we can live with that ;)
>
Thanks Andrew.
>
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -3327,7 +3313,7 @@ void show_free_areas(unsigned int filter)
>>
>> for_each_populated_zone(zone) {
>> unsigned long nr[MAX_ORDER], flags, order, total = 0;
>> - unsigned char types[MAX_ORDER];
>> + unsigned long nr_free[MAX_ORDER][MIGRATE_TYPES], mtype;
>>
>> if (skip_free_areas_node(filter, zone_to_nid(zone)))
>> continue;
>
> nr_free[][] is an 8x11 array of 8, I think? That's 704 bytes of stack,
> and show_free_areas() is called from very deep call stacks - from the
> oom-killer, for example. We shouldn't do this.
>
> I think we can eliminate nr_free[][]:
what about make it as global variable?
>
>> + for (mtype = 0; mtype < MIGRATE_TYPES; mtype++) {
>> + printk("%12s: ", migratetype_names[mtype]);
>> + for (order = 0; order < MAX_ORDER; order++)
>> + printk("%6lu ", nr_free[order][mtype]);
>> + printk("\n");
>> + }
>
> In the above loop, take zone->lock and calculate the nr_free for this
> particular order/mtype, then release zone->lock.
>
> That will be slower, but show_free_areas() doesn't need to be fast.
Yes, it mainly be called in oom killer.
Best Regards,
Neil Zhang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] mm: show free pages per each migrate type
2015-04-10 4:16 ` ZhangNeil
@ 2015-04-10 4:24 ` Andrew Morton
2015-04-12 7:17 ` ZhangNeil
0 siblings, 1 reply; 10+ messages in thread
From: Andrew Morton @ 2015-04-10 4:24 UTC (permalink / raw)
To: ZhangNeil; +Cc: linux-mm, linux-kernel
On Fri, 10 Apr 2015 04:16:15 +0000 ZhangNeil <neilzhang1123@hotmail.com> wrote:
> > I think we can eliminate nr_free[][]:
>
> what about make it as global__variable?
That isn't as good - it permanently consumes memory and really requires
new locking to protect the array from concurrent callers.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] mm: show free pages per each migrate type
2015-04-09 2:19 [PATCH v2] mm: show free pages per each migrate type Neil Zhang
2015-04-09 20:47 ` Andrew Morton
@ 2015-04-11 2:50 ` David Rientjes
2015-04-12 7:02 ` ZhangNeil
2015-04-12 7:08 ` ZhangNeil
1 sibling, 2 replies; 10+ messages in thread
From: David Rientjes @ 2015-04-11 2:50 UTC (permalink / raw)
To: Neil Zhang; +Cc: linux-mm, linux-kernel, akpm
On Thu, 9 Apr 2015, Neil Zhang wrote:
> show detailed free pages per each migrate type in show_free_areas.
>
> After apply this patch, the log printed out will be changed from
>
> [ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
> [ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
>
> to
>
> [ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
> [ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> [ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
> [ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
> [ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
> [ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
> [ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
> [ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> [ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
> [ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
> [ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
> [ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> [ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
>
> Signed-off-by: Neil Zhang <neilzhang1123@hotmail.com>
Sorry, this is just way too verbose. This output is emitted to the kernel
log on oom kill and since we lack a notification mechanism on system oom,
the _only_ way for userspace to detect oom kills that have occurred is by
scraping the kernel log. This is exactly what we do, and we have missed
oom kill events because they scroll from the ring buffer due to excessive
output such as this, which is why output was limited with the
show_free_areas() filter in the first place. Just because oom kill output
is much less than it has been in the past, for precisely this reason,
doesn't mean we can make it excessive again.
So nack on this patch, and if we really need to have this information (I
don't know your motivation for adding it since you list none in your
changelog), then we need to consider an oom verbosity sysctl or, better,
an actual system oom notification to userspace based on eventfd() without
requiring memcg.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] mm: show free pages per each migrate type
2015-04-11 2:50 ` David Rientjes
@ 2015-04-12 7:02 ` ZhangNeil
2015-04-12 7:08 ` ZhangNeil
1 sibling, 0 replies; 10+ messages in thread
From: ZhangNeil @ 2015-04-12 7:02 UTC (permalink / raw)
To: David Rientjes; +Cc: linux-mm, linux-kernel, akpm
[-- Attachment #1: Type: text/plain, Size: 4182 bytes --]
> Date: Fri, 10 Apr 2015 19:50:07 -0700
> From: rientjes@google.com
> To: neilzhang1123@hotmail.com
> CC: linux-mm@kvack.org; linux-kernel@vger.kernel.org; akpm@linux-foundation.org
> Subject: Re: [PATCH v2] mm: show free pages per each migrate type
>
> On Thu, 9 Apr 2015, Neil Zhang wrote:
>
> > show detailed free pages per each migrate type in show_free_areas.
> >
> > After apply this patch, the log printed out will be changed from
> >
> > [ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
> > [ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
> >
> > to
> >
> > [ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
> > [ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> > [ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
> > [ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
> > [ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
> > [ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
> > [ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> > [ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
> > [ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
> > [ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
> > [ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
> > [ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
> > [ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
> > [ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
> > [ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
> > [ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
> >
> > Signed-off-by: Neil Zhang <neilzhang1123@hotmail.com>
>
> Sorry, this is just way too verbose. This output is emitted to the kernel
> log on oom kill and since we lack a notification mechanism on system oom,
> the _only_ way for userspace to detect oom kills that have occurred is by
> scraping the kernel log. This is exactly what we do, and we have missed
> oom kill events because they scroll from the ring buffer due to excessive
> output such as this, which is why output was limited with the
> show_free_areas() filter in the first place. Just because oom kill output
> is much less than it has been in the past, for precisely this reason,
> doesn't mean we can make it excessive again.
>
Just like you said, OOM kill is much less than before, but we still need to analyze it when it happens on a mobile device. It can give more detailed info for us when debugging. Besides OOM kill, we also can check the memory usages in runtime by echo 'm' to sysRq.It can help us to find out code defect sometimes, for example, we even found that the NR_FREE_CMAmemory was not align with the total CMA pages in the free list showed by this patch.
> So nack on this patch, and if we really need to have this information (I
> don't know your motivation for adding it since you list none in your
> changelog), then we need to consider an oom verbosity sysctl or, better,
> an actual system oom notification to userspace based on eventfd() without
> requiring memcg.
[-- Attachment #2: Type: text/html, Size: 4905 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] mm: show free pages per each migrate type
2015-04-11 2:50 ` David Rientjes
2015-04-12 7:02 ` ZhangNeil
@ 2015-04-12 7:08 ` ZhangNeil
2015-04-13 23:42 ` David Rientjes
1 sibling, 1 reply; 10+ messages in thread
From: ZhangNeil @ 2015-04-12 7:08 UTC (permalink / raw)
To: David Rientjes; +Cc: linux-mm, linux-kernel, akpm
----------------------------------------
> Date: Fri, 10 Apr 2015 19:50:07 -0700
> From: rientjes@google.com
> To: neilzhang1123@hotmail.com
> CC: linux-mm@kvack.org; linux-kernel@vger.kernel.org; akpm@linux-foundation.org
> Subject: Re: [PATCH v2] mm: show free pages per each migrate type
>
> On Thu, 9 Apr 2015, Neil Zhang wrote:
>
>> show detailed free pages per each migrate type in show_free_areas.
>>
>> After apply this patch, the log printed out will be changed from
>>
>> [ 558.212844@0] Normal: 218*4kB (UEMC) 207*8kB (UEMC) 126*16kB (UEMC) 21*32kB (UC) 5*64kB (C) 3*128kB (C) 1*256kB (C) 1*512kB (C) 0*1024kB 0*2048kB 1*4096kB (R) = 10784kB
>> [ 558.227840@0] HighMem: 3*4kB (UMR) 3*8kB (UMR) 2*16kB (UM) 3*32kB (UMR) 0*64kB 1*128kB (M) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 548kB
>>
>> to
>>
>> [ 806.506450@1] Normal: 8969*4kB 4370*8kB 2*16kB 3*32kB 2*64kB 3*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 74804kB
>> [ 806.517456@1] orders: 0 1 2 3 4 5 6 7 8 9 10
>> [ 806.527077@1] Unmovable: 8287 4370 0 0 0 0 0 0 0 0 0
>> [ 806.536699@1] Reclaimable: 681 0 0 0 0 0 0 0 0 0 0
>> [ 806.546321@1] Movable: 1 0 0 0 0 0 0 0 0 0 0
>> [ 806.555942@1] Reserve: 0 0 2 3 2 3 3 1 0 1 0
>> [ 806.565564@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.575187@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.584810@1] HighMem: 80*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
>> [ 806.595383@1] orders: 0 1 2 3 4 5 6 7 8 9 10
>> [ 806.605004@1] Unmovable: 12 0 0 0 0 0 0 0 0 0 0
>> [ 806.614626@1] Reclaimable: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.624248@1] Movable: 11 15 0 0 0 0 0 0 0 0 0
>> [ 806.633869@1] Reserve: 57 0 0 0 0 0 0 0 0 0 0
>> [ 806.643491@1] CMA: 0 0 0 0 0 0 0 0 0 0 0
>> [ 806.653113@1] Isolate: 0 0 0 0 0 0 0 0 0 0 0
>>
>> Signed-off-by: Neil Zhang <neilzhang1123@hotmail.com>
>
> Sorry, this is just way too verbose. This output is emitted to the kernel
> log on oom kill and since we lack a notification mechanism on system oom,
> the _only_ way for userspace to detect oom kills that have occurred is by
> scraping the kernel log. This is exactly what we do, and we have missed
> oom kill events because they scroll from the ring buffer due to excessive
> output such as this, which is why output was limited with the
> show_free_areas() filter in the first place. Just because oom kill output
> is much less than it has been in the past, for precisely this reason,
> doesn't mean we can make it excessive again.
>
Just like you said, OOM kill is much less than before, but we still need to analyze it when
it happens on a mobile device. It can give more detailed info for us when debugging.
Besides OOM kill, we also can check the memory usages in runtime by echo 'm' to sysRq.
It can help us to find out code defect sometimes, for example, we even found that the NR_FREE_CMA
memory was not align with the total CMA pages in the free list showed by this patch.
> So nack on this patch, and if we really need to have this information (I
> don't know your motivation for adding it since you list none in your
> changelog), then we need to consider an oom verbosity sysctl or, better,
> an actual system oom notification to userspace based on eventfd() without
> requiring memcg.
Best Regards,
Neil Zhang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] mm: show free pages per each migrate type
2015-04-10 4:24 ` Andrew Morton
@ 2015-04-12 7:17 ` ZhangNeil
2015-04-12 7:25 ` Andrew Morton
0 siblings, 1 reply; 10+ messages in thread
From: ZhangNeil @ 2015-04-12 7:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel
----------------------------------------
> Date: Thu, 9 Apr 2015 21:24:41 -0700
> From: akpm@linux-foundation.org
> To: neilzhang1123@hotmail.com
> CC: linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2] mm: show free pages per each migrate type
>
> On Fri, 10 Apr 2015 04:16:15 +0000 ZhangNeil <neilzhang1123@hotmail.com> wrote:
>
>>> I think we can eliminate nr_free[][]:
>>
>> what about make it as global__variable?
>
> That isn't as good - it permanently consumes memory and really requires
> new locking to protect the array from concurrent callers.
>
Then it may need to change the code a lot.
BTW:
I calculate the nr_free[][] again, it is an 6x11 array of 8 in the worst case, that is 528B, is it acceptable?
Best Regards,
Neil Zhang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] mm: show free pages per each migrate type
2015-04-12 7:17 ` ZhangNeil
@ 2015-04-12 7:25 ` Andrew Morton
0 siblings, 0 replies; 10+ messages in thread
From: Andrew Morton @ 2015-04-12 7:25 UTC (permalink / raw)
To: ZhangNeil; +Cc: linux-mm, linux-kernel
On Sun, 12 Apr 2015 07:17:11 +0000 ZhangNeil <neilzhang1123@hotmail.com> wrote:
> I calculate the__nr_free[][] again, it is an 6x11 array of 8 in the worst case, that is 528B, is it acceptable?
I don't think so. This is hardly a critical change and that's a high cost.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] mm: show free pages per each migrate type
2015-04-12 7:08 ` ZhangNeil
@ 2015-04-13 23:42 ` David Rientjes
0 siblings, 0 replies; 10+ messages in thread
From: David Rientjes @ 2015-04-13 23:42 UTC (permalink / raw)
To: ZhangNeil; +Cc: Andrew Morton, linux-kernel, linux-mm
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1920 bytes --]
On Sun, 12 Apr 2015, ZhangNeil wrote:
> > Sorry, this is just way too verbose. This output is emitted to the kernel
> > log on oom kill and since we lack a notification mechanism on system oom,
> > the _only_ way for userspace to detect oom kills that have occurred is by
> > scraping the kernel log. This is exactly what we do, and we have missed
> > oom kill events because they scroll from the ring buffer due to excessive
> > output such as this, which is why output was limited with the
> > show_free_areas() filter in the first place. Just because oom kill output
> > is much less than it has been in the past, for precisely this reason,
> > doesn't mean we can make it excessive again.
> >
>
> Just like you said, OOM kill is much less than before, but we still need to analyze it when
> it happens on a mobile device. It can give more detailed info for us when debugging.
>
There is a very large amount of data that would be true for, and we simply
can't make oom killing more verbose in the kernel log because it is the
_only_ mechanism we have to determine that the kernel killed a user
process and what that process was. You could make the same argument for
dumping all of /proc/slabinfo, which people have proposed before and it's
been nacked, to discover slab leaks. We simply can't make it more
verbose, it's that easy.
> Besides OOM kill, we also can check the memory usages in runtime by echo 'm' to sysRq.
> It can help us to find out code defect sometimes, for example, we even found that the NR_FREE_CMA
> memory was not align with the total CMA pages in the free list showed by this patch.
>
Sysrq is an entirely different usecase and the natural response would be
to export this information for sysrq but not oom kill, but in this case
there is no compelling reason to dump it in the ring buffer in the first
place: it should be in procfs where it can easily be read and parsed.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-04-13 23:42 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-09 2:19 [PATCH v2] mm: show free pages per each migrate type Neil Zhang
2015-04-09 20:47 ` Andrew Morton
2015-04-10 4:16 ` ZhangNeil
2015-04-10 4:24 ` Andrew Morton
2015-04-12 7:17 ` ZhangNeil
2015-04-12 7:25 ` Andrew Morton
2015-04-11 2:50 ` David Rientjes
2015-04-12 7:02 ` ZhangNeil
2015-04-12 7:08 ` ZhangNeil
2015-04-13 23:42 ` David Rientjes
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox