[PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Rafael Aquini]

This patch introduces 'kernelpagesize_kB' line element to /proc/<pid>/numa_maps
report file in order to help identifying the size of pages that are backing
memory areas mapped by a given task. This is specially useful to
help differentiating between HUGE and GIGANTIC page backed VMAs.

This patch is based on Dave Hansen's proposal and reviewer's follow-ups
taken from the following dicussion threads:
 * https://lkml.org/lkml/2011/9/21/454
 * https://lkml.org/lkml/2014/12/20/66

Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
* v2 changelog:
  . print kernel page size unconditionally (jweiner, dhansen)
  . rename pagesize to match smaps terminology (dhansen, drientjes)

 fs/proc/task_mmu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 246eae8..3688d64 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1533,6 +1533,8 @@ static int show_numa_map(struct seq_file *m, void *v, int is_pid)
 	if (!md->pages)
 		goto out;
 
+	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);
+
 	if (md->anon)
 		seq_printf(m, " anon=%lu", md->anon);
 
-- 
1.9.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Andrew Morton]

On Mon,  5 Jan 2015 12:44:31 -0500 Rafael Aquini <aquini@redhat.com> wrote:

> This patch introduces 'kernelpagesize_kB' line element to /proc/<pid>/numa_maps
> report file in order to help identifying the size of pages that are backing
> memory areas mapped by a given task. This is specially useful to
> help differentiating between HUGE and GIGANTIC page backed VMAs.
> 
> This patch is based on Dave Hansen's proposal and reviewer's follow-ups
> taken from the following dicussion threads:
>  * https://lkml.org/lkml/2011/9/21/454

Dave's changelog contains useful information which this one lacked.  I
stole some of it.

: The output of /proc/$pid/numa_maps is in terms of number of pages like
: anon=22 or dirty=54.  Here's some output:
: 
: 7f4680000000 default file=/hugetlb/bigfile anon=50 dirty=50 N0=50
: 7f7659600000 default file=/anon_hugepage\040(deleted) anon=50 dirty=50 N0=50
: 7fff8d425000 default stack anon=50 dirty=50 N0=50
: Looks like we have a stack and a couple of anonymous hugetlbfs
: areas page which both use the same amount of memory.  They don't.
: 
: The 'bigfile' uses 1GB pages and takes up ~50GB of space.  The
: anon_hugepage uses 2MB pages and takes up ~100MB of space while the stack
: uses normal 4k pages.  You can go over to smaps to figure out what the
: page size _really_ is with KernelPageSize or MMUPageSize.  But, I think
: this is a pretty nasty and counterintuitive interface as it stands.
: 
: This patch introduces 'kernelpagesize_kB' line element to
: /proc/<pid>/numa_maps report file in order to help identifying the size of
: pages that are backing memory areas mapped by a given task.  This is
: specially useful to help differentiating between HUGE and GIGANTIC page
: backed VMAs.
: 
: This patch is based on Dave Hansen's proposal and reviewer's follow-ups
: taken from the following dicussion threads:
:  * https://lkml.org/lkml/2011/9/21/454
:  * https://lkml.org/lkml/2014/12/20/66


> +	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);

This changes the format of the numa_maps file and can potentially break
existing parsers.  Please discuss.

I'd complain about the patch's failure to update the documentation,
except numa_maps appears to be undocumented.  Sigh.  What the heck is "N0"?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Rafael Aquini]

On Mon, Jan 05, 2015 at 01:35:00PM -0800, Andrew Morton wrote:
> On Mon,  5 Jan 2015 12:44:31 -0500 Rafael Aquini <aquini@redhat.com> wrote:
> 
> > This patch introduces 'kernelpagesize_kB' line element to /proc/<pid>/numa_maps
> > report file in order to help identifying the size of pages that are backing
> > memory areas mapped by a given task. This is specially useful to
> > help differentiating between HUGE and GIGANTIC page backed VMAs.
> > 
> > This patch is based on Dave Hansen's proposal and reviewer's follow-ups
> > taken from the following dicussion threads:
> >  * https://lkml.org/lkml/2011/9/21/454
> 
> Dave's changelog contains useful information which this one lacked.  I
> stole some of it.
> 
> : The output of /proc/$pid/numa_maps is in terms of number of pages like
> : anon=22 or dirty=54.  Here's some output:
> : 
> : 7f4680000000 default file=/hugetlb/bigfile anon=50 dirty=50 N0=50
> : 7f7659600000 default file=/anon_hugepage\040(deleted) anon=50 dirty=50 N0=50
> : 7fff8d425000 default stack anon=50 dirty=50 N0=50
> : Looks like we have a stack and a couple of anonymous hugetlbfs
> : areas page which both use the same amount of memory.  They don't.
> : 
> : The 'bigfile' uses 1GB pages and takes up ~50GB of space.  The
> : anon_hugepage uses 2MB pages and takes up ~100MB of space while the stack
> : uses normal 4k pages.  You can go over to smaps to figure out what the
> : page size _really_ is with KernelPageSize or MMUPageSize.  But, I think
> : this is a pretty nasty and counterintuitive interface as it stands.
> : 
> : This patch introduces 'kernelpagesize_kB' line element to
> : /proc/<pid>/numa_maps report file in order to help identifying the size of
> : pages that are backing memory areas mapped by a given task.  This is
> : specially useful to help differentiating between HUGE and GIGANTIC page
> : backed VMAs.
> : 
> : This patch is based on Dave Hansen's proposal and reviewer's follow-ups
> : taken from the following dicussion threads:
> :  * https://lkml.org/lkml/2011/9/21/454
> :  * https://lkml.org/lkml/2014/12/20/66
> 
> 
> > +	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);
> 
> This changes the format of the numa_maps file and can potentially break
> existing parsers.  Please discuss.
> 
> I'd complain about the patch's failure to update the documentation,
> except numa_maps appears to be undocumented.  Sigh.  What the heck is "N0"?
>
That's a nice opportunity to attempt to sharp my doc writing skills.
Sorry for the total failure to identify it earlier.
I just took it as a TODO note to send a patch to document this interface soon.

Happy new year.
-- Rafael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Rafael Aquini]

On Mon, Jan 05, 2015 at 05:55:04PM -0500, Rafael Aquini wrote:
> > > +	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);
> > 
> > This changes the format of the numa_maps file and can potentially break
> > existing parsers.  Please discuss.
> > 
> > I'd complain about the patch's failure to update the documentation,
> > except numa_maps appears to be undocumented.  Sigh.  What the heck is "N0"?
> >
> That's a nice opportunity to attempt to sharp my doc writing skills.
> Sorry for the total failure to identify it earlier.
> I just took it as a TODO note to send a patch to document this interface soon.
>

Or perhaps that's a sign we should move the numa node locality
information to /proc/$pid/smaps and start printing a deprecation warning
for /proc/$pid/numa_maps users preparing them for a future removal?

 
> Happy new year.
> -- Rafael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Rafael Aquini]

On Mon, Jan 05, 2015 at 03:20:37PM -0800, Andrew Morton wrote:
> On Mon, 5 Jan 2015 17:55:05 -0500 Rafael Aquini <aquini@redhat.com> wrote:
> 
> > On Mon, Jan 05, 2015 at 01:35:00PM -0800, Andrew Morton wrote:
> > > On Mon,  5 Jan 2015 12:44:31 -0500 Rafael Aquini <aquini@redhat.com> wrote:
> > > 
> > > > This patch introduces 'kernelpagesize_kB' line element to /proc/<pid>/numa_maps
> > > > report file in order to help identifying the size of pages that are backing
> > > > memory areas mapped by a given task. This is specially useful to
> > > > help differentiating between HUGE and GIGANTIC page backed VMAs.
> > > > 
> > > > This patch is based on Dave Hansen's proposal and reviewer's follow-ups
> > > > taken from the following dicussion threads:
> > > >  * https://lkml.org/lkml/2011/9/21/454
> > > 
> > > Dave's changelog contains useful information which this one lacked.  I
> > > stole some of it.
> > > 
> > > : The output of /proc/$pid/numa_maps is in terms of number of pages like
> > > : anon=22 or dirty=54.  Here's some output:
> > > : 
> > > : 7f4680000000 default file=/hugetlb/bigfile anon=50 dirty=50 N0=50
> > > : 7f7659600000 default file=/anon_hugepage\040(deleted) anon=50 dirty=50 N0=50
> > > : 7fff8d425000 default stack anon=50 dirty=50 N0=50
> > > : Looks like we have a stack and a couple of anonymous hugetlbfs
> > > : areas page which both use the same amount of memory.  They don't.
> > > : 
> > > : The 'bigfile' uses 1GB pages and takes up ~50GB of space.  The
> > > : anon_hugepage uses 2MB pages and takes up ~100MB of space while the stack
> > > : uses normal 4k pages.  You can go over to smaps to figure out what the
> > > : page size _really_ is with KernelPageSize or MMUPageSize.  But, I think
> > > : this is a pretty nasty and counterintuitive interface as it stands.
> > > : 
> > > : This patch introduces 'kernelpagesize_kB' line element to
> > > : /proc/<pid>/numa_maps report file in order to help identifying the size of
> > > : pages that are backing memory areas mapped by a given task.  This is
> > > : specially useful to help differentiating between HUGE and GIGANTIC page
> > > : backed VMAs.
> > > : 
> > > : This patch is based on Dave Hansen's proposal and reviewer's follow-ups
> > > : taken from the following dicussion threads:
> > > :  * https://lkml.org/lkml/2011/9/21/454
> > > :  * https://lkml.org/lkml/2014/12/20/66
> > > 
> > > 
> > > > +	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);
> > > 
> > > This changes the format of the numa_maps file and can potentially break
> > > existing parsers.  Please discuss.
> 
> ^^ ??
>
Sorry I overlooked it.

Parsers indeed would have to be adjusted to cope with an extra line element
(they already have to do so, similarly, for the conditional 'huge' hint).
Despite I don't think of it as a showstopper (as is), I think we can consider
moving it to EOL, if its actual printout position turns out to be an issue.
 

For instance, with this patch a numa_maps line would look like the following:

 7ff965200000 default file=/anon_hugepage\040(deleted) huge kernelpagesize_kB=2048 anon=5 dirty=5 N0=5


or it could look like this, if we decide to switch kernelpagesize_kB position to EOL,
for the sake of parsers:

 7ff965200000 default file=/anon_hugepage\040(deleted) huge anon=5 dirty=5 N0=5 kernelpagesize_kB=2048
 


Cheers,
-- Rafael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] fs: proc: task_mmu: show page size in /proc/<pid>/numa_maps

[Andrew Morton]

On Mon, 5 Jan 2015 19:21:35 -0500 Rafael Aquini <aquini@redhat.com> wrote:

> > > > 
> > > > 
> > > > > +	seq_printf(m, " kernelpagesize_kB=%lu", vma_kernel_pagesize(vma) >> 10);
> > > > 
> > > > This changes the format of the numa_maps file and can potentially break
> > > > existing parsers.  Please discuss.
> > 
> > ^^ ??
> >
> Sorry I overlooked it.
> 
> Parsers indeed would have to be adjusted to cope with an extra line element
> (they already have to do so, similarly, for the conditional 'huge' hint).
> Despite I don't think of it as a showstopper (as is), I think we can consider
> moving it to EOL, if its actual printout position turns out to be an issue.
>  
> 
> For instance, with this patch a numa_maps line would look like the following:
> 
>  7ff965200000 default file=/anon_hugepage\040(deleted) huge kernelpagesize_kB=2048 anon=5 dirty=5 N0=5
> 
> 
> or it could look like this, if we decide to switch kernelpagesize_kB position to EOL,
> for the sake of parsers:
> 
>  7ff965200000 default file=/anon_hugepage\040(deleted) huge anon=5 dirty=5 N0=5 kernelpagesize_kB=2048

hm, OK, numa_maps is actually a pretty complex thing: it's a
combination of `name' and `name=value'.  Some fields may be absent. 
Parsers need to fully parse each field and can't make assumptions based
on field number.

So yes, I agree that any parser which actually works is unlikely to
break due to this change.  However it's probably a bit safer to put the
new field at the end of the line.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>