Re: JPEG image creation to display physical page state and memory fragmentation

Re: JPEG image creation to display physical page state and memory fragmentation

[Pintu Agarwal]

[-- Attachment #1: Type: text/plain, Size: 5801 bytes --]

                            
Dear All,
 
I wanted to represent the physical page state of memory in the form of an image.
The physical page state information such as free pages, movable pages, reclaimable pages are taken from /proc/pagetypeinfo.
(I know the buddyinfo and pagetypeinfo data may not be accurate/insufficient but that is ok for me)
 
These data will be used to create a YUV image and then can be converted to a jpeg image.
The image should clearly display the following(for each zone):-
- total free pages available --> in grey color
- total movable pages        --> in Red color- total reclaimable pages    --> in Blue color
Here each page will be represented by a pixel dot of different color.
A separate image will be created for each zone.
I know how to create a raw YUV image, but I am confused how to prepare an image layout based on the page-order and the total free page available.
It could be as similar as the anti-fragmentation page distribution image from below article from Mel Gorman.
http://lwn.net/Articles/224835/
http://www.skynet.ie/~mel/anti-frag/2007-02-28/page_type_distribution.jpg
But I am not able to understand how this image is formed.
From these image I am not able to find how many order-n pages are free, movable and reclaimable.
In the above image the block size is 32x32 and the image layout is (32x32) x (32x32) = 1024x1024
Also each pixel represent one page in memory.
But I am not able to understand how this pixels are filled into the blocks.
 
I wanted to create an image to show memory fragmentation under each zone.
If anybody has done similar kind of work please let me know.
 
I have already developed a sample utility in linux to create a dummy yuv420 image but could not able to figure out how to fill the pixels into the image.
My yuv program takes input as totalfreepages, totalmovablepages, totalreclaimablepages.
 
My system details are as folows:
- linux mobile running on ARM Cortex-A9.
- total available RAM = 888MB
- Page size = 4K
- Page block size = 1024 (from pagetypeinfo)
- page-order (0 to 10)    (from buddyinfo or pagetypeinfo)
 
 
Thanks, Regards,
Pintu
 

From: Mel Gorman <mgorman@suse.de>
To: Pintu Agarwal <pintu_agarwal@yahoo.com>
Sent: Friday, 1 July 2011 12:21 PM
Subject: Re: JPEG image creation for memory fragmentation

On Thu, Jun 30, 2011 at 08:02:15AM -0700, Pintu Agarwal wrote:

> I need to create a sample jpeg image for displaying page states
> and fragmentation info across each zone by collecting data from
> /proc/buddyinfo and /proc/pagetypeinfo a one instance of time.
>
> I saw your article about anti-fragmentation and page type distribution in the form of image from the following links:-
> http://lwn.net/Articles/224835/
> http://www.skynet.ie/~mel/anti-frag/2007-02-28/page_type_distribution.jpg
>  
> I wanted to understand, I to create the similar image using my data
> from buddyinfo and pagetypeinfo in my case.

There is insufficient data from buddyinfo and pagetypeinfo to generate
the image linked above.

> What logic you used for creating these images?

I used a perl script reading information via a kernel module and
creating the image with libgd. If I was doing it again, I would
read the necessary information from /proc/kpageflags. If I needed
a greater degree of accuracy, I would patch the kernel to expand on
the information available in /proc/kpageflags.

> How did you use the buddyinfo data to form this image?

I didn't use buddyinfo.

> Can you give some details about it?
>  
> Like your aticle I am also defining the following:-
> - Each block size is 1024
> - Each page is one pixel
> - Free page will be gray color
> - Movable page will be green color
> - Reclaimable page will be blue color
> - And (I dont want zoneboundary, percpu, pinned pages etc in the image)
>  
> Now suppose my system page state at some point of time looks like this:-
> -----------------------------------------------------------------------------------------
> Node:1, Zone:DMA
> Order    FreePages       MovablePages    ReclaimablePages        Fragmentation[%]
>   0         38               36                 1                         0%
>   1         59               55                 2                         0%
>   2         40               32                 1                         2%
>   3         19               16                 0                         4%
>   4         12               10                 1                         6%
>   5          4                2                 1                         9%
>   6          6                4                 1                        11%
>   7          7                4                 0                        16%
>   8          7                0                 1                        29%
>   9          2                1                 0                        55%
>  10          2                2                 0                        70%
> TotalFreePages: 6932
> TotalMovablePages: 3954
> TotalReclaimablePages: 377
> Overall Fragmentation: 18%
> -----------------------------------------------------------------------------------------
> How to represent this in the form of a jpeg image?

There is insufficient data there to generate the same jpeg but I doubt
it'd be very helpful anyway. Generating the images was a handy way of
validating anti-fragmentation was working as expected and convincing
people that it was behaving in the right way but I never used it to
formally evaluate how well fragmentation was being handled. For that
I used unusable free space index, a fragmentation index, allocation
latencies and success rates for allocating huge pages.

-- 
Mel Gorman
SUSE Labs

[-- Attachment #2: Type: text/html, Size: 14076 bytes --]