PageBlocks and Migrate Types

PageBlocks and Migrate Types

[Pankaj Suryawanshi]

[-- Attachment #1: Type: text/plain, Size: 2374 bytes --]

Hello,

1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?
How many movable/unmovable pages are defined by default?
With 2GB of memory, how many movable and unmovable pages are defined by
default ?
Example - cat /proc/pagetypeinfo :- Kernel Version - 4.14 CPU Architecture
- ARM 32-bit

Page block order: 12
Pages per block:  4096

Free pages count per migrate type at order       0      1      2      3
 4      5      6      7      8      9     10     11     12
Node    0, zone      DMA, type    Unmovable     71    235    137     30
17      7      2      0      0      1      1      0      0
Node    0, zone      DMA, type      Movable      0      1      1      0
 1      0      0      1      0      1      0      3      9
Node    0, zone      DMA, type  Reclaimable      0      0      1      2
 0      0      0      0      1      1      1      1      0
Node    0, zone      DMA, type   HighAtomic      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone      DMA, type          CMA      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone      DMA, type      Isolate      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone  HighMem, type    Unmovable     55     47     16      8
 6      2      1      0      0      0      0      1      0
Node    0, zone  HighMem, type      Movable      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone  HighMem, type  Reclaimable      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone  HighMem, type   HighAtomic      0      0      0      0
 0      0      0      0      0      0      0      0      0
Node    0, zone  HighMem, type          CMA    261    915    778    204
48     20      5      0      0      0      0      1     13
Node    0, zone  HighMem, type      Isolate      0      0      0      0
 0      0      0      0      0      0      0      0      0


Number of blocks type     Unmovable      Movable  Reclaimable   HighAtomic
         CMA      Isolate

Node 0, zone      DMA           10           14            4            0
         0            0
Node 0, zone  HighMem            3           32            0            0
        57            0


Regards,
Pankaj

[-- Attachment #2: Type: text/html, Size: 3298 bytes --]

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> Hello,
> 
> 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?

Pageblocks are a simple grouping of physically contiguous pages with
common set of flags. I haven't checked closely recently so I might
misremember but my recollection is that only the migrate type is stored
there. Normally we would store that information into page flags but
there is not enough room there.

MIGRATE_CMA represent pages allocated for the CMA allocator. There are
other migrate types denoting unmovable/movable allocations or pages that
are isolated from the page allocator.

Very broadly speaking, the migrate type groups pages with similar
movability properties to reduce fragmentation that compaction cannot
do anything about because there are objects of different properties
around. Please note that pageblock might contain objects of a different
migrate type in some cases (e.g. low on memory).

Have a look at gfpflags_to_migratetype and how the gfp mask is converted
to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
to see how it is used in the code.

> How many movable/unmovable pages are defined by default?

There is nothing like that. It depends on how many objects of a specific
type are allocated.

HTH
-- 
Michal Hocko
SUSE Labs

Re: PageBlocks and Migrate Types

[Pankaj Suryawanshi]

On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
>
> On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > Hello,
> >
> > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?
>
> Pageblocks are a simple grouping of physically contiguous pages with
> common set of flags. I haven't checked closely recently so I might
> misremember but my recollection is that only the migrate type is stored
> there. Normally we would store that information into page flags but
> there is not enough room there.
>
> MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> other migrate types denoting unmovable/movable allocations or pages that
> are isolated from the page allocator.
>
> Very broadly speaking, the migrate type groups pages with similar
> movability properties to reduce fragmentation that compaction cannot
> do anything about because there are objects of different properti
> around. Please note that pageblock might contain objects of a different
> migrate type in some cases (e.g. low on memory).
>
> Have a look at gfpflags_to_migratetype and how the gfp mask is converted
> to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
> to see how it is used in the code.
>
> > How many movable/unmovable pages are defined by default?
>
> There is nothing like that. It depends on how many objects of a specific
> type are allocated.


It means that it started creating pageblocks after allocation of
different objects, but from which block it allocate initially when
there is nothing like pageblocks ? (when memory subsystem up)

Pageblocks and its type dynamically changes ?
>
>
> HTH
> --
> Michal Hocko
> SUSE Labs

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Thu 22-08-19 23:54:19, Pankaj Suryawanshi wrote:
> On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> >
> > On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > > Hello,
> > >
> > > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?
> >
> > Pageblocks are a simple grouping of physically contiguous pages with
> > common set of flags. I haven't checked closely recently so I might
> > misremember but my recollection is that only the migrate type is stored
> > there. Normally we would store that information into page flags but
> > there is not enough room there.
> >
> > MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> > other migrate types denoting unmovable/movable allocations or pages that
> > are isolated from the page allocator.
> >
> > Very broadly speaking, the migrate type groups pages with similar
> > movability properties to reduce fragmentation that compaction cannot
> > do anything about because there are objects of different properti
> > around. Please note that pageblock might contain objects of a different
> > migrate type in some cases (e.g. low on memory).
> >
> > Have a look at gfpflags_to_migratetype and how the gfp mask is converted
> > to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
> > to see how it is used in the code.
> >
> > > How many movable/unmovable pages are defined by default?
> >
> > There is nothing like that. It depends on how many objects of a specific
> > type are allocated.
> 
> 
> It means that it started creating pageblocks after allocation of
> different objects, but from which block it allocate initially when
> there is nothing like pageblocks ? (when memory subsystem up)

Pageblocks are just a way to group physically contiguous pages. They
just exist along with the physically contiguous memory. The migrate type
for most of the memory is set to MIGRATE_MOVABLE. Portion of the memory
might be reserved by CMA then that memory has MIGRATE_CMA. Following
set_pageblock_migratetype call paths will give you a good picture. 

> Pageblocks and its type dynamically changes ?

Yes as the purpose of the underlying memory for the block changes.
-- 
Michal Hocko
SUSE Labs

Re: PageBlocks and Migrate Types

[Pankaj Suryawanshi]

[-- Attachment #1: Type: text/plain, Size: 2603 bytes --]

On Mon, Aug 26, 2019 at 12:34 PM Michal Hocko <mhocko@kernel.org> wrote:
>
> On Thu 22-08-19 23:54:19, Pankaj Suryawanshi wrote:
> > On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> > >
> > > On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > > > Hello,
> > > >
> > > > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux
memory ?
> > >
> > > Pageblocks are a simple grouping of physically contiguous pages with
> > > common set of flags. I haven't checked closely recently so I might
> > > misremember but my recollection is that only the migrate type is
stored
> > > there. Normally we would store that information into page flags but
> > > there is not enough room there.
> > >
> > > MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> > > other migrate types denoting unmovable/movable allocations or pages
that
> > > are isolated from the page allocator.
> > >
> > > Very broadly speaking, the migrate type groups pages with similar
> > > movability properties to reduce fragmentation that compaction cannot
> > > do anything about because there are objects of different properti
> > > around. Please note that pageblock might contain objects of a
different
> > > migrate type in some cases (e.g. low on memory).
> > >
> > > Have a look at gfpflags_to_migratetype and how the gfp mask is
converted
> > > to a migratetype for the allocation. Also follow different
MIGRATE_$TYPE
> > > to see how it is used in the code.
> > >
> > > > How many movable/unmovable pages are defined by default?
> > >
> > > There is nothing like that. It depends on how many objects of a
specific
> > > type are allocated.
> >
> >
> > It means that it started creating pageblocks after allocation of
> > different objects, but from which block it allocate initially when
> > there is nothing like pageblocks ? (when memory subsystem up)
>
> Pageblocks are just a way to group physically contiguous pages. They
> just exist along with the physically contiguous memory. The migrate type
> for most of the memory is set to MIGRATE_MOVABLE. Portion of the memory
> might be reserved by CMA then that memory has MIGRATE_CMA. Following
> set_pageblock_migratetype call paths will give you a good picture.

it means if i have 4096 continuous pages = 1 pageblock
then all the 4096 pages of same type. but if any one page is different than
block type then ? it changed the block type or something else ?
>
>
> > Pageblocks and its type dynamically changes ?
>
> Yes as the purpose of the underlying memory for the block changes.
okay
>
> --
> Michal Hocko
> SUSE Labs

[-- Attachment #2: Type: text/html, Size: 3280 bytes --]

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Mon 26-08-19 22:35:08, Pankaj Suryawanshi wrote:
> On Mon, Aug 26, 2019 at 12:34 PM Michal Hocko <mhocko@kernel.org> wrote:
> >
> > On Thu 22-08-19 23:54:19, Pankaj Suryawanshi wrote:
> > > On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> > > >
> > > > On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > > > > Hello,
> > > > >
> > > > > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux
> memory ?
> > > >
> > > > Pageblocks are a simple grouping of physically contiguous pages with
> > > > common set of flags. I haven't checked closely recently so I might
> > > > misremember but my recollection is that only the migrate type is
> stored
> > > > there. Normally we would store that information into page flags but
> > > > there is not enough room there.
> > > >
> > > > MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> > > > other migrate types denoting unmovable/movable allocations or pages
> that
> > > > are isolated from the page allocator.
> > > >
> > > > Very broadly speaking, the migrate type groups pages with similar
> > > > movability properties to reduce fragmentation that compaction cannot
> > > > do anything about because there are objects of different properti
> > > > around. Please note that pageblock might contain objects of a
> different
> > > > migrate type in some cases (e.g. low on memory).
> > > >
> > > > Have a look at gfpflags_to_migratetype and how the gfp mask is
> converted
> > > > to a migratetype for the allocation. Also follow different
> MIGRATE_$TYPE
> > > > to see how it is used in the code.
> > > >
> > > > > How many movable/unmovable pages are defined by default?
> > > >
> > > > There is nothing like that. It depends on how many objects of a
> specific
> > > > type are allocated.
> > >
> > >
> > > It means that it started creating pageblocks after allocation of
> > > different objects, but from which block it allocate initially when
> > > there is nothing like pageblocks ? (when memory subsystem up)
> >
> > Pageblocks are just a way to group physically contiguous pages. They
> > just exist along with the physically contiguous memory. The migrate type
> > for most of the memory is set to MIGRATE_MOVABLE. Portion of the memory
> > might be reserved by CMA then that memory has MIGRATE_CMA. Following
> > set_pageblock_migratetype call paths will give you a good picture.
> 
> it means if i have 4096 continuous pages = 1 pageblock
> then all the 4096 pages of same type. but if any one page is different than
> block type then ? it changed the block type or something else ?

That really depends on the specific migrate type. CMA, ISOLATE migrate
types are all or nothing IIRC. I would have to check the code to tell
exactly when MOVABLE/UNMOVABLE pageblocks transitions are done.
steal_suitable_fallback sounds like a good start to look at.
-- 
Michal Hocko
SUSE Labs