[LSF/MM TOPIC] CMA and larger page sizes

[LSF/MM TOPIC] CMA and larger page sizes

[Laura Abbott]

CMA as it's currently designed requires alignment to the pageblock size c.f.

         /*
          * Sanitise input arguments.
          * Pages both ends in CMA area could be merged into adjacent unmovable
          * migratetype page by page allocator's buddy algorithm. In the case,
          * you couldn't get a contiguous memory, which is not what we want.
          */
         alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
                           max_t(unsigned long, MAX_ORDER - 1, pageblock_order));


On arm64 with 64K page size and transparent huge page, this gives an alignment
of 512MB. This is quite restrictive and can eat up significant portions of
memory on smaller memory targets. Adjusting the configuration options really
isn't ideal for distributions that aim to have a single image which runs on
all targets.

Approaches I've thought about:
- Making CMA alignment less restrictive (and dealing with the fallout from
the comment above)
- Command line option to force a reasonable alignment

There's been some interest in other CMA topics so this might go along well.

Thanks,
Laura

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Michal Hocko]

[Ccing Joonsoo]

On Fri 26-01-18 02:08:14, Laura Abbott wrote:
> CMA as it's currently designed requires alignment to the pageblock size c.f.
> 
>         /*
>          * Sanitise input arguments.
>          * Pages both ends in CMA area could be merged into adjacent unmovable
>          * migratetype page by page allocator's buddy algorithm. In the case,
>          * you couldn't get a contiguous memory, which is not what we want.
>          */
>         alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
>                           max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
> 
> 
> On arm64 with 64K page size and transparent huge page, this gives an alignment
> of 512MB. This is quite restrictive and can eat up significant portions of
> memory on smaller memory targets. Adjusting the configuration options really
> isn't ideal for distributions that aim to have a single image which runs on
> all targets.
> 
> Approaches I've thought about:
> - Making CMA alignment less restrictive (and dealing with the fallout from
> the comment above)
> - Command line option to force a reasonable alignment
> 
> There's been some interest in other CMA topics so this might go along well.
> 
> Thanks,
> Laura
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Joonsoo Kim]

Hello, Laura.
Sorry for a late response.


On Fri, Jan 26, 2018 at 06:25:27PM +0100, Michal Hocko wrote:
> [Ccing Joonsoo]

Thanks! Michal.

> 
> On Fri 26-01-18 02:08:14, Laura Abbott wrote:
> > CMA as it's currently designed requires alignment to the pageblock size c.f.
> > 
> >         /*
> >          * Sanitise input arguments.
> >          * Pages both ends in CMA area could be merged into adjacent unmovable
> >          * migratetype page by page allocator's buddy algorithm. In the case,
> >          * you couldn't get a contiguous memory, which is not what we want.
> >          */
> >         alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
> >                           max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
> > 
> > 
> > On arm64 with 64K page size and transparent huge page, this gives an alignment
> > of 512MB. This is quite restrictive and can eat up significant portions of
> > memory on smaller memory targets. Adjusting the configuration options really
> > isn't ideal for distributions that aim to have a single image which runs on
> > all targets.
> > 
> > Approaches I've thought about:
> > - Making CMA alignment less restrictive (and dealing with the fallout from
> > the comment above)
> > - Command line option to force a reasonable alignment

If the patchset 'manage the memory of the CMA area by using the ZONE_MOVABLE' is
merged, this restriction can be removed since there is no unmovable
pageblock in ZONE_MOVABLE. Just quick thought. :)

Thanks.

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Laura Abbott]

On 04/03/2018 10:11 PM, Joonsoo Kim wrote:
> Hello, Laura.
> Sorry for a late response.
> 
> 
> On Fri, Jan 26, 2018 at 06:25:27PM +0100, Michal Hocko wrote:
>> [Ccing Joonsoo]
> 
> Thanks! Michal.
> 
>>
>> On Fri 26-01-18 02:08:14, Laura Abbott wrote:
>>> CMA as it's currently designed requires alignment to the pageblock size c.f.
>>>
>>>          /*
>>>           * Sanitise input arguments.
>>>           * Pages both ends in CMA area could be merged into adjacent unmovable
>>>           * migratetype page by page allocator's buddy algorithm. In the case,
>>>           * you couldn't get a contiguous memory, which is not what we want.
>>>           */
>>>          alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
>>>                            max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
>>>
>>>
>>> On arm64 with 64K page size and transparent huge page, this gives an alignment
>>> of 512MB. This is quite restrictive and can eat up significant portions of
>>> memory on smaller memory targets. Adjusting the configuration options really
>>> isn't ideal for distributions that aim to have a single image which runs on
>>> all targets.
>>>
>>> Approaches I've thought about:
>>> - Making CMA alignment less restrictive (and dealing with the fallout from
>>> the comment above)
>>> - Command line option to force a reasonable alignment
> 
> If the patchset 'manage the memory of the CMA area by using the ZONE_MOVABLE' is
> merged, this restriction can be removed since there is no unmovable
> pageblock in ZONE_MOVABLE. Just quick thought. :)
> 
> Thanks.
> 

Thanks for that pointer. What's the current status of that patchset? Was that
one that needed more review/testing?

Thanks,
Laura

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Vlastimil Babka]

On 04/11/2018 09:55 PM, Laura Abbott wrote:
> On 04/03/2018 10:11 PM, Joonsoo Kim wrote:
>> If the patchset 'manage the memory of the CMA area by using the ZONE_MOVABLE' is
>> merged, this restriction can be removed since there is no unmovable
>> pageblock in ZONE_MOVABLE. Just quick thought. :)
>>
>> Thanks.
>>
> 
> Thanks for that pointer. What's the current status of that patchset? Was that
> one that needed more review/testing?

It was merged by Linus today, see around commit bad8c6c0b114 ("mm/cma:
manage the memory of the CMA area by using the ZONE_MOVABLE")

Congrats, Joonsoo :)

> Thanks,
> Laura
> 

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Laura Abbott]

On 04/11/2018 01:02 PM, Vlastimil Babka wrote:
> On 04/11/2018 09:55 PM, Laura Abbott wrote:
>> On 04/03/2018 10:11 PM, Joonsoo Kim wrote:
>>> If the patchset 'manage the memory of the CMA area by using the ZONE_MOVABLE' is
>>> merged, this restriction can be removed since there is no unmovable
>>> pageblock in ZONE_MOVABLE. Just quick thought. :)
>>>
>>> Thanks.
>>>
>>
>> Thanks for that pointer. What's the current status of that patchset? Was that
>> one that needed more review/testing?
> 
> It was merged by Linus today, see around commit bad8c6c0b114 ("mm/cma:
> manage the memory of the CMA area by using the ZONE_MOVABLE")
> 
> Congrats, Joonsoo :)
> 

I took a look at this a little bit more and while it's true we don't
have the unmovable restriction anymore, CMA is still tied to the pageblock
size (512MB) because we still have MIGRATE_CMA. I guess making the
pageblock smaller seems like the most plausible approach?

Thanks,
Laura

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Michal Hocko]

On Wed 11-04-18 18:06:59, Laura Abbott wrote:
> On 04/11/2018 01:02 PM, Vlastimil Babka wrote:
> > On 04/11/2018 09:55 PM, Laura Abbott wrote:
> > > On 04/03/2018 10:11 PM, Joonsoo Kim wrote:
> > > > If the patchset 'manage the memory of the CMA area by using the ZONE_MOVABLE' is
> > > > merged, this restriction can be removed since there is no unmovable
> > > > pageblock in ZONE_MOVABLE. Just quick thought. :)
> > > > 
> > > > Thanks.
> > > > 
> > > 
> > > Thanks for that pointer. What's the current status of that patchset? Was that
> > > one that needed more review/testing?
> > 
> > It was merged by Linus today, see around commit bad8c6c0b114 ("mm/cma:
> > manage the memory of the CMA area by using the ZONE_MOVABLE")
> > 
> > Congrats, Joonsoo :)
> > 
> 
> I took a look at this a little bit more and while it's true we don't
> have the unmovable restriction anymore, CMA is still tied to the pageblock
> size (512MB) because we still have MIGRATE_CMA. I guess making the
> pageblock smaller seems like the most plausible approach?

Maybe I am wrong but my take on what Joonsoo said is that we really do
not have to care about page blocks and MIGRATE_CMA because GFP_MOVABLE
can be allocated from that migrate type as it is by definition movable.
The size of the page block shouldn't matter.
-- 
Michal Hocko
SUSE Labs

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Vlastimil Babka]

On 04/12/2018 07:51 AM, Michal Hocko wrote:
> On Wed 11-04-18 18:06:59, Laura Abbott wrote:
>>
>> I took a look at this a little bit more and while it's true we don't
>> have the unmovable restriction anymore, CMA is still tied to the pageblock
>> size (512MB) because we still have MIGRATE_CMA. I guess making the
>> pageblock smaller seems like the most plausible approach?
> 
> Maybe I am wrong but my take on what Joonsoo said is that we really do
> not have to care about page blocks and MIGRATE_CMA because GFP_MOVABLE
> can be allocated from that migrate type as it is by definition movable.
> The size of the page block shouldn't matter.

Agree, CMA itself doesn't need mark pageblocks with MIGRATE_CMA anymore.
The only user is now hardened usercopy via check_page_span() ->
is_migrate_cma_page(). If we could give up the CMA check there (or
recognize CMA differently?), MIGRATE_CMA could be removed completely,
together with the pageblock alignment code.

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Michal Hocko]

On Fri 26-01-18 02:08:14, Laura Abbott wrote:
> CMA as it's currently designed requires alignment to the pageblock size c.f.
> 
>         /*
>          * Sanitise input arguments.
>          * Pages both ends in CMA area could be merged into adjacent unmovable
>          * migratetype page by page allocator's buddy algorithm. In the case,
>          * you couldn't get a contiguous memory, which is not what we want.
>          */
>         alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
>                           max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
> 
> 
> On arm64 with 64K page size and transparent huge page, this gives an alignment
> of 512MB. This is quite restrictive and can eat up significant portions of
> memory on smaller memory targets. Adjusting the configuration options really
> isn't ideal for distributions that aim to have a single image which runs on
> all targets.
> 
> Approaches I've thought about:
> - Making CMA alignment less restrictive (and dealing with the fallout from
> the comment above)
> - Command line option to force a reasonable alignment

Laura, are you still interested discussing this or other CMA related
topic?

-- 
Michal Hocko
SUSE Labs

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Laura Abbott]

On 04/17/2018 04:36 AM, Michal Hocko wrote:
> On Fri 26-01-18 02:08:14, Laura Abbott wrote:
>> CMA as it's currently designed requires alignment to the pageblock size c.f.
>>
>>          /*
>>           * Sanitise input arguments.
>>           * Pages both ends in CMA area could be merged into adjacent unmovable
>>           * migratetype page by page allocator's buddy algorithm. In the case,
>>           * you couldn't get a contiguous memory, which is not what we want.
>>           */
>>          alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
>>                            max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
>>
>>
>> On arm64 with 64K page size and transparent huge page, this gives an alignment
>> of 512MB. This is quite restrictive and can eat up significant portions of
>> memory on smaller memory targets. Adjusting the configuration options really
>> isn't ideal for distributions that aim to have a single image which runs on
>> all targets.
>>
>> Approaches I've thought about:
>> - Making CMA alignment less restrictive (and dealing with the fallout from
>> the comment above)
>> - Command line option to force a reasonable alignment
> 
> Laura, are you still interested discussing this or other CMA related
> topic?
> 

In light of Joonsoo's patches, I don't think we need a lot of time
but I'd still like some chance to discuss. I think there was some
other interest in CMA topics so it can be combined with those if
they are happening as well.

Thanks,
Laura

Re: [LSF/MM TOPIC] CMA and larger page sizes

[Michal Hocko]

On Tue 17-04-18 08:01:53, Laura Abbott wrote:
> On 04/17/2018 04:36 AM, Michal Hocko wrote:
> > On Fri 26-01-18 02:08:14, Laura Abbott wrote:
> > > CMA as it's currently designed requires alignment to the pageblock size c.f.
> > > 
> > >          /*
> > >           * Sanitise input arguments.
> > >           * Pages both ends in CMA area could be merged into adjacent unmovable
> > >           * migratetype page by page allocator's buddy algorithm. In the case,
> > >           * you couldn't get a contiguous memory, which is not what we want.
> > >           */
> > >          alignment = max(alignment,  (phys_addr_t)PAGE_SIZE <<
> > >                            max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
> > > 
> > > 
> > > On arm64 with 64K page size and transparent huge page, this gives an alignment
> > > of 512MB. This is quite restrictive and can eat up significant portions of
> > > memory on smaller memory targets. Adjusting the configuration options really
> > > isn't ideal for distributions that aim to have a single image which runs on
> > > all targets.
> > > 
> > > Approaches I've thought about:
> > > - Making CMA alignment less restrictive (and dealing with the fallout from
> > > the comment above)
> > > - Command line option to force a reasonable alignment
> > 
> > Laura, are you still interested discussing this or other CMA related
> > topic?
> > 
> 
> In light of Joonsoo's patches, I don't think we need a lot of time
> but I'd still like some chance to discuss. I think there was some
> other interest in CMA topics so it can be combined with those if
> they are happening as well.

OK, so I've put a placeholder for a CMA discussion. You have won the
lead on that session ;) We can change that later of course.
-- 
Michal Hocko
SUSE Labs