Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[Michal Hocko]

[CC linux-mm and some usual suspects]

On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
> CMA region by user space will cause an error on our ARM 3.18 kernel
> platform with a 32MB CMA.
> 
> It seems that the malloc in CMA region will be aligned to
> CMA_ALIGNMENT everytime even if the requested malloc size is very
> small so the CMA region is not available after the malloc operations.
> 
> Is there any configuraiton that can change this behavior??
> 
> Thanks
> 
> Cheers
> Ken

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[lunar12 lunartwix]

2016-05-18 16:48 GMT+08:00 Michal Hocko <mhocko@kernel.org>:
> [CC linux-mm and some usual suspects]
>
> On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
>> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
>> CMA region by user space will cause an error on our ARM 3.18 kernel
>> platform with a 32MB CMA.
>>
>> It seems that the malloc in CMA region will be aligned to
>> CMA_ALIGNMENT everytime even if the requested malloc size is very
>> small so the CMA region is not available after the malloc operations.
>>
>> Is there any configuraiton that can change this behavior??
>>
>> Thanks
>>
>> Cheers
>> Ken
>
> --
> Michal Hocko
> SUSE Labs

Update more information and any comment would be very appreciated

CMA region (from boot message):
Reserved memory: created CMA memory pool at 0x22e00000, size 80 MiB

User space test program:

    do
    {

        addr = malloc(2*1024);
        memset((void *)addr,2*1024,0x5A);
        vaddr=(unsigned int)addr;

        //get_user_page & page_to_phys in kernel
        ioctl(devfd, IOCTL_MSYS_USER_TO_PHYSICAL, &addr)

        count++;
        paddr=(unsigned int)addr;

        if(paddr>0x22E00000)
        {
            printf("USR:0x%08X 0x%08X %d\n",vaddr,paddr,count);
        }
    } while(addr!=NULL);


System print out:

USR:0x0164B248 0x27C00000 11337
USR:0x0164BA50 0x27C00000 11338
USR:0x0164C258 0x27800000 11339
USR:0x0164CA60 0x27800000 11340
USR:0x0164D268 0x27600000 11341
USR:0x0164DA70 0x27600000 11342
USR:0x0164E278 0x27400000 11343
USR:0x0164EA80 0x27400000 11344
USR:0x0164F288 0x27200000 11345
USR:0x0164FA90 0x27200000 11346
....
It seems that an 2MB CMA would be occpuied every 2 malloc()

Cheers
Ken

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[Joonsoo Kim]

On Wed, May 18, 2016 at 09:15:13PM +0800, lunar12 lunartwix wrote:
> 2016-05-18 16:48 GMT+08:00 Michal Hocko <mhocko@kernel.org>:
> > [CC linux-mm and some usual suspects]

Michal, Thanks.

> >
> > On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
> >> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
> >> CMA region by user space will cause an error on our ARM 3.18 kernel
> >> platform with a 32MB CMA.
> >>
> >> It seems that the malloc in CMA region will be aligned to
> >> CMA_ALIGNMENT everytime even if the requested malloc size is very
> >> small so the CMA region is not available after the malloc operations.
> >>
> >> Is there any configuraiton that can change this behavior??
> >>
> >> Thanks
> >>
> >> Cheers
> >> Ken
> >
> > --
> > Michal Hocko
> > SUSE Labs
> 
> Update more information and any comment would be very appreciated
> 
> CMA region (from boot message):
> Reserved memory: created CMA memory pool at 0x22e00000, size 80 MiB
> 
> User space test program:
> 
>     do
>     {
> 
>         addr = malloc(2*1024);
>         memset((void *)addr,2*1024,0x5A);
>         vaddr=(unsigned int)addr;
> 
>         //get_user_page & page_to_phys in kernel
>         ioctl(devfd, IOCTL_MSYS_USER_TO_PHYSICAL, &addr)
> 
>         count++;
>         paddr=(unsigned int)addr;
> 
>         if(paddr>0x22E00000)
>         {
>             printf("USR:0x%08X 0x%08X %d\n",vaddr,paddr,count);
>         }
>     } while(addr!=NULL);
> 
> 
> System print out:
> 
> USR:0x0164B248 0x27C00000 11337
> USR:0x0164BA50 0x27C00000 11338
> USR:0x0164C258 0x27800000 11339
> USR:0x0164CA60 0x27800000 11340
> USR:0x0164D268 0x27600000 11341
> USR:0x0164DA70 0x27600000 11342
> USR:0x0164E278 0x27400000 11343
> USR:0x0164EA80 0x27400000 11344
> USR:0x0164F288 0x27200000 11345
> USR:0x0164FA90 0x27200000 11346
> ....
> It seems that an 2MB CMA would be occpuied every 2 malloc()

I'm not familiar with device part of CMA but try to analyze.

Above output means that your device maps 2 MB CMA mem to 1 page. I guess
that your device requires such alignment. Could you check
CONFIG_CMA_ALIGNMENT? And, insert to log to below snippet
in drivers/base/dma-contiguous.c to check your device align requirement?

struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
                                       unsigned int align)
{                                                                                          
        if (align > CONFIG_CMA_ALIGNMENT)
                align = CONFIG_CMA_ALIGNMENT;
        return cma_alloc(dev_get_cma_area(dev), count, align);
}

I guess changing CONFIG_CMA_ALIGNMENT works for you, but, since it
ignore your device align requirement, I'm not sure that it is right solution.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>