* [v4 1/4] ARM: dma-mapping: atomic_pool with struct page **pages
2012-08-28 5:13 [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Hiroshi Doyu
@ 2012-08-28 5:13 ` Hiroshi Doyu
2012-08-28 5:13 ` [v4 2/4] ARM: dma-mapping: Refactor out to introduce __in_atomic_pool Hiroshi Doyu
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Hiroshi Doyu @ 2012-08-28 5:13 UTC (permalink / raw)
To: m.szyprowski
Cc: Hiroshi Doyu, linux-arm-kernel, linaro-mm-sig, linux-mm,
linux-kernel, kyungmin.park, arnd, linux, chunsang.jeong, vdumpa,
subashrp, minchan, pullip.cho, konrad.wilk, linux-tegra
struct page **pages is necessary to align with non atomic path in
__iommu_get_pages(). atomic_pool() has the intialized **pages instead
of just *page.
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
---
arch/arm/mm/dma-mapping.c | 17 ++++++++++++++---
1 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 1123808..51d5e2b 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -296,7 +296,7 @@ struct dma_pool {
unsigned long *bitmap;
unsigned long nr_pages;
void *vaddr;
- struct page *page;
+ struct page **pages;
};
static struct dma_pool atomic_pool = {
@@ -335,6 +335,7 @@ static int __init atomic_pool_init(void)
unsigned long nr_pages = pool->size >> PAGE_SHIFT;
unsigned long *bitmap;
struct page *page;
+ struct page **pages;
void *ptr;
int bitmap_size = BITS_TO_LONGS(nr_pages) * sizeof(long);
@@ -342,21 +343,31 @@ static int __init atomic_pool_init(void)
if (!bitmap)
goto no_bitmap;
+ pages = kzalloc(nr_pages * sizeof(struct page *), GFP_KERNEL);
+ if (!pages)
+ goto no_pages;
+
if (IS_ENABLED(CONFIG_CMA))
ptr = __alloc_from_contiguous(NULL, pool->size, prot, &page);
else
ptr = __alloc_remap_buffer(NULL, pool->size, GFP_KERNEL, prot,
&page, NULL);
if (ptr) {
+ int i;
+
+ for (i = 0; i < nr_pages; i++)
+ pages[i] = page + i;
+
spin_lock_init(&pool->lock);
pool->vaddr = ptr;
- pool->page = page;
+ pool->pages = pages;
pool->bitmap = bitmap;
pool->nr_pages = nr_pages;
pr_info("DMA: preallocated %u KiB pool for atomic coherent allocations\n",
(unsigned)pool->size / 1024);
return 0;
}
+no_pages:
kfree(bitmap);
no_bitmap:
pr_err("DMA: failed to allocate %u KiB pool for atomic coherent allocation\n",
@@ -481,7 +492,7 @@ static void *__alloc_from_pool(size_t size, struct page **ret_page)
if (pageno < pool->nr_pages) {
bitmap_set(pool->bitmap, pageno, count);
ptr = pool->vaddr + PAGE_SIZE * pageno;
- *ret_page = pool->page + pageno;
+ *ret_page = pool->pages[pageno];
} else {
pr_err_once("ERROR: %u KiB atomic DMA coherent pool is too small!\n"
"Please increase it with coherent_pool= kernel parameter!\n",
--
1.7.5.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* [v4 2/4] ARM: dma-mapping: Refactor out to introduce __in_atomic_pool
2012-08-28 5:13 [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Hiroshi Doyu
2012-08-28 5:13 ` [v4 1/4] ARM: dma-mapping: atomic_pool with struct page **pages Hiroshi Doyu
@ 2012-08-28 5:13 ` Hiroshi Doyu
2012-08-28 5:13 ` [v4 3/4] ARM: dma-mapping: Introduce __atomic_get_pages() for __iommu_get_pages() Hiroshi Doyu
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Hiroshi Doyu @ 2012-08-28 5:13 UTC (permalink / raw)
To: m.szyprowski
Cc: Hiroshi Doyu, linux-arm-kernel, linaro-mm-sig, linux-mm,
linux-kernel, kyungmin.park, arnd, linux, chunsang.jeong, vdumpa,
subashrp, minchan, pullip.cho, konrad.wilk, linux-tegra
Check the given range("start", "size") is included in "atomic_pool" or not.
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
---
arch/arm/mm/dma-mapping.c | 26 ++++++++++++++++++++------
1 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 51d5e2b..a62f552 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -503,20 +503,34 @@ static void *__alloc_from_pool(size_t size, struct page **ret_page)
return ptr;
}
+static bool __in_atomic_pool(void *start, size_t size)
+{
+ struct dma_pool *pool = &atomic_pool;
+ void *end = start + size;
+ void *pool_start = pool->vaddr;
+ void *pool_end = pool->vaddr + pool->size;
+
+ if (start < pool_start || start > pool_end)
+ return false;
+
+ if (end <= pool_end)
+ return true;
+
+ WARN(1, "Wrong coherent size(%p-%p) from atomic pool(%p-%p)\n",
+ start, end - 1, pool_start, pool_end - 1);
+
+ return false;
+}
+
static int __free_from_pool(void *start, size_t size)
{
struct dma_pool *pool = &atomic_pool;
unsigned long pageno, count;
unsigned long flags;
- if (start < pool->vaddr || start > pool->vaddr + pool->size)
+ if (!__in_atomic_pool(start, size))
return 0;
- if (start + size > pool->vaddr + pool->size) {
- WARN(1, "freeing wrong coherent size from pool\n");
- return 0;
- }
-
pageno = (start - pool->vaddr) >> PAGE_SHIFT;
count = size >> PAGE_SHIFT;
--
1.7.5.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* [v4 3/4] ARM: dma-mapping: Introduce __atomic_get_pages() for __iommu_get_pages()
2012-08-28 5:13 [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Hiroshi Doyu
2012-08-28 5:13 ` [v4 1/4] ARM: dma-mapping: atomic_pool with struct page **pages Hiroshi Doyu
2012-08-28 5:13 ` [v4 2/4] ARM: dma-mapping: Refactor out to introduce __in_atomic_pool Hiroshi Doyu
@ 2012-08-28 5:13 ` Hiroshi Doyu
2012-08-28 5:13 ` [v4 4/4] ARM: dma-mapping: IOMMU allocates pages from atomic_pool with GFP_ATOMIC Hiroshi Doyu
2012-08-28 16:51 ` [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Marek Szyprowski
4 siblings, 0 replies; 6+ messages in thread
From: Hiroshi Doyu @ 2012-08-28 5:13 UTC (permalink / raw)
To: m.szyprowski
Cc: Hiroshi Doyu, linux-arm-kernel, linaro-mm-sig, linux-mm,
linux-kernel, kyungmin.park, arnd, linux, chunsang.jeong, vdumpa,
subashrp, minchan, pullip.cho, konrad.wilk, linux-tegra
Support atomic allocation in __iommu_get_pages().
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
---
arch/arm/mm/dma-mapping.c | 12 ++++++++++++
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index a62f552..4ef2d7b 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -503,6 +503,15 @@ static void *__alloc_from_pool(size_t size, struct page **ret_page)
return ptr;
}
+static struct page **__atomic_get_pages(void *addr)
+{
+ struct dma_pool *pool = &atomic_pool;
+ struct page **pages = pool->pages;
+ int offs = (addr - pool->vaddr) >> PAGE_SHIFT;
+
+ return pages + offs;
+}
+
static bool __in_atomic_pool(void *start, size_t size)
{
struct dma_pool *pool = &atomic_pool;
@@ -1288,6 +1297,9 @@ static struct page **__iommu_get_pages(void *cpu_addr, struct dma_attrs *attrs)
{
struct vm_struct *area;
+ if (__in_atomic_pool(cpu_addr, PAGE_SIZE))
+ return __atomic_get_pages(cpu_addr);
+
if (dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs))
return cpu_addr;
--
1.7.5.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* [v4 4/4] ARM: dma-mapping: IOMMU allocates pages from atomic_pool with GFP_ATOMIC
2012-08-28 5:13 [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Hiroshi Doyu
` (2 preceding siblings ...)
2012-08-28 5:13 ` [v4 3/4] ARM: dma-mapping: Introduce __atomic_get_pages() for __iommu_get_pages() Hiroshi Doyu
@ 2012-08-28 5:13 ` Hiroshi Doyu
2012-08-28 16:51 ` [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Marek Szyprowski
4 siblings, 0 replies; 6+ messages in thread
From: Hiroshi Doyu @ 2012-08-28 5:13 UTC (permalink / raw)
To: m.szyprowski
Cc: Hiroshi Doyu, linux-arm-kernel, linaro-mm-sig, linux-mm,
linux-kernel, kyungmin.park, arnd, linux, chunsang.jeong, vdumpa,
subashrp, minchan, pullip.cho, konrad.wilk, linux-tegra
Make use of the same atomic pool as DMA does, and skip a kernel page
mapping which can involve sleep'able operations at allocating a kernel
page table.
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
---
arch/arm/mm/dma-mapping.c | 36 ++++++++++++++++++++++++++++++++++++
1 files changed, 36 insertions(+), 0 deletions(-)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 4ef2d7b..f5024f9 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -1309,6 +1309,34 @@ static struct page **__iommu_get_pages(void *cpu_addr, struct dma_attrs *attrs)
return NULL;
}
+static void *__iommu_alloc_atomic(struct device *dev, size_t size,
+ dma_addr_t *handle)
+{
+ struct page *page;
+ void *addr;
+
+ addr = __alloc_from_pool(size, &page);
+ if (!addr)
+ return NULL;
+
+ *handle = __iommu_create_mapping(dev, &page, size);
+ if (*handle == DMA_ERROR_CODE)
+ goto err_mapping;
+
+ return addr;
+
+err_mapping:
+ __free_from_pool(addr, size);
+ return NULL;
+}
+
+static void __iommu_free_atomic(struct device *dev, struct page **pages,
+ dma_addr_t handle, size_t size)
+{
+ __iommu_remove_mapping(dev, handle, size);
+ __free_from_pool(page_address(pages[0]), size);
+}
+
static void *arm_iommu_alloc_attrs(struct device *dev, size_t size,
dma_addr_t *handle, gfp_t gfp, struct dma_attrs *attrs)
{
@@ -1326,6 +1354,9 @@ static void *arm_iommu_alloc_attrs(struct device *dev, size_t size,
*handle = DMA_ERROR_CODE;
size = PAGE_ALIGN(size);
+ if (gfp & GFP_ATOMIC)
+ return __iommu_alloc_atomic(dev, size, handle);
+
pages = __iommu_alloc_buffer(dev, size, gfp);
if (!pages)
return NULL;
@@ -1392,6 +1423,11 @@ void arm_iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
return;
}
+ if (__in_atomic_pool(cpu_addr, size)) {
+ __iommu_free_atomic(dev, pages, handle, size);
+ return;
+ }
+
if (!dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs)) {
unmap_kernel_range((unsigned long)cpu_addr, size);
vunmap(cpu_addr);
--
1.7.5.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation
2012-08-28 5:13 [v4 0/4] ARM: dma-mapping: IOMMU atomic allocation Hiroshi Doyu
` (3 preceding siblings ...)
2012-08-28 5:13 ` [v4 4/4] ARM: dma-mapping: IOMMU allocates pages from atomic_pool with GFP_ATOMIC Hiroshi Doyu
@ 2012-08-28 16:51 ` Marek Szyprowski
4 siblings, 0 replies; 6+ messages in thread
From: Marek Szyprowski @ 2012-08-28 16:51 UTC (permalink / raw)
To: Hiroshi Doyu
Cc: linux-arm-kernel, linaro-mm-sig, linux-mm, linux-kernel,
kyungmin.park, arnd, linux, chunsang.jeong, vdumpa, subashrp,
minchan, pullip.cho, konrad.wilk, linux-tegra
Hello,
On 8/28/2012 7:13 AM, Hiroshi Doyu wrote:
> The commit e9da6e9 "ARM: dma-mapping: remove custom consistent dma
> region" breaks the compatibility with existing drivers. This causes
> the following kernel oops(*1). That driver has called dma_pool_alloc()
> to allocate memory from the interrupt context, and it hits
> BUG_ON(in_interrpt()) in "get_vm_area_caller()". This patch seris
> fixes this problem with making use of the pre-allocate atomic memory
> pool which DMA is using in the same way as DMA does now.
>
> Any comment would be really appreciated.
Looks fine now. I will do some tests and apply them to my fixes-for-3.6
branch. Thanks for Your contribution!
Best regards
--
Marek Szyprowski
Samsung Poland R&D Center
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread