mmap sync issue

mmap sync issue

[Gil Weber]

Hello,
I am experiencing an issue with my device driver. I am using mmap and ioctl to share information with my user space application.
The thing is that the shared memory does not seems to be synced. Do check this, I have done a simple test:

int fd = open("/dev/test", O_RDWR | O_SYNC);
int * addr = mmap(0, 4, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	
for (i=0 ; i<100 ; i++ )
{
	*addr = i;
	ioctl(fd, 0, 0);
}


In my device driver, the only thing I do in ioctl is to display the content of the shared memory, and here is the result:

[ 5158.967000] Value : 0
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.967000] Value : 1
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
[ 5158.968000] Value : 11
...


So, clearly, memory is not synced...
Here is the code in my device driver:

static int test_open(struct inode *inode, struct file *filp)
{
	return 0;
}

static int test_release(struct inode *inode, struct file *filp)
{
	return 0;
}

static int test_mmap(struct file *filp, struct vm_area_struct *vma)
{
	int ret;
	unsigned long start = vma->vm_start;
	unsigned long pfn;

	pfn = vmalloc_to_pfn(vmalloc_area);
	if ((ret = remap_pfn_range(vma, start, pfn, PAGE_SIZE, PAGE_SHARED)) < 0) {
		return ret;
	}
	
	return 0;
}

static long test_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
	printk (KERN_INFO "Value : %d\n", vmalloc_area[0]);
	return 0;
}

static struct file_operations test_fops = {
	.open = test_open,
	.release = test_release,
	.mmap = test_mmap,
	.unlocked_ioctl = test_ioctl,
	.owner = THIS_MODULE,
};

This is done on an arm architecture (AT91 SAM9X5) with a kernel 3.5.
I have done the test, with the same code, on a powerpc target, with a kernel 2.6.27, and it seems to work (but maybe by chance?)
Am I missing something?
Maybe I need to implement the sync function in file operations, but in that case, how can I know that all mapped memory is synced?

Thanks in advance,
Gil Weber

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mmap sync issue

[Will Huck]

Cc experts
On 03/15/2013 07:39 PM, Gil Weber wrote:
> Hello,
> I am experiencing an issue with my device driver. I am using mmap and ioctl to share information with my user space application.
> The thing is that the shared memory does not seems to be synced. Do check this, I have done a simple test:
>
> int fd = open("/dev/test", O_RDWR | O_SYNC);
> int * addr = mmap(0, 4, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
> 	
> for (i=0 ; i<100 ; i++ )
> {
> 	*addr = i;
> 	ioctl(fd, 0, 0);
> }
>
>
> In my device driver, the only thing I do in ioctl is to display the content of the shared memory, and here is the result:
>
> [ 5158.967000] Value : 0
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.967000] Value : 1
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> [ 5158.968000] Value : 11
> ...
>
>
> So, clearly, memory is not synced...
> Here is the code in my device driver:
>
> static int test_open(struct inode *inode, struct file *filp)
> {
> 	return 0;
> }
>
> static int test_release(struct inode *inode, struct file *filp)
> {
> 	return 0;
> }
>
> static int test_mmap(struct file *filp, struct vm_area_struct *vma)
> {
> 	int ret;
> 	unsigned long start = vma->vm_start;
> 	unsigned long pfn;
>
> 	pfn = vmalloc_to_pfn(vmalloc_area);
> 	if ((ret = remap_pfn_range(vma, start, pfn, PAGE_SIZE, PAGE_SHARED)) < 0) {
> 		return ret;
> 	}
> 	
> 	return 0;
> }
>
> static long test_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
> {
> 	printk (KERN_INFO "Value : %d\n", vmalloc_area[0]);
> 	return 0;
> }
>
> static struct file_operations test_fops = {
> 	.open = test_open,
> 	.release = test_release,
> 	.mmap = test_mmap,
> 	.unlocked_ioctl = test_ioctl,
> 	.owner = THIS_MODULE,
> };
>
> This is done on an arm architecture (AT91 SAM9X5) with a kernel 3.5.
> I have done the test, with the same code, on a powerpc target, with a kernel 2.6.27, and it seems to work (but maybe by chance?)
> Am I missing something?
> Maybe I need to implement the sync function in file operations, but in that case, how can I know that all mapped memory is synced?
>
> Thanks in advance,
> Gil Weber
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=ilto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mmap sync issue

[Michel Lespinasse]

On Sat, Mar 16, 2013 at 4:39 PM, Will Huck <will.huckk@gmail.com> wrote:
> On 03/15/2013 07:39 PM, Gil Weber wrote:
>> I am experiencing an issue with my device driver. I am using mmap and
>> ioctl to share information with my user space application.
>> The thing is that the shared memory does not seems to be synced. Do check
>> this, I have done a simple test:

So if I got this right, the issue is that the vmalloc_area is
virtually aliased between the kernel and the user space mapping, so
that coherency is not guaranteed on architectures that use virtually
aliased caches.

fs/aio.c does something similar to what you want with their ring
buffer. The kernel doesn't access the ring buffer through a vmalloc
area like you're trying to do; instead it uses kmap_atomic() ..
kunmap_atomic() whenever it wants to access it.

I don't actually consider myself an expert in this area but I believe
the above should solve your problem :)

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mmap sync issue

[Hugh Dickins]

On Sat, 16 Mar 2013, Michel Lespinasse wrote:
> On Sat, Mar 16, 2013 at 4:39 PM, Will Huck <will.huckk@gmail.com> wrote:
> > On 03/15/2013 07:39 PM, Gil Weber wrote:
> >> I am experiencing an issue with my device driver. I am using mmap and
> >> ioctl to share information with my user space application.
> >> The thing is that the shared memory does not seems to be synced. Do check
> >> this, I have done a simple test:
> 
> So if I got this right, the issue is that the vmalloc_area is
> virtually aliased between the kernel and the user space mapping, so
> that coherency is not guaranteed on architectures that use virtually
> aliased caches.
> 
> fs/aio.c does something similar to what you want with their ring
> buffer. The kernel doesn't access the ring buffer through a vmalloc
> area like you're trying to do; instead it uses kmap_atomic() ..
> kunmap_atomic() whenever it wants to access it.
> 
> I don't actually consider myself an expert in this area but I believe
> the above should solve your problem :)

I don't think so: kmap_atomic() provides a temporary kernel mapping for
a page when not all memory is direct-mapped, but it's close to a no-op
when there's no highmem.  This question isn't about highmem.

I can't point to what solves the problem for the aio ringbuffer:
for all I know, that's not even used on such architectures.

The usual solution is flush_dcache_page(): see Documentation/cachetlb.txt.
Which mostly describes the common page cache case, but a driver like
yours may also need it.

Each architecture has its own implementation, and often its own way of
minimizing the overhead of flush_dcache_page(): if you're using it in
a new context, you might need to be careful about such optimizations,
and the page flags used to control them.

But better to ask on the (moderated) linux-arm-kernel list if it's not
clear to you how to use it: being a no-op on x86, those of us who know
little beyond x86 are apt to give bad advice.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>