kernel bug: mmap, XIP, page faults, multiple threads

kernel bug: mmap, XIP, page faults, multiple threads

[Louis Alex Eisner]

[-- Attachment #1.1: Type: text/plain, Size: 2683 bytes --]

Hello,

   I hope I'm sending this to the right people, but I wasn't sure who to
send it to, since I'm not entirely sure exactly where the bug lives.
 Without further ado:

Summary:
When multiple threads simultaneously attempt to write to the same page of a
file which has been mmapped using XIP for the first time, an unhandled
EBUSY signal causes the kernel to panic.

Full description:
I have created an ext2 file system on /dev/ram0 using the XIP flag.  I
create and populate a single large file on the device, like so (see
attached: prep_device_file.sh):
1.  Insert the brd kernel module;
2.  Format /dev/ram0 as ext2;
3.  Mount /dev/ram0 with -o xip
4.  Initialize a large file on the mounted file system using dd.

I have a test program (see attached: ramdiskbreaker.c) which does the
following:
1.  Creates a shared map of the entire file;
2.  Creates some child threads which attempt to write to the same location
in the map;
3.  Allows the child threads to simultaneously write to the same page in
the map.
4.  This triggers a kernel panic.  The full output of the panic is included
(panic.txt).

Keywords: ramdisk, xip, brd, mm, mmap

Kernel version:
$ cat /proc/version
Linux version 3.2.1-io (root@bb-23.calit2.net) (gcc version 4.1.2 20071124
(Red Hat 4.1.2-42)) #17 SMP Tue Jan 24 23:44:15 PST 2012

Software: see attached: ver_linux.txt

Environment: see attached: env.txt

Proc info: see attached: proc_*.txt

Additional info:
Two threads are enough to cause a panic, but more threads cause more
panics.  One thread does not cause a panic.
Pointing the threads to different pages does not cause a panic.
Turning off -o xip results in no panics.
Turning off O_DIRECT does NOT prevent panics.
MAP_POPULATE does not fix the problem.  This may be an additional bug?
Panics cause the process to become a zombie.
The actual panic occurs here:
http://lxr.linux.no/linux+v3.2.2/mm/filemap_xip.c#L266 when
vm_insert_mixed<http://lxr.linux.no/linux+v3.2.2/+code=vm_insert_mixed>
returns
-EBUSY, which isn't handled.
When I run ramdiskbreaker, I see the following output:
[leisner@bb-23 PosixNVM]$ gcc ramdiskbreaker.c -O2 -fPIC -pthread -o
rdb.exe && ./rdb.exe
Attempting to cause a kernel panic.
Thred count: 2
File: /tmp/memuram0/xddtestfile.txt
Thread 0 waiting...
Thread 1 waiting...
Starting threads.

Message from syslogd@bb-23 at Thu Jan 26 17:41:14 2012 ...
bb-23 kernel: Stack:
...


Thank you for your attention.  Please let me know if there's anything else
of use I can provide you with.  I will continue to try and narrow down the
bug further.  Please let me know if you discover a fix, or any other useful
information.

Regards,
Alex Eisner
http://nvsl.ucsd.edu/

[-- Attachment #1.2: Type: text/html, Size: 4324 bytes --]

[-- Attachment #2: bugreport.tar --]
[-- Type: application/x-tar, Size: 99328 bytes --]

Re: kernel bug: mmap, XIP, page faults, multiple threads

[Hugh Dickins]

On Thu, 26 Jan 2012, Louis Alex Eisner wrote:
> 
>    I hope I'm sending this to the right people, but I wasn't sure who to
> send it to, since I'm not entirely sure exactly where the bug lives.
>  Without further ado:
> 
> Summary:
> When multiple threads simultaneously attempt to write to the same page of a
> file which has been mmapped using XIP for the first time, an unhandled
> EBUSY signal causes the kernel to panic.

Thanks a lot for your report, and all the info you carefully gathered.

I confess that I haven't looked at it at all!  Because I was thinking
maybe I should take a look, and when did we last hear from Carsten?
And though I now see more recent postings from him in other fields,
what came first to my eye was this nugget below.

It was white-space-damaged and wouldn't apply (I bet that's why it
got lost), so I've fixed that up and reformatted the description,
and added you as a Reporter - but otherwise it's as Carsten posted.

Hugh