Re: [RFC v1][PATCH]page_fault retry with NOPAGE_RETRY

[Nick Piggin <npiggin@suse.de>]

On Sun, Nov 30, 2008 at 09:38:18PM +0200, Torok Edwin wrote:
> On 2008-11-28 14:10, Nick Piggin wrote:
> > This is what I have.
> >
> > It does two things. Firstly, it switches x86-64 over to use the xadd
> > algorithm rather than the spinlock algorithm. This is actually significant
> > in high contention situations, because the spinlock algorithm doesn't allow
> > concurrent operations on the lock while the queue of waiters is being
> > manipulated.
> >
> > Secondly, it moves wakeups out from underneath the waiter queue lock. This
> > is more significant on bigger machines where wakeup latency is worse and/or
> > runqueue locks are very heavily contended.
> >
> > Now both these changes are going to help *mainly* for the case when there are
> > a significant number of readers and writers, I think. So your write-heavy
> > workload may not win anything. I noticed some speedup a long time ago on
> > some weird java (volanomark) workload.
> 
> Hi,
> 
> I just tested your patch on top of tip/master, and my testprogram has
> segfaulted :(
> It is either something wrong in tip/master or the patch, or my program.
> This is the first time this testprogram segfaults, and it doesn't have a
> reason to segfault there.
> 
> 
> [  140.624155] scalability[4995]: segfault at 7f9ce137f000 ip
> 0000000000401a62 sp 00000000454950a0 error 4 in scalability[400000+3000]
> [  401.640738] scalability[5398]: segfault at 7fdbffba3000 ip
> 0000000000401a62 sp 00000000423d70a0 error 4 in scalability[400000+3000]
> 
> Here is the relevant portion, at 401a62 I read from the mapping:
> 
> static void mmap_worker_fn(int fd, off_t len)
> {
>     char *data = mmap(NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
>   401a4f:    48 89 c7                 mov    %rax,%rdi
>     if(data == MAP_FAILED) {
>   401a52:    74 36                    je     401a8a <mmap_worker_fn+0x5a>
>         perror("mmap");
>         abort();
>   401a54:    31 d2                    xor    %edx,%edx
>   401a56:    31 c9                    xor    %ecx,%ecx
> static pthread_mutex_t thrtime_mtx = PTHREAD_MUTEX_INITIALIZER;
> 
> static size_t execute(const char *data, size_t len)
> {
>     size_t sum = 0, i;
>     for(i=0;i<len;++i)
>   401a58:    48 85 db                 test   %rbx,%rbx
>   401a5b:    74 28                    je     401a85 <mmap_worker_fn+0x55>
>   401a5d:    0f 1f 00                 nopl   (%rax)
>         if(data[i] == 'd')
>             ++sum;
>   401a60:    31 c0                    xor    %eax,%eax
>   401a62:    80 3c 17 64              cmpb   $0x64,(%rdi,%rdx,1)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> This simply reads from the mapping
> 
>   401a66:    0f 94 c0                 sete   %al
> static pthread_mutex_t thrtime_mtx = PTHREAD_MUTEX_INITIALIZER;
> 
> Steps to reproduce:
> # sync; echo 3 >/proc/sys/vm/drop_caches; sync
> # echo 0 >/proc/lock_stat
> $ sudo ./scalability 16 /usr/bin/
> ... prints out results for read, and while running mmap_worker ...
> ... a message about segmentation fault ....
> 
> The testprogram is available here:
> http://edwintorok.googlepages.com/tst.tar.gz
> 
> My .config:
> http://edwintorok.googlepages.com/config
> 
> Can you reproduce the crash on your box?
> Can I help debugging the problem?

BTW. I think your source code (I see you updated it since last posting)
should be very easy to give good hints to the kernel about the IO. I
will try a few simple tricks and we can see if they help. (this pattern
of touching memory corresponds well to how your app works?)

Thanks,
Nick

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>