From mboxrd@z Thu Jan 1 00:00:00 1970 Message-Id: <20080529122602.557135000@nick.local0.net> References: <20080529122050.823438000@nick.local0.net> Date: Thu, 29 May 2008 22:20:55 +1000 From: npiggin@suse.de Subject: [patch 5/5] splice: use get_user_pages_fast Content-Disposition: inline; filename=splice-get_user_pages_fast.patch Sender: owner-linux-mm@kvack.org Return-Path: To: akpm@linux-foundation.org Cc: shaggy@austin.ibm.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, apw@shadowen.org List-ID: Use get_user_pages_fast in splice. This reverts some mmap_sem batching there, however the biggest problem with mmap_sem tends to be hold times blocking out other threads rather than cacheline bouncing. Further: on architectures that implement get_user_pages_fast without locks, mmap_sem can be avoided completely anyway. Signed-off-by: Nick Piggin Cc: shaggy@austin.ibm.com Cc: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org Cc: apw@shadowen.org --- fs/splice.c | 41 +++-------------------------------------- 1 file changed, 3 insertions(+), 38 deletions(-) Index: linux-2.6/fs/splice.c =================================================================== --- linux-2.6.orig/fs/splice.c +++ linux-2.6/fs/splice.c @@ -1147,36 +1147,6 @@ static long do_splice(struct file *in, l } /* - * Do a copy-from-user while holding the mmap_semaphore for reading, in a - * manner safe from deadlocking with simultaneous mmap() (grabbing mmap_sem - * for writing) and page faulting on the user memory pointed to by src. - * This assumes that we will very rarely hit the partial != 0 path, or this - * will not be a win. - */ -static int copy_from_user_mmap_sem(void *dst, const void __user *src, size_t n) -{ - int partial; - - if (!access_ok(VERIFY_READ, src, n)) - return -EFAULT; - - pagefault_disable(); - partial = __copy_from_user_inatomic(dst, src, n); - pagefault_enable(); - - /* - * Didn't copy everything, drop the mmap_sem and do a faulting copy - */ - if (unlikely(partial)) { - up_read(¤t->mm->mmap_sem); - partial = copy_from_user(dst, src, n); - down_read(¤t->mm->mmap_sem); - } - - return partial; -} - -/* * Map an iov into an array of pages and offset/length tupples. With the * partial_page structure, we can map several non-contiguous ranges into * our ones pages[] map instead of splitting that operation into pieces. @@ -1189,8 +1159,6 @@ static int get_iovec_page_array(const st { int buffers = 0, error = 0; - down_read(¤t->mm->mmap_sem); - while (nr_vecs) { unsigned long off, npages; struct iovec entry; @@ -1199,7 +1167,7 @@ static int get_iovec_page_array(const st int i; error = -EFAULT; - if (copy_from_user_mmap_sem(&entry, iov, sizeof(entry))) + if (copy_from_user(&entry, iov, sizeof(entry))) break; base = entry.iov_base; @@ -1233,9 +1201,8 @@ static int get_iovec_page_array(const st if (npages > PIPE_BUFFERS - buffers) npages = PIPE_BUFFERS - buffers; - error = get_user_pages(current, current->mm, - (unsigned long) base, npages, 0, 0, - &pages[buffers], NULL); + error = get_user_pages_fast((unsigned long)base, npages, + 0, &pages[buffers]); if (unlikely(error <= 0)) break; @@ -1274,8 +1241,6 @@ static int get_iovec_page_array(const st iov++; } - up_read(¤t->mm->mmap_sem); - if (buffers) return buffers; -- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org