[RFC][PATCH] hugetlbfs read support

[RFC][PATCH] hugetlbfs read support

[Badari Pulavarty]

Comments/flames ?

Thanks,
Badari

Support for reading from hugetlbfs files. libhugetlbfs lets application
text/data to be placed in large pages. When we do that, oprofile doesn't
work - since it tries to read from it.

This code is very similar to what do_generic_mapping_read() does, but
I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
Lamater's cleanup to pagecache would hopefully give me all of this.

Signed-off-by: Badari Pulavarty <pbadari@us.ibm.com>

 fs/hugetlbfs/inode.c |  109 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 109 insertions(+)

Index: linux-2.6.22/fs/hugetlbfs/inode.c
===================================================================
--- linux-2.6.22.orig/fs/hugetlbfs/inode.c	2007-07-08 16:32:17.000000000 -0700
+++ linux-2.6.22/fs/hugetlbfs/inode.c	2007-07-09 13:37:00.000000000 -0700
@@ -156,6 +156,114 @@ full_search:
 }
 #endif
 
+static int
+hugetlbfs_read_actor(struct page *page, unsigned long offset,
+			char __user *buf, unsigned long count,
+			unsigned long size)
+{
+	char *kaddr;
+	unsigned long to_copy;
+	int i, chunksize;
+
+	if (size > count)
+		size = count;
+
+	/* Find which 4k chunk and offset with in that chunk */
+	i = offset >> PAGE_CACHE_SHIFT;
+	offset = offset & ~PAGE_CACHE_MASK;
+	to_copy = size;
+
+	while (to_copy) {
+		chunksize = PAGE_CACHE_SIZE;
+		if (offset)
+			chunksize -= offset;
+		if (chunksize > to_copy)
+			chunksize = to_copy;
+		kaddr = kmap(&page[i]);
+		memcpy(buf, kaddr + offset, chunksize);
+		kunmap(&page[i]);
+		offset = 0;
+		to_copy -= chunksize;
+		buf += chunksize;
+		i++;
+	}
+	return size;
+}
+
+/*
+ * Support for read() - Find the page attached to f_mapping and copy out the
+ * data. Its *very* similar to do_generic_mapping_read(), we can't use that
+ * since it has PAGE_CACHE_SIZE assumptions.
+ */
+ssize_t
+hugetlbfs_read(struct file *filp, char __user *buf, size_t len, loff_t *ppos)
+{
+	struct address_space *mapping = filp->f_mapping;
+	struct inode *inode = mapping->host;
+	unsigned long index = *ppos >> HPAGE_SHIFT;
+	unsigned long end_index;
+	loff_t isize;
+	unsigned long offset;
+	ssize_t retval = 0;
+
+	/* validate len */
+	if (len == 0)
+		goto out;
+
+	isize = i_size_read(inode);
+	if (!isize)
+		goto out;
+
+	offset = *ppos & ~HPAGE_MASK;
+	end_index = (isize - 1) >> HPAGE_SHIFT;
+	for (;;) {
+		struct page *page;
+		unsigned long nr, ret;
+
+		/* nr is the maximum number of bytes to copy from this page */
+		nr = HPAGE_SIZE;
+		if (index >= end_index) {
+			if (index > end_index)
+				goto out;
+			nr = ((isize - 1) & ~HPAGE_MASK) + 1;
+			if (nr <= offset) {
+				goto out;
+			}
+		}
+		nr = nr - offset;
+
+		/* Find the page */
+		page = find_get_page(mapping, index);
+		if (unlikely(page == NULL)) {
+			/*
+			 * We can't find the page in the cache - bail out ?
+			 */
+			goto out;
+		}
+		/*
+		 * Ok, we have the page, so now we can copy it to user space...
+		 */
+		ret = hugetlbfs_read_actor(page, offset, buf, len, nr);
+		if (ret < 0) {
+			retval = retval ? : ret;
+			goto out;
+		}
+
+		offset += ret;
+		retval += ret;
+		len -= ret;
+		index += offset >> HPAGE_SHIFT;
+		offset &= ~HPAGE_MASK;
+
+		page_cache_release(page);
+		if (ret == nr && len)
+			continue;
+		goto out;
+	}
+out:
+	return retval;
+}
+
 /*
  * Read a page. Again trivial. If it didn't already exist
  * in the page cache, it is zero-filled.
@@ -560,6 +668,7 @@ static void init_once(void *foo, struct 
 }
 
 const struct file_operations hugetlbfs_file_operations = {
+	.read			= hugetlbfs_read,
 	.mmap			= hugetlbfs_file_mmap,
 	.fsync			= simple_sync_file,
 	.get_unmapped_area	= hugetlb_get_unmapped_area,


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Christoph Hellwig]

On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
> Comments/flames ?
> 
> Thanks,
> Badari
> 
> Support for reading from hugetlbfs files. libhugetlbfs lets application
> text/data to be placed in large pages. When we do that, oprofile doesn't
> work - since it tries to read from it.
> 
> This code is very similar to what do_generic_mapping_read() does, but
> I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
> Lamater's cleanup to pagecache would hopefully give me all of this.

The code looks fine, but I really hate that we need it all all.  We really
should make the general VM/FS code large page aware and get rid of this
whole hack called hugetlbfs..

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Nishanth Aravamudan]

On 10.07.2007 [10:17:20 +0100], Christoph Hellwig wrote:
> On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
> > Comments/flames ?
> > 
> > Thanks,
> > Badari
> > 
> > Support for reading from hugetlbfs files. libhugetlbfs lets
> > application text/data to be placed in large pages. When we do that,
> > oprofile doesn't work - since it tries to read from it.
> > 
> > This code is very similar to what do_generic_mapping_read() does,
> > but I can't use it since it has PAGE_CACHE_SIZE assumptions.
> > Christoph Lamater's cleanup to pagecache would hopefully give me all
> > of this.
> 
> The code looks fine, but I really hate that we need it all all.  We
> really should make the general VM/FS code large page aware and get rid
> of this whole hack called hugetlbfs..

I agree and sounds like something to bring up at KS (again?) or the
VM/FS summit. But, for now, hugetlbfs is the supported interface and
libhugetlbfs has run into this issue supporting one of its features. So
I would like to see this make it in.

Just my $0.02 as a libhuge developer.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Bill Irwin]

On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
>> This code is very similar to what do_generic_mapping_read() does, but
>> I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
>> Lamater's cleanup to pagecache would hopefully give me all of this.

On Tue, Jul 10, 2007 at 10:17:20AM +0100, Christoph Hellwig wrote:
> The code looks fine, but I really hate that we need it all all.  We really
> should make the general VM/FS code large page aware and get rid of this
> whole hack called hugetlbfs..

That needs to be taken up with Linus. If it's any consolation, I'm in
favor of such myself.


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Bill Irwin]

On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
> Support for reading from hugetlbfs files. libhugetlbfs lets application
> text/data to be placed in large pages. When we do that, oprofile doesn't
> work - since it tries to read from it.
> This code is very similar to what do_generic_mapping_read() does, but
> I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
> Lamater's cleanup to pagecache would hopefully give me all of this.
> Signed-off-by: Badari Pulavarty <pbadari@us.ibm.com>

What's the testing status of all this? I thoroughly approve of the
concept, of course.


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Nishanth Aravamudan]

On 10.07.2007 [08:37:52 -0700], Bill Irwin wrote:
> On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
> > Support for reading from hugetlbfs files. libhugetlbfs lets application
> > text/data to be placed in large pages. When we do that, oprofile doesn't
> > work - since it tries to read from it.
> > This code is very similar to what do_generic_mapping_read() does, but
> > I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
> > Lamater's cleanup to pagecache would hopefully give me all of this.
> > Signed-off-by: Badari Pulavarty <pbadari@us.ibm.com>
> 
> What's the testing status of all this? I thoroughly approve of the
> concept, of course.

With this change, OProfile is able to do symbol lookup (which is
achieved via libbfd, which does reads() of the appropriate files) with
relinked binaries in post-processing. The file utility is also able to
recognize persistent text segments as ELF executables.

If you would like further testing, let me know what.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Bill Irwin]

On Tue, Jul 10, 2007 at 08:28:46AM -0700, Nishanth Aravamudan wrote:
> I agree and sounds like something to bring up at KS (again?) or the
> VM/FS summit. But, for now, hugetlbfs is the supported interface and
> libhugetlbfs has run into this issue supporting one of its features. So
> I would like to see this make it in.
> Just my $0.02 as a libhuge developer.

It should also be there to make it more like a normal filesystem,
though your use case is of vastly higher priority than such concerns.

As far as large pages for the generic VM/VFS, I cast my absentee
vote(s) in favor of such, not that anyone gives a damn what I think.


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Bill Irwin]

On 10.07.2007 [08:37:52 -0700], Bill Irwin wrote:
>> What's the testing status of all this? I thoroughly approve of the
>> concept, of course.

On Tue, Jul 10, 2007 at 08:43:12AM -0700, Nishanth Aravamudan wrote:
> With this change, OProfile is able to do symbol lookup (which is
> achieved via libbfd, which does reads() of the appropriate files) with
> relinked binaries in post-processing. The file utility is also able to
> recognize persistent text segments as ELF executables.
> If you would like further testing, let me know what.

That's good enough for me.

Acked-by: William Irwin <bill.irwin@oracle.com>


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Badari Pulavarty]

Bill Irwin wrote:

>On 10.07.2007 [08:37:52 -0700], Bill Irwin wrote:
>
>>>What's the testing status of all this? I thoroughly approve of the
>>>concept, of course.
>>>
>
>On Tue, Jul 10, 2007 at 08:43:12AM -0700, Nishanth Aravamudan wrote:
>
>>With this change, OProfile is able to do symbol lookup (which is
>>achieved via libbfd, which does reads() of the appropriate files) with
>>relinked binaries in post-processing. The file utility is also able to
>>recognize persistent text segments as ELF executables.
>>If you would like further testing, let me know what.
>>
>
>That's good enough for me.
>
>Acked-by: William Irwin <bill.irwin@oracle.com>
>
Thanks. I may have to handle memcpy() failures. I will fix that and send 
out a patch.

Thanks,
Badari




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Badari Pulavarty]

Christoph Hellwig wrote:

>On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
>
>>Comments/flames ?
>>
>>Thanks,
>>Badari
>>
>>Support for reading from hugetlbfs files. libhugetlbfs lets application
>>text/data to be placed in large pages. When we do that, oprofile doesn't
>>work - since it tries to read from it.
>>
>>This code is very similar to what do_generic_mapping_read() does, but
>>I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
>>Lamater's cleanup to pagecache would hopefully give me all of this.
>>
>
>The code looks fine, but I really hate that we need it all all.  We really
>should make the general VM/FS code large page aware and get rid of this
>whole hack called hugetlbfs..
>
I would love to see *atleast* generic filemap handler routines does not 
assume PAGE_SIZE.
Clameter's cleanup patches hopefully would all of that for us - but I am 
not sure how it handles
largepages with kmap() to copy out the data.

But getting rid of hugetlbfs completely, needs bigger effort :(

Thanks,
Badari



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Christoph Lameter]

On Tue, 10 Jul 2007, Bill Irwin wrote:

> On Mon, Jul 09, 2007 at 12:28:11PM -0700, Badari Pulavarty wrote:
> > Support for reading from hugetlbfs files. libhugetlbfs lets application
> > text/data to be placed in large pages. When we do that, oprofile doesn't
> > work - since it tries to read from it.
> > This code is very similar to what do_generic_mapping_read() does, but
> > I can't use it since it has PAGE_CACHE_SIZE assumptions. Christoph
> > Lamater's cleanup to pagecache would hopefully give me all of this.
> > Signed-off-by: Badari Pulavarty <pbadari@us.ibm.com>
> 
> What's the testing status of all this? I thoroughly approve of the
> concept, of course.

The status is that Andrew has doubts about the antifrag approach etc 
which will stall all of this if the antifrag does not get merged. See 
the mm-merge discussion on lkml. Please make noise. Andrew asked for it.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] hugetlbfs read support

[Bill Irwin]

On Tue, 10 Jul 2007, Bill Irwin wrote:
>> What's the testing status of all this? I thoroughly approve of the
>> concept, of course.

On Tue, Jul 10, 2007 at 11:53:08AM -0700, Christoph Lameter wrote:
> The status is that Andrew has doubts about the antifrag approach etc 
> which will stall all of this if the antifrag does not get merged. See 
> the mm-merge discussion on lkml. Please make noise. Andrew asked for it.

Not quite what I was asking about, but good to hear.


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>