MM question

MM question

[Jason Titus]

I know this must be on a FAQ or some such, but after many hours of 
newsgroup/web searching - nothing.

Is there a way to turn off/down the page caching and buffering?  I'm doing
database work and am having a really time benchmarking other elements of the
system due to Linux's friendly caching....

I have tried editing /proc/sys/vm/pagecache and buffermem, but the changes
don't seem to do anything (2.2.0-pre5 - x86).  Is there something you have
to do to activate the changes?  Is there some other file to edit to set the
variables at boot time?

It sure would be nice to have more control over the caching, like being able
to have a /etc/cache.conf file where you could set parameters and mark
certain files/filetypes as priority cache items...

Anyway, any help would be much appreciated, and sorry for the ignorant
question,

Jason Titus
jason@iatlas.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Stephen C. Tweedie]

Hi,

On Mon, 15 Feb 1999 21:30:22 -0500, "Jason Titus" <jason@iatlas.com>
said:

> Is there a way to turn off/down the page caching and buffering?  I'm doing
> database work and am having a really time benchmarking other elements of the
> system due to Linux's friendly caching....

No.

You can tune a few different aspects of the VM's management of the
caches, but there is really no way to disable them completely.

> It sure would be nice to have more control over the caching, like being able
> to have a /etc/cache.conf file where you could set parameters and mark
> certain files/filetypes as priority cache items...

Why exactly do you need it?  For plain benchmarking, the standard
technique to defeat caching is to benchmark on files much larger than
physical memory.  

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

MM question

[Magnus Ahltorp]

I'm working on a Linux port of Arla, a free AFS (a distributed file
system) client. Arla caches whole files, which means that the read and
write file system calls are tunneled through to the cache file system
(usually ext2).

I implement the inode operation readpage for reads and the file
operation write for writes. readpage gets tunneled by creating a new
struct file and filling in the cache inode. write gets tunneled in
much the same way. Though, I have been seeing problems when a file
gets written to. The written data weren't always seen when doing
reads. Someone then suggested that the folloing code be added to
write:

	page = get_free_page(GFP_KERNEL);
	if (!page)
	    invalidate_inode_pages(inode);
	else {
	    int pos = file->f_pos;
	    int cnt = count;
	    int blk;
	    char *data=(char *)page;

	    while (cnt > 0) {
		blk = cnt;
		if (blk > PAGE_SIZE)
		    blk = PAGE_SIZE;
		copy_from_user(data, buf, blk);
		update_vm_cache(inode, pos, data, blk);
		pos += blk;
		cnt -= blk;
	    }
	    free_page(page);
	}

I inserted this piece of code, and things worked quite well. After a
while, I was seeing new problems. Writes were not propagating properly
to the cache file.

Does anyone have any suggestions on how this really should be done?

/Magnus
map@stacken.kth.se
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Benjamin C.R. LaHaise]

On 21 Feb 1999, Magnus Ahltorp wrote:
...
> I inserted this piece of code, and things worked quite well. After a
> while, I was seeing new problems. Writes were not propagating properly
> to the cache file.

The code listed is just plain wrong -- it doesn't take into account pages
that are already present in the page cache.  If you need to use this
technique, take a look at mm/filemap.c:generic_file_write for an example
of how to do this properly (mostly, find the page in the page cache, if
not found add it; lock page in page cache). 

> Does anyone have any suggestions on how this really should be done?

Could you clairify how you're doing things?  Are pages cached in the
kernel owned by your filesystem's inode or ext2's?  The hint I'm getting
from the code you quoted is the both are storing the data, which is
inefficient.  The easiest thing to do would be to tunnel all operations to
the ext2 inode -- your filesystem should not have a readpage function.
Rather, mmap(), read() and write() all receive the ext2 inode of the file,
so that all pages in memory are owned by the ext2 inode, and your inode is
merely an indirect handle that validates the cache.  How's that sound?

		-ben



--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Magnus Ahltorp]

> The code listed is just plain wrong -- it doesn't take into account pages
> that are already present in the page cache.  If you need to use this
> technique, take a look at mm/filemap.c:generic_file_write for an example
> of how to do this properly (mostly, find the page in the page cache, if
> not found add it; lock page in page cache). 

Since I didn't write the code myself, it doesn't express my
intentions, but rather someone else's, so don't look at it too much.

> Could you clairify how you're doing things?  Are pages cached in the
> kernel owned by your filesystem's inode or ext2's?  The hint I'm getting
> from the code you quoted is the both are storing the data, which is
> inefficient.  The easiest thing to do would be to tunnel all operations to
> the ext2 inode -- your filesystem should not have a readpage function.
> Rather, mmap(), read() and write() all receive the ext2 inode of the file,
> so that all pages in memory are owned by the ext2 inode, and your inode is
> merely an indirect handle that validates the cache.  How's that sound?

Right now, an Arla inode has some extra information, containing a
dentry for the cache file. The readpage() function just validates the
cache information, fills in a struct file (with the ext2 inode) and
calls ext2's readpage(). The struct page pointer is passed along to
ext2's readpage() without any modifications.

The write() does pretty much the same thing, just that here the data
is passed via a pointer and a length.

I don't really know what's supposed to happen during a readpage()
call, and what I want is a way to make write() affect the pages that
readpage() has read.

(I might sound somewhat disoriented, but that is because I am)

If you want to have a look at the particular source, the relevant file
is http://www.stacken.kth.se/~map/src/cvs/arla/xfs/linux/xfs_inodeops.c
(xfs_readpage() and xfs_write_file()).

/Magnus
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Benjamin C.R. LaHaise]

On 22 Feb 1999, Magnus Ahltorp wrote:

> Right now, an Arla inode has some extra information, containing a
> dentry for the cache file. The readpage() function just validates the
> cache information, fills in a struct file (with the ext2 inode) and
> calls ext2's readpage(). The struct page pointer is passed along to
> ext2's readpage() without any modifications.

Okay, you probably don't want to implement readpage, just read and write,
so your read will look like:

my_read(...)
{
	validate_cache();
	return cache_inode->read(cache_inode, ...)
}

The write operation should be something like:

my_write(...)
{
	validate_cache_for_write();
	down(&cache_inode->i_sem);
	cache_inode->write(cache_inode, ...)
	up(&cache_inode->i_sem);
}

This will make your inodes relatively lightweight, and avoid having in
memory pages attached to your inode which would be duplicates of those
attached to the ext2 inode.

> I don't really know what's supposed to happen during a readpage()
> call, and what I want is a way to make write() affect the pages that
> readpage() has read.

Readpage is called by generic_file_read and page fault handlers to pull
the page into the page cache.  In the case of writing, you need to update
the page cache, as well as commit the write to whatever backstore is used. 
Since you've got the entire file cached (right?), just making use of the
ext2 inode's read & write will keep the cache coherent and reduce the
amount work you need to do. 

> (I might sound somewhat disoriented, but that is because I am)

You're asking questions, so the hard part is over =)

		-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Magnus Ahltorp]

> Okay, you probably don't want to implement readpage, just read and write,
> so your read will look like:
> 
> This will make your inodes relatively lightweight, and avoid having in
> memory pages attached to your inode which would be duplicates of those
> attached to the ext2 inode.

Doesn't this mean that the read functions will be called every time
something has to be read? What about mmap?

> Readpage is called by generic_file_read and page fault handlers to pull
> the page into the page cache.  In the case of writing, you need to update
> the page cache, as well as commit the write to whatever backstore is used. 
> Since you've got the entire file cached (right?), just making use of the
> ext2 inode's read & write will keep the cache coherent and reduce the
> amount work you need to do. 

At the moment, we do whole file caching, but that might change in the
future.

/Magnus
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: MM question

[Stephen C. Tweedie]

Hi,

On 22 Feb 1999 22:13:09 +0100, Magnus Ahltorp <map@stacken.kth.se> said:

> Right now, an Arla inode has some extra information, containing a
> dentry for the cache file. The readpage() function just validates the
> cache information, fills in a struct file (with the ext2 inode) and
> calls ext2's readpage(). The struct page pointer is passed along to
> ext2's readpage() without any modifications.

This sounds like the source of the problem: the Arla inode's readpage
function will not be called if the page cache for the Arla page is
already present.  If you write to the underlying file, it will update
any ext2fs page cache present for the page, but will not touch the Arla
page itself.  You need to call update_vm_cache() against the appropriate
Arla inode for that to happen.  (Ext2 will already call
update_vm_cache() for the ext2fs inode, so the ext2 page cache will
remain consistent internally.)

Of course, you really want to take a step back at this point and work
out if this is really the best way forward, since you just end up
caching things twice if you are not careful...

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/