Re: 2.2.6_andrea2.bz2

Re: 2.2.6_andrea2.bz2

[Andrea Arcangeli]

On 26 Apr 1999, Eric W. Biederman wrote:

>The real gain of this is not so much in the current cases we are fast at
>but for things like network, and compressed files (in general anything that

Could you produce an example (also "look at file.c" will be ok ;) to allow
me to see which are the issues with network and compressed files? Thanks.

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: 2.2.6_andrea2.bz2

[Eric W. Biederman]

>>>>> "AA" == Andrea Arcangeli <andrea@e-mind.com> writes:

AA> On 26 Apr 1999, Eric W. Biederman wrote:
>> The real gain of this is not so much in the current cases we are fast at
>> but for things like network, and compressed files (in general anything that

AA> Could you produce an example (also "look at file.c" will be ok ;) to allow
AA> me to see which are the issues with network and compressed files? Thanks.

The primary one is they can't use the buffer cache so they must roll their
own caching mechanism.

Look at the smbfs version of updatepage. (write through!)
Look at e2compr which doesn't compress data until the file is closed!
    It writes the data uncompressed, and then at iput time
    rewrites the file compressed.
Run a moderately early version of fat_cvf/dmsdos, before they found out how
    to use the buffer cache, and watch it crawl on a read-only fs.
nfs doesn't periodically flush dirty data, to keep the volume low.
    It does flush on file close (which helps, and is as correct as possible
    for nfs), but you can still find

I haven't seen any provision in any of these roll your own solutions
for flushing the dirty buffers when the system is low on memory.
Etc.

The point is that because there isn't a caching subsystem
all of the filesystems above have to roll their own.  And because
they roll their own the code is less polished than a solution which
would work for all of them would be.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: 2.2.6_andrea2.bz2

[Andi Kleen]

On Mon, Apr 26, 1999 at 10:05:56AM +0200, Eric W. Biederman wrote:
> >>>>> "AA" == Andrea Arcangeli <andrea@e-mind.com> writes:
> 
> >>> o	update_shared_mappings (will greatly improve performances while
> >>> writing from many task to the same shared memory).
> >> 
> >> do you have performance numbers on this?
> 
> AA> The performance optimization can be huge.
> 
> AA> The reason this my code is not in the kernel is not because it's buggy but
> AA> simple because there are plans for 2.3.x (no-way for 2.2.x) to allow the
> AA> file cache to be dirty (to cache also writes and not only read in the page
> AA> cache).
> 
> Andrea.  The plan (at least my plan) is not to have 2 layers of buffers.
> Instead it is to do all of the caching (except for perhaps superblocks, and their
> kin in the page cache).  brw_page will be used for both reads and writes, with
> anonymouse buffer heads (at least for a start).

Stupid question: do you plan to cache fs metadata in the page cache too? 
If yes, it is rather wasteful to use a 4K page for the usually block sized
directories and other fs data like indirect blocks. How do you plan to 
address this problem?


-Andi
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: 2.2.6_andrea2.bz2

[Eric W. Biederman]

>>>>> "AK" == Andi Kleen <ak@muc.de> writes:

AK> On Mon, Apr 26, 1999 at 10:05:56AM +0200, Eric W. Biederman wrote:
>> >>>>> "AA" == Andrea Arcangeli <andrea@e-mind.com> writes:
>> 
>> >>> o	update_shared_mappings (will greatly improve performances while
>> >>> writing from many task to the same shared memory).
>> >> 
>> >> do you have performance numbers on this?
>> 
AA> The performance optimization can be huge.
>> 
AA> The reason this my code is not in the kernel is not because it's buggy but
AA> simple because there are plans for 2.3.x (no-way for 2.2.x) to allow the
AA> file cache to be dirty (to cache also writes and not only read in the page
AA> cache).
>> 
>> Andrea.  The plan (at least my plan) is not to have 2 layers of buffers.
>> Instead it is to do all of the caching (except for perhaps superblocks, and their
>> kin in the page cache).  brw_page will be used for both reads and writes, with
>> anonymouse buffer heads (at least for a start).

AK> Stupid question: do you plan to cache fs metadata in the page cache too? 
AK> If yes, it is rather wasteful to use a 4K page for the usually block sized
AK> directories and other fs data like indirect blocks. How do you plan to 
AK> address this problem?

I certainly plan on investigating it.  I currently have the buffer
pointer in struct page, set up as a generic pointer.  So you can either
use it's bits directly to keep track of what is dirty on a page.  Or
you can allocate a structure to have such things as a per page list
of dirty places (like nfs does now).

For the start however the buffer cache will remain for the fs metadata.

But the fs metadata should be small enough that even if
cached a little inefficiently we shouldn't have space problems.
At least that's my hunch.

Eric


--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/