[RFC][PATCH] dcache and rmap

[RFC][PATCH] dcache and rmap

[Ed Tomlinson]

Hi, 

I got tired of finding my box with 50-60% percent of memory tied up in dentry/inode 
caches every morning after update-db runs or after doing a find / -name "*" to generate
a list of files for backups.  So I decided to make a stab at fixing this.

The problem is that when there is not much memory pressure the vm is happy to let the
above caches expand and expand...  What I did was factored the shrink calls out of 
do_try_to_free_pages and placed an additional call to shrink in kswapd which can get
called if kswapd does not need to use do_try_to_free_pages.

The issue then becomes when to call the new shrink_caches function?  I changed the
dcache logic to estimate and track the number of new pages alloced to dentries.  Once a
threshold is exceeded, kswapd calls shrink_caches.   Using a threshold of 32 pages works
well here.

Patch is against 19-pre7-ac2. 

Comments?
Ed Tomlinson

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.422   -> 1.425  
#	         fs/dcache.c	1.18    -> 1.21   
#	         mm/vmscan.c	1.60    -> 1.61   
#	include/linux/dcache.h	1.11    -> 1.12   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/05/04	ed@oscar.et.ca	1.423
# Modify the cache shrinking logic to apply pressure when the dcache
# grows by more than <n> pages (currently 32).  Note the slab growth
# detection code, though not perfect, is okay for this use.
# --------------------------------------------
# 02/05/04	ed@oscar.et.ca	1.424
# Fix comments
# --------------------------------------------
# 02/05/04	ed@oscar.et.ca	1.425
# grammar
# --------------------------------------------
#
diff -Nru a/fs/dcache.c b/fs/dcache.c
--- a/fs/dcache.c	Sat May  4 23:17:36 2002
+++ b/fs/dcache.c	Sat May  4 23:17:36 2002
@@ -305,6 +305,22 @@
 	spin_lock(&dcache_lock);
 }
 
+
+/**
+ * Have we allocated over n pages worth of
+ * dentries entries? 
+ */
+
+#define DENTRIES_PER_PAGE (PAGE_SIZE/(sizeof(struct dentry)+8))
+
+static int dcache_alloc_count = 0;
+
+int try_to_shrink(int pages)
+{ 
+	return dcache_alloc_count > pages; 
+}
+
+
 /**
  * prune_dcache - shrink the dcache
  * @count: number of entries to try and free
@@ -567,7 +583,13 @@
 
 	count = dentry_stat.nr_unused / priority;
 
-	prune_dcache(count);
+	prune_dcache(count); 
+
+	/*
+	 * relieve some pressure...
+	 */
+	dcache_alloc_count /= 2;
+
 	return kmem_cache_shrink_nr(dentry_cache);
 }
 
@@ -585,8 +607,17 @@
  
 struct dentry * d_alloc(struct dentry * parent, const struct qstr *name)
 {
+	static int nr_entry_base = 0;
 	char * str;
 	struct dentry *dentry;
+ 
+	if (dentry_stat.nr_dentry < nr_entry_base) 
+		nr_entry_base = dentry_stat.nr_dentry;
+		  
+	if (dentry_stat.nr_dentry-nr_entry_base > DENTRIES_PER_PAGE) {
+	        dcache_alloc_count++;
+		nr_entry_base = dentry_stat.nr_dentry;
+	}
 
 	dentry = kmem_cache_alloc(dentry_cache, GFP_KERNEL); 
 	if (!dentry)
diff -Nru a/include/linux/dcache.h b/include/linux/dcache.h
--- a/include/linux/dcache.h	Sat May  4 23:17:36 2002
+++ b/include/linux/dcache.h	Sat May  4 23:17:36 2002
@@ -173,6 +173,7 @@
 /* dcache memory management */
 extern int shrink_dcache_memory(int, unsigned int);
 extern void prune_dcache(int);
+extern int try_to_shink(int);
 
 /* icache memory management (defined in linux/fs/inode.c) */
 extern int shrink_icache_memory(int, int);
diff -Nru a/mm/vmscan.c b/mm/vmscan.c
--- a/mm/vmscan.c	Sat May  4 23:17:36 2002
+++ b/mm/vmscan.c	Sat May  4 23:17:36 2002
@@ -562,6 +562,24 @@
 		if (inactive_high(zone) > 0)
 			refill_inactive_zone(zone, priority);
 }
+	
+static int shrink_caches(unsigned int pri, unsigned int gfp_mask)
+{
+	int ret = 0;
+
+        /*
+         * Eat memory from 
+         * dentry, inode and filesystem quota caches.
+         */
+        ret += shrink_dcache_memory(pri, gfp_mask);
+        ret += shrink_icache_memory(1, gfp_mask);
+#ifdef CONFIG_QUOTA
+        ret += shrink_dqcache_memory(pri, gfp_mask);
+#endif
+
+	return ret;
+}
+
 
 /*
  * Worker function for kswapd and try_to_free_pages, we get
@@ -571,6 +589,7 @@
  * This function will also move pages to the inactive list,
  * if needed.
  */
+
 static int do_try_to_free_pages(unsigned int gfp_mask)
 {
 	int ret = 0;
@@ -580,11 +599,7 @@
 	 * dentry, inode and filesystem quota caches.
 	 */
 	ret += page_launder(gfp_mask);
-	ret += shrink_dcache_memory(DEF_PRIORITY, gfp_mask);
-	ret += shrink_icache_memory(1, gfp_mask);
-#ifdef CONFIG_QUOTA
-	ret += shrink_dqcache_memory(DEF_PRIORITY, gfp_mask);
-#endif
+	ret += shrink_caches(DEF_PRIORITY, gfp_mask);
 
 	/*
 	 * Move pages from the active list to the inactive list.
@@ -653,6 +668,9 @@
  * If there are applications that are active memory-allocators
  * (most normal use), this basically shouldn't matter.
  */
+
+#define DCACHE_PAGES (32) 
+
 int kswapd(void *unused)
 {
 	struct task_struct *tsk = current;
@@ -691,6 +709,9 @@
 		 */
 		if (free_high(ALL_ZONES) >= 0 || free_low(ANY_ZONE) > 0)
 			do_try_to_free_pages(GFP_KSWAPD);
+		else 
+			if (try_to_shrink(DCACHE_PAGES))
+				shrink_caches(DEF_PRIORITY, GFP_KSWAPD);
 
 		refill_freelist();

 -----------------------------


 
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Martin J. Bligh]

> I got tired of finding my box with 50-60% percent of memory tied 
> up in dentry/inode caches every morning after update-db runs or 
> after doing a find / -name "*" to generate a list of files for 
> backups.  So I decided to make a stab at fixing this.

Are you actually out of memory at this point, and they're consuming
space you really need?

> The issue then becomes when to call the new shrink_caches function?  
> I changed the dcache logic to estimate and track the number of new 
> pages alloced to dentries.  Once a threshold is exceeded, kswapd 
> calls shrink_caches.   Using a threshold of 32 pages works well 
> here.

Why would you want to shrink the cache if you had plenty of free 
memory? Maybe I'm mistunderstanding what you're trying to do,
but that's what your explanation sounds like ....

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Ed Tomlinson]

On May 5, 2002 10:55 pm, Martin J. Bligh wrote:
> > I got tired of finding my box with 50-60% percent of memory tied
> > up in dentry/inode caches every morning after update-db runs or
> > after doing a find / -name "*" to generate a list of files for
> > backups.  So I decided to make a stab at fixing this.
>
> Are you actually out of memory at this point, and they're consuming
> space you really need?

Think of this another way.  There are 100000+ dentry/inodes in memory
comsuming 250M or so.  Meanwhile load is light and the background
aging is able to supply pages for the freelist.  We do not reclaim this
storage until we have vm pressure.  Usually this pressure is artifical, 
if we had reclaimed the storage it would not have occured, our caches
would have more useful data in them, and half the memory would not
sit idle for half a day.  

We age the rest of the memory to keep it hot.   Rmap does a good job 
and keeps the freelist heathly.  In this case nothing ages the dentries
and they get very cold.  My code ensures that the memory consumed 
by the, potentially cold, dentries/inodes is not excessive.

(I hate getting paged at 3 in the morning)

Ed Tomlinson

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Martin J. Bligh]

>> > I got tired of finding my box with 50-60% percent of memory tied
>> > up in dentry/inode caches every morning after update-db runs or
>> > after doing a find / -name "*" to generate a list of files for
>> > backups.  So I decided to make a stab at fixing this.
>> 
>> Are you actually out of memory at this point, and they're consuming
>> space you really need?
> 
> Think of this another way.  There are 100000+ dentry/inodes in memory
> comsuming 250M or so.  Meanwhile load is light and the background
> aging is able to supply pages for the freelist.  We do not reclaim this
> storage until we have vm pressure.  Usually this pressure is artifical, 
> if we had reclaimed the storage it would not have occured, our caches
> would have more useful data in them, and half the memory would not
> sit idle for half a day.  
> 
> We age the rest of the memory to keep it hot.   Rmap does a good job 
> and keeps the freelist heathly.  In this case nothing ages the dentries
> and they get very cold.  My code ensures that the memory consumed 
> by the, potentially cold, dentries/inodes is not excessive.

If there's no pressure on memory, then using it for caches is a good
thing. Why throw away data before we're out of space? If we are under
pressure on memory then dcache should shrink easily and rapidly. If
it's not, then make it shrink properly, don't just limit it to an
arbitrary size that may be totally unsuitable for some workloads.
You could even age it instead ... that'd make more sense than
restricting it to a static size.

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Ed Tomlinson]

On May 6, 2002 10:40 am, Martin J. Bligh wrote:
> >> > I got tired of finding my box with 50-60% percent of memory tied
> >> > up in dentry/inode caches every morning after update-db runs or
> >> > after doing a find / -name "*" to generate a list of files for
> >> > backups.  So I decided to make a stab at fixing this.
> >>
> >> Are you actually out of memory at this point, and they're consuming
> >> space you really need?
> >
> > Think of this another way.  There are 100000+ dentry/inodes in memory
> > comsuming 250M or so.  Meanwhile load is light and the background
> > aging is able to supply pages for the freelist.  We do not reclaim this
> > storage until we have vm pressure.  Usually this pressure is artifical,
> > if we had reclaimed the storage it would not have occured, our caches
> > would have more useful data in them, and half the memory would not
> > sit idle for half a day.
> >
> > We age the rest of the memory to keep it hot.   Rmap does a good job
> > and keeps the freelist heathly.  In this case nothing ages the dentries
> > and they get very cold.  My code ensures that the memory consumed
> > by the, potentially cold, dentries/inodes is not excessive.
>
> If there's no pressure on memory, then using it for caches is a good
> thing. Why throw away data before we're out of space? If we are under

The point is there is always memory pressure.  Sometimes kswapd can
supply the pages needed without calling do_try_to_free_pages.  When this
happens the dcache/icache can grow since we never try to shrink it.  My
patch changes this.

> pressure on memory then dcache should shrink easily and rapidly. If
> it's not, then make it shrink properly, don't just limit it to an
> arbitrary size that may be totally unsuitable for some workloads.

There is _no_ arbitrary limit.  All that happens is that if the dcache grows
by more than n pages we try to shrink it once.   If the system its actually
using the dentries, the dcache/icache will grow as needed.  If its not using 
them, which is the case here more often than not, they get dropped and
storage is freed to be better used by the rest of the vm.

> You could even age it instead ... that'd make more sense than
> restricting it to a static size.

Again I apply pressure.  There are no static limits.  Aging would be nice but
that would probably require rather massive changes to the way the slab cache
works.

Ed Tomlinson




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[William Lee Irwin III]

On Sun, May 05, 2002 at 09:17:16PM -0400, Ed Tomlinson wrote:
> I got tired of finding my box with 50-60% percent of memory tied up
> in dentry/inode caches every morning after update-db runs or after
> doing a find / -name "*" to generate a list of files for backups.
> So I decided to make a stab at fixing this.

I would only consider this a problem if the size of the cache were so
poorly controlled that it would trigger pageout of in-core user data.
Also, the cacheing algorithm doesn't seem to take into account turnover
rates and expansion rate.. e.g. if turnover is too rapid, expand, if
expansion is too fast, try directly reclaiming largely unused entries.


On Sun, May 05, 2002 at 09:17:16PM -0400, Ed Tomlinson wrote:
> The problem is that when there is not much memory pressure the vm is
> happy to let the above caches expand and expand...  What I did was
> factored the shrink calls out of > do_try_to_free_pages and placed an
> additional call to shrink in kswapd which can get called if kswapd
> does not need to use do_try_to_free_pages.

Well, the VM doesn't really have anything to do with this; it's just a
random structure floating out in filesystem code that needs notification
of when memory gets low to shrink itself. But anyway, all the VM really
cares about is memory is running low and it wants to get some from
somewhere. Bad space behavior is either the cacheing algorithm's fault
or its implementation's.

Abstracting this out to a "cache shrinking driver" might open up useful
possibilities, for instance, dynamic registration for per-object caches.


On Sun, May 05, 2002 at 09:17:16PM -0400, Ed Tomlinson wrote:
> The issue then becomes when to call the new shrink_caches function?
> I changed the dcache logic to estimate and track the number of new
> pages alloced to dentries.  Once a threshold is exceeded, kswapd
> calls shrink_caches. Using a threshold of 32 pages works well here.

Well, I think there are three major design issues. The first is the
magic number of 32 pages. This (compile-time!) tunable is almost
guaranteed not to work for everyone. The second is that in order to
address the issue you're actually concerned about, it seems you would
have to present some method for caches to know their allocation requests
would require evicting other useful data to satisfy; for instance,
evicting pagecache holding useful (but clean) user data to allocate
dcache. Third, the distinguished position of the dcache is suspicious
to me; I feel that a greater degree of generality is in order.

In short, I don't think you went far enough. How do you feel about
GFP_SPECULATIVE (a.k.a. GFP_DONT_TRY_TOO_HARD), cache priorities and
cache shrinking drivers?


Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Ed Tomlinson]

On May 6, 2002 09:44 pm, William Lee Irwin III wrote

stuff omitted

> Well, I think there are three major design issues. The first is the
> magic number of 32 pages. This (compile-time!) tunable is almost
> guaranteed not to work for everyone. The second is that in order to

Yes.  I figured I would keep it simple for now.   This could be made a
proc tuneable or kswapd could be modifed to autotune this.

> address the issue you're actually concerned about, it seems you would
> have to present some method for caches to know their allocation requests
> would require evicting other useful data to satisfy; for instance,

This is an interesting idea.  Might be as simple as hooking into the slab.c
code and checking if the number of freepages is to small.  Then we could
increment a counter/flag (which counter/flag set at cache create time, 
associated with a 'driver'.  See below).  If this works it would be generic
too...

> evicting pagecache holding useful (but clean) user data to allocate
> dcache. Third, the distinguished position of the dcache is suspicious
> to me; I feel that a greater degree of generality is in order.

In two years I have never seen any cache other than the dcache/icache
cause the problem.  I short, yes its suspicious, but warented (here) by 
experience.

> In short, I don't think you went far enough. How do you feel about
> GFP_SPECULATIVE (a.k.a. GFP_DONT_TRY_TOO_HARD), cache priorities and
> cache shrinking drivers?

Think I will sprinkle slab.c with a printk or two to see if we detect when
it's allocations are eating other caches.  If this works we should be able to
let the vm know when to shrink the slab cache and to let it know which
caches need shrinking (ie shrink_caches becomes a 'driver' to shrink the
dcache/icache family.  kmem_cache_reap being the generic 'driver')

Thanks for the feedback and interesting idea,

Ed Tomlinson

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[William Lee Irwin III]

At some point in the past, I wrote:
>> In short, I don't think you went far enough. How do you feel about
>> GFP_SPECULATIVE (a.k.a. GFP_DONT_TRY_TOO_HARD), cache priorities and
>> cache shrinking drivers?

On Tue, May 07, 2002 at 07:41:52AM -0400, Ed Tomlinson wrote:
> Think I will sprinkle slab.c with a printk or two to see if we detect when
> it's allocations are eating other caches.  If this works we should be able to
> let the vm know when to shrink the slab cache and to let it know which
> caches need shrinking (ie shrink_caches becomes a 'driver' to shrink the
> dcache/icache family.  kmem_cache_reap being the generic 'driver')
> Thanks for the feedback and interesting idea,

Well, the trick is kmem_cache_reap() doesn't know how to prune
references to things within the cache like prune_dcache() does. It is
in essence its own cache in front of another cache for allocations. I'm
not sure making kmem_cache_reap() trigger reaping of the caches it's
parked in front of is a great idea. It seems that it would go the other
direction: reaping a cache parked in front of a slab would want to call
kmem_cache_reap() sometime afterward (so the memory is actually
reclaimed instead of sitting in the slab cache). IIRC the VM actually
does this at some point after calling the assorted cache shrink functions.
kmem_cache_reap() may well be needed in contexts where the caches are
doing fine jobs of keeping their space under control or shrinking
themselves just fine, without intervention from outside callers.


Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Christoph Hellwig]

On Tue, May 07, 2002 at 05:57:12AM -0700, William Lee Irwin III wrote:
> in essence its own cache in front of another cache for allocations. I'm
> not sure making kmem_cache_reap() trigger reaping of the caches it's
> parked in front of is a great idea. It seems that it would go the other
> direction: reaping a cache parked in front of a slab would want to call
> kmem_cache_reap() sometime afterward (so the memory is actually
> reclaimed instead of sitting in the slab cache). IIRC the VM actually
> does this at some point after calling the assorted cache shrink functions.
> kmem_cache_reap() may well be needed in contexts where the caches are
> doing fine jobs of keeping their space under control or shrinking
> themselves just fine, without intervention from outside callers.

<hint>
In newer Solaris versions (at least SunOS 5.7/5.8) kmem_cache_t has a new
method to allow reclaiming of objects on memory pressure.
</hint>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[William Lee Irwin III]

On Tue, May 07, 2002 at 03:10:57PM +0100, Christoph Hellwig wrote:
> <hint>
> In newer Solaris versions (at least SunOS 5.7/5.8) kmem_cache_t has a new
> method to allow reclaiming of objects on memory pressure.
> </hint>

Well, it can't be that bad of an idea then. I still like keeping the
things separate, but maybe there just isn't that much difference.

Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Nikita Danilov]

William Lee Irwin III writes:
 > At some point in the past, I wrote:
 > >> In short, I don't think you went far enough. How do you feel about
 > >> GFP_SPECULATIVE (a.k.a. GFP_DONT_TRY_TOO_HARD), cache priorities and
 > >> cache shrinking drivers?
 > 
 > On Tue, May 07, 2002 at 07:41:52AM -0400, Ed Tomlinson wrote:
 > > Think I will sprinkle slab.c with a printk or two to see if we detect when
 > > it's allocations are eating other caches.  If this works we should be able to
 > > let the vm know when to shrink the slab cache and to let it know which
 > > caches need shrinking (ie shrink_caches becomes a 'driver' to shrink the
 > > dcache/icache family.  kmem_cache_reap being the generic 'driver')
 > > Thanks for the feedback and interesting idea,
 > 
 > Well, the trick is kmem_cache_reap() doesn't know how to prune
 > references to things within the cache like prune_dcache() does. It is
 > in essence its own cache in front of another cache for allocations. I'm
 > not sure making kmem_cache_reap() trigger reaping of the caches it's
 > parked in front of is a great idea. It seems that it would go the other
 > direction: reaping a cache parked in front of a slab would want to call
 > kmem_cache_reap() sometime afterward (so the memory is actually
 > reclaimed instead of sitting in the slab cache). IIRC the VM actually
 > does this at some point after calling the assorted cache shrink functions.
 > kmem_cache_reap() may well be needed in contexts where the caches are
 > doing fine jobs of keeping their space under control or shrinking
 > themselves just fine, without intervention from outside callers.

I remember Linus once mentioned an idea that slab pages should have
->writepage() that triggers shrink of the front-end cache.

Along this way, VM would just manage one big cache---physical memory.

 > 
 > 
 > Cheers,
 > Bill

Nikita.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Ed Tomlinson]

Hi,

I did something similiar in the patch I posted under the subject:

[RFC][PATCH] cache shrinking via page age

Only I used the same method we now use to shrink caches but triggered them
using page aging, and at the same time making the trigger cache specific.

Another though I had was to put the 'freeable' slab pages onto the inactive
clean list and reclaim them when they reach the head of the list.  It gets a 
little tricky since slabs can contain multiple pages...   Before trying this
I want to see how well what I have posted works.

Ed Tomlinson

On May 13, 2002 07:07 am, Nikita Danilov wrote:
> William Lee Irwin III writes:
>  > At some point in the past, I wrote:
>  > >> In short, I don't think you went far enough. How do you feel about
>  > >> GFP_SPECULATIVE (a.k.a. GFP_DONT_TRY_TOO_HARD), cache priorities and
>  > >> cache shrinking drivers?
>  >
>  > On Tue, May 07, 2002 at 07:41:52AM -0400, Ed Tomlinson wrote:
>  > > Think I will sprinkle slab.c with a printk or two to see if we detect
>  > > when it's allocations are eating other caches.  If this works we
>  > > should be able to let the vm know when to shrink the slab cache and to
>  > > let it know which caches need shrinking (ie shrink_caches becomes a
>  > > 'driver' to shrink the dcache/icache family.  kmem_cache_reap being
>  > > the generic 'driver') Thanks for the feedback and interesting idea,
>  >
>  > Well, the trick is kmem_cache_reap() doesn't know how to prune
>  > references to things within the cache like prune_dcache() does. It is
>  > in essence its own cache in front of another cache for allocations. I'm
>  > not sure making kmem_cache_reap() trigger reaping of the caches it's
>  > parked in front of is a great idea. It seems that it would go the other
>  > direction: reaping a cache parked in front of a slab would want to call
>  > kmem_cache_reap() sometime afterward (so the memory is actually
>  > reclaimed instead of sitting in the slab cache). IIRC the VM actually
>  > does this at some point after calling the assorted cache shrink
>  > functions. kmem_cache_reap() may well be needed in contexts where the
>  > caches are doing fine jobs of keeping their space under control or
>  > shrinking themselves just fine, without intervention from outside
>  > callers.
>
> I remember Linus once mentioned an idea that slab pages should have
> ->writepage() that triggers shrink of the front-end cache.
>
> Along this way, VM would just manage one big cache---physical memory.
>
>  > Cheers,
>  > Bill
>
> Nikita.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: [RFC][PATCH] dcache and rmap

[Andrew Morton]

Ed Tomlinson wrote:
> 
> Hi,
> 
> I did something similiar in the patch I posted under the subject:
> 
> [RFC][PATCH] cache shrinking via page age
> 
> Only I used the same method we now use to shrink caches but triggered them
> using page aging, and at the same time making the trigger cache specific.
> 
> Another though I had was to put the 'freeable' slab pages onto the inactive
> clean list and reclaim them when they reach the head of the list.  It gets a
> little tricky since slabs can contain multiple pages...   Before trying this
> I want to see how well what I have posted works.
> 

Using the VM would be better...

It means that you'll need to create an address_space (and
possibly, at this stage, an inode) to back the slab pages.

Probably there is no need to create a radix tree, nor to give
those pages an ->index.  Just bump page->count to indicate that
the page is "sort-of" in the pagecache.  The kernel presently
assumes, in __remove_inode_page() and __add_to_page_cache()
that these pages are in a radix-tree.  I don't think it needs
to.  Just set slab_space.page_tree to NULL and handle that in
the various places which go oops ;)

Probably there is no need to create a mapping per slab.
A global one should suffice.

There's no need to ever set those pages dirty, hence there's
no need for a ->writepage.

When pages are added to the slab you should put some slab
backpointer into page->private, set PG_private and
increment page->count.

Most of the work will be in slab_space->a_ops.releasepage().
In there you'll need to find the slab via page->private
and just start tossing away pages until you see the target
page come "free".  Then clear PG_private and drop page->count
and return success from ->releasepage().  The page is still
"in the pagecache" so it has an incremented ->count.  The
modified __remove_inode_page() will perform the final release.

That's all fairly straightforward.  The tricky bit is getting
the aging right.  These pages don't have referenced bits in the
pte's.  Possibly, running mark_page_accessed() inside kmem_cache_alloc
would be sufficient.  It would be more accurate to make every user of
a slab object "touch" that object's backing page but that's not
feasible.

-
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: [RFC][PATCH] dcache and rmap

[Martin J. Bligh]

> another good reason to keep these caches small is that
> their data structures are faster to traverse.  when
> they are larger than necessary they probably evict more
> important data from the L1 cache during a dcache or
> inode lookup.

I think a better way to fix this would be by not dumping
everything into one massive global hash table. 

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

RE: [RFC][PATCH] dcache and rmap

[Lever, Charles]

another good reason to keep these caches small is that
their data structures are faster to traverse.  when
they are larger than necessary they probably evict more
important data from the L1 cache during a dcache or
inode lookup.

the most important thing is to make sure ed's "aging"
process doesn't do more damage than good by being
expensive to run and by purging too aggressively.

> -----Original Message-----
> From: Ed Tomlinson [mailto:tomlins@cam.org]
> Sent: Monday, May 06, 2002 11:13 AM
> To: Martin J. Bligh; linux-mm@kvack.org
> Subject: Re: [RFC][PATCH] dcache and rmap
> 
> 
> On May 6, 2002 10:40 am, Martin J. Bligh wrote:
> > >> > I got tired of finding my box with 50-60% percent of 
> memory tied
> > >> > up in dentry/inode caches every morning after update-db runs or
> > >> > after doing a find / -name "*" to generate a list of files for
> > >> > backups.  So I decided to make a stab at fixing this.
> > >>
> > >> Are you actually out of memory at this point, and 
> they're consuming
> > >> space you really need?
> > >
> > > Think of this another way.  There are 100000+ 
> dentry/inodes in memory
> > > comsuming 250M or so.  Meanwhile load is light and the background
> > > aging is able to supply pages for the freelist.  We do 
> not reclaim this
> > > storage until we have vm pressure.  Usually this pressure 
> is artifical,
> > > if we had reclaimed the storage it would not have 
> occured, our caches
> > > would have more useful data in them, and half the memory would not
> > > sit idle for half a day.
> > >
> > > We age the rest of the memory to keep it hot.   Rmap does 
> a good job
> > > and keeps the freelist heathly.  In this case nothing 
> ages the dentries
> > > and they get very cold.  My code ensures that the memory consumed
> > > by the, potentially cold, dentries/inodes is not excessive.
> >
> > If there's no pressure on memory, then using it for caches is a good
> > thing. Why throw away data before we're out of space? If we 
> are under
> 
> The point is there is always memory pressure.  Sometimes kswapd can
> supply the pages needed without calling do_try_to_free_pages. 
>  When this
> happens the dcache/icache can grow since we never try to 
> shrink it.  My
> patch changes this.
> 
> > pressure on memory then dcache should shrink easily and rapidly. If
> > it's not, then make it shrink properly, don't just limit it to an
> > arbitrary size that may be totally unsuitable for some workloads.
> 
> There is _no_ arbitrary limit.  All that happens is that if 
> the dcache grows
> by more than n pages we try to shrink it once.   If the 
> system its actually
> using the dentries, the dcache/icache will grow as needed.  
> If its not using 
> them, which is the case here more often than not, they get dropped and
> storage is freed to be better used by the rest of the vm.
> 
> > You could even age it instead ... that'd make more sense than
> > restricting it to a static size.
> 
> Again I apply pressure.  There are no static limits.  Aging 
> would be nice but
> that would probably require rather massive changes to the way 
> the slab cache
> works.
> 
> Ed Tomlinson
> 
> 
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/
> 
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/