instrumentation patch for shrink_mmap to find cause of failures - it did :-)

instrumentation patch for shrink_mmap to find cause of failures - it did :-)

[Roger Larsson]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

When applying this patch and running it a typical output looks like
this:


Jun  7 00:41:10 dox kernel: shrink_mmap:     2     0     0     0  
212     0    0     0     0 

First 2 is PID
then comes the counters for different reasons to continue.

The fourth counter is 'count_noncarezone'

I interprete this as:


a) shrink_mmap hits - we need both Normal and DMA pages.
b) Normal pages are easily fulfilled.
c) DMA pages are rare among all Normal pages.
d) When a Normal page is found it is counted as tested but
  then discarded.
e) while loops ends (before finding any DMA pages)
f) shrink_mmap has failed
g) swapping starts...

What to do about it:
a) Move the zone check early - buffers may get old...
b) Undo counting before continuing due to wrong zone

I and others will examine our options.

/RogerL

--
Home page:
  http://www.norran.net/nra02596/

[-- Attachment #2: filemap.inst --]
[-- Type: text/plain, Size: 2969 bytes --]

--- filemap.c~	Sat Jun  3 19:09:16 2000
+++ filemap.c	Wed Jun  7 00:34:49 2000
@@ -311,6 +311,14 @@
 	int ret = 0, count, nr_dirty;
 	struct list_head * page_lru;
 	struct page * page = NULL;
+	int count_nonbuffers_w_page_gt_1 = 0;
+	int count_nonlockable = 0;
+	int count_failed_try_to_free_buffers = 0;
+	int count_noncarezone = 0;
+	int count_latepagecounterror = 0;
+	int count_async = 0;
+	int count_nonio = 0;
+	int count_mappingbut = 0;
 	
 	count = nr_lru_pages / (priority + 1);
 	nr_dirty = priority;
@@ -337,11 +345,15 @@
 		 * Avoid unscalable SMP locking for pages we can
 		 * immediate tell are untouchable..
 		 */
-		if (!page->buffers && page_count(page) > 1)
+		if (!page->buffers && page_count(page) > 1) {
+		  count_nonbuffers_w_page_gt_1++;
 			goto dispose_continue;
+		}
 
-		if (TryLockPage(page))
+		if (TryLockPage(page)) {
+		  count_nonlockable++;
 			goto dispose_continue;
+		}
 
 		/* Release the pagemap_lru lock even if the page is not yet
 		   queued in any lru queue since we have just locked down
@@ -359,8 +371,10 @@
 		 */
 		if (page->buffers) {
 			int wait = ((gfp_mask & __GFP_IO) && (nr_dirty-- < 0));
-			if (!try_to_free_buffers(page, wait))
+			if (!try_to_free_buffers(page, wait)) {
+			  count_failed_try_to_free_buffers++;
 				goto unlock_continue;
+			}
 			/* page was locked, inode can't go away under us */
 			if (!page->mapping) {
 				atomic_dec(&buffermem_pages);
@@ -372,8 +386,10 @@
 		 * Page is from a zone we don't care about.
 		 * Don't drop page cache entries in vain.
 		 */
-		if (page->zone->free_pages > page->zone->pages_high)
+		if (page->zone->free_pages > page->zone->pages_high) {
+		  count_noncarezone++;
 			goto unlock_continue;
+		}
 
 		/* Take the pagecache_lock spinlock held to avoid
 		   other tasks to notice the page while we are looking at its
@@ -385,8 +401,10 @@
 		 * We can't free pages unless there's just one user
 		 * (count == 2 because we added one ourselves above).
 		 */
-		if (page_count(page) != 2)
+		if (page_count(page) != 2) {
+		  count_latepagecounterror++;
 			goto cache_unlock_continue;
+		}
 
 		/*
 		 * Is it a page swap page? If so, we want to
@@ -407,8 +425,10 @@
 				rw_swap_page(WRITE, page, 0);
 				spin_lock(&pagemap_lru_lock);
 				page_cache_release(page);
+				count_async++;
 				goto dispose_continue;
 			}
+			count_nonio++;
 			goto cache_unlock_continue;
 		}
 
@@ -419,6 +439,7 @@
 				spin_unlock(&pagecache_lock);
 				goto made_inode_progress;
 			}
+			count_mappingbut++;
 			goto cache_unlock_continue;
 		}
 
@@ -447,6 +468,17 @@
 
 out:
 	spin_unlock(&pagemap_lru_lock);
+
+	printk("shrink_mmap: %5d %5d %5d %5d %5d %5d %5d %5d %5d\n",
+	       current->pid,
+	       count_nonbuffers_w_page_gt_1,
+	       count_nonlockable,
+	       count_failed_try_to_free_buffers,
+	       count_noncarezone,
+	       count_latepagecounterror,
+	       count_async,
+	       count_nonio,
+	       count_mappingbut);
 
 	return ret;
 }

Re: instrumentation patch for shrink_mmap to find cause of failures - it did :-)

[Rik van Riel]

On Wed, 7 Jun 2000, Roger Larsson wrote:

> What to do about it:
> a) Move the zone check early - buffers may get old...
> b) Undo counting before continuing due to wrong zone

a) is a non-issue since the buffers will get old just as
   well if there's no memory pressure and shrink_mmap
   isn't run

I think your patch (from the other email) is the correct one
for the current MM structure. I'm definately taking your
results into account for the active/inactive/scavenge list
thing I'm working on right now.

regards,

Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.

Wanna talk about the kernel?  irc.openprojects.net / #kernelnewbies
http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/