Re: [PATCH] Reclaim orphaned swap pages

Re: [PATCH] Reclaim orphaned swap pages

[Richard Jerrell]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 561 bytes --]

>From what I can see of the patch vm_enough_memory will still fail causing
premature oom merely because those pages haven't been accounted for as
free.  The checking should probably be done in page_launder, too, to
prevent the write at that level.  Also, the page count might be two,
depending on if you have buffers.  The attached patch should work just
fine and remove these pages as quickly as possible.  I sent it out to the
kernel list on the 27th in response to Stephen Tweedie's point that exit
path deletion was slow, but I guess it belongs here.

Rich

[-- Attachment #2: Type: TEXT/PLAIN, Size: 3485 bytes --]

diff -rNu linux-2.4.1/include/linux/swap.h linux-2.4.1-paging-fix/include/linux/swap.h
--- linux-2.4.1/include/linux/swap.h	Tue Jan 30 02:24:56 2001
+++ linux-2.4.1-paging-fix/include/linux/swap.h	Tue Mar 27 14:00:17 2001
@@ -69,6 +69,7 @@
 FASTCALL(unsigned int nr_free_buffer_pages(void));
 extern int nr_active_pages;
 extern int nr_inactive_dirty_pages;
+extern int nr_swap_cache_pages;
 extern atomic_t nr_async_pages;
 extern struct address_space swapper_space;
 extern atomic_t page_cache_size;
diff -rNu linux-2.4.1/mm/mmap.c linux-2.4.1-paging-fix/mm/mmap.c
--- linux-2.4.1/mm/mmap.c	Mon Jan 29 11:10:41 2001
+++ linux-2.4.1-paging-fix/mm/mmap.c	Tue Mar 27 10:38:17 2001
@@ -63,6 +63,7 @@
 	free += atomic_read(&page_cache_size);
 	free += nr_free_pages();
 	free += nr_swap_pages;
+	free += nr_swap_cache_pages;
 	return free > pages;
 }
 
diff -rNu linux-2.4.1/mm/swap_state.c linux-2.4.1-paging-fix/mm/swap_state.c
--- linux-2.4.1/mm/swap_state.c	Fri Dec 29 18:04:27 2000
+++ linux-2.4.1-paging-fix/mm/swap_state.c	Tue Mar 27 10:41:04 2001
@@ -17,6 +17,8 @@
 
 #include <asm/pgtable.h>
 
+int nr_swap_cache_pages = 0;
+
 static int swap_writepage(struct page *page)
 {
 	rw_swap_page(WRITE, page, 0);
@@ -58,6 +60,7 @@
 #ifdef SWAP_CACHE_INFO
 	swap_cache_add_total++;
 #endif
+	nr_swap_cache_pages++;
 	if (!PageLocked(page))
 		BUG();
 	if (PageTestandSetSwapCache(page))
@@ -96,6 +99,7 @@
 #ifdef SWAP_CACHE_INFO
 	swap_cache_del_total++;
 #endif
+	nr_swap_cache_pages--;
 	remove_from_swap_cache(page);
 	swap_free(entry);
 }
diff -rNu linux-2.4.1/mm/vmscan.c linux-2.4.1-paging-fix/mm/vmscan.c
--- linux-2.4.1/mm/vmscan.c	Mon Jan 15 15:36:49 2001
+++ linux-2.4.1-paging-fix/mm/vmscan.c	Tue Mar 27 14:37:18 2001
@@ -394,6 +394,21 @@
 	return page;
 }
 
+/** 
+ * Short-circuits the dead swap cache page to prevent
+ * unnecessary disk IO
+ */
+static inline void delete_dead_swap_cache_page(struct page *page) {
+	if (block_flushpage(page, 0))
+		lru_cache_del(page);
+	
+	ClearPageDirty(page);
+	spin_unlock(&pagemap_lru_lock);
+	__delete_from_swap_cache(page);
+	spin_lock(&pagemap_lru_lock);
+	page_cache_release(page);
+}
+
 /**
  * page_launder - clean dirty inactive pages, move to inactive_clean list
  * @gfp_mask: what operations we are allowed to do
@@ -467,6 +482,24 @@
 		}
 
 		/*
+		 * Prevent a dead process's pages from being
+		 * written to swap when no one else needs
+		 * them.  Lazy form of removing these pages
+		 * at exit time.
+		 */
+		if (PageSwapCache(page)) {
+			page_cache_get(page);
+			if(!is_page_shared(page)) {
+        			delete_dead_swap_cache_page(page);
+
+				UnlockPage(page);
+				page_cache_release(page);
+				continue;
+			}
+			page_cache_release(page);
+		}
+
+		/*
 		 * Dirty swap-cache page? Write it out if
 		 * last copy..
 		 */
@@ -650,6 +683,24 @@
 			list_del(page_lru);
 			nr_active_pages--;
 			continue;
+		}
+
+		/*
+		 * Prevent a dead process's pages from being
+		 * written to swap when no one else needs
+		 * them.  Lazy form of removing these pages
+		 * at exit time.
+		 */
+		if (PageSwapCache(page)) {
+			page_cache_get(page);
+			if(!is_page_shared(page) && !TryLockPage(page)) {
+				delete_dead_swap_cache_page(page);
+
+				UnlockPage(page);
+				page_cache_release(page);
+				continue;
+			}
+			page_cache_release(page);
 		}
 
 		/* Do aging on the pages. */

Re: [PATCH] Reclaim orphaned swap pages

[Szabolcs Szakacsits]

> From what I can see of the patch vm_enough_memory will still fail
> causing premature oom

Actually if vm_enough_memory fails that prevents oom, apps get ENOMEM
instead of killed by oom_kill later. Moreover vm_enough_memory is long
different and apparently it's just overestimating free pages that makes
people unhappy with the resulted higher oom_kill/ENOMEM rate. If you
want to prevent premature oom you should patch out_of_memory but be
careful if you overestimate you can/will lockup. Anyway I think the
right place for oom_kill would be in the page fault handler [just as for
early kernels] but this needs a small change in __alloc_pages otherwise
processes get stuck there [see goto try_again] when system is low on
memory.

	Szaka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Richard Jerrell]

> Actually if vm_enough_memory fails that prevents oom, apps get ENOMEM
> instead of killed by oom_kill later. Moreover vm_enough_memory is long
> different and apparently it's just overestimating free pages that makes
> people unhappy with the resulted higher oom_kill/ENOMEM rate. If you

That's not really what I'm getting at.  Currently if you run a memory
intensive application, quit after it's pages are on an lru, and try to
restart, you won't be able to get the memory.  This is because pages which
are sitting around in the swap cache are not counted as free, and they
should be, because they are freeable.  So the patch isn't to prevent the
oom killer from terminating your processes, it's to prevent the temporary
memory leak from making your system think it is low on memory when it
really isn't.  When vm_enough_memory says you can get the amount you
request, when you fault on those pages you can trigger the swapper,
launder, and reclaimer.  That will free up the pages bound to the swap
cache that no one cares about and you will get your memory.  If you don't
modify vm_enough_memory, you aren't going to get those pages back until
you happen to trigger the swapping code again.

Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Szabolcs Szakacsits]

On Mon, 2 Apr 2001, Richard Jerrell wrote:

> That's not really what I'm getting at.

Because you are running 2.4.1 kernel, at least your patch was against
it, see the changes since then, upgrade and retry everything [although
sure I don't say there isn't anything to fix/improve ;)].

	Szaka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Marcelo Tosatti]

On Mon, 2 Apr 2001, Richard Jerrell wrote:

> > Actually if vm_enough_memory fails that prevents oom, apps get ENOMEM
> > instead of killed by oom_kill later. Moreover vm_enough_memory is long
> > different and apparently it's just overestimating free pages that makes
> > people unhappy with the resulted higher oom_kill/ENOMEM rate. If you
> 
> That's not really what I'm getting at.  Currently if you run a memory
> intensive application, quit after it's pages are on an lru, and try to
> restart, you won't be able to get the memory.  This is because pages which
> are sitting around in the swap cache are not counted as free, and they
> should be, because they are freeable.

No. Dirty swapcache pages which have pte's referencing them are not
freeable.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Richard Jerrell]

> > That's not really what I'm getting at.  Currently if you run a memory
> > intensive application, quit after it's pages are on an lru, and try to
> > restart, you won't be able to get the memory.  This is because pages which
> > are sitting around in the swap cache are not counted as free, and they
> > should be, because they are freeable.
> 
> No. Dirty swapcache pages which have pte's referencing them are not
> freeable.

If you quit the application, it no longer has ptes which are referencing
the page.  If, in addition, this page no longer has any ptes referencing
it, then it is wasting space.  That is why we free the page (providing
that the swap entry is not shared either).  Otherwise, you will run out of
memory because everything is stuck in the swap cache until it gets
laundered, regardless of whether anyone is still referencing the
page.  That is not a good thing, which is what the patch fixes.

Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Marcelo Tosatti]

On Tue, 3 Apr 2001, Richard Jerrell wrote:

> > > That's not really what I'm getting at.  Currently if you run a memory
> > > intensive application, quit after it's pages are on an lru, and try to
> > > restart, you won't be able to get the memory.  This is because pages which
> > > are sitting around in the swap cache are not counted as free, and they
> > > should be, because they are freeable.
> > 
> > No. Dirty swapcache pages which have pte's referencing them are not
> > freeable.
> 
> If you quit the application, it no longer has ptes which are referencing
> the page.  If, in addition, this page no longer has any ptes referencing
> it, then it is wasting space.  That is why we free the page (providing
> that the swap entry is not shared either).  Otherwise, you will run out of
> memory because everything is stuck in the swap cache until it gets
> laundered, regardless of whether anyone is still referencing the
> page.  That is not a good thing, which is what the patch fixes.

Right.

But you should not count _all_ swapcache pages as freeable. 



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Richard Jerrell]

> But you should not count _all_ swapcache pages as freeable. 

I see what you mean now.  But still, swapcache pages are freeable.  If it
is still in the swapcache, then we still have space reserved on disk for
it.  If your page is being referenced by pte's, then eventually the
swapper will replace those pte's with a reference to the swap cell.  Then,
the swap cache page will be written to disk and reclaimed.  The pages are
not immediately freeable without any additional work, but they are there
because the page is able to be swapped out.  If you are trying to say we
can't count them because we don't know how much work it would take to free
that particular page, then why do we include buffermem pages in the total
of available memory.  We can't be guaranteed that those pages are
freeable, because block_flushpage might just leave them sitting around as
anonymous.  So, basically, I include the swap cache pages in the total
amount of free memory because they are taking up twice as much space as
they should be: one page on disk and one page in memory.

Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Bulent Abali]

>On Wed, 28 Mar 2001, Stephen Tweedie wrote:
>
>> Rik, the patch below tries to reclaim orphaned swap pages after
>> swapped processes exit.  I've only given it basic testing but I want
>> to get feedback on it sooner rather than later --- we need to do
>> _something_ about this problem!
>>
>> The patch works completely differently to the release-on-exit diffs:
>
>It looks good and simple enough to just plug into the
>kernel. I cannot see any problem with this patch, except
>that the PAGECACHE_LOCK macro doesn't seem to exist (yet)
>in my kernel tree ;))

Stephen,

I would like to test your patch.  Is there a resolution for the
non-existent
PAGECACHE_LOCK macro?

I believe I am running in to the orphaned swap pages problem in 2.4.3.
Killing a 900 MB process which is in the midst of being swapped out
leaves a huge chunk in the page cache and the swap file
long after process has exited.  Thanks in advance for any suggestions.

Bulent Abali


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

[PATCH] Reclaim orphaned swap pages

[Stephen Tweedie]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

Hi,

Rik, the patch below tries to reclaim orphaned swap pages after
swapped processes exit.  I've only given it basic testing but I want
to get feedback on it sooner rather than later --- we need to do
_something_ about this problem!

The patch works completely differently to the release-on-exit diffs:
this one works in refill_inactive(), so has zero impact on the hot
paths.  It also works by looking for such orphaned pages in the swap
cache, not by examining swap entries --- it is much cheaper to find
a swap entry for a given page than to find the swap cache page for a
given swap entry.

The patch should be fairly non-intrusive --- all it does is to clear
the age, referenced bit and dirty bit on pages which are swap cached
and which have page count and swap count == 1.  The normal reclaim
path will then recycle these pages rapidly.

We can also extend this to do the swap reclaim that we want.
Currently the recycling only happens if the page count is one, but in
theory we can also do this for pages with higher refcounts if we are
low on swap --- as long as the swap ref is one on a swap cached page,
we know that the swap cache is the only reference to the swap entry
and no ptes can possibly point to that location on disk, so it's safe
to reclaim the swap cache at that point.

Cheers,
 Stephen

[-- Attachment #2: linux-2.4.2-swapreclaim.patch --]
[-- Type: text/plain, Size: 1543 bytes --]

--- linux-2.4.2-0.1.42/mm/vmscan.c.~1~	Wed Mar 28 18:07:20 2001
+++ linux-2.4.2-0.1.42/mm/vmscan.c	Wed Mar 28 19:34:35 2001
@@ -704,6 +704,32 @@
 	return freed_pages + flushed_pages;
 }
 
+/* Check to see if a given page is a swap cache page with no further
+ * references to the swap entry nor to the page in memory.  Clear all of
+ * the aging information for such pages to force them immediately onto
+ * the inactive_clean list for page reclamation. */
+
+static void check_orphaned_swap(struct page *page)
+{
+	spinlock_t *pg_lock = PAGECACHE_LOCK(page);
+
+	if (spin_trylock(pg_lock)) {
+		/* Re-test the page state under protection of the proper
+		 * spinlock */
+		if (PageSwapCache(page) && atomic_read(&page->count) == 1) {
+			if (!TryLockPage(page)) {
+				if (swap_count(page) == 1) {
+					ClearPageDirty(page);
+					ClearPageReferenced(page);
+					page->age = 0;
+				}
+				UnlockPage(page);
+			}
+		}
+		spin_unlock(pg_lock);
+	}
+}
+
 /**
  * refill_inactive_scan - scan the active list and find pages to deactivate
  * @priority: the priority at which to scan
@@ -728,6 +754,12 @@
 		/* Wrong page on list?! (list corruption, should not happen) */
 		if (!PageActive(page))
 			BUG();
+
+		/* Special case: orphaned swap cache pages should be
+		 * reclaimed as quickly as possible, regardless of their
+		 * age or whether they are dirty or not. */
+		if (PageSwapCache(page) && atomic_read(&page->count) == 1)
+			check_orphaned_swap(page);
 
 		/* Do aging on the pages. */
 		if (PageTestandClearReferenced(page)) {

Re: [PATCH] Reclaim orphaned swap pages

[Patrick O'Rourke]

Stephen Tweedie wrote:

 
> The patch works completely differently to the release-on-exit diffs:
> this one works in refill_inactive(), so has zero impact on the hot
> paths.  It also works by looking for such orphaned pages in the swap
> cache, not by examining swap entries --- it is much cheaper to find
> a swap entry for a given page than to find the swap cache page for a
> given swap entry.

It seems we would still have the situation whereby a process will be unable
to allocate memory because vm_enough_memory() fails, even though there is
sufficient orphaned swap pages available to satisfy the request. Isn't it
possible there may be enough free memory such that refill_inactive_scan()
isn't running, but still not enough to satisfy vm_enough_memory()?

Pat

-- 
Patrick O'Rourke
978.606.0236
orourke@missioncriticallinux.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Rik van Riel]

On Wed, 28 Mar 2001, Stephen Tweedie wrote:

> Rik, the patch below tries to reclaim orphaned swap pages after
> swapped processes exit.  I've only given it basic testing but I want
> to get feedback on it sooner rather than later --- we need to do
> _something_ about this problem!
> 
> The patch works completely differently to the release-on-exit diffs:

It looks good and simple enough to just plug into the
kernel. I cannot see any problem with this patch, except
that the PAGECACHE_LOCK macro doesn't seem to exist (yet)
in my kernel tree ;))

cheers,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com.br/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [PATCH] Reclaim orphaned swap pages

[Stephen Tweedie]

Hi,

On Fri, Mar 30, 2001 at 07:16:28PM -0300, Rik van Riel wrote:
> 
> It looks good and simple enough to just plug into the
> kernel. I cannot see any problem with this patch, except
> that the PAGECACHE_LOCK macro doesn't seem to exist (yet)
> in my kernel tree ;))

Yep, I built this on a tree which had Ingo's Tux patches applied and
I'd forgotten that he had added a fine-grained page cache lock to his
code.  Will fix.

Cheers,
 Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/