From: Peter Zijlstra <a.p.zijlstra@chello.nl>
To: linux-mm@kvack.org, linux-kernel@vger.kernel.org
Cc: Bob Picco <bob.picco@hp.com>, Andrew Morton <akpm@osdl.org>,
IWAMOTO Toshihiro <iwamoto@valinux.co.jp>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Christoph Lameter <christoph@lameter.com>,
Wu Fengguang <wfg@mail.ustc.edu.cn>,
Nick Piggin <npiggin@suse.de>, Linus Torvalds <torvalds@osdl.org>,
Rik van Riel <riel@redhat.com>,
Marcelo Tosatti <marcelo.tosatti@cyclades.com>
Subject: [PATCH 09/34] mm: page-replace-move-isolate_lru_pages.patch
Date: Wed, 22 Mar 2006 23:33:10 +0100 [thread overview]
Message-ID: <20060322223238.12658.35814.sendpatchset@twins.localnet> (raw)
In-Reply-To: <20060322223107.12658.14997.sendpatchset@twins.localnet>
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
In anticipation that only the policy implementation will know anything about
the management of the pages, move isolate_lru_pages over to the policy
implementation.
API:
int page_replace_isolate(struct page*)
isolate a single page from the cache mgmt structures - used by page migration.
NOTE: this function leaves the reclaim page state untouched so that it can be
reinserted in the dest. zone at the correct place.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Marcelo Tosatti <marcelo.tosatti@cyclades.com>
---
include/linux/mm_page_replace.h | 6 ++
include/linux/mm_use_once_policy.h | 3 +
include/linux/swap.h | 2
mm/mempolicy.c | 2
mm/useonce.c | 81 +++++++++++++++++++++++++++++++++++++
mm/vmscan.c | 79 ------------------------------------
6 files changed, 92 insertions(+), 81 deletions(-)
Index: linux-2.6-git/mm/mempolicy.c
===================================================================
--- linux-2.6-git.orig/mm/mempolicy.c
+++ linux-2.6-git/mm/mempolicy.c
@@ -552,7 +552,7 @@ static void migrate_page_add(struct page
* Avoid migrating a page that is shared with others.
*/
if ((flags & MPOL_MF_MOVE_ALL) || page_mapcount(page) == 1) {
- if (isolate_lru_page(page))
+ if (page_replace_isolate(page))
list_add_tail(&page->lru, pagelist);
}
}
Index: linux-2.6-git/mm/useonce.c
===================================================================
--- linux-2.6-git.orig/mm/useonce.c
+++ linux-2.6-git/mm/useonce.c
@@ -75,3 +75,84 @@ int page_replace_add_drain_all(void)
return 0;
}
#endif
+
+#ifdef CONFIG_MIGRATION
+/*
+ * Isolate one page from the LRU lists and put it on the
+ * indicated list with elevated refcount.
+ *
+ * Result:
+ * 0 = page not on LRU list
+ * 1 = page removed from LRU list and added to the specified list.
+ */
+int page_replace_isolate(struct page *page)
+{
+ int ret = 0;
+
+ if (PageLRU(page)) {
+ struct zone *zone = page_zone(page);
+ spin_lock_irq(&zone->lru_lock);
+ if (TestClearPageLRU(page)) {
+ ret = 1;
+ get_page(page);
+ if (PageActive(page))
+ del_page_from_active_list(zone, page);
+ else
+ del_page_from_inactive_list(zone, page);
+ }
+ spin_unlock_irq(&zone->lru_lock);
+ }
+
+ return ret;
+}
+#endif
+
+/*
+ * zone->lru_lock is heavily contended. Some of the functions that
+ * shrink the lists perform better by taking out a batch of pages
+ * and working on them outside the LRU lock.
+ *
+ * For pagecache intensive workloads, this function is the hottest
+ * spot in the kernel (apart from copy_*_user functions).
+ *
+ * Appropriate locks must be held before calling this function.
+ *
+ * @nr_to_scan: The number of pages to look through on the list.
+ * @src: The LRU list to pull pages off.
+ * @dst: The temp list to put pages on to.
+ * @scanned: The number of pages that were scanned.
+ *
+ * returns how many pages were moved onto *@dst.
+ */
+int isolate_lru_pages(int nr_to_scan, struct list_head *src,
+ struct list_head *dst, int *scanned)
+{
+ int nr_taken = 0;
+ struct page *page;
+ int scan = 0;
+
+ while (scan++ < nr_to_scan && !list_empty(src)) {
+ page = lru_to_page(src);
+ prefetchw_prev_lru_page(page, src, flags);
+
+ if (!TestClearPageLRU(page))
+ BUG();
+ list_del(&page->lru);
+ if (get_page_testone(page)) {
+ /*
+ * It is being freed elsewhere
+ */
+ __put_page(page);
+ SetPageLRU(page);
+ list_add(&page->lru, src);
+ continue;
+ } else {
+ list_add(&page->lru, dst);
+ nr_taken++;
+ }
+ }
+
+ *scanned = scan;
+ return nr_taken;
+}
+
Index: linux-2.6-git/mm/vmscan.c
===================================================================
--- linux-2.6-git.orig/mm/vmscan.c
+++ linux-2.6-git/mm/vmscan.c
@@ -509,6 +509,7 @@ keep:
}
#ifdef CONFIG_MIGRATION
+
static inline void move_to_lru(struct page *page)
{
list_del(&page->lru);
@@ -936,87 +937,9 @@ next:
return nr_failed + retry;
}
-
-/*
- * Isolate one page from the LRU lists and put it on the
- * indicated list with elevated refcount.
- *
- * Result:
- * 0 = page not on LRU list
- * 1 = page removed from LRU list and added to the specified list.
- */
-int isolate_lru_page(struct page *page)
-{
- int ret = 0;
-
- if (PageLRU(page)) {
- struct zone *zone = page_zone(page);
- spin_lock_irq(&zone->lru_lock);
- if (TestClearPageLRU(page)) {
- ret = 1;
- get_page(page);
- if (PageActive(page))
- del_page_from_active_list(zone, page);
- else
- del_page_from_inactive_list(zone, page);
- }
- spin_unlock_irq(&zone->lru_lock);
- }
-
- return ret;
-}
#endif
/*
- * zone->lru_lock is heavily contended. Some of the functions that
- * shrink the lists perform better by taking out a batch of pages
- * and working on them outside the LRU lock.
- *
- * For pagecache intensive workloads, this function is the hottest
- * spot in the kernel (apart from copy_*_user functions).
- *
- * Appropriate locks must be held before calling this function.
- *
- * @nr_to_scan: The number of pages to look through on the list.
- * @src: The LRU list to pull pages off.
- * @dst: The temp list to put pages on to.
- * @scanned: The number of pages that were scanned.
- *
- * returns how many pages were moved onto *@dst.
- */
-static int isolate_lru_pages(int nr_to_scan, struct list_head *src,
- struct list_head *dst, int *scanned)
-{
- int nr_taken = 0;
- struct page *page;
- int scan = 0;
-
- while (scan++ < nr_to_scan && !list_empty(src)) {
- page = lru_to_page(src);
- prefetchw_prev_lru_page(page, src, flags);
-
- if (!TestClearPageLRU(page))
- BUG();
- list_del(&page->lru);
- if (get_page_testone(page)) {
- /*
- * It is being freed elsewhere
- */
- __put_page(page);
- SetPageLRU(page);
- list_add(&page->lru, src);
- continue;
- } else {
- list_add(&page->lru, dst);
- nr_taken++;
- }
- }
-
- *scanned = scan;
- return nr_taken;
-}
-
-/*
* shrink_cache() adds the number of pages reclaimed to sc->nr_reclaimed
*/
static void shrink_cache(struct zone *zone, struct scan_control *sc)
Index: linux-2.6-git/include/linux/mm_use_once_policy.h
===================================================================
--- linux-2.6-git.orig/include/linux/mm_use_once_policy.h
+++ linux-2.6-git/include/linux/mm_use_once_policy.h
@@ -76,5 +76,8 @@ static inline int page_replace_activate(
return 1;
}
+extern int isolate_lru_pages(int nr_to_scan, struct list_head *src,
+ struct list_head *dst, int *scanned);
+
#endif /* __KERNEL__ */
#endif /* _LINUX_MM_USEONCE_POLICY_H */
Index: linux-2.6-git/include/linux/mm_page_replace.h
===================================================================
--- linux-2.6-git.orig/include/linux/mm_page_replace.h
+++ linux-2.6-git/include/linux/mm_page_replace.h
@@ -88,6 +88,12 @@ typedef enum {
/* int page_replace_activate(struct page *page); */
+#ifdef CONFIG_MIGRATION
+extern int page_replace_isolate(struct page *p);
+#else
+static inline int page_replace_isolate(struct page *p) { return -ENOSYS; }
+#endif
+
#ifdef CONFIG_MM_POLICY_USEONCE
#include <linux/mm_use_once_policy.h>
#else
Index: linux-2.6-git/include/linux/swap.h
===================================================================
--- linux-2.6-git.orig/include/linux/swap.h
+++ linux-2.6-git/include/linux/swap.h
@@ -186,7 +186,6 @@ static inline int zone_reclaim(struct zo
#endif
#ifdef CONFIG_MIGRATION
-extern int isolate_lru_page(struct page *p);
extern int putback_lru_pages(struct list_head *l);
extern int migrate_page(struct page *, struct page *);
extern void migrate_page_copy(struct page *, struct page *);
@@ -195,7 +194,6 @@ extern int migrate_pages(struct list_hea
struct list_head *moved, struct list_head *failed);
extern int fail_migrate_page(struct page *, struct page *);
#else
-static inline int isolate_lru_page(struct page *p) { return -ENOSYS; }
static inline int putback_lru_pages(struct list_head *l) { return 0; }
static inline int migrate_pages(struct list_head *l, struct list_head *t,
struct list_head *moved, struct list_head *failed) { return -ENOSYS; }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2006-03-22 22:33 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-03-22 22:31 [PATCH 00/34] mm: Page Replacement Policy Framework Peter Zijlstra
2006-03-22 22:31 ` [PATCH 01/34] mm: kill-page-activate.patch Peter Zijlstra
2006-03-22 22:32 ` [PATCH 02/34] mm: page-replace-kconfig-makefile.patch Peter Zijlstra
2006-03-22 23:03 ` Jeff Garzik
2006-03-22 22:32 ` [PATCH 03/34] mm: page-replace-insert.patch Peter Zijlstra
2006-03-22 22:32 ` [PATCH 04/34] mm: page-replace-use_once.patch Peter Zijlstra
2006-03-22 22:32 ` [PATCH 05/34] mm: page-replace-generic-pagevec.patch Peter Zijlstra
2006-03-22 22:32 ` [PATCH 06/34] mm: page-replace-activate.patch Peter Zijlstra
2006-03-22 22:32 ` [PATCH 07/34] mm: page-replace-move-macros.patch Peter Zijlstra
2006-03-22 22:33 ` [PATCH 08/34] mm: page-replace-move-scan_control.patch Peter Zijlstra
2006-03-22 22:33 ` Peter Zijlstra [this message]
2006-03-22 22:33 ` [PATCH 10/34] mm: page-replace-reinsert.patch Peter Zijlstra
2006-03-22 22:33 ` [PATCH 11/34] mm: page-replace-should_reclaim_mapped.patch Peter Zijlstra
2006-03-22 22:33 ` [PATCH 12/34] mm: page-replace-shrink.patch Peter Zijlstra
2006-03-22 22:33 ` [PATCH 13/34] mm: page-replace-mark-accessed.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 14/34] mm: page-replace-remove-mm_inline.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 15/34] mm: page-replace-rotate.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 16/34] mm: page-replace-init.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 17/34] mm: page-replace-info.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 18/34] mm: page-replace-counts.patch Peter Zijlstra
2006-03-22 22:34 ` [PATCH 19/34] mm: page-replace-data.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 20/34] mm: page-replace-pg_flags.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 21/34] mm: page-replace-nonresident.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 22/34] mm: page-replace-shrink-new.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 23/34] mm: page-replace-documentation.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 24/34] mm: sum_cpu_var.patch Peter Zijlstra
2006-03-22 22:35 ` [PATCH 25/34] mm: kswapd-writeout-wait.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 26/34] mm: clockpro-nonresident.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 27/34] mm: clockpro-ignore_token.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 28/34] mm: clockpro-PG_reclaim2.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 29/34] mm: clockpro-clockpro.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 30/34] mm: cart-nonresident.patch Peter Zijlstra
2006-03-22 22:36 ` [PATCH 31/34] mm: cart-PG_reclaim3.patch Peter Zijlstra
2006-03-22 22:37 ` [PATCH 32/34] mm: cart-cart.patch Peter Zijlstra
2006-03-22 22:37 ` [PATCH 33/34] mm: cart-r.patch Peter Zijlstra
2006-03-22 22:37 ` [PATCH 34/34] mm: random.patch Peter Zijlstra
2006-03-22 22:51 ` [PATCH 00/34] mm: Page Replacement Policy Framework Andrew Morton
2006-03-23 2:21 ` Nick Piggin
2006-03-23 21:13 ` Marcelo Tosatti
2006-03-23 4:01 ` Rik van Riel
2006-03-23 20:53 ` Marcelo Tosatti
2006-03-23 18:15 ` Linus Torvalds
2006-03-23 18:26 ` Rik van Riel
2006-03-23 18:48 ` Diego Calleja
2006-03-23 19:03 ` Peter Zijlstra
2006-03-23 22:30 ` Marcelo Tosatti
2006-03-23 20:49 ` Linus Torvalds
2006-03-23 20:59 ` Rik van Riel
2006-03-24 15:06 ` Helge Hafting
2006-03-28 23:05 ` Elladan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060322223238.12658.35814.sendpatchset@twins.localnet \
--to=a.p.zijlstra@chello.nl \
--cc=akpm@osdl.org \
--cc=bob.picco@hp.com \
--cc=christoph@lameter.com \
--cc=iwamoto@valinux.co.jp \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=marcelo.tosatti@cyclades.com \
--cc=npiggin@suse.de \
--cc=riel@redhat.com \
--cc=torvalds@osdl.org \
--cc=wfg@mail.ustc.edu.cn \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox