* [PATCH 1/7] Direct Migration V1: CONFIG_MIGRATION for swap migration
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 2/7] Direct Migration V1: PageSwapCache checks Christoph Lameter
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Marcelo Tosatti,
linux-mm, torvalds, Christoph Lameter, Hirokazu Takahashi,
Magnus Damm, Paul Jackson, Dave Hansen, KAMEZAWA Hiroyuki
Add CONFIG_MIGRATION for page migration support
Include page migration if the system is NUMA or having a memory model
that allows distinct areas of memory (SPARSEMEM, DISCONTIGMEM).
And:
- Only include lru_add_drain_per_cpu if building for an SMP system.
This patch is independent of the rest of the direct migration patchset
and is also needed to complete the swap migration patchset.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/include/linux/swap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/swap.h 2005-11-02 11:39:01.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/swap.h 2005-11-02 11:42:00.000000000 -0800
@@ -179,7 +179,9 @@ extern int vm_swappiness;
extern int isolate_lru_page(struct page *p);
extern int putback_lru_pages(struct list_head *l);
+#ifdef CONFIG_MIGRATION
extern int migrate_pages(struct list_head *l, struct list_head *t);
+#endif
#ifdef CONFIG_MMU
/* linux/mm/shmem.c */
Index: linux-2.6.14-rc5-mm1/mm/Kconfig
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/Kconfig 2005-10-31 14:10:53.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/Kconfig 2005-11-02 11:44:57.000000000 -0800
@@ -132,3 +132,11 @@ config SPLIT_PTLOCK_CPUS
default "4096" if ARM && !CPU_CACHE_VIPT
default "4096" if PARISC && DEBUG_SPINLOCK && !64BIT
default "4"
+
+#
+# support for page migration
+#
+config MIGRATION
+ def_bool y if NUMA || SPARSEMEM || DISCONTIGMEM
+
+
Index: linux-2.6.14-rc5-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/vmscan.c 2005-11-02 11:39:01.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/vmscan.c 2005-11-02 11:54:00.000000000 -0800
@@ -572,6 +572,7 @@ keep:
return reclaimed;
}
+#ifdef CONFIG_MIGRATION
/*
* swapout a single page
* page is locked upon entry, unlocked on exit
@@ -721,6 +722,7 @@ retry_later:
return nr_failed + retry;
}
+#endif
/*
* zone->lru_lock is heavily contended. Some of the functions that
@@ -766,10 +768,12 @@ static int isolate_lru_pages(struct zone
return scanned;
}
+#ifdef CONFIG_SMP
static void lru_add_drain_per_cpu(void *dummy)
{
lru_add_drain();
}
+#endif
/*
* Isolate one page from the LRU lists and put it on the
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 2/7] Direct Migration V1: PageSwapCache checks
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
2005-11-04 23:37 ` [PATCH 1/7] Direct Migration V1: CONFIG_MIGRATION for swap migration Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 3/7] Direct Migration V1: migrate_pages() extension for direct migration Christoph Lameter
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Dave Hansen, linux-mm,
torvalds, Christoph Lameter, Hirokazu Takahashi, Magnus Damm,
Paul Jackson, Marcelo Tosatti, KAMEZAWA Hiroyuki
Check for PageSwapCache after looking up and locking a swap page.
The page migration code may change a swap pte to point to a different page
under lock_page().
If that happens then the vm must retry the lookup operation in the swap
space to find the correct page number. There are a couple of locations
in the VM where a lock_page() is done on a swap page. In these locations
we need to check afterwards if the page was migrated. If the page was migrated
then the old page that was looked up before was freed and no longer has the
PageSwapCache bit set.
Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp>
Signed-off-by: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Christoph Lameter <clameter@@sgi.com>
Index: linux-2.6.14-rc5-mm1/mm/memory.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/memory.c 2005-10-24 10:27:12.000000000 -0700
+++ linux-2.6.14-rc5-mm1/mm/memory.c 2005-10-28 16:09:11.000000000 -0700
@@ -1665,6 +1665,7 @@ static int do_swap_page(struct mm_struct
goto out;
entry = pte_to_swp_entry(orig_pte);
+again:
page = lookup_swap_cache(entry);
if (!page) {
swapin_readahead(entry, address, vma);
@@ -1688,6 +1689,12 @@ static int do_swap_page(struct mm_struct
mark_page_accessed(page);
lock_page(page);
+ if (!PageSwapCache(page)) {
+ /* Page migration has occured */
+ unlock_page(page);
+ page_cache_release(page);
+ goto again;
+ }
/*
* Back out if somebody else already faulted in this pte.
Index: linux-2.6.14-rc5-mm1/mm/shmem.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/shmem.c 2005-10-24 10:27:12.000000000 -0700
+++ linux-2.6.14-rc5-mm1/mm/shmem.c 2005-10-28 16:08:20.000000000 -0700
@@ -1013,6 +1013,14 @@ repeat:
page_cache_release(swappage);
goto repeat;
}
+ if (!PageSwapCache(swappage)) {
+ /* Page migration has occured */
+ shmem_swp_unmap(entry);
+ spin_unlock(&info->lock);
+ unlock_page(swappage);
+ page_cache_release(swappage);
+ goto repeat;
+ }
if (PageWriteback(swappage)) {
shmem_swp_unmap(entry);
spin_unlock(&info->lock);
Index: linux-2.6.14-rc5-mm1/mm/swapfile.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/swapfile.c 2005-10-24 10:27:36.000000000 -0700
+++ linux-2.6.14-rc5-mm1/mm/swapfile.c 2005-10-28 16:08:20.000000000 -0700
@@ -624,6 +624,7 @@ static int try_to_unuse(unsigned int typ
*/
swap_map = &si->swap_map[i];
entry = swp_entry(type, i);
+again:
page = read_swap_cache_async(entry, NULL, 0);
if (!page) {
/*
@@ -658,6 +659,12 @@ static int try_to_unuse(unsigned int typ
wait_on_page_locked(page);
wait_on_page_writeback(page);
lock_page(page);
+ if (!PageSwapCache(page)) {
+ /* Page migration has occured */
+ unlock_page(page);
+ page_cache_release(page);
+ goto again;
+ }
wait_on_page_writeback(page);
/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 3/7] Direct Migration V1: migrate_pages() extension for direct migration
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
2005-11-04 23:37 ` [PATCH 1/7] Direct Migration V1: CONFIG_MIGRATION for swap migration Christoph Lameter
2005-11-04 23:37 ` [PATCH 2/7] Direct Migration V1: PageSwapCache checks Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 4/7] Direct Migration V1: remove_from_swap() to remove swap ptes Christoph Lameter
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Marcelo Tosatti,
linux-mm, torvalds, Christoph Lameter, Hirokazu Takahashi,
Magnus Damm, Paul Jackson, Dave Hansen, KAMEZAWA Hiroyuki
Add direct migration support with fall back to swap.
Direct migration support on top of the swap based page migration facility.
This allows the direct migration of anonymous pages and the migration of
file backed pages by dropping the associated buffers (requires writeout).
Fall back to swap out if necessary.
The patch is based on lots of patches from the hotplug project but the code
was restructured, documented and simplified as much as possible.
Note that an additional patch that defines the migrate_page() method
for filesystems is necessary in order to avoid writeback for anonymous
and file backed pages.
Signed-off-by: Mike Kravetz <kravetz@us.ibm.com>
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/vmscan.c 2005-11-04 09:53:30.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/vmscan.c 2005-11-04 10:23:58.000000000 -0800
@@ -576,10 +576,6 @@ keep:
/*
* swapout a single page
* page is locked upon entry, unlocked on exit
- *
- * return codes:
- * 0 = complete
- * 1 = retry
*/
static int swap_page(struct page *page)
{
@@ -595,42 +591,201 @@ static int swap_page(struct page *page)
case PAGE_KEEP:
case PAGE_ACTIVATE:
goto unlock_retry;
+
case PAGE_SUCCESS:
- goto retry;
+ return -EAGAIN;
+
case PAGE_CLEAN:
; /* try to free the page below */
}
}
if (PagePrivate(page)) {
- if (!try_to_release_page(page, GFP_KERNEL))
+ if (!try_to_release_page(page, GFP_KERNEL) ||
+ (!mapping && page_count(page) == 1))
goto unlock_retry;
- if (!mapping && page_count(page) == 1)
- goto free_it;
}
- if (!remove_mapping(mapping, page))
- goto unlock_retry; /* truncate got there first */
+ if (remove_mapping(mapping, page)) {
+ unlock_page(page);
+ return 0;
+ }
+
+unlock_retry:
+ unlock_page(page);
+
+ return -EAGAIN;
+}
+
+static inline void move_to_lru(struct page *page)
+{
+ list_del(&page->lru);
+ if (PageActive(page)) {
+ /*
+ * lru_cache_add_active checks that
+ * the PG_active bit is off.
+ */
+ ClearPageActive(page);
+ lru_cache_add_active(page);
+ } else
+ lru_cache_add(page);
+ put_page(page);
+}
+
+/*
+ * Page migration was developed in the context of the memory hotplug project.
+ * The main authors of the migration code are:
+ *
+ * IWAMOTO Toshihiro <iwamoto@valinux.co.jp>
+ * Hirokazu Takahashi <taka@valinux.co.jp>
+ * Dave Hansen <haveblue@us.ibm.com>
+ * Christoph Lameter <clameter@sgi.com>
+ */
+
+/*
+ * Remove references for a page and establish the new page with the correct
+ * basic settings to be able to stop accesses to the page.
+ */
+int migrate_page_remove_references(struct page *newpage, struct page *page, int nr_refs)
+{
+ struct address_space *mapping = page_mapping(page);
+ struct page **radix_pointer;
+
+ /* Must have added this page to swap so mapping must exist */
+ BUG_ON(!mapping);
+
+ /* Bail out if there are other users of the page */
+ if (page_mapcount(page) + nr_refs != page_count(page))
+ return 1;
+
+ read_lock_irq(&mapping->tree_lock);
+
+ radix_pointer = (struct page **)radix_tree_lookup_slot(
+ &mapping->page_tree,
+ page_index(page));
+ if (*radix_pointer != page) {
+ /*
+ * Someone else modified the radix tree
+ */
+ read_unlock_irq(&mapping->tree_lock);
+ return 1;
+ }
+
+ /*
+ * Certain minimal information about a page must be available
+ * in order for other subsystems to properly handle the page if they
+ * find it through some of the links that we soon may establish.
+ */
+ get_page(newpage);
+ newpage->index = page_index(page);
+ if (PageSwapCache(page)) {
+ SetPageSwapCache(newpage);
+ set_page_private(newpage, page_private(page));
+ } else
+ newpage->mapping = page->mapping;
+
+ *radix_pointer = newpage;
+ read_unlock_irq(&mapping->tree_lock);
+
+ /*
+ * We are now in the critical section where there is no easy way
+ * out since other processes accessing newpage may have followed
+ * the mapping that we have exstablished above. We need to succeed!
+ */
+ while (page_mapped(page)) {
+ int rc = try_to_unmap(page);
+
+ if (rc == SWAP_SUCCESS)
+ break;
+ /*
+ * If there are other runnable processes then running
+ * them may make it possible to unmap the page
+ */
+ schedule();
+
+ /*
+ * A really unswappable page should not occur here
+ * since we should have checked for the page
+ * not being in a vma that is unswappable
+ * before entering this function.
+ *
+ * Currently we will simply hang if such a page
+ * is encountered here.
+ */
+ }
+
+ __put_page(page); /* mapping removed from page */
+
+ return 0;
+}
+
+/*
+ * Copy the page to its new location
+ */
+void migrate_page_copy(struct page *newpage, struct page *page)
+{
+
+ copy_highpage(newpage, page);
+
+ if (PageError(page))
+ SetPageError(newpage);
+ if (PageReferenced(page))
+ SetPageReferenced(newpage);
+ if (PageUptodate(page))
+ SetPageUptodate(newpage);
+ if (PageActive(page))
+ SetPageActive(newpage);
+ if (PageChecked(page))
+ SetPageChecked(newpage);
+ if (PageMappedToDisk(page))
+ SetPageMappedToDisk(newpage);
+
+ if (PageDirty(page)) {
+ clear_page_dirty_for_io(page);
+ set_page_dirty(newpage);
+ }
-free_it:
/*
- * We may free pages that were taken off the active list
- * by isolate_lru_page. However, free_hot_cold_page will check
- * if the active bit is set. So clear it.
+ * Make the old page a zombie page
*/
+ ClearPageSwapCache(page);
ClearPageActive(page);
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ page->mapping = NULL;
- list_del(&page->lru);
- unlock_page(page);
- put_page(page);
- return 0;
+ /*
+ * If any waiters have accumulated on the new page then
+ * wake them up.
+ */
+ if (PageWriteback(newpage))
+ end_page_writeback(newpage);
+}
-unlock_retry:
- unlock_page(page);
+/*
+ * Common logic to directly migrate a single page suitable for
+ * pages that do not use PagePrivate.
+ *
+ * Pages are locked upon entry and exit.
+ *
+ * It has been verified that the page is not
+ * 1. Undergoing writeback.
+ * 2. Having any additional references besides the radix tree,
+ * page tables and the reference from isolate_lru_page().
+ * 3. Part of a vma that is not swappable
+ */
+int migrate_page(struct page *newpage, struct page *page)
+{
+ BUG_ON(PageWriteback(page)); /* Writeback must be complete */
+
+ if (migrate_page_remove_references(newpage, page, 2))
+ return -EAGAIN;
-retry:
- return 1;
+ migrate_page_copy(newpage, page);
+
+ return 0;
}
+
/*
* migrate_pages
*
@@ -644,10 +799,7 @@ retry:
* are movable anymore because t has become empty
* or no retryable pages exist anymore.
*
- * SIMPLIFIED VERSION: This implementation of migrate_pages
- * is only swapping out pages and never touches the second
- * list. The direct migration patchset
- * extends this function to avoid the use of swap.
+ * Return: Number of pages not migrated when t ran empty.
*/
int migrate_pages(struct list_head *l, struct list_head *t)
{
@@ -658,6 +810,7 @@ int migrate_pages(struct list_head *l, s
struct page *page;
struct page *page2;
int swapwrite = current->flags & PF_SWAPWRITE;
+ int rc = 0;
if (!swapwrite)
current->flags |= PF_SWAPWRITE;
@@ -666,32 +819,48 @@ redo:
retry = 0;
list_for_each_entry_safe(page, page2, l, lru) {
+ struct page *newpage = NULL;
+ struct address_space *mapping;
+
cond_resched();
+ if (t && list_empty(t))
+ break;
+
+ if (page_count(page) == 1) {
+ /* page was freed from under us */
+ move_to_lru(page);
+ continue;
+ }
/*
* Skip locked pages during the first two passes to give the
- * functions holding the lock time to release the page. Later we use
- * lock_page to have a higher chance of acquiring the lock.
+ * functions holding the lock time to release the page.
+ * Later we use lock_page() to have a higher chance of
+ * acquiring the lock.
*/
+ rc = -EAGAIN;
if (pass > 2)
lock_page(page);
else
if (TestSetPageLocked(page))
- goto retry_later;
+ goto next;
/*
- * Only wait on writeback if we have already done a pass where
- * we we may have triggered writeouts for lots of pages.
+ * Only wait on writeback if we have already done a pass
+ * where we we may have triggered writeouts.
*/
if (pass > 0)
wait_on_page_writeback(page);
else
- if (PageWriteback(page)) {
- unlock_page(page);
- goto retry_later;
- }
+ if (PageWriteback(page))
+ goto unlock_page;
#ifdef CONFIG_SWAP
+ /*
+ * Anonymous pages must have swap cache references otherwise
+ * the information contained in the page maps cannot be
+ * preserved.
+ */
if (PageAnon(page) && !PageSwapCache(page)) {
if (!add_to_swap(page)) {
unlock_page(page);
@@ -702,13 +871,80 @@ redo:
}
#endif /* CONFIG_SWAP */
+ if (!t) {
+ rc = swap_page(page);
+ goto next;
+ }
+
+ newpage = lru_to_page(t);
+ lock_page(newpage);
+
/*
* Page is properly locked and writeback is complete.
* Try to migrate the page.
*/
- if (swap_page(page)) {
-retry_later:
+ mapping = page_mapping(page);
+
+ /*
+ * Trigger writeout if page is dirty
+ */
+ if (PageDirty(page)) {
+ switch (pageout(page, mapping)) {
+ case PAGE_KEEP:
+ case PAGE_ACTIVATE:
+ goto unlock_both;
+
+ case PAGE_SUCCESS:
+ unlock_page(newpage);
+ goto next;
+
+ case PAGE_CLEAN:
+ ; /* try to migrate the page below */
+ }
+ }
+
+ /*
+ * If we have no buffer or can release the buffers
+ * then do a simple migration.
+ */
+ if (!page_has_buffers(page) ||
+ try_to_release_page(page, GFP_KERNEL)) {
+ rc = migrate_page(newpage, page);
+ goto unlock_both;
+ }
+
+ /*
+ * On early passes with mapped pages simply
+ * retry. There may be a lock held for some
+ * buffers that may go away later. Later
+ * swap them out.
+ */
+ if (pass > 2) {
+ unlock_page(newpage);
+ newpage = NULL;
+ rc = swap_page(page);
+ goto next;
+ }
+
+unlock_both:
+ unlock_page(newpage);
+
+unlock_page:
+ unlock_page(page);
+
+next:
+ if (rc == -EAGAIN)
+ /* Page should be retried later */
retry++;
+
+ else if (rc) {
+ /* Permanent failure to migrate the page */
+ list_move(&page->lru, &failed);
+ nr_failed++;
+ }
+ else if (newpage) {
+ /* Successful migration. Return page to LRU */
+ move_to_lru(newpage);
}
}
if (retry && pass++ < 10)
@@ -869,21 +1105,6 @@ done:
pagevec_release(&pvec);
}
-static inline void move_to_lru(struct page *page)
-{
- list_del(&page->lru);
- if (PageActive(page)) {
- /*
- * lru_cache_add_active checks that
- * the PG_active bit is off.
- */
- ClearPageActive(page);
- lru_cache_add_active(page);
- } else
- lru_cache_add(page);
- put_page(page);
-}
-
/*
* Add isolated pages on the list back to the LRU
*
Index: linux-2.6.14-rc5-mm1/include/linux/swap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/swap.h 2005-11-04 09:53:30.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/swap.h 2005-11-04 10:04:46.000000000 -0800
@@ -181,6 +181,10 @@ extern int putback_lru_pages(struct list
#ifdef CONFIG_MIGRATION
extern int migrate_pages(struct list_head *l, struct list_head *t);
+
+extern int migrate_page(struct page *, struct page *);
+extern int migrate_page_remove_references(struct page *, struct page *, int);
+extern void migrate_page_copy(struct page *, struct page *);
#endif
#ifdef CONFIG_MMU
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 4/7] Direct Migration V1: remove_from_swap() to remove swap ptes
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
` (2 preceding siblings ...)
2005-11-04 23:37 ` [PATCH 3/7] Direct Migration V1: migrate_pages() extension for direct migration Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 5/7] Direct Migration V1: upgrade MPOL_MF_MOVE and sys_migrate_pages() Christoph Lameter
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Dave Hansen, linux-mm,
torvalds, Christoph Lameter, Hirokazu Takahashi, Magnus Damm,
Paul Jackson, Marcelo Tosatti, KAMEZAWA Hiroyuki
Add remove_from_swap
remove_from_swap() allows the restoration of the pte entries that existed
before page migration occurred for anonymous pages by walking the reverse
maps. This reduces swap use and establishes regular pte's without the need
for page faults.
It may also fix a leak of swap entries that could occur if a page
is freed without locking it first (zap_pte_range?) while migration occurs.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/include/linux/swap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/swap.h 2005-11-04 10:04:46.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/swap.h 2005-11-04 10:24:09.000000000 -0800
@@ -265,6 +265,7 @@ extern int remove_exclusive_swap_page(st
struct backing_dev_info;
extern spinlock_t swap_lock;
+extern int unuse_vma(struct vm_area_struct *vma, swp_entry_t entry, struct page *page);
/* linux/mm/thrash.c */
extern struct mm_struct * swap_token_mm;
Index: linux-2.6.14-rc5-mm1/mm/swapfile.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/swapfile.c 2005-11-04 09:55:21.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/swapfile.c 2005-11-04 10:24:09.000000000 -0800
@@ -477,7 +477,7 @@ static inline int unuse_pud_range(struct
return 0;
}
-static int unuse_vma(struct vm_area_struct *vma,
+int unuse_vma(struct vm_area_struct *vma,
swp_entry_t entry, struct page *page)
{
pgd_t *pgd;
Index: linux-2.6.14-rc5-mm1/mm/rmap.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/rmap.c 2005-10-31 14:10:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/rmap.c 2005-11-04 10:24:09.000000000 -0800
@@ -206,6 +206,25 @@ out:
}
/*
+ * Remove an anonymous page from swap replacing the swap pte's
+ * through real pte's pointing to valid pages.
+ */
+void remove_from_swap(struct page *page)
+{
+ struct anon_vma *anon_vma;
+ struct vm_area_struct *vma;
+
+ anon_vma = page_lock_anon_vma(page);
+ if (!anon_vma)
+ return;
+ list_for_each_entry(vma, &anon_vma->head, anon_vma_node) {
+ swp_entry_t entry = { .val = page_private(page) };
+
+ unuse_vma(vma, entry, page);
+ }
+}
+
+/*
* At what user virtual address is page expected in vma?
*/
static inline unsigned long
Index: linux-2.6.14-rc5-mm1/include/linux/rmap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/rmap.h 2005-10-31 14:10:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/rmap.h 2005-11-04 10:24:09.000000000 -0800
@@ -91,6 +91,7 @@ static inline void page_dup_rmap(struct
*/
int page_referenced(struct page *, int is_locked, int ignore_token);
int try_to_unmap(struct page *);
+void remove_from_swap(struct page *page);
/*
* Called from mm/filemap_xip.c to unmap empty zero page
Index: linux-2.6.14-rc5-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/vmscan.c 2005-11-04 10:23:58.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/vmscan.c 2005-11-04 10:24:57.000000000 -0800
@@ -939,11 +939,13 @@ next:
else if (rc) {
/* Permanent failure to migrate the page */
+ remove_from_swap(page);
list_move(&page->lru, &failed);
nr_failed++;
}
else if (newpage) {
/* Successful migration. Return page to LRU */
+ remove_from_swap(newpage);
move_to_lru(newpage);
}
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 5/7] Direct Migration V1: upgrade MPOL_MF_MOVE and sys_migrate_pages()
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
` (3 preceding siblings ...)
2005-11-04 23:37 ` [PATCH 4/7] Direct Migration V1: remove_from_swap() to remove swap ptes Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 6/7] Direct Migration V1: Avoid writeback using page_migrate() method Christoph Lameter
2005-11-04 23:37 ` [PATCH 7/7] Direct Migration V1: Add gfp_t to add_to_swap() to avoid atomic allocs Christoph Lameter
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Marcelo Tosatti,
linux-mm, torvalds, Christoph Lameter, Hirokazu Takahashi,
Magnus Damm, Paul Jackson, Dave Hansen, KAMEZAWA Hiroyuki
Modify policy layer to support direct page migration
- Add migrate_pages_to() allowing the migration of a list of pages to a
a specified node or to vma with a specific allocation policy.
- Modify do_migrate_pages() to do a staged move of pages from the
source nodes to the target nodes.
The patch depends on Paul Jackson's newest cpuset patchset
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/mm/mempolicy.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/mempolicy.c 2005-11-02 11:39:07.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/mempolicy.c 2005-11-04 14:47:28.000000000 -0800
@@ -89,6 +89,7 @@
/* Internal MPOL_MF_xxx flags */
#define MPOL_MF_DISCONTIG_OK (1<<20) /* Skip checks for continuous vmas */
+#define MPOL_MF_INVERT (1<<21) /* Invert check for nodemask */
static kmem_cache_t *policy_cache;
static kmem_cache_t *sn_cache;
@@ -217,6 +218,7 @@ static void migrate_page_add(struct vm_a
* Avoid migrating a page that is shared by others and not writable.
*/
if ((flags & MPOL_MF_MOVE_ALL) ||
+ !page->mapping ||
PageAnon(page) ||
mapping_writably_mapped(page->mapping) ||
single_mm_mapping(vma->vm_mm, page->mapping)
@@ -257,7 +259,7 @@ static int check_pte_range(struct vm_are
continue;
}
nid = pfn_to_nid(pfn);
- if (!node_isset(nid, *nodes)) {
+ if (!node_isset(nid, *nodes) == !(flags & MPOL_MF_INVERT)) {
if (pagelist) {
struct page *page = pfn_to_page(pfn);
@@ -359,7 +361,8 @@ check_range(struct mm_struct *mm, unsign
first = find_vma(mm, start);
if (!first)
return ERR_PTR(-EFAULT);
- if (first->vm_flags & VM_RESERVED)
+ if (first->vm_flags & VM_RESERVED &&
+ !(flags & MPOL_MF_DISCONTIG_OK))
return ERR_PTR(-EACCES);
prev = NULL;
for (vma = first; vma && vma->vm_start < end; vma = vma->vm_next) {
@@ -446,6 +449,44 @@ static int contextualize_policy(int mode
return mpol_check_policy(mode, nodes);
}
+
+/*
+ * Migrate the list 'pagelist' of pages to a certain destination.
+ *
+ * Specify destination with either non-NULL vma or dest_node >= 0
+ * Return the number of pages not migrated or error code
+ */
+static int migrate_pages_to(struct list_head *pagelist,
+ struct vm_area_struct *vma, int dest)
+{
+ LIST_HEAD(newlist);
+ int err = 0;
+ struct page *page;
+ struct list_head *p;
+
+ list_for_each(p, pagelist) {
+ if (vma)
+ page = alloc_page_vma(GFP_HIGHUSER, vma,
+ vma->vm_start);
+ else
+ page = alloc_pages_node(dest, GFP_HIGHUSER, 0);
+
+ if (!page) {
+ err = -ENOMEM;
+ goto out;
+ }
+ list_add(&page->lru, &newlist);
+ }
+ err = migrate_pages(pagelist, &newlist);
+out:
+ while (!list_empty(&newlist)) {
+ page = list_entry(newlist.next, struct page, lru);
+ list_del(&page->lru);
+ __free_page(page);
+ }
+ return err;
+}
+
long do_mbind(unsigned long start, unsigned long len,
unsigned long mode, nodemask_t *nmask, unsigned long flags)
{
@@ -496,14 +537,22 @@ long do_mbind(unsigned long start, unsig
down_write(&mm->mmap_sem);
vma = check_range(mm, start, end, nmask, flags,
(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ? &pagelist : NULL);
+
err = PTR_ERR(vma);
if (!IS_ERR(vma)) {
+
err = mbind_range(vma, start, end, new);
- if (!list_empty(&pagelist))
- migrate_pages(&pagelist, NULL);
- if (!err && !list_empty(&pagelist) && (flags & MPOL_MF_STRICT))
+
+ if (!err) {
+ if (!list_empty(&pagelist))
+ migrate_pages_to(&pagelist, vma, -1);
+
+ if (!list_empty(&pagelist) && (flags & MPOL_MF_STRICT))
err = -EIO;
+ }
+
}
+
if (!list_empty(&pagelist))
putback_lru_pages(&pagelist);
@@ -631,10 +680,37 @@ long do_get_mempolicy(int *policy, nodem
}
/*
- * For now migrate_pages simply swaps out the pages from nodes that are in
- * the source set but not in the target set. In the future, we would
- * want a function that moves pages between the two nodesets in such
- * a way as to preserve the physical layout as much as possible.
+ * Migrate pages from one node to a target node.
+ * Returns error or the number of pages not migrated.
+ */
+int migrate_to_node(struct mm_struct *mm, int source,
+ int dest, int flags)
+{
+ nodemask_t nmask;
+ LIST_HEAD(pagelist);
+ int err = 0;
+
+ nodes_clear(nmask);
+ node_set(source, nmask);
+
+ check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+ flags | MPOL_MF_DISCONTIG_OK | MPOL_MF_INVERT,
+ &pagelist);
+
+ if (!list_empty(&pagelist)) {
+
+ err = migrate_pages_to(&pagelist, NULL, dest);
+
+ if (!list_empty(&pagelist))
+ putback_lru_pages(&pagelist);
+
+ }
+ return err;
+}
+
+/*
+ * Move pages between the two nodesets so as to preserve the physical
+ * layout as much as possible.
*
* Returns the number of page that could not be moved.
*/
@@ -642,22 +718,76 @@ int do_migrate_pages(struct mm_struct *m
const nodemask_t *from_nodes, const nodemask_t *to_nodes, int flags)
{
LIST_HEAD(pagelist);
- int count = 0;
- nodemask_t nodes;
-
- nodes_andnot(nodes, *from_nodes, *to_nodes);
- nodes_complement(nodes, nodes);
+ int err = 0;
+ nodemask_t tmp;
+ int busy = 0;
down_read(&mm->mmap_sem);
- check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nodes,
- flags | MPOL_MF_DISCONTIG_OK, &pagelist);
- if (!list_empty(&pagelist)) {
- migrate_pages(&pagelist, NULL);
- if (!list_empty(&pagelist))
- count = putback_lru_pages(&pagelist);
+
+/* Find a 'source' bit set in 'tmp' whose corresponding 'dest'
+ * bit in 'to' is not also set in 'tmp'. Clear the found 'source'
+ * bit in 'tmp', and return that <source, dest> pair for migration.
+ * The pair of nodemasks 'to' and 'from' define the map.
+ *
+ * If no pair of bits is found that way, fallback to picking some
+ * pair of 'source' and 'dest' bits that are not the same. If the
+ * 'source' and 'dest' bits are the same, this represents a node
+ * that will be migrating to itself, so no pages need move.
+ *
+ * If no bits are left in 'tmp', or if all remaining bits left
+ * in 'tmp' correspond to the same bit in 'to', return false
+ * (nothing left to migrate).
+ *
+ * This lets us pick a pair of nodes to migrate between, such that
+ * if possible the dest node is not already occupied by some other
+ * source node, minimizing the risk of overloading the memory on a
+ * node that would happen if we migrated incoming memory to a node
+ * before migrating outgoing memory source that same node.
+ *
+ * A single scan of tmp is sufficient. As we go, we remember the
+ * most recent <s, d> pair that moved (s != d). If we find a pair
+ * that not only moved, but what's better, moved to an empty slot
+ * (d is not set in tmp), then we break out then, with that pair.
+ * Otherwise when we finish scannng from_tmp, we at least have the
+ * most recent <s, d> pair that moved. If we get all the way through
+ * the scan of tmp without finding any node that moved, much less
+ * moved to an empty node, then there is nothing left worth migrating.
+ */
+
+ tmp = *from_nodes;
+ while (!nodes_empty(tmp)) {
+ int s,d;
+ int source = -1;
+ int dest = 0;
+
+ for_each_node_mask(s, tmp) {
+
+ d = node_remap(s, from_nodes, to_nodes);
+ if (s == d)
+ continue;
+
+ source = s; /* Node moved. Memorize */
+ dest = d;
+
+ /* dest not in remaining from nodes? */
+ if (!node_isset(dest, tmp))
+ break;
+ }
+ if (source == -1)
+ break;
+
+ node_clear(source, tmp);
+ err = migrate_to_node(mm, source, dest, flags);
+ if (err > 0)
+ busy += err;
+ if (err < 0)
+ break;
}
+
up_read(&mm->mmap_sem);
- return count;
+ if (err < 0)
+ return err;
+ return busy;
}
/*
@@ -754,9 +884,6 @@ asmlinkage long sys_set_mempolicy(int mo
return do_set_mempolicy(mode, &nodes);
}
-/* Macro needed until Paul implements this function in kernel/cpusets.c */
-#define cpuset_mems_allowed(task) node_online_map
-
asmlinkage long sys_migrate_pages(pid_t pid, unsigned long maxnode,
const unsigned long __user *old_nodes,
const unsigned long __user *new_nodes)
@@ -800,8 +927,8 @@ asmlinkage long sys_migrate_pages(pid_t
*
* The permission check was taken from check_kill_permission()
*/
- if ((current->euid ^ task->suid) && (current->euid ^ task->uid) &&
- (current->uid ^ task->suid) && (current->uid ^ task->uid) &&
+ if ((current->euid != task->suid) && (current->euid != task->uid) &&
+ (current->uid != task->suid) && (current->uid != task->uid) &&
!capable(CAP_SYS_ADMIN)) {
err = -EPERM;
goto out;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 6/7] Direct Migration V1: Avoid writeback using page_migrate() method
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
` (4 preceding siblings ...)
2005-11-04 23:37 ` [PATCH 5/7] Direct Migration V1: upgrade MPOL_MF_MOVE and sys_migrate_pages() Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
2005-11-04 23:37 ` [PATCH 7/7] Direct Migration V1: Add gfp_t to add_to_swap() to avoid atomic allocs Christoph Lameter
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Dave Hansen, linux-mm,
torvalds, Christoph Lameter, Hirokazu Takahashi, Magnus Damm,
Paul Jackson, Marcelo Tosatti, KAMEZAWA Hiroyuki
Migrate a page with buffers without requiring writeback
This introduces a new address space operation migrate_page() that
may be used by a filesystem to implement its own version of page migration.
A version is provided that migrates buffers attached to pages. Some
filesystems (ext2, ext3, xfs) are modified to utilize this feature.
The swapper address space operation are modified so that a regular
migrate_pages() will occur for anonymous pages without writeback
(migrate_pages forces every anonymous page to have a swap entry).
Signed-off-by: Mike Kravetz <kravetz@us.ibm.com>
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/include/linux/fs.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/fs.h 2005-10-31 14:11:15.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/fs.h 2005-11-04 11:50:29.000000000 -0800
@@ -327,6 +327,8 @@ struct address_space_operations {
loff_t offset, unsigned long nr_segs);
struct page* (*get_xip_page)(struct address_space *, sector_t,
int);
+ /* migrate the contents of a page to the specified target */
+ int (*migrate_page) (struct page *, struct page *);
};
struct backing_dev_info;
Index: linux-2.6.14-rc5-mm1/mm/swap_state.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/swap_state.c 2005-10-31 14:11:17.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/swap_state.c 2005-11-04 11:50:29.000000000 -0800
@@ -26,6 +26,7 @@ static struct address_space_operations s
.writepage = swap_writepage,
.sync_page = block_sync_page,
.set_page_dirty = __set_page_dirty_nobuffers,
+ .migrate_page = migrate_page,
};
static struct backing_dev_info swap_backing_dev_info = {
Index: linux-2.6.14-rc5-mm1/fs/xfs/linux-2.6/xfs_aops.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/fs/xfs/linux-2.6/xfs_aops.c 2005-10-19 23:23:05.000000000 -0700
+++ linux-2.6.14-rc5-mm1/fs/xfs/linux-2.6/xfs_aops.c 2005-11-04 11:50:29.000000000 -0800
@@ -1356,4 +1356,5 @@ struct address_space_operations linvfs_a
.commit_write = generic_commit_write,
.bmap = linvfs_bmap,
.direct_IO = linvfs_direct_IO,
+ .migrate_page = buffer_migrate_page,
};
Index: linux-2.6.14-rc5-mm1/fs/buffer.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/fs/buffer.c 2005-10-31 14:11:00.000000000 -0800
+++ linux-2.6.14-rc5-mm1/fs/buffer.c 2005-11-04 14:43:55.000000000 -0800
@@ -3024,6 +3024,73 @@ asmlinkage long sys_bdflush(int func, lo
}
/*
+ * Migration function for pages with buffers. This function can only be used
+ * if the underlying filesystem guarantees that no other references to "page"
+ * exist.
+ */
+int buffer_migrate_page(struct page *newpage, struct page *page)
+{
+#ifdef CONFIG_MIGRATION
+ struct address_space *mapping = page->mapping;
+ struct buffer_head *bh, *head;
+
+ if (!mapping)
+ return -EAGAIN;
+
+ if (!page_has_buffers(page))
+ return migrate_page(newpage, page);
+
+ head = page_buffers(page);
+
+ /*
+ * Remove the mapping so the page can be migrated.
+ */
+ if (migrate_page_remove_references(newpage, page, 3))
+ return -EAGAIN;
+
+ spin_lock(&mapping->private_lock);
+
+ bh = head;
+ do {
+ get_bh(bh);
+ lock_buffer(bh);
+ bh = bh->b_this_page;
+
+ } while (bh != head);
+
+ ClearPagePrivate(page);
+ set_page_private(newpage, page_private(page));
+ set_page_private(page, 0);
+ put_page(page);
+ get_page(newpage);
+
+ bh = head;
+ do {
+ set_bh_page(bh, newpage, bh_offset(bh));
+ bh = bh->b_this_page;
+
+ } while (bh != head);
+
+ SetPagePrivate(newpage);
+ spin_unlock(&mapping->private_lock);
+
+ migrate_page_copy(newpage, page);
+
+ spin_lock(&mapping->private_lock);
+ bh = head;
+ do {
+ unlock_buffer(bh);
+ put_bh(bh);
+ bh = bh->b_this_page;
+
+ } while (bh != head);
+ spin_unlock(&mapping->private_lock);
+
+#endif
+ return 0;
+}
+
+/*
* Buffer-head allocation
*/
static kmem_cache_t *bh_cachep;
Index: linux-2.6.14-rc5-mm1/fs/ext3/inode.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/fs/ext3/inode.c 2005-10-31 14:11:03.000000000 -0800
+++ linux-2.6.14-rc5-mm1/fs/ext3/inode.c 2005-11-04 11:50:29.000000000 -0800
@@ -1557,6 +1557,7 @@ static struct address_space_operations e
.invalidatepage = ext3_invalidatepage,
.releasepage = ext3_releasepage,
.direct_IO = ext3_direct_IO,
+ .migrate_page = buffer_migrate_page,
};
static struct address_space_operations ext3_writeback_aops = {
@@ -1570,6 +1571,7 @@ static struct address_space_operations e
.invalidatepage = ext3_invalidatepage,
.releasepage = ext3_releasepage,
.direct_IO = ext3_direct_IO,
+ .migrate_page = buffer_migrate_page,
};
static struct address_space_operations ext3_journalled_aops = {
Index: linux-2.6.14-rc5-mm1/fs/ext2/inode.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/fs/ext2/inode.c 2005-10-31 14:11:03.000000000 -0800
+++ linux-2.6.14-rc5-mm1/fs/ext2/inode.c 2005-11-04 11:50:29.000000000 -0800
@@ -706,6 +706,7 @@ struct address_space_operations ext2_aop
.bmap = ext2_bmap,
.direct_IO = ext2_direct_IO,
.writepages = ext2_writepages,
+ .migrate_page = buffer_migrate_page,
};
struct address_space_operations ext2_aops_xip = {
@@ -723,6 +724,7 @@ struct address_space_operations ext2_nob
.bmap = ext2_bmap,
.direct_IO = ext2_direct_IO,
.writepages = ext2_writepages,
+ .migrate_page = buffer_migrate_page,
};
/*
Index: linux-2.6.14-rc5-mm1/fs/xfs/linux-2.6/xfs_buf.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/fs/xfs/linux-2.6/xfs_buf.c 2005-10-31 14:10:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/fs/xfs/linux-2.6/xfs_buf.c 2005-11-04 11:50:29.000000000 -0800
@@ -1633,6 +1633,7 @@ xfs_mapping_buftarg(
struct address_space *mapping;
static struct address_space_operations mapping_aops = {
.sync_page = block_sync_page,
+ .migrate_page = fail_migrate_page,
};
inode = new_inode(bdev->bd_inode->i_sb);
Index: linux-2.6.14-rc5-mm1/include/linux/buffer_head.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/buffer_head.h 2005-10-31 14:10:59.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/buffer_head.h 2005-11-04 11:50:29.000000000 -0800
@@ -210,6 +210,7 @@ int nobh_truncate_page(struct address_sp
int nobh_writepage(struct page *page, get_block_t *get_block,
struct writeback_control *wbc);
+int buffer_migrate_page(struct page *, struct page *);
/*
* inline definitions
Index: linux-2.6.14-rc5-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/vmscan.c 2005-11-04 10:24:57.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/vmscan.c 2005-11-04 11:50:29.000000000 -0800
@@ -572,6 +572,15 @@ keep:
return reclaimed;
}
+/*
+ * Non migratable page
+ */
+int fail_migrate_page(struct page *newpage, struct page *page)
+{
+ return -EIO;
+}
+
+
#ifdef CONFIG_MIGRATION
/*
* swapout a single page
@@ -885,6 +894,11 @@ redo:
*/
mapping = page_mapping(page);
+ if (mapping->a_ops->migrate_page) {
+ rc = mapping->a_ops->migrate_page(newpage, page);
+ goto unlock_both;
+ }
+
/*
* Trigger writeout if page is dirty
*/
Index: linux-2.6.14-rc5-mm1/include/linux/swap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/swap.h 2005-11-04 10:24:09.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/swap.h 2005-11-04 11:50:29.000000000 -0800
@@ -185,6 +185,7 @@ extern int migrate_pages(struct list_hea
extern int migrate_page(struct page *, struct page *);
extern int migrate_page_remove_references(struct page *, struct page *, int);
extern void migrate_page_copy(struct page *, struct page *);
+extern int fail_migrate_page(struct page *, struct page *);
#endif
#ifdef CONFIG_MMU
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 7/7] Direct Migration V1: Add gfp_t to add_to_swap() to avoid atomic allocs
2005-11-04 23:37 [PATCH 0/7] Direct Migration V1: Overview Christoph Lameter
` (5 preceding siblings ...)
2005-11-04 23:37 ` [PATCH 6/7] Direct Migration V1: Avoid writeback using page_migrate() method Christoph Lameter
@ 2005-11-04 23:37 ` Christoph Lameter
6 siblings, 0 replies; 8+ messages in thread
From: Christoph Lameter @ 2005-11-04 23:37 UTC (permalink / raw)
To: akpm
Cc: Hugh Dickins, Mike Kravetz, linux-kernel, Marcelo Tosatti,
linux-mm, torvalds, Christoph Lameter, Hirokazu Takahashi,
Magnus Damm, Paul Jackson, Dave Hansen, KAMEZAWA Hiroyuki
Add gfp_mask to add_to_swap
The migration code calls the function with GFP_KERNEL
while the swap code calls it with GFP_ATOMIC, because
the migration code can ask the swap code to free some pages
when we're in a low memory situation.
Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp>
Signed-off-by: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.14-rc5-mm1/include/linux/swap.h
===================================================================
--- linux-2.6.14-rc5-mm1.orig/include/linux/swap.h 2005-11-04 10:27:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/include/linux/swap.h 2005-11-04 10:34:36.000000000 -0800
@@ -237,7 +237,7 @@ extern int rw_swap_page_sync(int, swp_en
extern struct address_space swapper_space;
#define total_swapcache_pages swapper_space.nrpages
extern void show_swap_cache_info(void);
-extern int add_to_swap(struct page *);
+extern int add_to_swap(struct page *, gfp_t);
extern void __delete_from_swap_cache(struct page *);
extern void delete_from_swap_cache(struct page *);
extern int move_to_swap_cache(struct page *, swp_entry_t);
Index: linux-2.6.14-rc5-mm1/mm/swap_state.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/swap_state.c 2005-11-04 10:27:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/swap_state.c 2005-11-04 10:35:10.000000000 -0800
@@ -143,7 +143,7 @@ void __delete_from_swap_cache(struct pag
* Allocate swap space for the page and add the page to the
* swap cache. Caller needs to hold the page lock.
*/
-int add_to_swap(struct page * page)
+int add_to_swap(struct page * page, gfp_t gfp_mask)
{
swp_entry_t entry;
int err;
@@ -171,7 +171,7 @@ int add_to_swap(struct page * page)
* Add it to the swap cache and mark it dirty
*/
err = __add_to_swap_cache(page, entry,
- GFP_ATOMIC|__GFP_NOMEMALLOC|__GFP_NOWARN);
+ gfp_mask|__GFP_NOMEMALLOC|__GFP_NOWARN);
switch (err) {
case 0: /* Success */
Index: linux-2.6.14-rc5-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.14-rc5-mm1.orig/mm/vmscan.c 2005-11-04 10:27:52.000000000 -0800
+++ linux-2.6.14-rc5-mm1/mm/vmscan.c 2005-11-04 10:33:38.000000000 -0800
@@ -457,7 +457,7 @@ static int shrink_list(struct list_head
if (PageAnon(page) && !PageSwapCache(page)) {
if (!sc->may_swap)
goto keep_locked;
- if (!add_to_swap(page))
+ if (!add_to_swap(page, GFP_ATOMIC))
goto activate_locked;
}
#endif /* CONFIG_SWAP */
@@ -871,7 +871,7 @@ redo:
* preserved.
*/
if (PageAnon(page) && !PageSwapCache(page)) {
- if (!add_to_swap(page)) {
+ if (!add_to_swap(page, GFP_KERNEL)) {
unlock_page(page);
list_move(&page->lru, &failed);
nr_failed++;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 8+ messages in thread