[RFC PATCH 0/4] Make MIGRATE_ISOLATE a standalone bit

[Zi Yan <ziy@nvidia.com>]

Hi all,

This patchset moves MIGRATE_ISOLATE to a standalone bit to avoid
being overwritten during pageblock isolation process. Currently,
MIGRATE_ISOLATE is part of enum migratetype (in include/linux/mmzone.h),
thus, setting a pageblock to MIGRATE_ISOLATE overwrites its original
migratetype. This causes pageblock migratetype loss during
alloc_contig_range() and memory offline, especially when the process
fails due to a failed pageblock isolation and the code tries to undo the
finished pageblock isolations.

I am also trying to collect feedback on how to handle MIGRATE_ISOLATE in
existing code. As MIGRATE_ISOLATE becomes a standalone bit like
PB_migrate_skip (pageblock skipped by compaction), in theory, existing code
could be changed to use {get,clear,set}_pageblock_isolate() like
{get,clear,set}_pageblock_skip() and MIGRATE_ISOLATE could be removed
from enum migratetype. But free list has a separate MIGRATE_ISOLATE list
and the memory info code used by OOM also shows free memory with
different migratetypes. I wonder if we want more extensive changes to
existing code to remove MIGRATE_ISOLATE. If not, what can I do to improve
the MIGRATE_ISOLATE specialized code in Patch 1? For example:


void set_pageblock_migratetype(struct page *page, int migratetype)
{
	if (unlikely(page_group_by_mobility_disabled &&
		     migratetype < MIGRATE_PCPTYPES))
		migratetype = MIGRATE_UNMOVABLE;

#ifdef CONFIG_MEMORY_ISOLATION
	if (migratetype == MIGRATE_ISOLATE)
		set_pageblock_isolate(page);
	else
#endif
	{
		if (get_pageblock_isolate(page))
			clear_pageblock_isolate(page);
		set_pfnblock_flags_mask(page, (unsigned long)migratetype,
				page_to_pfn(page), MIGRATETYPE_MASK);
	}
}


Design
===

Pageblock flags are read in words to achieve good performance and existing
pageblock flags take 4 bits per pageblock. To avoid a substantial change
to the pageblock flag code, pageblock flag bits are expanded to use 8
and MIGRATE_ISOLATE is moved to use the last bit (bit 7).

It might look like the pageblock flags have doubled the overhead, but in
reality, the overhead is only 1 byte per 2MB/4MB (based on pageblock config),
or 0.0000476 %.

In terms of performance for changing pageblock types, I did a very
simple test by offlining and onlining all memory of a 16GB VM 10 times
and did not see a noticeable runtime difference with the patchset.


TODOs
===

1. improve pageblock migratetype handling code to be less hacky (see the
   example above).
2. more performance tests on pageblock migratetype change.


Any comment and/or suggestion is welcome. Thanks.


Zi Yan (4):
  mm/page_isolation: make page isolation a standalone bit.
  mm/page_isolation: remove migratetype from
    move_freepages_block_isolate()
  mm/page_isolation: remove migratetype from undo_isolate_page_range()
  mm/page_isolation: remove migratetype parameter from more functions.

 include/linux/mmzone.h          | 24 +++++++++++---
 include/linux/page-isolation.h  | 11 +++----
 include/linux/pageblock-flags.h | 29 ++++++++++++++++-
 mm/memory_hotplug.c             |  5 ++-
 mm/page_alloc.c                 | 55 +++++++++++++++++++++++++++------
 mm/page_isolation.c             | 55 ++++++++++++++-------------------
 6 files changed, 123 insertions(+), 56 deletions(-)

-- 
2.45.2