Re: [PATCH] Add __GFP_MOVABLE for callers to flag allocations that may be migrated

[Christoph Lameter <clameter@sgi.com>]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3029 bytes --]

On Tue, 5 Dec 2006, Mel Gorman wrote:

> Portions of it sure, but to offline the DIMM, all pages must be removed from
> it. To guarantee the offlining, that means only __GFP_MOVABLE allocations
> are allowed within that area and a zone is the easiest way to do it.

We were talking about the memory map (page struct array) not the page in 
the DIMM per se. The memory map could also be made movable to deal with 
pages overlapping boundaries (I am not sure that there is really a problem 
at this time, we can probably afford to require the complete removal of a 
page full of page structs. This is true in the the vmemmap case. 
Sparsemem has large memmap chunks that may span multiple pages but 
that could be fixed by changing the chunk size.

> Now, that said, if anti-fragmentation only uses lower PFNs, the number
> of active unmovable pages has to be large enough to span all DIMMs
> before the offlining would fail. This problem will be hit in some
> situations.
> 
> > Set a flag in the page struct of those page struct pages 
> > straddling the border and free the page struct pages describing only
> > memory in the DIMM.
> > 
> 
> I'm not sure what you mean by this. If I wanted to offline a DIMM and I had
> anti-frag (specifically the portion of it that allows a flag that affects a
> whole block of pages), I would mark all the MAX_ORDER_NR_PAGES blocks there
> as going offline so that the pages will not be reallocated. Some time in
> the future, the DIMM will be offlined but it could be an indefinte length
> of time. If the DIMM consisted of just ZONE_MOVABLE, it could be offlined
> in the length of time it takes to migrate all pages elsewhere or page them out.

You have a block full of page structs (that is placed on other memory than 
the DIMM). Some of the pages belonging to the page structs are in the area 
to be offlined and other are not. Then you can remove the pages to be 
offlined from the freelist (if they are on it) and from usage (by 
migration or recleaim) and then mark them as unused. Marking them as 
unused could then be as simple as setting PG_reserved.
> 
> > This is *node* hotplug and we already have a node/zone structure etc where 
> > we could set some option to require only movable allocations.
> 
> True. It would be a bit of a hack, but it's work without needing zones.

We must have some means of marking a node as removalbe anyways in order 
to support node hotplug® What is so hackish about it?

> > I still do not see a need for additional zones.
> 
> It's needed if you want to 100% guarantee the ability to offline a DIMM under
> all circumstances. However, ZONE_MOVABLE comes with it's own problems such
> as not allowing kernel allocations like network buffers.

You cannot offline all DIMMS since (at least at this point in time) we 
need memory that is not movable. If you have multiple DIMMS then the 
additional DIMMS may be placed in areas of a zone that cannot take 
unmovable MAX_ORDER_NR_PAGES blocks.