Re: Page allocator: Single Zone optimizations

[mel@skynet.ie (Mel Gorman)]

On (29/10/06 22:32), Nick Piggin didst pronounce:
> Christoph Lameter wrote:
> >On Sat, 28 Oct 2006, Andrew Morton wrote:
> >
> >
> >>>We (and I personally with the prezeroing patches) have been down 
> >>>this road several times and did not like what we saw. 
> >>
> >>Details?
> >
> >
> >The most important issues that come to my mind right now  (this has 
> >been discussed frequently in various contexts so I may be missing 
> >some things) are:
> >
> >1. Duplicate the caches (pageset structures). This reduces cache hit 
> >   rates. Duplicates lots of information in the page allocator.
> 
> You would have to do the same thing to get an O(1) per-CPU allocation
> for a specific zone/reclaim type/etc regardless whether or not you use
> zones.
> 
> >2. Necessity of additional load balancing across multiple zones.
> 
> a. we have to do this anyway for eg. dma32 and NUMA, and b. it is much
> better than the highmem problem was because all the memory is kernel
> addressable.
> 
> If you use another scheme (eg. lists within zones within nodes, rather
> than just more zones within nodes), then you still fundamentally have
> to balance somehow.
> 
> >3. The NUMA layer can only support memory policies for a single zone.
> 
> That's broken. The VM had zones long before it had nodes or memory
> policies.
> 
> >4. You may have to duplicate the slab allocator caches for that
> >   purpose.
> 
> If you want specific allocations from a given zone, yes. So you may
> have to do the same if you want a specific slab allcoation from a
> list within a zone.
> 
> >5. More bits used in the page flags.
> 
> Aren't there patches to move the bits out of the page flags? A list
> within zones approach would have to use either page flags or some
> external info (eg. page pfn) to determine what list for the page to
> go back to anyway, wouldn't you?
> 
> >6. ZONES have to be sized at bootup which creates more dangers of runinng
> >   out of memory, possibly requiring more complex load balancing.
> 
> Mel's list based defrag approach requires complex load balancing too.
> 

I never really got this objection. With list-based anti-frag, the
zone-balancing logic remains the same. There are patches from Andy
Whitcroft that reclaims pages in contiguous blocks, but still with the same
zone-ordering. It doesn't affect load balancing between zones as such.

With zone-based anti-fragmentation, the load balancing was a bit more
entertaining all right.

In the context of memory hot-unplug though, list-based anti-fragmentation
only really helps you if you can unplug regions of size MAX_ORDER_NR_PAGES. If
you go over that, you need zones.

> >>Again.  On the whole, that was a pretty useless email.  Please give us
> >>something we can use.
> >
> >
> >Well review the discussions that we had regarding Mel Gorman's defrag 
> >approaches. We discussed this in detail at the VM summit and decided to 
> >not create additional zones but instead separate the free lists. You and 
> >Linus seemed to be in agreement with this. I am a bit surprised .... 
> >Is this a Google effect?
> >
> >Moreover the discussion here is only remotely connected to the issue at 
> >hand. We all agree that ZONE_DMA is bad and we want to have an alternate 
> >scheme. Why not continue making it possible to not compile ZONE_DMA 
> >dependent code into the kernel?
> >
> >Single zone patches would increase VM performance. That would in turn 
> >make it more difficult to get approaches in that require multiple zones 
> >since the performance drop would be more significant.
> 
> node->zone->many lists vs node->many zones? I guess the zones approach is
> faster?
> 

Not really. If I have a zone with two sets of free lists or two zones with
one set of free lists each, there are the same number of lists.  However, for
anti-fragmentation with additional lists, you frequently use the preferred list
because they size themselves based on allocator usage patterns.  With zones,
you *must* get the zone sizes right or the performance hit for zone
fallbacks starts becoming noticeable.

> Not that I am any more convinced that defragmentation is a good idea than
> I was a year ago, but I think it is naive to think we can instantly be rid
> of all the problems associated with zones by degenerating that layer of the
> VM and introducing a new one that does basically the same things.
> 
> It is true that zones may not be a perfect fit for what some people want to
> do, but until they have shown a) what they want to do is a good idea, and
> b) zones can't easily be adapted, then using the infrastructure we already
> have throughout the entire mm seems like a good idea.
> 
> IMO, Andrew's idea to have 1..N zones in a node seems sane and it would be
> a good generalisation of even the present code.
> 
> -- 
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>