Re: [PATCH] GFP_THISNODE for the slab allocator

[Paul Jackson <pj@sgi.com>]

Andrew wrote:
> Could cache a zone* and the cpu number.  If the cpu number has changed
> since last time, do another lookup.

Hmmm ... getting closer.  This doesn't work as stated, because
consecutive requests to allocate a page could use different zonelists,
perhaps from MPOL_BIND, while still coming from the same cpu number.
The cached zone* would be in the wrong zonelist in that case.

How about two struct zone pointers in the task struct?

One caching the zonelist pointer passed into get_page_from_freelist(),
and the other caching the pointer you've been suggesting all along,
to the zone where we found free memory last time we looked.

If that same task tries to allocate a page with a different zonelist
then we fallback to a brute force lookup and reset the cached state.

(Note to self) The cpuset_update_task_memory_state() routine will
have to zap these two cached zone pointers.  That's easy.


Also, as you noticed earlier, we need a way to notice if a once full
zone that we've been skipping over gets some more free memory.

One way to do that would be to add one more (a third) zone* to the
task struct.  This third zone* would point to the next zone to retry
for free memory.

  Once each time we call get_page_from_freelist(), we'd retry one
  zone, to see if it had gained some free memory.

    If it still had no free memory, increment the retry pointer,
    wrapping when it got up to the zone* we were currently getting
    memory from.

    If we discovered some new free memory on the retried node, then
    start using that zone* instead of the one we were using.


Now we're up to three zone* pointers in the task struct:
  base  -- the base zonelist pointer passed to get_page_from_freelist()
  cur   -- the current zone we're getting memory from
  retry -- the next zone to recheck for free memory

If we make the cur and retry pointers be 32 bit indices, instead of
pointers, this saves 64 bits in the task struct on 64 bit arch's.

Calls to get_page_from_freelist() with GFP_HARDWALL -not- set, and
those with ALLOC_CPUSET -not- set, must bypass this cached state.

The micro-optimizations I had in mind to the cpuset_zone_allowed()
call from get_page_from_freelist() are probably still worth doing,
as that code path, from a linear search of the zonelist, is still
necessary in various situations.

How does this sound?

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>