Re: [PATCH] GFP_THISNODE for the slab allocator

[Paul Jackson <pj@sgi.com>]

Andrew wrote:
> IOW: in which operational scenarios and configurations would you view this
> go-back-to-the-earlier-zone-if-some-memory-came-free-in-it approach
> to be needed?

On fake numa systems, I agree that going back to earlier zones is
not needed.  As you have stated, all nodes are equally good on such
a system.

And besides, right now, I could not give you -any- operational scenario
in which the fake numa approach would be needed.  Perhaps you have
some in mind ...?  I'd be interested to learn how you view these fake
numa based memory containers being used.


On real numa systems, if we don't go back to earlier zones fairly
soon after it is possible to do so, then we are significantly changing
the memory placement behaviour of the system.  That can be risky and is
better not done without good motivation.

If some app running for a while on one cpu, allowed to use memory
on several nodes, had its allocations temporarilly pushed off its
local node, further down its zonelist, it might expect to have its
allocations go back to its local node, just by freeing up memory there.

Many of our most important HPC (High Performance Computing) apps rely
on what they call 'first touch' placement.  That means to them that
memory will be allocated on the node associated with the allocating
thread, or on the closest node thereto.  They will run massive jobs,
with sometimes just a few of the many threads in the job allocating
massive amounts of memory, by the simple expedient of controlling
on which cpu the allocator thread is running as it allocates by
touching the memory pages for the first time.

Their performance can depend critically on getting that memory
placement correct, so that the computational threads are, on average,
as close as can be to their data.

This is the sort of memory placement change that has a decent chance
of coming back around and biting me in the backside, a year or two
down the road, when some app that happened, perhaps unwittingly,
to be sensitive to this change, tripped over it.

I am certainly not saying for sure such a problem would arise.
Good programming practices would suggest not relying on such node
overflow to get memory placed.  But good programming practices are
not always perfectly followed.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>