Here's the latest version of my code to allow page tables to be
relocated.  The point of
doinghttps://mail.google.com/a/google.com/?ui=2&ik=d060386038&view=att&th=1174583ed6bc6e0a&attid=0.2
this is two fold.  The first is when a process is migrated from one
numa node to another, with these changes, the page tables can be
migrated as well.  The second is to allow page tables to be relocated
to defragment memory.

These changes cause a roughly 3.5% increase in the lmbench page fault
benchmark time on my x86_64 system.  The data summary is attached.
relocate.lmbench is the lmbench summary of several runs on a kernel
with the changes.  generic.lmbench is the lmbench summary of several
runs on a stock kernel.org 2.6.23 kernel.

This loss in performance is caused by having to rewalk the page tables
after grabbing the spin lock before updating anything. The only way
around this I've thought of is to set a flag in page struct to
indicate if the rewalk is necessary.  I believe this will cause a
cache miss most of the time and end up being worse.  I have not been
able to come up with any better way, so I hope the 3.5% change in the
micro benchmark is work the gain.  It would also not be too difficult
to make these changes a config option.

Comments are welcome.

    Ross