On Wed, 27 May 2009, Mel Gorman wrote: > On x86 and x86-64, it is possible that page tables are shared beween shared > mappings backed by hugetlbfs. As part of this, page_table_shareable() checks > a pair of vma->vm_flags and they must match if they are to be shared. All > VMA flags are taken into account, including VM_LOCKED. > > The problem is that VM_LOCKED is cleared on fork(). When a process with a > shared memory segment forks() to exec() a helper, there will be shared VMAs > with different flags. The impact is that the shared segment is sometimes > considered shareable and other times not, depending on what process is > checking. > > What happens is that the segment page tables are being shared but the count is > inaccurate depending on the ordering of events. As the page tables are freed > with put_page(), bad pmd's are found when some of the children exit. The > hugepage counters also get corrupted and the Total and Free count will > no longer match even when all the hugepage-backed regions are freed. This > requires a reboot of the machine to "fix". > > This patch addresses the problem by comparing all flags except VM_LOCKED when > deciding if pagetables should be shared or not for hugetlbfs-backed mapping. > > Signed-off-by: Mel Gorman > Acked-by: Hugh Dickins I tested this patch using 2.6.30-rc7 and the libhugetlbfs test suite on x86_64. Everything looks good to me. Acked-by: Eric B Munson Tested-by: Eric B Munson