From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx167.postini.com [74.125.245.167]) by kanga.kvack.org (Postfix) with SMTP id D57A96B00E7 for ; Fri, 16 Mar 2012 10:53:03 -0400 (EDT) Message-Id: <20120316144240.763518310@chello.nl> Date: Fri, 16 Mar 2012 15:40:38 +0100 From: Peter Zijlstra Subject: [RFC][PATCH 10/26] mm, mpol: Make mempolicy home-node aware References: <20120316144028.036474157@chello.nl> Content-Disposition: inline; filename=numa-foo-2.patch Sender: owner-linux-mm@kvack.org List-ID: To: Linus Torvalds , Andrew Morton , Thomas Gleixner , Ingo Molnar , Paul Turner , Suresh Siddha , Mike Galbraith , "Paul E. McKenney" , Lai Jiangshan , Dan Smith , Bharata B Rao , Lee Schermerhorn , Andrea Arcangeli , Rik van Riel , Johannes Weiner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Zijlstra Add another layer of fallback policy to make the home node concept useful from a memory allocation PoV. This changes the mpol order to: - vma->vm_ops->get_policy [if applicable] - vma->vm_policy [if applicable] - task->mempolicy - tsk_home_node() preferred [NEW] - default_policy Note that the tsk_home_node() policy has Migrate-on-Fault enabled to facilitate efficient on-demand memory migration. Signed-off-by: Peter Zijlstra --- mm/mempolicy.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -117,6 +117,22 @@ static struct mempolicy default_policy = .flags = MPOL_F_LOCAL, }; +static struct mempolicy preferred_node_policy[MAX_NUMNODES]; + +static struct mempolicy *get_task_policy(struct task_struct *p) +{ + struct mempolicy *pol = p->mempolicy; + int node; + + if (!pol) { + node = tsk_home_node(p); + if (node != -1) + pol = &preferred_node_policy[node]; + } + + return pol; +} + static const struct mempolicy_operations { int (*create)(struct mempolicy *pol, const nodemask_t *nodes); /* @@ -1478,7 +1494,7 @@ asmlinkage long compat_sys_mbind(compat_ struct mempolicy *get_vma_policy(struct task_struct *task, struct vm_area_struct *vma, unsigned long addr) { - struct mempolicy *pol = task->mempolicy; + struct mempolicy *pol = get_task_policy(task); if (vma) { if (vma->vm_ops && vma->vm_ops->get_policy) { @@ -1856,7 +1872,7 @@ alloc_pages_vma(gfp_t gfp, int order, st */ struct page *alloc_pages_current(gfp_t gfp, unsigned order) { - struct mempolicy *pol = current->mempolicy; + struct mempolicy *pol = get_task_policy(current); struct page *page; if (!pol || in_interrupt() || (gfp & __GFP_THISNODE)) @@ -2302,6 +2318,15 @@ void __init numa_policy_init(void) sizeof(struct sp_node), 0, SLAB_PANIC, NULL); + for_each_node(nid) { + preferred_node_policy[nid] = (struct mempolicy) { + .refcnt = ATOMIC_INIT(1), + .mode = MPOL_PREFERRED, + .flags = MPOL_F_MOF, + .v = { .preferred_node = nid, }, + }; + } + /* * Set interleaving policy for system init. Interleaving is only * enabled across suitably sized nodes (default is >= 16MB), or -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org