From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5130BC54791 for ; Mon, 11 Mar 2024 01:39:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68FFD6B0074; Sun, 10 Mar 2024 21:39:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63FAD6B0075; Sun, 10 Mar 2024 21:39:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52E5E6B0078; Sun, 10 Mar 2024 21:39:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 425FE6B0074 for ; Sun, 10 Mar 2024 21:39:40 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D0F8B140207 for ; Mon, 11 Mar 2024 01:39:39 +0000 (UTC) X-FDA: 81883051278.07.153F8DD Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) by imf30.hostedemail.com (Postfix) with ESMTP id 1B3CF80002 for ; Mon, 11 Mar 2024 01:39:36 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=eTy5xV1g; spf=pass (imf30.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.7 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710121178; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=weoYIyWSXjgTWranweffbvJDQ2Tws/LJi4LwMTJKCCg=; b=pw+sdeSPeF5Y1KEiEv9WYc3iM11U4NkR67w8oBAiJEY7jEarADm/abVFt3rQB8BQnjjIlH xKikkUqx4FZs7sopbc4CmwDe2RyrnN00hAOuMtlDiQnO5UCO1R1JTjM16LTaZgiFpQ2zdq MRyraM4Y1B/uIZw1oPSYNry6zNWbl8s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710121178; a=rsa-sha256; cv=none; b=SQuuEmwxoIqnfzTpOKPgW2dkG4vxKcpute8MEhvNaEGaCXnqWn3g1RN24eWn71B087hsFO n56VRVcVRA8D/EpwM90v4BH2kxA2yNC2oLWQnyiJIcwTsIqEEcsxJBvzKI9rotENTqKn1b 0o+Wb4/vW0rcpPkHml26vaCcvBhjhM4= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=eTy5xV1g; spf=pass (imf30.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.7 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710121177; x=1741657177; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=RH97JCVDoNwe8SVC6X+BVVE8z2EgxsVYbKPbHPNPJLY=; b=eTy5xV1gXxSeEatdxEwtXRgVj0PBZENYdXGl7GKHe3RA8dcIGuPwGsBm 8yxt0PPLOvD43o7cv88FbZXgS0nmzo26Lbh7l2HG7wOq4R+hLVRK2dMmp whwTQZvrqQHf1QJ6ErgwrHcjD9C2qdlTlaGvEfdbFUVJhPGUoRsw0O173 zdnFWqei4OAMQAKuZ69+GlL4U5z1zAXP4fb0+PMvDSUVesES7d2HSpIBY LDMHGC1QPGl38AyvO138URny98+4bbbncj+5VdC9o7PQP6atkr2NSb7mG aCm8YLA6ERWRX3CyqOeZopHPojhLOHXMbE/Nl85jpxNx4dHmZkVfz514V g==; X-IronPort-AV: E=McAfee;i="6600,9927,11009"; a="30212907" X-IronPort-AV: E=Sophos;i="6.07,115,1708416000"; d="scan'208";a="30212907" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Mar 2024 18:39:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,115,1708416000"; d="scan'208";a="15665180" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Mar 2024 18:39:30 -0700 From: "Huang, Ying" To: Donet Tom Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Aneesh Kumar , Michal Hocko , Dave Hansen , Mel Gorman , Feng Tang , Andrea Arcangeli , Peter Zijlstra , Ingo Molnar , Rik van Riel , Johannes Weiner , Matthew Wilcox , Vlastimil Babka , Dan Williams , Hugh Dickins , Kefeng Wang , Suren Baghdasaryan Subject: Re: [PATCH v2 2/2] mm/numa_balancing:Allow migrate on protnone reference with MPOL_PREFERRED_MANY policy In-Reply-To: <369d6a58758396335fd1176d97bbca4e7730d75a.1709909210.git.donettom@linux.ibm.com> (Donet Tom's message of "Fri, 8 Mar 2024 09:15:38 -0600") References: <369d6a58758396335fd1176d97bbca4e7730d75a.1709909210.git.donettom@linux.ibm.com> Date: Mon, 11 Mar 2024 09:37:36 +0800 Message-ID: <874jdd5z1b.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 1B3CF80002 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: u9m93tjeqerocadx9nnw9xs7wao8op3e X-HE-Tag: 1710121176-387124 X-HE-Meta: U2FsdGVkX1/smrI+uIq4Qjo4yKJ6Hw+Baer8wGs69xZDT2SIsTrMRntEWx76uEOnsoDwxBRY89EhExzl7pDyW019ujxsD78NH/h49ddHcgJB94HcnUne/lks8FRss3iJuGPjb9ydyDg5hqEWX9H2+7+ERQu+Enw55vTRvSJZ8ZaF4ISYRo/RlTqZ7VZ0cr4bfKi50sOcY6lcU/sWCHvTy/v5Ek2288KhnM0JdYI1Sahcljn87CHA8UI3oJ+fmMwBp1i0AQ9RKcqTZisBZd6vxT46Mnrsfr45p6CNQBEdOGtV2qqMTEJicGnDGJv+Q7fm+kIptjkpiJ7CCVLgmoKfIayu4Jup/ULoi+zq7DJ1yndfIskVvBCpZz8sbkJked0zzfZ7k+uHIMlXt0N5Bj1x1YuoY5dwdACZoEULMZKe58n90u1Yrq0kLpQVe2/3GEwegDA3mrrkf1b2WwEtFlEhboedytJiPj9OCf/OJ4OAC9ytH7QExB5g7FMVFxvQWda6u8wVtJivUgdHZnp3cGwPBri/k8PakJEWhl+RRWYcEbGg/WLjMF1fr3YGsUPkCZkzhygc0uZKHHzjk23k2e3y1tNspTalQBDmtZYnkcnNI+PwDmK+Qgm2+oG2vzo7TKrWZwyFWZfxss1v3XNmgH5EBMYHeZ+9itRUzLoFoYpCk2i71+3uD9xDvyMnRAjbVqaD9Tkq9GzO8TD2YUJn5MKcA3GYLMt+KuJgRiODjivqUQVH3AlM+HNinp/tPsXENg9jPmJJawtnEk0h0HJEvpHN1xXbDFJn7O8hdvUvQdmB34HASGoeqmprecuRHupDyfpZCmeD4Kpw/g3mpJOMz2sWFhvWcqcyTV1r2/984gDlqRstE8hktgeadAcCUq4nJ3YfpKhGchjl88FzAx1vedMUPw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Donet Tom writes: > commit bda420b98505 ("numa balancing: migrate on fault among multiple bound > nodes") added support for migrate on protnone reference with MPOL_BIND > memory policy. This allowed numa fault migration when the executing node > is part of the policy mask for MPOL_BIND. This patch extends migration > support to MPOL_PREFERRED_MANY policy. > > Currently, we cannot specify MPOL_PREFERRED_MANY with the mempolicy flag > MPOL_F_NUMA_BALANCING. This causes issues when we want to use > NUMA_BALANCING_MEMORY_TIERING. To effectively use the slow memory tier, > the kernel should not allocate pages from the slower memory tier via > allocation control zonelist fallback. Instead, we should move cold pages > from the faster memory node via memory demotion. For a page allocation, > kswapd is only woken up after we try to allocate pages from all nodes in > the allocation zone list. This implies that, without using memory > policies, we will end up allocating hot pages in the slower memory tier. > > MPOL_PREFERRED_MANY was added by commit b27abaccf8e8 ("mm/mempolicy: add > MPOL_PREFERRED_MANY for multiple preferred nodes") to allow better > allocation control when we have memory tiers in the system. With > MPOL_PREFERRED_MANY, the user can use a policy node mask consisting only > of faster memory nodes. When we fail to allocate pages from the faster > memory node, kswapd would be woken up, allowing demotion of cold pages > to slower memory nodes. > > With the current kernel, such usage of memory policies implies we can't > do page promotion from a slower memory tier to a faster memory tier > using numa fault. This patch fixes this issue. > > For MPOL_PREFERRED_MANY, if the executing node is in the policy node > mask, we allow numa migration to the executing nodes. If the executing > node is not in the policy node mask, we do not allow numa migration. > > Signed-off-by: Aneesh Kumar K.V (IBM) > Signed-off-by: Donet Tom > --- > mm/mempolicy.c | 22 +++++++++++++++++----- > 1 file changed, 17 insertions(+), 5 deletions(-) > > diff --git a/mm/mempolicy.c b/mm/mempolicy.c > index e635d7ed501b..ccd9c6c5fcf5 100644 > --- a/mm/mempolicy.c > +++ b/mm/mempolicy.c > @@ -1458,9 +1458,10 @@ static inline int sanitize_mpol_flags(int *mode, unsigned short *flags) > if ((*flags & MPOL_F_STATIC_NODES) && (*flags & MPOL_F_RELATIVE_NODES)) > return -EINVAL; > if (*flags & MPOL_F_NUMA_BALANCING) { > - if (*mode != MPOL_BIND) > + if (*mode == MPOL_BIND || *mode == MPOL_PREFERRED_MANY) > + *flags |= (MPOL_F_MOF | MPOL_F_MORON); > + else > return -EINVAL; > - *flags |= (MPOL_F_MOF | MPOL_F_MORON); > } > return 0; > } > @@ -2515,15 +2516,26 @@ int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, > break; > > case MPOL_BIND: > - /* Optimize placement among multiple nodes via NUMA balancing */ > + case MPOL_PREFERRED_MANY: > + /* > + * Even though MPOL_PREFERRED_MANY can allocate pages outside > + * policy nodemask we don't allow numa migration to nodes > + * outside policy nodemask for now. This is done so that if we > + * want demotion to slow memory to happen, before allocating > + * from some DRAM node say 'x', we will end up using a > + * MPOL_PREFERRED_MANY mask excluding node 'x'. In such scenario > + * we should not promote to node 'x' from slow memory node. > + */ This is a little hard to digest for me. And, I don't think that we need to put this policy choice in code comments. It's better to put it in patch description. Where we can give more background, for example, to avoid cross-socket traffic, etc. Otherwise, the patchset looks good to me. Thanks! > if (pol->flags & MPOL_F_MORON) { > + /* > + * Optimize placement among multiple nodes > + * via NUMA balancing > + */ > if (node_isset(thisnid, pol->nodes)) > break; > goto out; > } > - fallthrough; > > - case MPOL_PREFERRED_MANY: > /* > * use current page if in policy nodemask, > * else select nearest allowed node, if any. -- Best Regards, Huang, Ying