From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2452DC433EF for ; Sat, 16 Apr 2022 13:24:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E4C56B0073; Sat, 16 Apr 2022 09:24:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 795296B0074; Sat, 16 Apr 2022 09:24:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E6AD6B0075; Sat, 16 Apr 2022 09:24:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 4B0356B0073 for ; Sat, 16 Apr 2022 09:24:18 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0EDAC1234 for ; Sat, 16 Apr 2022 13:24:18 +0000 (UTC) X-FDA: 79362810996.18.E141E17 Received: from mail-yb1-f181.google.com (mail-yb1-f181.google.com [209.85.219.181]) by imf31.hostedemail.com (Postfix) with ESMTP id 79FF720005 for ; Sat, 16 Apr 2022 13:24:17 +0000 (UTC) Received: by mail-yb1-f181.google.com with SMTP id j2so18659234ybu.0 for ; Sat, 16 Apr 2022 06:24:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oU+AwHHHxQPd62TLBBNEIyttpdorCRVi3Dj+2Sxq6dE=; b=o54bG7Wy3WvrnmeIlTKWAVh2brKXaUAF9ROBOfMw5zOiPg7578L7pEc/ykwK0S8kvt AYFu5f0HOhQ/iSiDBDcwiobE2lRwsNGvDDAVeh9zGKKAU2EN17l4J1zQTX1ptFpxYyQs R/19dhbS+CyO6O7l1Uub2ICgeFwGb8gRBOa/2CtaczFexpSuEWBpTa+zG7dI81G/dxOy YvKu0srxJRfzxtXI4j5ClBCWixB1UcEPyy+n2vIg9D5oiw96aITKZKcgAKFKe0muKUrH uUoxVtECRRHI0rdJ6EL2iFy+eO3aCFKha1TCIbfBd9qbMBw9JHs0iXOPXU/HpVm3YSeY LMCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oU+AwHHHxQPd62TLBBNEIyttpdorCRVi3Dj+2Sxq6dE=; b=N2Ed4mQX8mL3jvyqNhNwBw8bDlvt6hMhV9iIEW+chBBIBNh/KUigB1c2Z/kxYu/VYH c0dLzidKJsKH2rR73uljTlhubI+GzJWr4DGsUN/L2+/sRfIFaRMSuetsbn8eZWks5+gc aaQcKd/Dp7gW2QmBxZJCdAQYzqztyMO4D+ltt7XNniDzpvP6nBBSpPpm7VqmhxzwfP51 Q5bw29AKzsOapER3qiIRBsn8lZYN/1dDOwiNYKwlDfllvKvciLreoDeq0PvVhC5ITc8V qcArioq4pm8+ttK9ggh8o6ezEZhG6QgLm1mGH2iNS3Rfc1pgv6AZdmtkJGgLHY/Ue46w WduQ== X-Gm-Message-State: AOAM530nXUIrBTy+s3qVb5ULyWTsCUeAUOOu3DBbkr7FVXaB8OueLi2P +1GjhtqNHF3dziWqUklWPxUwR7UhVf5GgRae1Zg= X-Google-Smtp-Source: ABdhPJw8aC5GotARdN/SpCXlaXgHLYEbOOY2W/sIKwghTYs6WnQxIjVfMD58dgsycB1wyQn0wlmXHBwWcx7O0BZEziY= X-Received: by 2002:a5b:488:0:b0:61d:a178:68c9 with SMTP id n8-20020a5b0488000000b0061da17868c9mr3126073ybp.554.1650115456707; Sat, 16 Apr 2022 06:24:16 -0700 (PDT) MIME-Version: 1.0 References: <20220416123930.5956-1-dthex5d@gmail.com> In-Reply-To: <20220416123930.5956-1-dthex5d@gmail.com> From: dthex5d Date: Sat, 16 Apr 2022 22:24:05 +0900 Message-ID: Subject: Re: [PATCH] mm/mmzone: Introduce a new macro for_each_node_zonelist() To: Andrew Morton , Mike Kravetz , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin Cc: Ohhoon Kwon , JaeSang Yoo , Wonhyuk Yang , Jiyoup Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: multipart/alternative; boundary="000000000000a2b03f05dcc571fd" X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 79FF720005 X-Stat-Signature: 8af73xftduuqtzbfm81tn3ree9hwbidg Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=o54bG7Wy; spf=pass (imf31.hostedemail.com: domain of dthex5d@gmail.com designates 209.85.219.181 as permitted sender) smtp.mailfrom=dthex5d@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1650115457-455595 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000a2b03f05dcc571fd Content-Type: text/plain; charset="UTF-8" There was a compiler error that I missed. Sorry for bothering you guys. I just sent a v2 for this patch. On Sat, Apr 16, 2022 at 9:39 PM Donghyeok Kim wrote: > There are some codes using for_each_zone_zonelist() even when only > iterating each node is needed. This commit introduces a new macro > for_each_node_zonelist() which iterates through valid nodes in the > zonelist. > > By using this new macro, code can be written in a much simpler form. > Also, slab/slub can now skip trying to allocate from the node which was > previously tried and failed. > > Co-developed-by: Ohhoon Kwon > Signed-off-by: Ohhoon Kwon > Signed-off-by: Donghyeok Kim > --- > include/linux/mmzone.h | 36 ++++++++++++++++++++++++++++++++++++ > mm/hugetlb.c | 17 +++++++---------- > mm/mmzone.c | 17 +++++++++++++++++ > mm/slab.c | 7 ++----- > mm/slub.c | 8 ++++---- > mm/vmscan.c | 15 ++++++--------- > 6 files changed, 72 insertions(+), 28 deletions(-) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index 9aaa04ac862f..cb2ddd0b4c95 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -1464,6 +1464,42 @@ static inline struct zoneref > *first_zones_zonelist(struct zonelist *zonelist, > #define for_each_zone_zonelist(zone, z, zlist, highidx) \ > for_each_zone_zonelist_nodemask(zone, z, zlist, highidx, NULL) > > + > +struct zoneref *next_node_zones_zonelist(struct zoneref *z, > + int prev_nid, > + enum zone_type highest_zoneidx, > + nodemask_t *nodes); > + > +/** > + * for_each_node_zonelist_nodemask - helper macro to iterate over valid > nodes in a zonelist which have at least one zone at or below a given zone > index and within a nodemask > + * @node: The current node in the iterator > + * @z: First matched zoneref within current node > + * @zlist: The zonelist being iterated > + * @highidx: The zone index of the highest zone in the node > + * @nodemask: Nodemask allowed by the allocator > + * > + * This iterator iterates through all nodes which have at least one zone > at or below a given zone index and > + * within a given nodemask > + */ > +#define for_each_node_zonelist_nodemask(node, z, zlist, highidx, > nodemask) \ > + for (z = first_zones_zonelist(zlist, highidx, nodemask), > \ > + node = zonelist_zone(z) ? zonelist_node_idx(z) : > NUMA_NO_NODE; \ > + zonelist_zone(z); > \ > + z = next_node_zones_zonelist(++z, node, highidx, > nodemask), \ > + node = zonelist_zone(z) ? zonelist_node_idx(z) : > NUMA_NO_NODE) > + > +/** > + * for_each_node_zonelist - helper macro to iterate over nodes in a > zonelist which have at least one zone at or below a given zone index > + * @node: The current node in the iterator > + * @z: First matched zoneref within current node > + * @zlist: The zonelist being iterated > + * @highidx: The zone index of the highest zone in the node > + * > + * This iterator iterates through all nodes which have at least one zone > at or below a given zone index. > + */ > +#define for_each_node_zonelist(node, z, zlist, highidx) \ > + for_each_node_zonelist_nodemask(node, z, zlist, highidx, NULL) > + > /* Whether the 'nodes' are all movable nodes */ > static inline bool movable_only_nodes(nodemask_t *nodes) > { > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index daa4bdd6c26c..283f28f1aca8 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1157,7 +1157,6 @@ static struct page > *dequeue_huge_page_nodemask(struct hstate *h, gfp_t gfp_mask, > { > unsigned int cpuset_mems_cookie; > struct zonelist *zonelist; > - struct zone *zone; > struct zoneref *z; > int node = NUMA_NO_NODE; > > @@ -1165,18 +1164,16 @@ static struct page > *dequeue_huge_page_nodemask(struct hstate *h, gfp_t gfp_mask, > > retry_cpuset: > cpuset_mems_cookie = read_mems_allowed_begin(); > - for_each_zone_zonelist_nodemask(zone, z, zonelist, > gfp_zone(gfp_mask), nmask) { > + > + /* > + * no need to ask again on the same node. Pool is node rather than > + * zone aware > + */ > + for_each_node_zonelist_nodemask(node, z, zonelist, > gfp_zone(gfp_mask), nmask) { > struct page *page; > > - if (!cpuset_zone_allowed(zone, gfp_mask)) > - continue; > - /* > - * no need to ask again on the same node. Pool is node > rather than > - * zone aware > - */ > - if (zone_to_nid(zone) == node) > + if (!cpuset_node_allowed(node, gfp_mask)) > continue; > - node = zone_to_nid(zone); > > page = dequeue_huge_page_node_exact(h, node); > if (page) > diff --git a/mm/mmzone.c b/mm/mmzone.c > index 68e1511be12d..8b7d6286056e 100644 > --- a/mm/mmzone.c > +++ b/mm/mmzone.c > @@ -72,6 +72,23 @@ struct zoneref *__next_zones_zonelist(struct zoneref *z, > return z; > } > > +/* Returns the zone in the next node and at or below highest_zoneidx in a > zonelist */ > +struct zoneref *next_node_zones_zonelist(struct zoneref *z, > + int prev_nid, > + enum zone_type highest_zoneidx, > + nodemask_t *nodes) > +{ > + if (likely(nodes == NULL)) > + while (z->zone && (zonelist_node_idx(z) == prev_nid || > zonelist_zone_idx(z) > highest_zoneidx)) > + z++; > + else > + while (z->zone && (zonelist_node_idx(z) == prev_nid || > zonelist_zone_idx(z) > highest_zoneidx || > + !zref_in_nodemask(z, nodes))) > + z++; > + > + return z; > +} > + > void lruvec_init(struct lruvec *lruvec) > { > enum lru_list lru; > diff --git a/mm/slab.c b/mm/slab.c > index a301f266efd1..b374fb88f80e 100644 > --- a/mm/slab.c > +++ b/mm/slab.c > @@ -3077,7 +3077,6 @@ static void *fallback_alloc(struct kmem_cache > *cache, gfp_t flags) > { > struct zonelist *zonelist; > struct zoneref *z; > - struct zone *zone; > enum zone_type highest_zoneidx = gfp_zone(flags); > void *obj = NULL; > struct slab *slab; > @@ -3096,10 +3095,8 @@ static void *fallback_alloc(struct kmem_cache > *cache, gfp_t flags) > * Look through allowed nodes for objects available > * from existing per node queues. > */ > - for_each_zone_zonelist(zone, z, zonelist, highest_zoneidx) { > - nid = zone_to_nid(zone); > - > - if (cpuset_zone_allowed(zone, flags) && > + for_each_node_zonelist(nid, z, zonelist, highest_zoneidx) { > + if (cpuset_node_allowed(nid, flags) && > get_node(cache, nid) && > get_node(cache, nid)->free_objects) { > obj = ____cache_alloc_node(cache, > diff --git a/mm/slub.c b/mm/slub.c > index 6dc703488d30..3e8b4aa98b84 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -2192,7 +2192,7 @@ static void *get_any_partial(struct kmem_cache *s, > gfp_t flags, > #ifdef CONFIG_NUMA > struct zonelist *zonelist; > struct zoneref *z; > - struct zone *zone; > + int nid; > enum zone_type highest_zoneidx = gfp_zone(flags); > void *object; > unsigned int cpuset_mems_cookie; > @@ -2222,12 +2222,12 @@ static void *get_any_partial(struct kmem_cache *s, > gfp_t flags, > do { > cpuset_mems_cookie = read_mems_allowed_begin(); > zonelist = node_zonelist(mempolicy_slab_node(), flags); > - for_each_zone_zonelist(zone, z, zonelist, highest_zoneidx) > { > + for_each_node_zonelist(nid, z, zonelist, highest_zoneidx) { > struct kmem_cache_node *n; > > - n = get_node(s, zone_to_nid(zone)); > + n = get_node(s, nid); > > - if (n && cpuset_zone_allowed(zone, flags) && > + if (n && cpuset_node_allowed(nid, flags) && > n->nr_partial > s->min_partial) { > object = get_partial_node(s, n, ret_slab, > flags); > if (object) { > diff --git a/mm/vmscan.c b/mm/vmscan.c > index d4a7d2bd276d..f25b71bf8f61 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -6176,9 +6176,9 @@ static unsigned long do_try_to_free_pages(struct > zonelist *zonelist, > struct scan_control *sc) > { > int initial_priority = sc->priority; > - pg_data_t *last_pgdat; > + pg_data_t *pgdat; > struct zoneref *z; > - struct zone *zone; > + int nid; > retry: > delayacct_freepages_start(); > > @@ -6206,19 +6206,16 @@ static unsigned long do_try_to_free_pages(struct > zonelist *zonelist, > } while (--sc->priority >= 0); > > last_pgdat = NULL; > - for_each_zone_zonelist_nodemask(zone, z, zonelist, sc->reclaim_idx, > + for_each_node_zonelist_nodemask(nid, z, zonelist, sc->reclaim_idx, > sc->nodemask) { > - if (zone->zone_pgdat == last_pgdat) > - continue; > - last_pgdat = zone->zone_pgdat; > + pgdat = NODE_DATA(nid); > > - snapshot_refaults(sc->target_mem_cgroup, zone->zone_pgdat); > + snapshot_refaults(sc->target_mem_cgroup, pgdat); > > if (cgroup_reclaim(sc)) { > struct lruvec *lruvec; > > - lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, > - zone->zone_pgdat); > + lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, > pgdat); > clear_bit(LRUVEC_CONGESTED, &lruvec->flags); > } > } > -- > 2.17.1 > > --000000000000a2b03f05dcc571fd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
There was a compiler error that I missed.
Sorry=C2=A0f= or=C2=A0bothering you guys.
=C2=A0I just sent a v2 for this patch= .

On Sat, Apr 16, 2022 at 9:39 PM Donghyeok Kim <dthex5d@gmail.com> wrote:
There are some codes using for_each_zon= e_zonelist() even when only
iterating each node is needed. This commit introduces a new macro
for_each_node_zonelist() which iterates through valid nodes in the
zonelist.

By using this new macro, code can be written in a much simpler form.
Also, slab/slub can now skip trying to allocate from the node which was
previously tried and failed.

Co-developed-by: Ohhoon Kwon <ohkwon1043@gmail.com>
Signed-off-by: Ohhoon Kwon <ohkwon1043@gmail.com>
Signed-off-by: Donghyeok Kim <dthex5d@gmail.com>
---
=C2=A0include/linux/mmzone.h | 36 ++++++++++++++++++++++++++++++++++++
=C2=A0mm/hugetlb.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 17 +++++++----= ------
=C2=A0mm/mmzone.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | 17 +++++++++++= ++++++
=C2=A0mm/slab.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 7 += +-----
=C2=A0mm/slub.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 8 += +++----
=C2=A0mm/vmscan.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | 15 ++++++-----= ----
=C2=A06 files changed, 72 insertions(+), 28 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 9aaa04ac862f..cb2ddd0b4c95 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1464,6 +1464,42 @@ static inline struct zoneref *first_zones_zonelist(s= truct zonelist *zonelist,
=C2=A0#define for_each_zone_zonelist(zone, z, zlist, highidx) \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 for_each_zone_zonelist_nodemask(zone, z, zlist,= highidx, NULL)

+
+struct zoneref *next_node_zones_zonelist(struct zoneref *z,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0int prev_= nid,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0enum zone= _type highest_zoneidx,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nodemask_= t *nodes);
+
+/**
+ * for_each_node_zonelist_nodemask - helper macro to iterate over valid no= des in a zonelist which have at least one zone at or below a given zone ind= ex and within a nodemask
+ * @node: The current node in the iterator
+ * @z: First matched zoneref within current node
+ * @zlist: The zonelist being iterated
+ * @highidx: The zone index of the highest zone in the node
+ * @nodemask: Nodemask allowed by the allocator
+ *
+ * This iterator iterates through all nodes which have at least one zone a= t or below a given zone index and
+ * within a given nodemask
+ */
+#define for_each_node_zonelist_nodemask(node, z, zlist, highidx, nodemask)= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0\
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for (z =3D first_zones_zonelist(zlist, highidx,= nodemask),=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 \
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0node =3D zonelist_zone(z) ? zonelist_node_idx(z) : NUMA_NO_NODE;= =C2=A0 \
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0zonelist_zone(z);= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0\
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0z =3D next_node_zon= es_zonelist(++z, node, highidx, nodemask),=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0\
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0node =3D zonelist_zone(z) ? zonelist_node_idx(z) : NUMA_NO_NODE)<= br> +
+/**
+ * for_each_node_zonelist - helper macro to iterate over nodes in a zoneli= st which have at least one zone at or below a given zone index
+ * @node: The current node in the iterator
+ * @z: First matched zoneref within current node
+ * @zlist: The zonelist being iterated
+ * @highidx: The zone index of the highest zone in the node
+ *
+ * This iterator iterates through all nodes which have at least one zone a= t or below a given zone index.
+ */
+#define for_each_node_zonelist(node, z, zlist, highidx) \
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_node_zonelist_nodemask(node, z, zlist,= highidx, NULL)
+
=C2=A0/* Whether the 'nodes' are all movable nodes */
=C2=A0static inline bool movable_only_nodes(nodemask_t *nodes)
=C2=A0{
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index daa4bdd6c26c..283f28f1aca8 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1157,7 +1157,6 @@ static struct page *dequeue_huge_page_nodemask(struct= hstate *h, gfp_t gfp_mask,
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 unsigned int cpuset_mems_cookie;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zonelist *zonelist;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct zone *zone;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zoneref *z;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int node =3D NUMA_NO_NODE;

@@ -1165,18 +1164,16 @@ static struct page *dequeue_huge_page_nodemask(stru= ct hstate *h, gfp_t gfp_mask,

=C2=A0retry_cpuset:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpuset_mems_cookie =3D read_mems_allowed_begin(= );
-=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_zone_zonelist_nodemask(zone, z, zoneli= st, gfp_zone(gfp_mask), nmask) {
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/*
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * no need to ask again on the same node. Pool = is node rather than
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * zone aware
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_node_zonelist_nodemask(node, z, zoneli= st, gfp_zone(gfp_mask), nmask) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct page *page;<= br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!cpuset_zone_al= lowed(zone, gfp_mask))
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0continue;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/*
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 * no need to ask a= gain on the same node. Pool is node rather than
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 * zone aware
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (zone_to_nid(zon= e) =3D=3D node)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!cpuset_node_al= lowed(node, gfp_mask))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 continue;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0node =3D zone_to_ni= d(zone);

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 page =3D dequeue_hu= ge_page_node_exact(h, node);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (page)
diff --git a/mm/mmzone.c b/mm/mmzone.c
index 68e1511be12d..8b7d6286056e 100644
--- a/mm/mmzone.c
+++ b/mm/mmzone.c
@@ -72,6 +72,23 @@ struct zoneref *__next_zones_zonelist(struct zoneref *z,=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return z;
=C2=A0}

+/* Returns the zone in the next node and at or below highest_zoneidx in a = zonelist */
+struct zoneref *next_node_zones_zonelist(struct zoneref *z,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0int prev_= nid,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0enum zone= _type highest_zoneidx,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nodemask_= t *nodes)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (likely(nodes =3D=3D NULL))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0while (z->zone &= amp;& (zonelist_node_idx(z) =3D=3D prev_nid || zonelist_zone_idx(z) >= ; highest_zoneidx))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0z++;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0else
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0while (z->zone &= amp;& (zonelist_node_idx(z) =3D=3D prev_nid || zonelist_zone_idx(z) >= ; highest_zoneidx ||
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0!zref_in_nodemask(z, nodes)))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0z++;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0return z;
+}
+
=C2=A0void lruvec_init(struct lruvec *lruvec)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 enum lru_list lru;
diff --git a/mm/slab.c b/mm/slab.c
index a301f266efd1..b374fb88f80e 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -3077,7 +3077,6 @@ static void *fallback_alloc(struct kmem_cache *cache,= gfp_t flags)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zonelist *zonelist;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zoneref *z;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct zone *zone;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 enum zone_type highest_zoneidx =3D gfp_zone(fla= gs);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 void *obj =3D NULL;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct slab *slab;
@@ -3096,10 +3095,8 @@ static void *fallback_alloc(struct kmem_cache *cache= , gfp_t flags)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* Look through allowed nodes for objects = available
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* from existing per node queues.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/
-=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_zone_zonelist(zone, z, zonelist, highe= st_zoneidx) {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nid =3D zone_to_nid= (zone);
-
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (cpuset_zone_all= owed(zone, flags) &&
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_node_zonelist(nid, z, zonelist, highes= t_zoneidx) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (cpuset_node_all= owed(nid, flags) &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 get_node(cache, nid) &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 get_node(cache, nid)->free_objects) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 obj =3D ____cache_alloc_node(cache,<= br> diff --git a/mm/slub.c b/mm/slub.c
index 6dc703488d30..3e8b4aa98b84 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2192,7 +2192,7 @@ static void *get_any_partial(struct kmem_cache *s, gf= p_t flags,
=C2=A0#ifdef CONFIG_NUMA
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zonelist *zonelist;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zoneref *z;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct zone *zone;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0int nid;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 enum zone_type highest_zoneidx =3D gfp_zone(fla= gs);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 void *object;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 unsigned int cpuset_mems_cookie;
@@ -2222,12 +2222,12 @@ static void *get_any_partial(struct kmem_cache *s, = gfp_t flags,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 do {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cpuset_mems_cookie = =3D read_mems_allowed_begin();
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 zonelist =3D node_z= onelist(mempolicy_slab_node(), flags);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_zone_zonel= ist(zone, z, zonelist, highest_zoneidx) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_node_zonel= ist(nid, z, zonelist, highest_zoneidx) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 struct kmem_cache_node *n;

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0n =3D get_node(s, zone_to_nid(zone));
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0n =3D get_node(s, nid);

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0if (n && cpuset_zone_allowed(zone, flags) &&
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0if (n && cpuset_node_allowed(nid, flags) &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 n->nr= _partial > s->min_partial) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 object =3D get_partial_node(s, n, re= t_slab, flags);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (object) {
diff --git a/mm/vmscan.c b/mm/vmscan.c
index d4a7d2bd276d..f25b71bf8f61 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -6176,9 +6176,9 @@ static unsigned long do_try_to_free_pages(struct zone= list *zonelist,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 s= truct scan_control *sc)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int initial_priority =3D sc->priority;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0pg_data_t *last_pgdat;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0pg_data_t *pgdat;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 struct zoneref *z;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct zone *zone;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0int nid;
=C2=A0retry:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 delayacct_freepages_start();

@@ -6206,19 +6206,16 @@ static unsigned long do_try_to_free_pages(struct zo= nelist *zonelist,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 } while (--sc->priority >=3D 0);

=C2=A0 =C2=A0 =C2=A0 =C2=A0 last_pgdat =3D NULL;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_zone_zonelist_nodemask(zone, z, zoneli= st, sc->reclaim_idx,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for_each_node_zonelist_nodemask(nid, z, zonelis= t, sc->reclaim_idx,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sc->n= odemask) {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (zone->zone_p= gdat =3D=3D last_pgdat)
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0continue;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0last_pgdat =3D zone= ->zone_pgdat;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pgdat =3D NODE_DATA= (nid);

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0snapshot_refaults(s= c->target_mem_cgroup, zone->zone_pgdat);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0snapshot_refaults(s= c->target_mem_cgroup, pgdat);

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (cgroup_reclaim(= sc)) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 struct lruvec *lruvec;

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0lruvec =3D mem_cgroup_lruvec(sc->target_mem_cgroup,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 zone->zone_pgdat);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0lruvec =3D mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 clear_bit(LRUVEC_CONGESTED, &lruvec->flags);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
--
2.17.1

--000000000000a2b03f05dcc571fd--