From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CA1CC433E2 for ; Mon, 7 Sep 2020 16:36:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F034521D80 for ; Mon, 7 Sep 2020 16:36:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F034521D80 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 81ADB6B005C; Mon, 7 Sep 2020 12:36:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 625476B005A; Mon, 7 Sep 2020 12:36:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 428DE6B005D; Mon, 7 Sep 2020 12:36:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id 174FC6B005A for ; Mon, 7 Sep 2020 12:36:45 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CD11A3620 for ; Mon, 7 Sep 2020 16:36:44 +0000 (UTC) X-FDA: 77236819128.14.bit48_1609baa270cd Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id A49EB18229818 for ; Mon, 7 Sep 2020 16:36:44 +0000 (UTC) X-HE-Tag: bit48_1609baa270cd X-Filterd-Recvd-Size: 4425 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Sep 2020 16:36:44 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 3C425ACB5; Mon, 7 Sep 2020 16:36:43 +0000 (UTC) From: Vlastimil Babka To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michal Hocko , Pavel Tatashin , David Hildenbrand , Oscar Salvador , Joonsoo Kim , Vlastimil Babka Subject: [RFC 4/5] mm, page_alloc: cache pageset high and batch in struct zone Date: Mon, 7 Sep 2020 18:36:27 +0200 Message-Id: <20200907163628.26495-5-vbabka@suse.cz> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200907163628.26495-1-vbabka@suse.cz> References: <20200907163628.26495-1-vbabka@suse.cz> MIME-Version: 1.0 X-Rspamd-Queue-Id: A49EB18229818 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All per-cpu pagesets for a zone use the same high and batch values, that = are duplicated there just for performance (locality) reasons. This patch adds= the same variables also to struct zone as 'central' ones. This will be useful later for making possible to disable pcplists tempora= rily by setting high value to 0, while remembering the values for restoring th= em later. But we can also immediately benefit from not updating pagesets of = all possible cpus in case the newly recalculated values (after sysctl change = or memory online/offline) are actually unchanged from the previous ones. Signed-off-by: Vlastimil Babka --- include/linux/mmzone.h | 2 ++ mm/page_alloc.c | 18 +++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 8379432f4f2f..15582ca368b9 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -431,6 +431,8 @@ struct zone { #endif struct pglist_data *zone_pgdat; struct per_cpu_pageset __percpu *pageset; + int pageset_high; + int pageset_batch; =20 #ifndef CONFIG_SPARSEMEM /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a0cab2c6055e..004350a2b6ca 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5834,7 +5834,7 @@ static void build_zonelists(pg_data_t *pgdat) * not check if the processor is online before following the pageset poi= nter. * Other parts of the kernel may not check if the zone is available. */ -static void setup_pageset(struct per_cpu_pageset *p); +static void pageset_init(struct per_cpu_pageset *p); static DEFINE_PER_CPU(struct per_cpu_pageset, boot_pageset); static DEFINE_PER_CPU(struct per_cpu_nodestat, boot_nodestats); =20 @@ -6237,7 +6237,7 @@ static void pageset_init(struct per_cpu_pageset *p) pcp->batch =3D 1; } =20 -static void zone_set_pageset_high_and_batch(struct zone *zone) +static void zone_set_pageset_high_and_batch(struct zone *zone, bool forc= e_update) { unsigned long new_high; unsigned long new_batch; @@ -6256,6 +6256,14 @@ static void zone_set_pageset_high_and_batch(struct= zone *zone) new_batch =3D max(1UL, 1 * new_batch); } =20 + if (zone->pageset_high !=3D new_high || + zone->pageset_batch !=3D new_batch) { + zone->pageset_high =3D new_high; + zone->pageset_batch =3D new_batch; + } else if (!force_update) { + return; + } + for_each_possible_cpu(cpu) { p =3D per_cpu_ptr(zone->pageset, cpu); pageset_update(&p->pcp, new_high, new_batch); @@ -6273,7 +6281,7 @@ void __meminit setup_zone_pageset(struct zone *zone= ) pageset_init(p); } =20 - zone_set_pageset_high_and_batch(zone); + zone_set_pageset_high_and_batch(zone, true); } =20 /* @@ -8038,7 +8046,7 @@ int percpu_pagelist_fraction_sysctl_handler(struct = ctl_table *table, int write, goto out; =20 for_each_populated_zone(zone) - zone_set_pageset_high_and_batch(zone); + zone_set_pageset_high_and_batch(zone, false); out: mutex_unlock(&pcp_batch_high_lock); return ret; @@ -8654,7 +8662,7 @@ EXPORT_SYMBOL(free_contig_range); void __meminit zone_pcp_update(struct zone *zone) { mutex_lock(&pcp_batch_high_lock); - zone_set_pageset_high_and_batch(zone); + zone_set_pageset_high_and_batch(zone, false); mutex_unlock(&pcp_batch_high_lock); } =20 --=20 2.28.0