From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76970C43217 for ; Wed, 13 Oct 2021 16:00:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EBBAE610FE for ; Wed, 13 Oct 2021 16:00:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EBBAE610FE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 8BF2C900002; Wed, 13 Oct 2021 12:00:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 86F986B0074; Wed, 13 Oct 2021 12:00:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73738900002; Wed, 13 Oct 2021 12:00:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 647916B0073 for ; Wed, 13 Oct 2021 12:00:43 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1C7BE184383EF for ; Wed, 13 Oct 2021 16:00:43 +0000 (UTC) X-FDA: 78691877166.12.1FE9D14 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf09.hostedemail.com (Postfix) with ESMTP id A01FF3000108 for ; Wed, 13 Oct 2021 16:00:42 +0000 (UTC) Received: by mail-qv1-f49.google.com with SMTP id m13so1986250qvk.1 for ; Wed, 13 Oct 2021 09:00:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pZexP7HogPfyV8yTiXGv7CUWiNgnazFkMB2vWUC/dbo=; b=GJh4rtFJ38Z6naQzZ8Xp9sIvZwwAZbpY9qdMZbmiWoeDWkKABOU3kTQVN508osXhEu VKW5jW1sgoSolLAwb1r0NnabzjFEaMiy32diG+QLlz+MU0cqbUlTmjbGhHjbTjLxpC1f w6mKKLf+9C7rfLd6c3wvqlGFznroCDRis44gi83jSdA2dW3JGegquav69UiEHLxKzNCO lBqQNR1LC15JS3OonZU+0n8/1DTjpf2Xnykr8W+bto0CmA+g5Sk2SsMZl5kMfpJ52yZZ KzfvEFFuqROwoQNATs36ElJxJO0y7KAv709wsjXk+j4vgqyJO55498hcVI9y9jOqbhzi dDxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pZexP7HogPfyV8yTiXGv7CUWiNgnazFkMB2vWUC/dbo=; b=Vg2Ka6dtXulr16ZApriSktPydGq2yHkxHRTZUHWCxl4gGEonM9Mma1d9ZYMpS+s1fp jYuWpK3yGu0DINNPABeWfqmZMkh3pu9IpCZcAABPa9tdmXCKblSFQghhGkbrhUO8Qu8O JhSE84G7E4+8U+lcQuDXMa0NQwzrPB0fqrDjF9cJng4dfb8KpUi3VRTeo/0X1CuqEGM9 TiyA1TYnXhBVOFlI1WqvVGlhFQZYkfimsWRGI0aQ4AoZokLr+D+rxmFIxbU/YQJTPySi 7heUp2UtUSG6UdheuvpT1oCDv5RleM92/zph5Sm9+cW333KSU9Z3ODMOvPAdOi6ygp2R kcTQ== X-Gm-Message-State: AOAM533BVZEUY8d0Eor4weFM3HAdaICJE9CGdN9KECPNWA2co4lSZhy4 o399+yk0ABvVGc9AIACgVA== X-Google-Smtp-Source: ABdhPJweZh80c8p2kCFXcJDAOtKDvmfyAuC0lF3eF4cM2TDclooX+UDe12liPR44rLkcgU6VXNe+iw== X-Received: by 2002:a05:6214:1c8d:: with SMTP id ib13mr36691652qvb.10.1634140841945; Wed, 13 Oct 2021 09:00:41 -0700 (PDT) Received: from moria.home.lan (c-73-219-103-14.hsd1.vt.comcast.net. [73.219.103.14]) by smtp.gmail.com with ESMTPSA id w17sm20161qts.53.2021.10.13.09.00.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Oct 2021 09:00:41 -0700 (PDT) From: Kent Overstreet To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, linux-raid@vger.kernel.org, linux-block@vger.kernel.org, axboe@kernel.dk Cc: Kent Overstreet , alexander.h.duyck@linux.intel.com Subject: [PATCH 1/5] mm: Make free_area->nr_free per migratetype Date: Wed, 13 Oct 2021 12:00:30 -0400 Message-Id: <20211013160034.3472923-2-kent.overstreet@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211013160034.3472923-1-kent.overstreet@gmail.com> References: <20211013160034.3472923-1-kent.overstreet@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A01FF3000108 X-Stat-Signature: wwmsh856fhxscchrks3zi5tg8g9symxp Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=GJh4rtFJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.219.49 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com X-HE-Tag: 1634140842-663685 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is prep work for introducing a struct page_free_list, which will have a list head and nr_free - it turns out a fair amount of the code looking at free_area->nr_free actually wants the number of elements on a particular freelist. Signed-off-by: Kent Overstreet --- include/linux/mmzone.h | 14 ++++++++++++-- mm/page_alloc.c | 30 +++++++++++++++++------------- mm/page_reporting.c | 2 +- mm/vmstat.c | 28 +++------------------------- 4 files changed, 33 insertions(+), 41 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 6a1d79d846..089587b918 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -96,7 +96,7 @@ extern int page_group_by_mobility_disabled; =20 struct free_area { struct list_head free_list[MIGRATE_TYPES]; - unsigned long nr_free; + unsigned long nr_free[MIGRATE_TYPES]; }; =20 static inline struct page *get_page_from_free_area(struct free_area *are= a, @@ -108,7 +108,17 @@ static inline struct page *get_page_from_free_area(s= truct free_area *area, =20 static inline bool free_area_empty(struct free_area *area, int migratety= pe) { - return list_empty(&area->free_list[migratetype]); + return area->nr_free[migratetype] =3D=3D 0; +} + +static inline size_t free_area_nr_free(struct free_area *area) +{ + unsigned migratetype; + size_t nr_free =3D 0; + + for (migratetype =3D 0; migratetype < MIGRATE_TYPES; migratetype++) + nr_free +=3D area->nr_free[migratetype]; + return nr_free; } =20 struct pglist_data; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b37435c274..8918c00a91 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -966,7 +966,7 @@ static inline void add_to_free_list(struct page *page= , struct zone *zone, struct free_area *area =3D &zone->free_area[order]; =20 list_add(&page->lru, &area->free_list[migratetype]); - area->nr_free++; + area->nr_free[migratetype]++; } =20 /* Used for pages not on another list */ @@ -976,7 +976,7 @@ static inline void add_to_free_list_tail(struct page = *page, struct zone *zone, struct free_area *area =3D &zone->free_area[order]; =20 list_add_tail(&page->lru, &area->free_list[migratetype]); - area->nr_free++; + area->nr_free[migratetype]++; } =20 /* @@ -993,7 +993,7 @@ static inline void move_to_free_list(struct page *pag= e, struct zone *zone, } =20 static inline void del_page_from_free_list(struct page *page, struct zon= e *zone, - unsigned int order) + unsigned int order, int migratetype) { /* clear reported state and update reported page count */ if (page_reported(page)) @@ -1002,7 +1002,7 @@ static inline void del_page_from_free_list(struct p= age *page, struct zone *zone, list_del(&page->lru); __ClearPageBuddy(page); set_page_private(page, 0); - zone->free_area[order].nr_free--; + zone->free_area[order].nr_free[migratetype]--; } =20 /* @@ -1098,7 +1098,7 @@ static inline void __free_one_page(struct page *pag= e, if (page_is_guard(buddy)) clear_page_guard(zone, buddy, order, migratetype); else - del_page_from_free_list(buddy, zone, order); + del_page_from_free_list(buddy, zone, order, migratetype); combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; @@ -2456,7 +2456,7 @@ struct page *__rmqueue_smallest(struct zone *zone, = unsigned int order, page =3D get_page_from_free_area(area, migratetype); if (!page) continue; - del_page_from_free_list(page, zone, current_order); + del_page_from_free_list(page, zone, current_order, migratetype); expand(zone, page, order, current_order, migratetype); set_pcppage_migratetype(page, migratetype); return page; @@ -3525,7 +3525,7 @@ int __isolate_free_page(struct page *page, unsigned= int order) =20 /* Remove page from free list */ =20 - del_page_from_free_list(page, zone, order); + del_page_from_free_list(page, zone, order, mt); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -6038,14 +6038,16 @@ void show_free_areas(unsigned int filter, nodemas= k_t *nodemask) struct free_area *area =3D &zone->free_area[order]; int type; =20 - nr[order] =3D area->nr_free; - total +=3D nr[order] << order; + nr[order] =3D 0; + types[order] =3D 0; =20 - types[order] =3D 0; for (type =3D 0; type < MIGRATE_TYPES; type++) { if (!free_area_empty(area, type)) types[order] |=3D 1 << type; + nr[order] +=3D area->nr_free[type]; } + + total +=3D nr[order] << order; } spin_unlock_irqrestore(&zone->lock, flags); for (order =3D 0; order < MAX_ORDER; order++) { @@ -6623,7 +6625,7 @@ static void __meminit zone_init_free_lists(struct z= one *zone) unsigned int order, t; for_each_migratetype_order(order, t) { INIT_LIST_HEAD(&zone->free_area[order].free_list[t]); - zone->free_area[order].nr_free =3D 0; + zone->free_area[order].nr_free[t] =3D 0; } } =20 @@ -9317,6 +9319,7 @@ void __offline_isolated_pages(unsigned long start_p= fn, unsigned long end_pfn) struct page *page; struct zone *zone; unsigned int order; + unsigned int migratetype; unsigned long flags; =20 offline_mem_sections(pfn, end_pfn); @@ -9346,7 +9349,8 @@ void __offline_isolated_pages(unsigned long start_p= fn, unsigned long end_pfn) BUG_ON(page_count(page)); BUG_ON(!PageBuddy(page)); order =3D buddy_order(page); - del_page_from_free_list(page, zone, order); + migratetype =3D get_pfnblock_migratetype(page, pfn); + del_page_from_free_list(page, zone, order, migratetype); pfn +=3D (1 << order); } spin_unlock_irqrestore(&zone->lock, flags); @@ -9428,7 +9432,7 @@ bool take_page_off_buddy(struct page *page) int migratetype =3D get_pfnblock_migratetype(page_head, pfn_head); =20 - del_page_from_free_list(page_head, zone, page_order); + del_page_from_free_list(page_head, zone, page_order, migratetype); break_down_buddy_pages(zone, page_head, page, 0, page_order, migratetype); if (!is_migrate_isolate(migratetype)) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 382958eef8..4e45ae95db 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -145,7 +145,7 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, * The division here should be cheap since PAGE_REPORTING_CAPACITY * should always be a power of 2. */ - budget =3D DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + budget =3D DIV_ROUND_UP(area->nr_free[mt], PAGE_REPORTING_CAPACITY * 16= ); =20 /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { diff --git a/mm/vmstat.c b/mm/vmstat.c index 8ce2620344..eb46f99c72 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1071,7 +1071,7 @@ static void fill_contig_page_info(struct zone *zone= , unsigned long blocks; =20 /* Count number of free blocks */ - blocks =3D zone->free_area[order].nr_free; + blocks =3D free_area_nr_free(&zone->free_area[order]); info->free_blocks_total +=3D blocks; =20 /* Count free base pages */ @@ -1445,7 +1445,7 @@ static void frag_show_print(struct seq_file *m, pg_= data_t *pgdat, =20 seq_printf(m, "Node %d, zone %8s ", pgdat->node_id, zone->name); for (order =3D 0; order < MAX_ORDER; ++order) - seq_printf(m, "%6lu ", zone->free_area[order].nr_free); + seq_printf(m, "%6zu ", free_area_nr_free(&zone->free_area[order])); seq_putc(m, '\n'); } =20 @@ -1470,29 +1470,7 @@ static void pagetypeinfo_showfree_print(struct seq= _file *m, zone->name, migratetype_names[mtype]); for (order =3D 0; order < MAX_ORDER; ++order) { - unsigned long freecount =3D 0; - struct free_area *area; - struct list_head *curr; - bool overflow =3D false; - - area =3D &(zone->free_area[order]); - - list_for_each(curr, &area->free_list[mtype]) { - /* - * Cap the free_list iteration because it might - * be really large and we are under a spinlock - * so a long time spent here could trigger a - * hard lockup detector. Anyway this is a - * debugging tool so knowing there is a handful - * of pages of this order should be more than - * sufficient. - */ - if (++freecount >=3D 100000) { - overflow =3D true; - break; - } - } - seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount); + seq_printf(m, "%6zu ", zone->free_area[order].nr_free[mtype]); spin_unlock_irq(&zone->lock); cond_resched(); spin_lock_irq(&zone->lock); --=20 2.33.0