From: Johannes Weiner <hannes@cmpxchg.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Vlastimil Babka <vbabka@suse.cz>,
Mel Gorman <mgorman@techsingularity.net>,
Miaohe Lin <linmiaohe@huawei.com>,
Kefeng Wang <wangkefeng.wang@huawei.com>, Zi Yan <ziy@nvidia.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: [PATCH V2 0/6] mm: page_alloc: freelist migratetype hygiene
Date: Mon, 11 Sep 2023 15:41:41 -0400 [thread overview]
Message-ID: <20230911195023.247694-1-hannes@cmpxchg.org> (raw)
V2:
- dropped the get_pfnblock_migratetype() optimization
patchlet since somebody else beat me to it (thanks Zi)
- broke out pcp bypass fix since somebody else reported the bug:
https://lore.kernel.org/linux-mm/20230911181108.GA104295@cmpxchg.org/
- fixed the CONFIG_UNACCEPTED_MEMORY build (lkp)
- rebased to v6.6-rc1
The series is based on v6.6-rc1 plus the pcp bypass fix above ^
---
This is a breakout series from the huge page allocator patches[1].
While testing and benchmarking the series incrementally, as per
reviewer request, it became apparent that there are several sources of
freelist migratetype violations that later patches in the series hid.
Those violations occur when pages of one migratetype end up on the
freelists of another type. This encourages incompatible page mixing
down the line, where allocation requests ask for one migrate type, but
receives pages of another. This defeats the mobility grouping.
The series addresses those causes. The last patch adds type checks on
all freelist movements to rule out any violations. I used these checks
to identify the violations fixed up in the preceding patches.
The series is a breakout, but has merit on its own: Less type mixing
means improved grouping, means less work for compaction, means higher
THP success rate and lower allocation latencies. The results can be
seen in a mixed workload that stresses the machine with a kernel build
job while periodically attempting to allocate batches of THP. The data
is averaged over 50 consecutive defconfig builds:
VANILLA PATCHED-CLEANLISTS
Hugealloc Time median 14642.00 ( +0.00%) 10506.00 ( -28.25%)
Hugealloc Time min 4820.00 ( +0.00%) 4783.00 ( -0.77%)
Hugealloc Time max 6786868.00 ( +0.00%) 6556624.00 ( -3.39%)
Kbuild Real time 240.03 ( +0.00%) 241.45 ( +0.59%)
Kbuild User time 1195.49 ( +0.00%) 1195.69 ( +0.02%)
Kbuild System time 96.44 ( +0.00%) 97.03 ( +0.61%)
THP fault alloc 11490.00 ( +0.00%) 11802.30 ( +2.72%)
THP fault fallback 782.62 ( +0.00%) 478.88 ( -38.76%)
THP fault fail rate % 6.38 ( +0.00%) 3.90 ( -33.52%)
Direct compact stall 297.70 ( +0.00%) 224.56 ( -24.49%)
Direct compact fail 265.98 ( +0.00%) 191.56 ( -27.87%)
Direct compact success 31.72 ( +0.00%) 33.00 ( +3.91%)
Direct compact success rate % 13.11 ( +0.00%) 17.26 ( +29.43%)
Compact daemon scanned migrate 1673661.58 ( +0.00%) 1591682.18 ( -4.90%)
Compact daemon scanned free 2711252.80 ( +0.00%) 2615217.78 ( -3.54%)
Compact direct scanned migrate 384998.62 ( +0.00%) 261689.42 ( -32.03%)
Compact direct scanned free 966308.94 ( +0.00%) 667459.76 ( -30.93%)
Compact migrate scanned daemon % 80.86 ( +0.00%) 83.34 ( +3.02%)
Compact free scanned daemon % 74.41 ( +0.00%) 78.26 ( +5.10%)
Alloc stall 338.06 ( +0.00%) 440.72 ( +30.28%)
Pages kswapd scanned 1356339.42 ( +0.00%) 1402313.42 ( +3.39%)
Pages kswapd reclaimed 581309.08 ( +0.00%) 587956.82 ( +1.14%)
Pages direct scanned 56384.18 ( +0.00%) 141095.04 ( +150.24%)
Pages direct reclaimed 17055.54 ( +0.00%) 22427.96 ( +31.50%)
Pages scanned kswapd % 96.38 ( +0.00%) 93.60 ( -2.86%)
Swap out 41528.00 ( +0.00%) 47969.92 ( +15.51%)
Swap in 6541.42 ( +0.00%) 9093.30 ( +39.01%)
File refaults 127666.50 ( +0.00%) 135766.84 ( +6.34%)
include/linux/mm.h | 18 +-
include/linux/page-isolation.h | 2 +-
include/linux/vmstat.h | 8 -
mm/debug_page_alloc.c | 12 +-
mm/internal.h | 5 -
mm/page_alloc.c | 357 ++++++++++++++++++---------------
mm/page_isolation.c | 23 ++-
7 files changed, 217 insertions(+), 208 deletions(-)
next reply other threads:[~2023-09-11 19:50 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-11 19:41 Johannes Weiner [this message]
2023-09-11 19:41 ` [PATCH 1/6] mm: page_alloc: remove pcppage migratetype caching Johannes Weiner
2023-09-11 19:59 ` Zi Yan
2023-09-11 21:09 ` Andrew Morton
2023-09-12 13:47 ` Vlastimil Babka
2023-09-12 14:50 ` Johannes Weiner
2023-09-13 9:33 ` Vlastimil Babka
2023-09-13 13:24 ` Johannes Weiner
2023-09-13 13:34 ` Vlastimil Babka
2023-09-12 15:03 ` Johannes Weiner
2023-09-14 7:29 ` Vlastimil Babka
2023-09-14 9:56 ` Mel Gorman
2023-09-27 5:42 ` Huang, Ying
2023-09-27 14:51 ` Johannes Weiner
2023-09-30 4:26 ` Huang, Ying
2023-10-02 14:58 ` Johannes Weiner
2023-09-11 19:41 ` [PATCH 2/6] mm: page_alloc: fix up block types when merging compatible blocks Johannes Weiner
2023-09-11 20:01 ` Zi Yan
2023-09-13 9:52 ` Vlastimil Babka
2023-09-14 10:00 ` Mel Gorman
2023-09-11 19:41 ` [PATCH 3/6] mm: page_alloc: move free pages when converting block during isolation Johannes Weiner
2023-09-11 20:17 ` Zi Yan
2023-09-11 20:47 ` Johannes Weiner
2023-09-11 20:50 ` Zi Yan
2023-09-13 14:31 ` Vlastimil Babka
2023-09-14 10:03 ` Mel Gorman
2023-09-11 19:41 ` [PATCH 4/6] mm: page_alloc: fix move_freepages_block() range error Johannes Weiner
2023-09-11 20:23 ` Zi Yan
2023-09-13 14:40 ` Vlastimil Babka
2023-09-14 13:37 ` Johannes Weiner
2023-09-14 10:03 ` Mel Gorman
2023-09-11 19:41 ` [PATCH 5/6] mm: page_alloc: fix freelist movement during block conversion Johannes Weiner
2023-09-13 19:52 ` Vlastimil Babka
2023-09-14 14:47 ` Johannes Weiner
2023-09-11 19:41 ` [PATCH 6/6] mm: page_alloc: consolidate free page accounting Johannes Weiner
2023-09-13 20:18 ` Vlastimil Babka
2023-09-14 4:11 ` Johannes Weiner
2023-09-14 23:52 ` [PATCH V2 0/6] mm: page_alloc: freelist migratetype hygiene Mike Kravetz
2023-09-15 14:16 ` Johannes Weiner
2023-09-15 15:05 ` Mike Kravetz
2023-09-16 19:57 ` Mike Kravetz
2023-09-16 20:13 ` Andrew Morton
2023-09-18 7:16 ` Vlastimil Babka
2023-09-18 14:52 ` Johannes Weiner
2023-09-18 17:40 ` Mike Kravetz
2023-09-19 6:49 ` Johannes Weiner
2023-09-19 12:37 ` Zi Yan
2023-09-19 15:22 ` Zi Yan
2023-09-19 18:47 ` Mike Kravetz
2023-09-19 20:57 ` Zi Yan
2023-09-20 0:32 ` Mike Kravetz
2023-09-20 1:38 ` Zi Yan
2023-09-20 6:07 ` Vlastimil Babka
2023-09-20 13:48 ` Johannes Weiner
2023-09-20 16:04 ` Johannes Weiner
2023-09-20 17:23 ` Zi Yan
2023-09-21 2:31 ` Zi Yan
2023-09-21 10:19 ` David Hildenbrand
2023-09-21 14:47 ` Zi Yan
2023-09-25 21:12 ` Zi Yan
2023-09-26 17:39 ` Johannes Weiner
2023-09-28 2:51 ` Zi Yan
2023-10-03 2:26 ` Zi Yan
2023-10-10 21:12 ` Johannes Weiner
2023-10-11 15:25 ` Johannes Weiner
2023-10-11 15:45 ` Johannes Weiner
2023-10-11 15:57 ` Zi Yan
2023-10-13 0:06 ` Zi Yan
2023-10-13 14:51 ` Zi Yan
2023-10-16 13:35 ` Zi Yan
2023-10-16 14:37 ` Johannes Weiner
2023-10-16 15:00 ` Zi Yan
2023-10-16 18:51 ` Johannes Weiner
2023-10-16 19:49 ` Zi Yan
2023-10-16 20:26 ` Johannes Weiner
2023-10-16 20:39 ` Johannes Weiner
2023-10-16 20:48 ` Zi Yan
2023-09-26 18:19 ` David Hildenbrand
2023-09-28 3:22 ` Zi Yan
2023-10-02 11:43 ` David Hildenbrand
2023-10-03 2:35 ` Zi Yan
2023-09-18 7:07 ` Vlastimil Babka
2023-09-18 14:09 ` Johannes Weiner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230911195023.247694-1-hannes@cmpxchg.org \
--to=hannes@cmpxchg.org \
--cc=akpm@linux-foundation.org \
--cc=linmiaohe@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@techsingularity.net \
--cc=vbabka@suse.cz \
--cc=wangkefeng.wang@huawei.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox