From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10CE3C25B10 for ; Mon, 13 May 2024 18:10:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 359756B019F; Mon, 13 May 2024 14:10:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 308F36B02BF; Mon, 13 May 2024 14:10:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D0F16B02C0; Mon, 13 May 2024 14:10:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id F28A06B019F for ; Mon, 13 May 2024 14:10:46 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 937C2140FE4 for ; Mon, 13 May 2024 18:10:46 +0000 (UTC) X-FDA: 82114163292.22.7BEF065 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf11.hostedemail.com (Postfix) with ESMTP id A70164000C for ; Mon, 13 May 2024 18:10:44 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kc8+B761; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715623844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1obXIpFTbK7Mcgk9nQSzyMnbMY4o4pmJSXSmOSDveAg=; b=fOPYFKnXrNbIlWC6c2+sBoBuJy4Z1hZiYJSEnd1WpLquRHmGngNZI1bC5wcMXhUgy51dun R2rZkbZdArsXgauqb3nZpS58IbTTbKpNyHekjGt7dAIJRpGXkO301hZVP0GIFcTkxZRL4k 3+S8y1E22TmTEo9Tz2sTprM51Ze+Vj8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kc8+B761; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715623844; a=rsa-sha256; cv=none; b=NR2x4E3g73FT5ybI5I1oMsECmIiemTUbxSLw0vbyF1FeDGQRPHjGLHAPtmH1BIU2tXv7h7 fKXjW+Js9JLbvrLkj4U8pA2BI85UzaxEhufphUIklJldVHaG5WP1bMU7yloCcSVjtoBQg6 wUFYXidDtxtEJDFzRwNpbMi0ZDQnCio= Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-41fef5dda72so110405e9.1 for ; Mon, 13 May 2024 11:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715623843; x=1716228643; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=1obXIpFTbK7Mcgk9nQSzyMnbMY4o4pmJSXSmOSDveAg=; b=kc8+B76100RzX/9ZL+qMd0Nmgf6BKAntapNL4CBHfgF+8EHvMeGYnL9wqh9k3ZrvSp oMK5FG9JIBAEx+M2P42lCW3bjPWy1a8oLQeVvN1VKQ5TvAp3Q1Sj/q5G7gxZ/0m4r+m+ ZpGf8PMX6fVHP9Mr6qLlGmPyrkmBp6FuXknVaF3LTlMd1AQrXBxjSLJf7FRydP5jIGWz G8pG7kTWKsCgjHlFlgeJWvfB7bCpJ0CATPJhtZKrdN4E0HA0jToMI6eOBx7JjVUd3Sj+ /bPP5739LpRFJ+VM2QNvE9N40Jj2rHlGH/SK4I/RyCL6SMdVwTfEuwsiGho3Nd7GQyqa cvGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715623843; x=1716228643; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1obXIpFTbK7Mcgk9nQSzyMnbMY4o4pmJSXSmOSDveAg=; b=jWxFYcUgzBHQwVHdhrqC7tv7bDFumO73B+QalfeRKqIzWwjkMLgWiJfHQs6SroKoKs K+wSAbsIdo/JABY1eMw3YJRfG5QuZpT9an3HcKFdZG1Ml2B49XclRkTOoj8sVN1TzGc3 nCNGR4WTlLHq6cTOUPiopnCXbBeCHPjIm+Pq7r9zXylaW5N9hz8IhmwCk05829V4SdOI M1f1E6CUDPOL5Xm9FgvEJqBl29/eGgrDfLei9BjCmOlKONTSI1s8PYmRXRau0cTHX/tO 1EPybNNpsu461wDKoFu7uAwBeJIjNhWRiOR1/sbXO2y06X8TsFXHdAA7RfVToc+ag0HV teaw== X-Forwarded-Encrypted: i=1; AJvYcCUkNIGfElx0cNC/lDOHTvFI1lsE0Kru38PNnxHVe2eLPA/liFDnafRYk4imUmBMSqZ9/mDuefZy6Hv15Z3uYpXKwh0= X-Gm-Message-State: AOJu0Yxv6U5Ruuefa3EMlvQuGruPRR45R/PsxKAha3zx6y0kW9/AUtjb tuXLn2n9uX4xmX9WH+5PkemArnI5+wA0ygf+3cgsyjh0hpmd6MeJX/HK4P30cYjNnqpwTj0v0Hd esDOuNk/HDMj/+KLbVunVh3NHTnD7bydJP7BN X-Google-Smtp-Source: AGHT+IGaiDNjVxJcmjCPG0ym9D8p7YvfB21qu6clVocHjA4sTpmeCIsCnVsfZHB7a6TZGV7ULHLYG8QQZLynnvXqBX4= X-Received: by 2002:a05:600c:3d0c:b0:41c:ab7:f9af with SMTP id 5b1f17b1804b1-4200f8ac9c8mr3859505e9.3.1715623842823; Mon, 13 May 2024 11:10:42 -0700 (PDT) MIME-Version: 1.0 References: <20240320180429.678181-1-hannes@cmpxchg.org> <20240513160331.GA320190@cmpxchg.org> In-Reply-To: <20240513160331.GA320190@cmpxchg.org> From: Yu Zhao Date: Mon, 13 May 2024 12:10:04 -0600 Message-ID: Subject: Re: [PATCH V4 00/10] mm: page_alloc: freelist migratetype hygiene To: Johannes Weiner Cc: Andrew Morton , Vlastimil Babka , Mel Gorman , Zi Yan , "Huang, Ying" , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kalesh Singh , Chun-Tse Shao Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A70164000C X-Stat-Signature: xwg6dnhdsdmgg3ptf4as7eoxucazk935 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1715623844-992298 X-HE-Meta: U2FsdGVkX18sg2R+2Sy/zUKOVr1AOYrvLeC0IfEeFTC0RBGTv3MrOLoGN2mZDHbaXYHST+7iaJ6w3G1umjmx1pw5VXFziIJKRGhZt/wLeZA4I0G911ctyHUDDtATUL4rCQbHW6yGxWNUtPI3n0QtxMUZBPQNxsOlK04i+1TjrPuIq2ocaWksEFO3mVGPpJt1e1oDkjL6KEFdZaOoMyY5KYTKrIK1oROHUlj4mdPM33AqRqpgkfrxgTvcSkhML012XrlqfWeqbe6EBah1TjJsR4Qx9207yqPZGauOsS6hhgj3VJGNTT79is0C6sqwDukb9nTBGUhZWJJLu6ERGkvxNRhfDzZmpKGydaojisykWM8i/Wdb+p9zqvssa27alkhAlyhcvyw7ajp9TRJ9er+fBZd1JcFUS3uVqzk8sbcwiJWPmkT/RVTVMbkY8TYIuOqeGsgc+7Udy3LDIpoPWk9oWlfc487xPi2i6AUre3gKlpCanmFgTc0C4ve3dGKVszQytpBwv+A5TzNtvLJD5oii0F+MR0DrqG/MIbHbag3PhsFweX/lZISdzAmKhePGrmc9aS64ehAn08STfrzvh01uq99OXzPbNMuBgopSJGKiQXBhq+s11cE6dqJcWe2vSFea+qMjQ1WRk8bqq7uwj77t2cwYzSKNoToAkzdYTKfXj1FyFDnOTcD6fiP3a+INCv63RzeVolLrM/Kch1Vc9gTL6dFg4mo9ELDZiNmHHSi1S55WP+gpUAF5DtTtCmvj/HWqi0cy2RGu+4lu3HuDYDXaNZBQ8eNxuW/bX863yXBFu9HkbzMXjvNDQa6wDL3jNbQoj8XJibjpmHs1m5r4rgEIEYCmqMATphB9PGFFl9MZ5xRR0Cl5MZl2p5F96Tc1pwbEete6ba88AHCaLR4bRkn7tTmgfg1MffudIenfrjdRxXsTjCpN6H40+Y49UdQDnKnuigDIVFQJJ5/rPzjLqLf 2mjTlC9W MlCUhf3si9YoTWZD+qSCZa807a9Ojl4Pjg6PcYzjPCWGXI9MVhlLKKBNtczsaLwG6EZtTgh5XcHGFzhS3Kk4wZy5l+28bMa4JRBuUiR1H++NAjXra0TNmw8ak6UQdlTEkpiKynstN4+R665Thwe3vZu1IGZrdcSvTTnT7GaqmnbOv6AvN3FQSGpEbB58t27V96XP0j3z1MNMsWOAe9GGR/6h0XlkZ98pDiChJgaxRZ1VBshGRnTKoqnqKE66DEfUlXbh2qArN1zgiAMzX3TGFTlCtNJUaZOq/m51jucE9v0HLBe4ORiEGOmTAWCR7ZeEw/v5Oy8wjpZbiGXc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, May 13, 2024 at 10:03=E2=80=AFAM Johannes Weiner wrote: > > On Fri, May 10, 2024 at 11:14:43PM -0600, Yu Zhao wrote: > > On Wed, Mar 20, 2024 at 12:04=E2=80=AFPM Johannes Weiner wrote: > > > > > > V4: > > > - fixed !pcp_order_allowed() case in free_unref_folios() > > > - reworded the patch 0 changelog a bit for the git log > > > - rebased to mm-everything-2024-03-19-23-01 > > > - runtime-tested again with various CONFIG_DEBUG_FOOs enabled > > > > > > --- > > > > > > The page allocator's mobility grouping is intended to keep unmovable > > > pages separate from reclaimable/compactable ones to allow on-demand > > > defragmentation for higher-order allocations and huge pages. > > > > > > Currently, there are several places where accidental type mixing > > > occurs: an allocation asks for a page of a certain migratetype and > > > receives another. This ruins pageblocks for compaction, which in turn > > > makes allocating huge pages more expensive and less reliable. > > > > > > The series addresses those causes. The last patch adds type checks on > > > all freelist movements to prevent new violations being introduced. > > > > > > The benefits can be seen in a mixed workload that stresses the machin= e > > > with a memcache-type workload and a kernel build job while > > > periodically attempting to allocate batches of THP. The following dat= a > > > is aggregated over 50 consecutive defconfig builds: > > > > > > VANILLA = PATCHED > > > Hugealloc Time mean 165843.93 ( +0.00%) 1130= 25.88 ( -31.85%) > > > Hugealloc Time stddev 158957.35 ( +0.00%) 1147= 16.07 ( -27.83%) > > > Kbuild Real time 310.24 ( +0.00%) 3= 00.73 ( -3.06%) > > > Kbuild User time 1271.13 ( +0.00%) 12= 59.42 ( -0.92%) > > > Kbuild System time 582.02 ( +0.00%) 5= 59.79 ( -3.81%) > > > THP fault alloc 30585.14 ( +0.00%) 408= 53.62 ( +33.57%) > > > THP fault fallback 36626.46 ( +0.00%) 263= 57.62 ( -28.04%) > > > THP fault fail rate % 54.49 ( +0.00%) = 39.22 ( -27.53%) > > > Pagealloc fallback 1328.00 ( +0.00%) = 1.00 ( -99.85%) > > > Pagealloc type mismatch 181009.50 ( +0.00%) = 0.00 ( -100.00%) > > > Direct compact stall 434.56 ( +0.00%) 2= 57.66 ( -40.61%) > > > Direct compact fail 421.70 ( +0.00%) 2= 49.94 ( -40.63%) > > > Direct compact success 12.86 ( +0.00%) = 7.72 ( -37.09%) > > > Direct compact success rate % 2.86 ( +0.00%) = 2.82 ( -0.96%) > > > Compact daemon scanned migrate 3370059.62 ( +0.00%) 36120= 54.76 ( +7.18%) > > > Compact daemon scanned free 7718439.20 ( +0.00%) 53863= 85.02 ( -30.21%) > > > Compact direct scanned migrate 309248.62 ( +0.00%) 1767= 21.04 ( -42.85%) > > > Compact direct scanned free 433582.84 ( +0.00%) 3157= 27.66 ( -27.18%) > > > Compact migrate scanned daemon % 91.20 ( +0.00%) = 94.48 ( +3.56%) > > > Compact free scanned daemon % 94.58 ( +0.00%) = 94.42 ( -0.16%) > > > Compact total migrate scanned 3679308.24 ( +0.00%) 37887= 75.80 ( +2.98%) > > > Compact total free scanned 8152022.04 ( +0.00%) 57021= 12.68 ( -30.05%) > > > Alloc stall 872.04 ( +0.00%) 51= 56.12 ( +490.71%) > > > Pages kswapd scanned 510645.86 ( +0.00%) 33= 94.94 ( -99.33%) > > > Pages kswapd reclaimed 134811.62 ( +0.00%) 27= 01.26 ( -98.00%) > > > Pages direct scanned 99546.06 ( +0.00%) 3764= 07.52 ( +278.12%) > > > Pages direct reclaimed 62123.40 ( +0.00%) 2895= 35.70 ( +366.06%) > > > Pages total scanned 610191.92 ( +0.00%) 3798= 02.46 ( -37.76%) > > > Pages scanned kswapd % 76.36 ( +0.00%) = 0.10 ( -98.58%) > > > Swap out 12057.54 ( +0.00%) 150= 22.98 ( +24.59%) > > > Swap in 209.16 ( +0.00%) 2= 56.48 ( +22.52%) > > > File refaults 17701.64 ( +0.00%) 117= 65.40 ( -33.53%) > > > > > > Huge page success rate is higher, allocation latencies are shorter an= d > > > more predictable. > > > > > > Stealing (fallback) rate is drastically reduced. Notably, while the > > > vanilla kernel keeps doing fallbacks on an ongoing basis, the patched > > > kernel enters a steady state once the distribution of block types is > > > adequate for the workload. Steals over 50 runs: > > > > > > VANILLA PATCHED > > > 1504.0 227.0 > > > 1557.0 6.0 > > > 1391.0 13.0 > > > 1080.0 26.0 > > > 1057.0 40.0 > > > 1156.0 6.0 > > > 805.0 46.0 > > > 736.0 20.0 > > > 1747.0 2.0 > > > 1699.0 34.0 > > > 1269.0 13.0 > > > 1858.0 12.0 > > > 907.0 4.0 > > > 727.0 2.0 > > > 563.0 2.0 > > > 3094.0 2.0 > > > 10211.0 3.0 > > > 2621.0 1.0 > > > 5508.0 2.0 > > > 1060.0 2.0 > > > 538.0 3.0 > > > 5773.0 2.0 > > > 2199.0 0.0 > > > 3781.0 2.0 > > > 1387.0 1.0 > > > 4977.0 0.0 > > > 2865.0 1.0 > > > 1814.0 1.0 > > > 3739.0 1.0 > > > 6857.0 0.0 > > > 382.0 0.0 > > > 407.0 1.0 > > > 3784.0 0.0 > > > 297.0 0.0 > > > 298.0 0.0 > > > 6636.0 0.0 > > > 4188.0 0.0 > > > 242.0 0.0 > > > 9960.0 0.0 > > > 5816.0 0.0 > > > 354.0 0.0 > > > 287.0 0.0 > > > 261.0 0.0 > > > 140.0 1.0 > > > 2065.0 0.0 > > > 312.0 0.0 > > > 331.0 0.0 > > > 164.0 0.0 > > > 465.0 1.0 > > > 219.0 0.0 > > > > > > Type mismatches are down too. Those count every time an allocation > > > request asks for one migratetype and gets another. This can still > > > occur minimally in the patched kernel due to non-stealing fallbacks, > > > but it's quite rare and follows the pattern of overall fallbacks - > > > once the block type distribution settles, mismatches cease as well: > > > > > > VANILLA: PATCHED: > > > 182602.0 268.0 > > > 135794.0 20.0 > > > 88619.0 19.0 > > > 95973.0 0.0 > > > 129590.0 0.0 > > > 129298.0 0.0 > > > 147134.0 0.0 > > > 230854.0 0.0 > > > 239709.0 0.0 > > > 137670.0 0.0 > > > 132430.0 0.0 > > > 65712.0 0.0 > > > 57901.0 0.0 > > > 67506.0 0.0 > > > 63565.0 4.0 > > > 34806.0 0.0 > > > 42962.0 0.0 > > > 32406.0 0.0 > > > 38668.0 0.0 > > > 61356.0 0.0 > > > 57800.0 0.0 > > > 41435.0 0.0 > > > 83456.0 0.0 > > > 65048.0 0.0 > > > 28955.0 0.0 > > > 47597.0 0.0 > > > 75117.0 0.0 > > > 55564.0 0.0 > > > 38280.0 0.0 > > > 52404.0 0.0 > > > 26264.0 0.0 > > > 37538.0 0.0 > > > 19671.0 0.0 > > > 30936.0 0.0 > > > 26933.0 0.0 > > > 16962.0 0.0 > > > 44554.0 0.0 > > > 46352.0 0.0 > > > 24995.0 0.0 > > > 35152.0 0.0 > > > 12823.0 0.0 > > > 21583.0 0.0 > > > 18129.0 0.0 > > > 31693.0 0.0 > > > 28745.0 0.0 > > > 33308.0 0.0 > > > 31114.0 0.0 > > > 35034.0 0.0 > > > 12111.0 0.0 > > > 24885.0 0.0 > > > > > > Compaction work is markedly reduced despite much better THP rates. > > > > > > In the vanilla kernel, reclaim seems to have been driven primarily by > > > watermark boosting that happens as a result of fallbacks. With those > > > all but eliminated, watermarks average lower and kswapd does less > > > work. The uptick in direct reclaim is because THP requests have to > > > fend for themselves more often - which is intended policy right > > > now. Aggregate reclaim activity is lowered significantly, though. > > > > This series significantly regresses Android and ChromeOS under memory > > pressure. THPs are virtually nonexistent on client devices, and IIRC, > > it was mentioned in the early discussions that potential regressions > > for such a case are somewhat expected? > > This is not expected for the 10 patches here. You might be referring > to the discussion around the huge page allocator series, which had > fallback restrictions and many changes to reclaim and compaction. Right, now I remember. > Can you confirm that you were testing the latest patches that are in > mm-stable as of today? There was a series of follow-up fixes. Here is what I have on top of 6.8.y, which I think includes all the follow-up fixes. The performance delta was measured between 5 & 22. 1 mm: convert free_unref_page_list() to use folios 2 mm: add free_unref_folios() 3 mm: handle large folios in free_unref_folios() 4 mm/page_alloc: remove unused fpi_flags in free_pages_prepare() 5 mm: add alloc_contig_migrate_range allocation statistics 6 mm: page_alloc: remove pcppage migratetype caching 7 mm: page_alloc: optimize free_unref_folios() 8 mm: page_alloc: fix up block types when merging compatible blocks 9 mm: page_alloc: move free pages when converting block during isolati= on 10 mm: page_alloc: fix move_freepages_block() range error 11 mm: page_alloc: fix freelist movement during block conversion 12 mm-page_alloc-fix-freelist-movement-during-block-conversion-fix 13 mm: page_alloc: close migratetype race between freeing and stealing 14 mm: page_alloc: set migratetype inside move_freepages() 15 mm: page_isolation: prepare for hygienic freelists 16 mm-page_isolation-prepare-for-hygienic-freelists-fix 17 mm: page_alloc: consolidate free page accounting 18 mm: page_alloc: consolidate free page accounting fix 19 mm: page_alloc: consolidate free page accounting fix 2 20 mm: page_alloc: consolidate free page accounting fix 3 21 mm: page_alloc: change move_freepages() to __move_freepages_block() 22 mm: page_alloc: batch vmstat updates in expand() > Especially, please double check you have the follow-up fixes to > compaction capturing and the CMA fallback policy. It sounds like the > behavior Baolin described before the CMA fix. Yes, that one was included. > Lastly, what's the base you backported this series to? It was 6.8, we can potentially try 6.9 this week and 6.10-rc in a few weeks when it's in good shape for performance benchmarks. > > On Android (ARMv8.2), app launch time regressed by about 7%; On > > ChromeOS (Intel ADL), tab switch time regressed by about 8%. Also PSI > > (full and some) on both platforms increased by over 20%. I could post > > the details of the benchmarks and the metrics they measure, but I > > doubt they would mean much to you. I did ask our test teams to save > > extra kernel logs that might be more helpful, and I could forward them > > to you. > > If the issue persists with the latest patches in -mm, a kernel config > and snapshots of /proc/vmstat, /proc/pagetypeinfo, /proc/zoneinfo > before/during/after the problematic behavior would be very helpful. Assuming all the fixes were included, do you want the logs from 6.8? We have them available now.