From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC76DC83F09 for ; Thu, 10 Jul 2025 03:37:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 637456B0095; Wed, 9 Jul 2025 23:37:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E7C86B009E; Wed, 9 Jul 2025 23:37:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FD6D6B009F; Wed, 9 Jul 2025 23:37:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3D4216B0095 for ; Wed, 9 Jul 2025 23:37:38 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D6C3080453 for ; Thu, 10 Jul 2025 03:37:37 +0000 (UTC) X-FDA: 83646945354.24.ECCA409 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by imf22.hostedemail.com (Postfix) with ESMTP id 02706C000F for ; Thu, 10 Jul 2025 03:37:35 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C8KC0+wp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752118656; a=rsa-sha256; cv=none; b=yF5jLkHf0j8KQj2dmnwj/nroWIRI8k90a/l9IcHjtFlY/pjHkqg/sCz9kkYtcgOj8zomx1 lw7L8cki7iqY3EI8J2Hf7ftmS4wJItw56x+MDTS2qSm2FeRTTENaySKc8QHK4SUabjzmug HnW5s9Bu/JUFDRgb6nKb+INTx/zVUX4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C8KC0+wp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752118656; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=L7oCOOALe1LrYdPFtNVGk/i/koqxWAk4W7m8zK9PpmM=; b=uVGr0YgBQbJF4H3C1hanTpcYm9Rpum2LwxHcSQQXMQwdzSfEjEVa+6FP1KBY1JPUU/Q4V/ FsFDxkUrpDTfpHvpboyWzlNc3HXM1VOWdnbjoIJ1v9TcGAiuSFhWnox+b8sDFL6d2TSbck elQ4zx2wwh66Rf2GMNuLLAl7XfNxRlI= Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-b31c84b8052so775944a12.1 for ; Wed, 09 Jul 2025 20:37:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752118654; x=1752723454; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=L7oCOOALe1LrYdPFtNVGk/i/koqxWAk4W7m8zK9PpmM=; b=C8KC0+wpsNBrEK8nrZeDbQB3PuuJxIiHk5qJKMJhvwcv58ji/vlXtrFq5R9rvu3R+Z l7JaMWn4foi6MZ8p4asGgZzm1eQg1pz/z3jaAAvW8iKiY2dxK9qf+r5S8GQC5uAet1Xf TiPaab7tzDJrag+N5Bc6m9EkI1zgnaZM5asinCvyDfOFcOaDcqVgGPTMH8K/Dv5+6Ds9 8B5MO5AuVJ1j0vor8Cwmmaom2Ea1XmLB6RWCPSKA+zuNQPeV5ATGgzLHz2oEIIyJUDdd yPQ6lst6/1EVOaLo77Z4FtLTI9miVXCIug2lNTT4Vgq0SIg8eAWmyFSL0suKkvpNqb4/ mIkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752118654; x=1752723454; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=L7oCOOALe1LrYdPFtNVGk/i/koqxWAk4W7m8zK9PpmM=; b=cSgDcEVkLia6dmupxuKRMb3SlGAF/DzFIW1wDHYSALgRPzzyStN4hFPamyeF29Bn8/ 7CcxZzspYrENzYXfHWDk9nu6UqhhlsupptuRSaU+VYHjuNzpa08zqLq9Ugo1rO0ktIe6 aljf3p3OlnlehxRJd/lpUpxk9upv63Zm9SSqh5sQ3vbEFmn8BbkDquQ66yj8ilyBO03h mA+sLGj+DkxXz1wDv+ICvDfJW4sK6pYuMobdkj5pRf1RAn2OLjPP1MdLmxccTGwUg+wp v7jwDtv4w7SSI2jkAZ0b5IZUyHHQgFRAcEvUhb1JE8IeAXtlvr37l4JuQhF+0mCgkPG/ kunQ== X-Gm-Message-State: AOJu0YwkXYkEBwT0PD7ZmAvL7eV+cTW+YKlkyPe6KDJ0T0tfWzJev+MQ aqtld+vgpxxQubehdLKgQ5rwN3p+1vGvbVxL3Qstj0ju5VHwkimM7tLIKsEf/e59rR8= X-Gm-Gg: ASbGncvHAGoVA2U5OYAE7qwLddPEekNK981a9NRmcfgkPR/4kfHljCVYngveBjKUneO NA6+RWfNYprA6ZwP7tPrTzL1vYN+QcrraebgJ5gx7vQYcf76fLQC2iU03Cv9GAfVGMmjjm08kZc o7/OC2hkrNzOaT1hzo7oqs/8I3ZG0cFcjrXqMwAg3qfrWN8/PGdNEpAMvjODVRKSUY47SU+5z4v 6K3BOuIIqhnVyM9asQbTyjSBGPDaujEsnJemJHPO8NUiLRXNwkzj9cXOnVv8i1rNCxMC8lmcT69 4g1Qzs0Pd+AWXk/jDAna/PGLad2RKTpaBZTVICCCgzjK0K5bTnfAWkgrjTTSLBgQFJSUEcLGosF A X-Google-Smtp-Source: AGHT+IG7cIHscPSKgzqgeYs1L9w153XiD/rnRx3GdHz3vwqkTKkgWIbvbXDVnLJfaWyDbJ38PgCx1A== X-Received: by 2002:a17:90a:c884:b0:313:f6fa:5bb3 with SMTP id 98e67ed59e1d1-31c2fdd0494mr6948497a91.26.1752118653616; Wed, 09 Jul 2025 20:37:33 -0700 (PDT) Received: from KASONG-MC4 ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-31c300689aasm3716320a91.13.2025.07.09.20.37.30 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Jul 2025 20:37:32 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Matthew Wilcox , Kemeng Shi , Chris Li , Nhat Pham , Baoquan He , Barry Song , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v5 0/8] mm/shmem, swap: bugfix and improvement of mTHP swap in Date: Thu, 10 Jul 2025 11:36:58 +0800 Message-ID: <20250710033706.71042-1-ryncsn@gmail.com> X-Mailer: git-send-email 2.50.0 Reply-To: Kairui Song MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 02706C000F X-Stat-Signature: twuxdu9wg73w17ziw5s5onye1mzo8ds4 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1752118655-285438 X-HE-Meta: U2FsdGVkX193O9ULHuwJ8KveQ9EqzC6qNqTa8iY/dAic1YjfByyLL9kZwYgJ6u2LIXWPOTT9LEX6Dj+C49SiJ0Ofc4NpoLFqJcybwhcJ/ANGG8kQ4NKjZPZLoGYnKxd+NOBWTc7ymNP+mY/fCQjBSgkEeVI5ChQ1cPSsDbqEfoboGO+ODPQVm3uz0dqGGGNlyVxwDekGYWEc8TVAptKrvU3LkXt9RfEUUSrLdzflTNgPRtWcSzy0mYKZihI4647RTpixLJwnbLqsfDGXNxxu+D8RgcfzxlBGSISv1gAEACQykwdwjoZayf3REfnWbRlLUWQFcDWNBH2yt48kIgP3dXq/UBUsaZU2t5nc0Zme4PE5HemZ6EFjfgSaCE7YeWW+afH0OVMzBNWhCwT68rE75QJl0jAwO8m70dcNtnfoLOIe3RMUnK9vPtBsPxjaTsV48xx1RwXqDMwUYiBIeTmF5cxhDv+8plVpbjksqTUdfttbEmcVLgXD71DMLc5AyF844BeHZ4OtzhtISXcoHBxZaQdTkhHernZ9Jtih+rV+15u2L5/dbV3HOxwDX9Fc3k2F8j9ezyeX4tlY4tc1PZFtmpseAh/HuMpYOL+Z4OgH38d+dwpwhiNiaqd61sYmxSpfFemPCgw+nptwdhuOm1TUsFVQtHLDhaQLBtM0v7h254SOVDwXvb4pnZsQHYyhSYpE4qsiAlu96BaDZKIRCyhnQfM/Ft1ohpZUrvZE+OwfziDLEqsc1RwcGagaAE6fZtE2NsQUAYiKi1cRgnCBnmpYCf50a4HTdT0qpt66r48MLyT3eqZblgc+9ALEpbRIeGgvnaIym4XrtEkfyifFdSWiiNH2SHH+/3uQAhawx1wdtpUK81j0durAnbH9rH1PykOBwX+9zaKr57UTaY+Z4KJuOyHW7M8fudA55hayoJsYF7nu1X/IcZHqEjSFmRiu+yj8wLuQJu6Y8qpTSS/Lp65 cYIgHX/6 GdK9NlwGS0CLL8sNnYQYvmaEXdxeUnCGeURiM9TmLTpiqSWW9YJMlnNEdA1DD6CaWO5kKQdvhC6gY5wTLEHvlgaQYl6ssEWTkqflIsOJmMb7lyKpJSmIfbchBcgQME4Slh+VUtT6A+tLUvU158epYTlO5y6mjbQHGrEZ/ApsrvJvEZpNPpRZ1sjhSsQlmsFWLT2uTssGlBBiRNU+InOr2o75HqvNp7cMe5lwurdLiswOelquSH45fisAjJiu+gL1f2PR60dw1q4yNObsiLmkH8fdsNk0eRiPHhagOmol3XUV10rEzUtqYrm/8OASSTrCM9tGELRWLieMhdLJ0RD5xdCkbrf0iV1JHrCAzYEHvSUgrVJskazrmn2ZIiQAHltokyxiBy3AFHxYIb/oc/DLusjgmhW+DfhJa5XwMr97igHAKq4KT2k+buVgCwHKLihHfmeHrMVBWKixjaTKkCQmMWm0GELAB5jzKl2AfuVNqWYYOKzDizNuGxSOhb3wS9OQ8ngax077mqH1kaGcibaB/9y1IZk5ll/CmI3WiphZzVJ4ZANPcpYP2wjMQaA9GlQlM31jt/XGw8adPSclMl7VDcacrqtWrqg2jHmBc95BVe2FgmWUky1GaM1eQpNS6L3EpMwp3qbe3/WL3k46OqBUbEXGKYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song The current mTHP swapin path have several problems. It may potentially hang, may cause redundant faults due to false positive swap cache lookup, and it will involve at least 4 Xarray tree walks (get order, get order again, confirm swap, insert folio). And for !CONFIG_TRANSPARENT_HUGEPAGE builds, it will performs some mTHP related checks. This series fixes all of the mentioned issues, and the code should be more robust and prepared for the swap table series. Now tree walks is reduced to twice (get order & confirm, insert folio), !CONFIG_TRANSPARENT_HUGEPAGE build overhead is also minimized, and comes with a sanity check now. The performance is slightly better after this series, sequential swap in of 24G data from ZRAM, using transparent_hugepage_tmpfs=always (24 samples each): Before: Avg: 10.67s, stddev: 0.04 After patch 1: Avg: 10.49s, stddev: 0.04 After patch 2: Avg: 10.42s, stddev: 0.05 After patch 3: Avg: 10.45s, stddev: 0.05 After patch 4: Avg: 10.49s, stddev: 0.04 After patch 5: Avg: 9.67s, stddev: 0.03 After patch 6: Avg: 9.67s, stddev: 0.04 After patch 7: Avg: 9.68s, stddev: 0.05 After patch 8: Avg: 9.66s, stddev: 0.04 Several patches improve the performance by a little, which is about ~10% faster in total. Build kernel test showed very slightly improvement, testing with make -j48 with defconfig in a 768M memcg also using ZRAM as swap, and transparent_hugepage_tmpfs=always (6 test runs): Before: avg: 3353.66s, stddev: 33.73 After patch 1: avg: 3354.19s, stddev: 42.54 After patch 2: avg: 3364.16s, stddev: 52.74 After patch 3: avg: 3355.73s, stddev: 36.17 After patch 4: avg: 3352.78s, stddev: 39.80 After patch 5: avg: 3355.19s, stddev: 50.78 After patch 6: avg: 3333.63s, stddev: 32.50 After patch 7: avg: 3297.70s, stddev: 38.93 After patch 8: avg: 3302.35s, stddev: 50.61 --- V4: https://lore.kernel.org/linux-mm/20250704181748.63181-1-ryncsn@gmail.com/ Updates: - Merge patch 5 into patch 8 to fix a intermediate code error. [ Baolin Wang ] - Instead of passing two swap entries, calculate the new order 0 entry in shmem_swap_alloc_folio, to improve code readability. [ Baolin Wang ] - Rebase on top of mm-new. V3: https://lore.kernel.org/linux-mm/20250627062020.534-1-ryncsn@gmail.com/ Updates: - Split and reorganize a few intermediate patches [ Baolin Wang ] - Fix a duplicated fault issue that may occur in one intermediate patch [ Baolin Wang ] - Improve variable naming [ Baolin Wang ] - Fix a wrong error value problem, return proper error value when direct swapin failed. - No major code change from V3. V2: https://lore.kernel.org/linux-mm/20250619175538.15799-1-ryncsn@gmail.com/ Updates: - Split the clean up patch into 3 individual patches [ Baolin Wang ] - Fix a code error in the first patch [ Baolin Wang ] - I found there are some other remaining issue that can be fixed after the clean up so includes these too: fix major fault counter, and clean up the goto labels. V1: https://lore.kernel.org/linux-mm/20250617183503.10527-1-ryncsn@gmail.com/ Updates: - Improve of funtion name and variable names, also commit message [ Kemeng Shi, Dev Jain ] - Correct Fixes: tag [ Andrew Morton ] - Collect Reviewed-by. Two of the patches in this series comes from the swap table series [1], and worth noting that the performance gain of this series is independent to the swap table series, we'll see another bigger performance gain and reduce of memory usage after the swap table series. I found these issues while trying to split the shmem changes out of the swap table series for easier reviewing, and found several more issues while doing stress tests for performance comparision. Barry also mentioned that CONFIG_TRANSPARENT_HUGEPAGE may have redundant checks [2] and I managed to clean them up properly too. No issue is found with a few days of stress testing. Link: https://lore.kernel.org/linux-mm/20250514201729.48420-1-ryncsn@gmail.com/ [1] Link: https://lore.kernel.org/linux-mm/CAMgjq7AsKFz7UN+seR5atznE_RBTDC9qjDmwN5saMe+KL3b1mQ@mail.gmail.com/ [2] Kairui Song (8): mm/shmem, swap: improve cached mTHP handling and fix potential hung mm/shmem, swap: avoid redundant Xarray lookup during swapin mm/shmem, swap: tidy up THP swapin checks mm/shmem, swap: tidy up swap entry splitting mm/shmem, swap: never use swap cache and readahead for SWP_SYNCHRONOUS_IO mm/shmem, swap: simplify swapin path and result handling mm/shmem, swap: rework swap entry and index calculation for large swapin mm/shmem, swap: fix major fault counting mm/shmem.c | 267 ++++++++++++++++++++++++++++------------------------- 1 file changed, 143 insertions(+), 124 deletions(-) -- 2.50.0