From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C025C25B74 for ; Fri, 24 May 2024 08:57:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 221C16B008C; Fri, 24 May 2024 04:57:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D32D6B0092; Fri, 24 May 2024 04:57:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0999D6B0093; Fri, 24 May 2024 04:57:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E13F56B008C for ; Fri, 24 May 2024 04:57:39 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8736D1A09DD for ; Fri, 24 May 2024 08:57:39 +0000 (UTC) X-FDA: 82152686238.08.9D07832 Received: from out-170.mta1.migadu.com (out-170.mta1.migadu.com [95.215.58.170]) by imf22.hostedemail.com (Postfix) with ESMTP id 1CF4DC0016 for ; Fri, 24 May 2024 08:57:36 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=gmsP42Ph; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf22.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716541057; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Ds3tXwBbM94eOORMZkrEUwB+zppk0KBTTGow/Bf7ObE=; b=cm2iqoIpQpDGEoupNdkuHSt3neLhd1irY+dhC6cqazzKA403hioZ4m03lRZrua70bZPpBP lUlPe9cIcUfTwMA6MZLM3SRnzOD+1SaQSccLRUwAi3BKDq2J58RoqaQo54ZXXneD06vHB6 jxI+Ky0WfMh2D6aUH+d5hkNpaM5ExSk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716541057; a=rsa-sha256; cv=none; b=huZ0UTcJ65ojNYA9pw/P0CeBUnxQ5M+lvgDlBxt/xpfsKbRkMOOESWWKiSX2w4zVHSktVH Mtw+nQTskvUV05oFSCl0VIhYd6lbW2CGtJZfVUYZlE0fJQqXJhuPGCFnieZULzColCwRal Ql8nxDHo5/RRWglIK3GFOn9o3SheVE8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=gmsP42Ph; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf22.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev X-Envelope-To: linux-mm@kvack.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716541054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ds3tXwBbM94eOORMZkrEUwB+zppk0KBTTGow/Bf7ObE=; b=gmsP42PhKt2uOV+RAkNceZG54YnzQJokMJoltMBQIAhoYJpf3d1WZb87ZIExln82ufv6Cx 66wGydR9WWv+rj5YYvkTvs7grhOAaUHQgz8g2MbzKj3bBC6OwJwfzCYUxa0d04Calj9ZTZ Y3EW5n4sNTBBeR56/6R1R5mRfrLGokA= X-Envelope-To: hughd@google.com X-Envelope-To: chengming.zhou@linux.dev X-Envelope-To: zhouchengming@bytedance.com X-Envelope-To: shr@devkernel.io X-Envelope-To: david@redhat.com X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: aarcange@redhat.com X-Envelope-To: linux-kernel@vger.kernel.org X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou Subject: [PATCH 0/4] mm/ksm: cmp_and_merge_page() optimizations and cleanup Date: Fri, 24 May 2024 16:56:49 +0800 Message-Id: <20240524-b4-ksm-scan-optimize-v1-0-053b31bd7ab4@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-B4-Tracking: v=1; b=H4sIAFFWUGYC/x3MQQ5AMBBA0avIrE1SVRGuIhbVDiaipCMixN01l m/x/wNCkUmgzR6IdLLwFhKKPAM32zARsk8GrbRRlTY4GFxkRXE24LYfvPJN6PXoG6MKV1Y1pHS PNPL1b7v+fT8bM4zFZgAAAA== To: Andrew Morton , david@redhat.com, aarcange@redhat.com, hughd@google.com, shr@devkernel.io Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com, Chengming Zhou X-Developer-Signature: v=1; a=ed25519-sha256; t=1716541051; l=4789; i=chengming.zhou@linux.dev; s=20240508; h=from:subject:message-id; bh=Qwch/DUifmAp31RvHL48cbakz6SI6sa/rOXdG37/3XU=; b=tc4ETyQ1obSkVNvp3j9PenRnW77MaL27u6kiBfgJ2EWDTRmFBjAxEsvIY9mP+ovHUK/uTGFhu QvlGJNeavitASKO7v+zSljH/WcCfK5JmN7Mtp02Mj6IfTzFo6MGYOYx X-Developer-Key: i=chengming.zhou@linux.dev; a=ed25519; pk=kx40VUetZeR6MuiqrM7kPCcGakk1md0Az5qHwb6gBdU= X-Migadu-Flow: FLOW_OUT X-Stat-Signature: uoh8r9n6jnbtosjfp97tdrc33xt59ma7 X-Rspamd-Queue-Id: 1CF4DC0016 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1716541056-899761 X-HE-Meta: U2FsdGVkX1/YaPzK9EAo8Be9jkAV6W13JSQrgW/U5IW9xKtjqCAc86+1o9go8oVMF5uAtZAQ5lZl9Pw0D9kMwCHR6dlNLNvJBy+Aa82eMDGicQfI0/FrMOjPZp3oRHTvsMxNbC5JMqIt7ncv54t2jXLuK5L5W1vEljKbds3zPdXhLT/UejCOaswk0YClsXDMfp3gO76GIwAiWXd0UEmtU3UCrJcB6XlZ7slUjzPy7B0RehhtOMA+so+Di/+FNFVnsuLp49V66l28DX5VaI7d2KULxLp7wVdr10CPm80/pcuixYiR1GULMU0P5zHrukPSkvZAtL/5EJJhArujICINaykrtRZaOiFX16EoA+nmW3+eaSCoRIUznDKP+m1MwEKjSqBkqPVE72QBqIUdbjWWjH5QJr5M/KXQUZ6eFTEX3CSEOilTIS9ysDsSphAkvQTFJgpVPn7QRQOTiLfExGEJE2Njc7hfkycFvKB2SLdBTCThYWWux39FrsKHFxEBzTjC27dpPenAKQHWDy9QDmbyvXQwAQsVYMGBz9mZmOnE+SFxngjJuRJVqpt/zzZrSsflV8VG+lxPCwbFVBX00DdBMgAh+DoYmfXa+zG5akg7cD7nxLXeEymjgMh6JCiYyDxqWWojld/mhzOwXVCgf3dLDvxAyNsfRiGGnLXOmfXJHZO4plQlOFjwJ0cRrE/JCxkAzAgvoZ/tUzSsSKJKjuvL9YLKKUtztJdlJeupcLIHN3WliDHY1lbkBGbjdL8RnjQNz32r0Az6mLiRlE+x4LrXSTzEUM5Lx/5agpfq8qrxz9y59pM2bKIG58uAK2qg9AM0Sp0I4kgy4JhzzWXcd8oUpBrj/T9kWri0SOzCtK8DOqLpur3vlg+q/SEMNwb3/MFualauPZWQp45arhTXxhKDhDvWNW5Ge63YUme6xKTwg/FNWwrFt3gJR1lTigiXaKcEeodbHfQrtS9MS4U5wQe rxK8vamS OEUFpbfaPLwh3EKOSeC7tqI1A7AwCRjxBTJnTwJRUXEFttbsjmslUypHC1+l24plPAPIgvlnZwBkk2rnMYv/Oi4dC1GTuvNRhgfm+oJVOHcpnSCWO3R5r3AkK9ep56nRX3BA5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello, This series mainly optimizes cmp_and_merge_page() to have more efficient separate code flow for ksm page and non-ksm anon page. - ksm page: don't need to calculate the checksum obviously. - anon page: don't need to search stable tree if changing fast and try to merge with zero page before searching ksm page on stable tree. Please see the patch-2 for details. Patch-3 is cleanup also a little optimization for the chain()/chain_prune interfaces, which made the stable_tree_search()/stable_tree_insert() over complex. In patch-4, fix behaviors in stable_tree_search() when handle migrating stable_node: return the migrated ksm page if no shareable ksm page found on the stable tree, so our rmap_item can be added directly. I have done simple testing using "hackbench -g 1 -l 300000" (maybe I need to use a better workload) on my machine, have seen a little CPU usage decrease of ksmd and some improvements of cmp_and_merge_page() latency: Before: - ksm page [128, 256) 21 | | [256, 512) 12509 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [512, 1K) 769 |@@@ | [1K, 2K) 99 | | [2K, 4K) 4 | | [4K, 8K) 2 | | [8K, 16K) 8 | | - anon page [512, 1K) 19 | | [1K, 2K) 7160 |@@@@@@@@@@@ | [2K, 4K) 33516 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [4K, 8K) 33172 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8K, 16K) 11305 |@@@@@@@@@@@@@@@@@ | [16K, 32K) 1303 |@@ | [32K, 64K) 16 | | [64K, 128K) 6 | | [128K, 256K) 6 | | [256K, 512K) 9 | | [512K, 1M) 3 | | [1M, 2M) 2 | | [2M, 4M) 1 | | After: - ksm page [128, 256) 9 | | [256, 512) 915 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [512, 1K) 41 |@@ | [1K, 2K) 1 | | [2K, 4K) 1 | | - anon page [512, 1K) 374 | | [1K, 2K) 5367 |@@@@ | [2K, 4K) 64362 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [4K, 8K) 27721 |@@@@@@@@@@@@@@@@@@@@@@ | [8K, 16K) 1047 | | [16K, 32K) 63 | | [32K, 64K) 7 | | [64K, 128K) 6 | | [128K, 256K) 5 | | [256K, 512K) 3 | | [512K, 1M) 1 | | We can see the latency of cmp_and_merge_page() when handling non-ksm anon pages has been improved. Thanks for review and comments! Signed-off-by: Chengming Zhou --- Chengming Zhou (4): mm/ksm: refactor out try_to_merge_with_zero_page() mm/ksm: don't waste time searching stable tree for fast changing page mm/ksm: optimize the chain()/chain_prune() interfaces mm/ksm: use ksm page itself if no another ksm page is found on stable tree mm/ksm.c | 266 ++++++++++++++++++++------------------------------------------- 1 file changed, 84 insertions(+), 182 deletions(-) --- base-commit: 2218eca02bc4203f68b8fb7e1116e5a2601506d1 change-id: 20240524-b4-ksm-scan-optimize-d2fd9401c357 Best regards, -- Chengming Zhou