From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F557C25B74 for ; Fri, 24 May 2024 08:57:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC0046B0095; Fri, 24 May 2024 04:57:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B6F2B6B0096; Fri, 24 May 2024 04:57:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E9436B0098; Fri, 24 May 2024 04:57:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7CF666B0095 for ; Fri, 24 May 2024 04:57:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 311EA80809 for ; Fri, 24 May 2024 08:57:44 +0000 (UTC) X-FDA: 82152686448.01.1D1B26B Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) by imf23.hostedemail.com (Postfix) with ESMTP id D919714000E for ; Fri, 24 May 2024 08:57:41 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=E5m6I8de; spf=pass (imf23.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716541062; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IrbgcBme4gf384UGODzdSyOVXcWHfhExwH5eqJcgItc=; b=w9lK6kYl00Ru+Jk0+fRCW7ObV8jTgLm+wSKn4HVtzOXLJA7nPyP+KMW6luY4JMa7FctSz/ TvRDXxdXq3yuydL89gHtBR3ZTRnKCMKpSEl4T/vqj0zJdVF9bE3nC5wH073EA9ZopWehII 015cOPHuKf9wH+Ey7bXOLakCr7wrRdw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716541062; a=rsa-sha256; cv=none; b=tAzrPBfNsp4GzTP9B0ViTgLi1lwVwGuqdw7ueesQzHmoK5poNCHmK00137ovwOtBvQo6/C P71gkDzTQR6zmqTapWSlMzA+uy8pXmLI0/E19nWKYu+zvWVwTeoWcTldiVaUXpe+5hqQQe MNrqUQ3EYuViDZInBjXtU6rEfys/T9E= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=E5m6I8de; spf=pass (imf23.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Envelope-To: linux-mm@kvack.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716541060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IrbgcBme4gf384UGODzdSyOVXcWHfhExwH5eqJcgItc=; b=E5m6I8deIoqw6vq/84vM96nA3u/7CWXC5h8vgVpVwMJUh9OwqPhQXgVDBobdnZCcRDd2Gc B5HXL/d0UK9N+vBSADHhJzUu5BnEttIHr756vbu3OZuMvZ340tmEvLOGgNm72mD3NyGRsH zGXnHf/1q761IrZcj6NTN3l/RrrjkK0= X-Envelope-To: hughd@google.com X-Envelope-To: chengming.zhou@linux.dev X-Envelope-To: zhouchengming@bytedance.com X-Envelope-To: shr@devkernel.io X-Envelope-To: david@redhat.com X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: aarcange@redhat.com X-Envelope-To: linux-kernel@vger.kernel.org X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou Date: Fri, 24 May 2024 16:56:51 +0800 Subject: [PATCH 2/4] mm/ksm: don't waste time searching stable tree for fast changing page MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20240524-b4-ksm-scan-optimize-v1-2-053b31bd7ab4@linux.dev> References: <20240524-b4-ksm-scan-optimize-v1-0-053b31bd7ab4@linux.dev> In-Reply-To: <20240524-b4-ksm-scan-optimize-v1-0-053b31bd7ab4@linux.dev> To: Andrew Morton , david@redhat.com, aarcange@redhat.com, hughd@google.com, shr@devkernel.io Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com, Chengming Zhou X-Developer-Signature: v=1; a=ed25519-sha256; t=1716541051; l=2614; i=chengming.zhou@linux.dev; s=20240508; h=from:subject:message-id; bh=1G8p4ukUYdMOk654nZZJtD5LJSaJbjnQHPtspc5amuo=; b=CYjqK00M2lLdR7+WKq2pN5yhtp9lGVF60p4EeSzgxNV31x3wCoi5EIyOTEKvBZMWRWTvsRgmg N2x1EYuvlGYA6VQKuU6yVelvTy0n7Rt4HTLV50/OImvqxM243FfBbNl X-Developer-Key: i=chengming.zhou@linux.dev; a=ed25519; pk=kx40VUetZeR6MuiqrM7kPCcGakk1md0Az5qHwb6gBdU= X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: yapjzudb7xwutqs5f34ydynnxmsd4qxu X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D919714000E X-HE-Tag: 1716541061-493091 X-HE-Meta: U2FsdGVkX18nTZFBtaB1JUKy58MPCEdbVZL+YjJKs18nBSoDRqQ64eY6ebnmhc5aO9RyNxhRhkdd2d5/XqI6G/H/6zEyR5swnJHo4NnjrVgFlOxE4Q8i0z5JFZEJC36XwjZ3XcJ71/rvMObfbeqKmekrM9vHMQpZBqH2Y0VpTsLN+fb8rVnL2k4/nsgUZrPGyzBwn54aQ/W8i3TekHCyLfOGYdrnWOCLi+4pDEHbi3tTEgfS9Bc4dlSd+BCzlK4Xpg2ijs3ztDORqYnnJJ4VMX8XYRZ8DGzZ+FM/5CvvAaRhXbFmZZOPOzJ8ncyfJczgUUIpjMCxkUzr1mg5ubcUVRicuExw9IpnZyddPh4gTI+HXavit17RTKssMj/GS/kbfmU1TIwiAbG6qZDndoCPBQ8DE5AzAAHKBICd+ATCp5/c7gYxvSnAuHHWGEn9DI58rOsINFtXUsYxfGPzSD9AN6grBttMBVpvGdH1HALIL3w2+uesP6LCIOYogD7OMHkA1N8c+vUhK2PVHS1DFpTsP/71uTelzxVaCpNhPBEQmlh9zZ1aPbO1epixxclFY3EaLNogRbNFOjAO8zUuVa1WvBjVKhips6uet62Q43gWvd2VrsJrDK5XnEvQmGiTwJu1bzR+wDvhBJ+KLbdvRhImVGJyAhAZ9nrBIBoBI/czswZ1RGbs1Cpf/9uc3nBxvWd5F9mqURDwIalFBpqRVwCcoOGTTCbcbZgYGnz40BIX8vjUHNAXWk7zji250/+bmAAq0VetYg+a9WEHigt5L8NNPH2ACsGl2YD1OA1hkUxdiuYHROuzzGKvqolbxbIPW7U7tEyKY/jnRwOtJhFkkz3lqlg2cL0SP0BtLk2HJYkt8WA7SEok+uwe1aR/iijdsNwzA7XDZsaXYtnw8ZtW2spm2XtSUlNsAzDOzvOzYkrAiOPde6mULQXAXHZ5dsPobE2egiNBxb5q6kYoGCcg9Yd mk3LGum3 bpt38Xr8DjYbnTeCFoDPkTSLAXIkB0uU/J9a8oaBK7zjnmsJ0ZSFJvW8TVdwVzqj/uCF24V67pnnoFni50AUMkTVOQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The code flow in cmp_and_merge_page() is suboptimal for handling the ksm page and non-ksm page at the same time. For example: - ksm page 1. Mostly just return if this ksm page is not migrated and this rmap_item has been on the rmap hlist. Or we have to fix this rmap_item mapping. 2. But we absolutely don't need to checksum for this ksm page, since it can't change. - non-ksm page 1. First don't need to waste time searching stable tree if fast changing. 2. Should try to merge with zero page before search the stable tree. 3. Then search stable tree to find mergeable ksm page. This patch optimizes the code flow so the handling differences between ksm page and non-ksm page become clearer and more efficient too. Signed-off-by: Chengming Zhou --- mm/ksm.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index cbd4ba7ea974..2424081f386e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2366,6 +2366,23 @@ static noinline void cmp_and_merge_page(struct page *page, struct ksm_rmap_item */ if (!is_page_sharing_candidate(stable_node)) max_page_sharing_bypass = true; + } else { + remove_rmap_item_from_tree(rmap_item); + + /* + * If the hash value of the page has changed from the last time + * we calculated it, this page is changing frequently: therefore we + * don't want to insert it in the unstable tree, and we don't want + * to waste our time searching for something identical to it there. + */ + checksum = calc_checksum(page); + if (rmap_item->oldchecksum != checksum) { + rmap_item->oldchecksum = checksum; + return; + } + + if (!try_to_merge_with_zero_page(rmap_item, page)) + return; } /* We first start with searching the page inside the stable tree */ @@ -2396,21 +2413,6 @@ static noinline void cmp_and_merge_page(struct page *page, struct ksm_rmap_item return; } - /* - * If the hash value of the page has changed from the last time - * we calculated it, this page is changing frequently: therefore we - * don't want to insert it in the unstable tree, and we don't want - * to waste our time searching for something identical to it there. - */ - checksum = calc_checksum(page); - if (rmap_item->oldchecksum != checksum) { - rmap_item->oldchecksum = checksum; - return; - } - - if (!try_to_merge_with_zero_page(rmap_item, page)) - return; - tree_rmap_item = unstable_tree_search_insert(rmap_item, page, &tree_page); if (tree_rmap_item) { -- 2.45.1